> For the complete documentation index, see [llms.txt](https://karini-ai.gitbook.io/karini-ai-documentation/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://karini-ai.gitbook.io/karini-ai-documentation/recipes/workflow-recipe/set-up-workflow-recipe.md). # Set up Workflow Recipe Workflow recipes can be initiated with either a **Chat node** or a **Webhook node**, depending on the specific use case and the desired interaction flow ### Chat This node is designed to initiate and facilitate user interactions, serving as the entry point for conversational engagement. #### Conversation History: Determine the number of messages to retain in the conversation history. This allows you to augment the prompt context with past conversation, improving the response quality. #### Generate follow up questions: Enable Generate follow-up questions to prompt the system to autonomously generate relevant questions based on the conversation context and generated answer. To use this option, the Follow Up question generator model must be configured in the[ Organization](https://docs.karini.ai/organization). You are provided with a sample prompt for this task; however, you can update it as required. ``` Generate follow-up questions based on the given input question and answer. Follow these guidelines: 1. Number of Questions: Include 2-3 short follow-up questions that directly expand on or clarify the main question. 2. Relevance is Key: Ensure these questions are relevant, showing an understanding of the initial question and its broader implications. 3. Emphasis on Follow-Ups: Always aim to provide follow-up questions to foster deeper dialogue, use the given answer as a basis for further exploration. Reference Question: {question} Reference Answer: {answer} Output Format: Question-1: [provide followup question here] Question-2: [provide followup question here] Question-3: [provide followup question here] ``` #### Enable Audio Mode When Audio mode is enabled, the audio option becomes available on Copilots. This allows users to interact through voice queries and receive responses in both text and audio formats, enhancing accessibility and user experience. You must have Speech to Text Model Endpoint and Text to Speech Model Endpoint configured in the[ Organization](https://docs.karini.ai/organization). {% hint style="info" %} To **integrate** the **Chat** node effectively, follow these configurations based on document handling requirements: * **With Document Handling** : Establish a connection between the Chat and Processing nodes when document uploading, processing, and querying are essential for interaction. * **Without Document Handling** : Directly link the Chat node to the Start node if the workflow does not require document-based references, enabling a standard conversational experience. {% endhint %} ### Webhook A **webhook** is a user-defined HTTP callback that allows the system to send automated messages or data to an external URL when an event occurs. Here are the key elements of a webhook node. **Label**: Serves as a unique identifier for the webhook, making it easy to reference or manage. **Webhook URL**: The endpoint to which the webhook sends HTTP requests. It directs the data to the appropriate destination. **Webhook Token**: Used for authentication to ensure that the request made by the webhook is valid and authorized to access the API. **Query Method**: Specifies the HTTP method (such as POST) used for the request to the API. **Webhook Query Headers**: Defines the headers included in the request, often containing metadata like content type, authorization, or other necessary information for the API to process the request. The webhook request must include the following headers for authentication and content type specification: ``` { "Content-Type": "application/json", "x-api-token": "token_value" } ``` **Payload Template:** A predefined structure or format for the data sent with the request. It helps in organizing the information and ensuring that the API receives the correct data structure. ``` { "files": [ { "content_type": "application/pdf", "file_name": "test1.pdf", "file_content": "(base64data)" }, { "content_type": "application/pdf", "file_name": "test2.pdf", "file_path": "s3://bucket/path" }, { "content_type": "text/plain", "file_name": "test.txt", "text": "this is a plain text" } ], "input_message": "", "metadata": {} } ``` **Generate curl command:** The **Generate curl command** enables users to automatically generate a curl command tailored to the specific configuration of their webhook setup within the recipe workflow. Upon clicking the button, the system displays the corresponding curl command, which replicates the HTTP request setup defined in the workflow. This includes the following components: * **HTTP Method**: The selected HTTP method (e.g., POST, GET) for initiating the webhook request. * **Headers**: Automatically includes required headers, such as `Content-Type` and the API token for authentication (if configured). * **Payload Template**: The request body content is generated based on the defined payload template, which may include file types, file paths, and textual content.
For webhook recipe invocation and webhook history details, refer to the [webhook recipe](/karini-ai-documentation/recipes/workflow-recipe/webhook-recipe.md) section. {% hint style="info" %} #### To ensure optimal functionality of the Webhook node, follow these configuration guidelines: * **With Document Processing:** Connect the **Webhook** node to the **Processing** node when documents must be uploaded, processed, or queried before transmitting data externally. * **Without Document Processing:** If no document handling is required, connect the **Webhook** node directly to the **Start** node. {% endhint %} ### Start The **Start node** serves as the entry point of the workflow, initiating the flow of tasks by connecting to various functional nodes based on specific process requirements. It can link to the **Knowledge base node** for information retrieval, the **Router node** for directing execution based on logic, the **Prompt node** for generating responses, the **Custom function node** for executing predefined tasks, the **Agent node** for intelligent automation, and the **Transform node** for enabling parallel processing. This flexibility allows workflows to be dynamically structured according to operational needs, ensuring efficient execution and automation. ### Knowledge Base A **Knowledge base** is a vector embedding structured repository designed to store, organize, and manage information, enabling applications to retrieve relevant data efficiently. The system supports the following two types of knowledge bases: #### **Native Knowledge Base:** Choose a dataset from the available dataset list. After selecting the dataset, the relevant prompt contexts can be configured to retrieve information from the vector store. For detailed guidance, refer to [Vector DB prompt](/karini-ai-documentation/recipes/knowledgebase-recipe/create-recipe.md#vector-db-prompt). #### **Bedrock Knowledge Base** The Bedrock Knowledge Base refers to an AI-powered retrieval system offered by Amazon Bedrock, a service by AWS (Amazon Web Services). It allows users to integrate enterprise knowledge bases with AI-powered applications, enabling natural language queries on stored data. For more details refer[ Amazon Bedrock Knowledge Bases.](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base.html) The configuration fields include: 1. **Knowledge Base ID:** Enter the ID of the knowledge base. 2. **Filter Key**: Specifies the criteria for filtering results. 3. **Filter Value**: Defines the specific value to filter by. 4. **Number of Results**: Specifies how many responses should be retrieved. 5. **Overwrite Credentials Option** : By default, the AWS credentials configured in[ Organization](https://docs.karini.ai/organization) settings will be used to invoke AWS resource. However, if needed, you can provide alternate AWS credentials. #### **Document Reader** **Document Reader** is designed to facilitate the extraction of information from various document formats, including PDFs, Word files, and other textual formats. This feature integrates seamlessly with the knowledge base, enabling efficient document processing, data extraction, and integration into workflows. 1. **Configuration Settings:** 1. **Connector Type:**\ The **Connector Type** setting defines the method of integrating external data sources. 2. The following connector options are available: * **Amazon S3**: Enables the system to retrieve and process documents stored in Amazon S3 buckets. * **Overwrite Credentials** :By default, the AWS credentials configured in[ Organization](https://docs.karini.ai/organization) settings will be used to invoke AWS resources. However, if needed, you can provide alternate AWS credentials * **Presigned URL**: Supports data retrieval through presigned URLs, which provide temporary. * **Email**: Facilitates the extraction of documents directly from email attachments. 2. **Preprocessing options:** Karini AI recipes support following preprocessing options: 1. **Enable Transcriptions:** Enables automatic transcription, facilitating the conversion of audio or speech-based files into text. 1. **Default:** Using the Default method, the OpenAI Whisper model is employed and should be selected as the Speech-to-Text Model Endpoint for transcription tasks in the [Organization ](https://docs.karini.ai/organization#:~:text=from%20the%20list-,Speech%20to%20Text,-Karini%20AI%20supports)page. 2. **Amazon Transcribe**: Amazon Transcribe is an automatic speech recognition service that uses machine learning models to convert audio to text. 2. **OCR Options:** This option provides various methods for extracting text from documents and images: 1. **Unstructured IO with Extract Images**: This method is used for extracting images from unstructured data sources. It processes unstructured documents, identifying and extracting images that can be further analyzed or used in different applications. 2. **PyMuPDF with Fallback to Amazon Textract:** This approach utilizes PyMuPDF to extract text and images from PDF documents. If PyMuPDF fails or is insufficient, the process falls back to Amazon Textract, ensuring a comprehensive extraction by leveraging Amazon's advanced OCR capabilities. 3. **Amazon Textract**: The selected OCR method is **Amazon Textract**, a cloud-based service that identifies and extracts text, structured data, and elements like tables and forms from documents. 1. **Extract Layouts:** 1. This option helps recognize the **structural layout** of a document, such as: 1. Headings 2. Paragraphs 3. Columns 4. Useful for document formatting retention. 2. **Extract Tables:** 1. This option allows **structured table extraction**, preserving row and column relationships. 2. Useful for processing invoices, reports, and tabular data. 4. **Tesseract**: An open-source OCR engine for extracting text from images and PDFs. 5. **VLM:** A specialized method for processing and extracting text from images or documents using visual language models. You must have VLM configured in the[ Organization](https://docs.karini.ai/organization). 1. The **VLM Prompt** provides a predefined instruction set guiding the AI model on **how to analyze the image** and **what details to extract**. 2. The **VLM Prompt** instructs the system to **analyze an image in-depth**, extracting all visible text while preserving structure and order. Additionally, it provides a **detailed description** of diagrams, graphs, or scenes, explaining components, relationships, and inferred meanings to ensure a **comprehensive textual representation** of the image. The **VLM prompt** is defined as follows: ``` Instruction: Please analyze the attached image thoroughly and provide a detailed textual report that includes the following: 1. Extracted Text: - Extract all text present in the image. - This includes any visible text in labels, signs, titles, annotations, captions, legends, or embedded within diagrams, flowcharts, graphs, or any other elements. - Preserve the order and structure as it appears in the image. 2. Detailed Description: - Describe everything that is happening or depicted in the image. - For flowcharts or diagrams: - Explain each component, including shapes (e.g., rectangles, diamonds), connectors (e.g., arrows, lines), and how they relate to each other. - Describe the process flow, decision points, inputs, outputs, and any loops or cycles. - For graphs or charts: - Identify the type of graph (e.g., bar chart, line graph, pie chart). - Describe the axes, labels, data points, trends, and any significant patterns or anomalies. - For scenes or images with objects: - Describe all objects, people, animals, and elements present. - Include details about their appearance, positions, actions, expressions, and interactions. - Mention any colors, shapes, and sizes that are relevant to understanding the image. - Describe the background and setting to provide context. 3. Interpretation and Contextual Information: - Provide any inferred meanings, implications, or conclusions that can be drawn from the image. - Explain the purpose or function of the diagram, flowchart, or scene if apparent. - If the image represents a concept, process, or data, elaborate on what it signifies. Formatting Guidelines: - Organize your response into clear sections with headings: "Extracted Text," "Detailed Description," and "Interpretation." - Use bullet points or numbered lists where appropriate for clarity. - Ensure that the description is comprehensive and allows someone to fully understand the image without seeing it. ``` **PII Masking Options**: To mask Personally Identifiable Information (PII) within your dataset, enable the PII Masking option. You can specify the entities to be masked by selecting from the available list, ensuring secure data preprocessing. For more details, refer to this [PII entities](https://docs.aws.amazon.com/comprehend/latest/dg/how-pii.html). The following **state flags** need to be configured based on the use case . #### State settings State settings **control data access** within the workflow, ensuring that nodes can retrieve and process relevant information. 1. **Messages:** Accesses the conversation history for processing, with options for full, last, or specific node messages. 1. **All Messages:** Accesses the entire conversation history, allowing the node to consider all previous interactions for context. 2. **Last Message**: Accesses only the most recent message in the conversation, useful for nodes that need to respond to the latest input. 3. **Node Message**: Accesses messages from a specific node in the workflow, ideal for retrieving targeted information shared by a particular node. 2. **Scratchpad**: The Scratchpad acts as a shared, temporary storage space (in JSON/dictionary format) accessible by all nodes in the workflow. Nodes can read from or write to this space as needed. This facilitates intermediate data retention and seamless data transfer between nodes without long-term persistence. 1. **Input**: Allows nodes to read data from the Scratchpad state by specifying a valid JSON path. The node retrieves data stored in the Scratchpad state for further processing. 2. **Output:** 1. Allows nodes to write data to the Scratchpad state by specifying a valid JSON path. You can choose to write the output of a node into the Scratchpad for future steps. \ Additional options include 2. **Extract JSON** : Attempts to extract the first valid JSON object from the node's output and store it in the Scratchpad state. 3. **Overwrite** : Replaces the existing value at the specified JSON path with the new value from the node’s output. (if disabled, it will try appending the value or persist the older value if appending is not possible.) ### Reranker The **Reranker** node is used to improve the relevance of search results returned from one or more connected **Knowledge Base** nodes. It takes the candidate results from upstream knowledge bases and reorders (and optionally filters) them based on a relevancy score produced by a reranking model. The re-ranked results are then passed to the next node in the recipe. #### **Messages**: Accesses the conversation history for processing, with options for full, last, or specific node messages. 1. **All Messages**: Accesses the entire conversation history, allowing the node to consider all previous interactions for context. 2. **Last Message**: Accesses only the most recent message in the conversation, useful for nodes that need to respond to the latest input. 3. **Node Message**: Accesses messages from a specific node in the workflow, ideal for retrieving targeted information shared by a particular node. #### **Reranker Query** This enables the user to select the **Node** and the **Message**. The configuration allows: * **Node**: Choose the specific node. * **Message**: Choose which message from that node to use (for example, Last). 1. **Enable Reranker:** By default, it’s enabled when configuring the node. 2. **Top-N** * Specifies the maximum number of top-ranking vectors (results) to return after reranking. * This value must be less than or equal to the top\_k parameter used when querying the knowledge base. 3. **Reranker Threshold** * Defines the minimum relevancy score a result must meet to be included in the final list. * The model selects up to Top-N results whose score is greater than or equal to this threshold. * A higher threshold yields fewer but more relevant results; a lower threshold is more permissive and can return a larger set of results. By configuring the Reranker node selecting the appropriate message context, query source, Top-N value, and relevancy threshold you ensure that downstream nodes receive only the most relevant knowledge base results, significantly improving the quality of responses generated by the recipe. {% hint style="info" %} Note:To use the Reranker node, a reranker model must be configured in the Organization settings. {% endhint %} ### Connector The Connector node functions as an interface facilitating data exchange between the system and external storage solutions. There are two available connector types, as listed below. * **Amazon S3** : Enables integration with Amazon Simple Storage Service (S3) for retrieving or storing data. * **Overwrite Credentials** :By default, the AWS credentials configured in[ Organization](https://docs.karini.ai/organization) settings will be used to invoke AWS resources. However, if needed, you can provide alternate AWS credentials * **Presigned URL**: Allows access to data stored at a presigned URL, providing temporary access to files hosted on cloud storage systems. * **In Memory Base64 Data** : Supports handling Base64-encoded data stored in memory for temporary or intermediate processing. * **Box**: Facilitates integration with **Box**, a cloud storage service, allowing seamless interaction with documents and files stored in Box. * **Email**: It allows the system to process and extract documents directly from **email attachments**. This is particularly useful when you need to work with incoming email data in workflows. * **Email Count:** Defines the number of email to process at once. * **Date:** Specifies that emails will only be pulled if they were received after the provided date * **Credentials** * **Overwrite Credentials**: Enables entering **specific IMAP server credentials**. * **IMAP Server**: The email server address (e.g., imap.gmail.com). * **Email**: The email address for access. * **Email Password**: The password for authentication. {% hint style="info" %} The **Connector** node can be linked to either the **Start** node or the **Processing** node, depending on the workflow requirements. {% endhint %} ### Processing The Processing node is employed in the workflow recipe to enable file uploads to Copilot for querying. It streamlines the processing of uploaded files by providing configurable options designed to support specific data extraction and privacy requirements. The Processing options are detailed in the [Document reader](#document-reader) section;refer to it for comprehensive information. {% hint style="info" %} Connect the **Processing** node to the **Start** node to initiate workflow execution. {% endhint %} ### Router Router directs the workflow to the next node based on conditions or input, enabling dynamic branching paths within the workflow recipe. This ensures that the workflow can adapt based on the given data or context, enhancing flexibility and decision-making in the process. Node Methods: The Router supports three node methods for decision-making. * **Default**: This method follows the standard routing logic, processing data without additional customization. It adheres to a predefined flow, ensuring consistency and simplicity for straightforward workflows that don't require dynamic decision-making. Using the **Default** method, routing conditions can be assigned to **edges** to determine the appropriate node for processing the request.
There are two available options: * **Default Routing:** which applies when no specific conditions are met. * **Custom Routing**-:where you can define explicit conditions for each edge in the provided text box. {% hint style="info" %} The Default Routing Condition can be assigned to only one edge within the routing configuration. {% endhint %} * **Prompt**: * The Prompt method enables the selection of a predefined prompt for the Router node from the available prompt list. * The selected prompt contains instructions that guide the router on how to direct the workflow. * The router will evaluate the input and route the workflow accordingly based on the prompt’s logic. * Here is the provided sample prompt: ```` You will receive a set of input messages containing a task. Your role is to analyze the latest (last) input message and provide the correct routing output based on that task. Use previous messages in the conversation as context only if necessary, but prioritize the latest (last) message for decision-making. Follow these instructions carefully: 1. If the latest input **starts with coding:**, respond with: ```{{"next": "Policy Agent"}}``` 2. If the latest input **starts with sales data:**, respond with: ```{{"next": "Enterprise Sales Agent"}}``` 3. If the latest input **starts with QNA:**, respond with: ```{{"next": "Bot QnA"}}``` Provide only the JSON output based on the identified task. Only response with route options, don't say anything else. ```` {% hint style="info" %} **Only published prompts, along with current versions and associated models, are visible within the recipe.** {% endhint %} * **Lambda**: The Lambda method integrates AWS Lambda functions to execute custom logic. * **Lambda ARN** : Enter the Amazon Resource Name (ARN) of the Lambda function you want to invoke. This uniquely identifies the Lambda function within AWS. * **Input test payload**: This is a sample payload that will be used to test the Lambda function. This helps ensure the function behaves as expected with the provided input. * **Test button** : Enables you to validate the function by executing the test payload. * **Overwrite credentials option** → By default, the AWS credentials configured in[ Organization](https://docs.karini.ai/organization) settings will be used to invoke AWS resource. However, if needed, you can provide alternate AWS credentials #### **Invoke Retries:** specifies the number of retry attempts when an execution fails, ensuring improved reliability and fault tolerance in processing. #### **State settings** **Document Cache**: Provides access to shared document information across nodes. 1. Retrieve Documents: Fetches the entire document based on a specified filename or file path. Useful for accessing full document content from the knowledge base 2. Retrieve Chunks: Fetches specific document chunks based on semantic similarity, ideal for retrieving only relevant parts of a document related to the query. 3. Ephemeral: Pass context as raw text directly with the query without storing it in the knowledge base. #### **Metadata**: Provides access to webhook metadata from connected APIs, enabling external data flow. To view more details about **Messages** and **Scratchpad**, refer to the preceding [state settings](#state-settings) section. {% hint style="info" %} Connect the Router node to Knowledge Base, Agent, Prompt, Custom Function or Transform nodes based on the specific use case. {% endhint %} ### Prompt The Prompt Node is enabling the system to execute logic-driven actions based on predefined prompts. You can select a prompt from the existing prompts in the prompt playground to add to the recipe. Once a prompt is selected, the system displays the **associated primary and fallback models**, along with **guardrails** attached if they were configured in the Prompt Playground. Additionally, you can navigate to the Prompt Playground by clicking the **redirect** icon, allowing them to view the prompt details and make necessary modifications. The following image illustrates the **version** and **redirect** icon.
To switch to **a specific version**, click on the d**isplayed version**. This will generate a list of all associated versions. Select the required version, and the system will load the complete prompt details corresponding to the selected version. The following image displays the associated versions.
{% hint style="info" %} **Only published prompts are visible within the recipe.** {% endhint %} Refer sample prompt. 
Answer the question based on the context below. Keep the answer short and concise. Respond "Unsure about answer " if not sure about the answer.

{context}

{question}.
The **Guardrail** option is available on this tile. You can choose from existing guardrails in the **Prompt Playground**, which will be reflected in the recipe upon prompt selection. Alternatively, you may enable the **default guardrail** configured at the organizational level. **Invoke Retries** specifies the number of retry attempts when an execution fails, ensuring improved reliability and fault tolerance in processing. The [**State settings**](#state-settings) are detailed in the preceding section; refer to it for comprehensive information. #### Metadata: Metadata refers to supplementary information that provides context, details, or additional insights about a specific process, event, or interaction. Within the framework of prompts, metadata enables the dynamic exchange of external data, facilitating more informed and context-aware decision-making. {% hint style="info" %} For metadata to be utilized, the prompt must explicitly reference the {metadata} variable within its body {% endhint %} Metadata includes user-specific data, enabling the agent to personalize interactions and enhance user experience. Metadata can be reviewed in the copilot history. Ensure that the scratchpad is enabled and that the prompt includes Scratchpad as a variable. #### Online Evaluation: **Online Evaluation** provides real-time quality assessment of LLM generated responses during runtime. It evaluates responses against multiple metrics to ensure quality, relevance, and adherence to guidelines without requiring separate evaluation datasets.\ This evaluation uses the **NLA (Natural Language Assistant)** model which is configured in the [organization ](/karini-ai-documentation/organization.md)page to analyze and score responses based on criteria such as relevancy, accuracy, and adherence to guidelines.
**Evaluation Metrics:**\ The system evaluates responses across the following key metrics:
MetricDescriptionScore Range
Answer RelevancyMeasures how relevant the response is to the user's query0-5
Answer FaithfulnessEvaluates if the response is faithful to the provided context, checking for hallucinations or errors0-5
Context SufficiencyAssesses whether the provided context is sufficient to answer the query, and identifies any missing info0-5
Guideline AdherenceChecks if the response follows specified guidelines and complies with custom rules0-5
**Viewing Evaluation Results:** Upon execution, evaluation results are made available in **Copilot History**. To access these results, follow the steps below: 1. Navigate to **Copilot**. 2. Select the **Action** button for the Copilot you want to view. 3. Click on **View History**. 4. The **View** button and **View trace** will become available. Click it to access the results. 5. Evaluations can be reviewed through both the **View** button and the **View Traces** button for further insights. **View:** Now you will be able to review the evaluation results and reasoning for each response. The results will include scores for each evaluation metric, along with a detailed reasoning summary for each assessed response.
**View Traces:**\ Evaluation results are also accessible within traces, where they are incorporated into the overall execution context. This integration allows users to analyze the evaluation results in conjunction with the broader execution flow and performance data, offering a comprehensive view of how the evaluation contributes to the overall system performance.
{% hint style="info" %} Connect **Prompt** node to **End** node in recipe. {% endhint %} ### Agent Select an agent prompt from the available options. Each prompt encompasses pre-configured tools and settings essential for processing inquiries and responding with effectiveness. Once you've selected the prompt, the canvas will reveal the tools and configurations integrated into the agent prompt. You can update the configurations for the preset tools. You cannot add or delete an agent tool from the recipe canvas. In order to edit tool types, or add or delete tools, you need to edit the[ agent prompt](https://docs.karini.ai/prompt-management/agentic-prompts) in the prompt playground. These tools empower the agent to analyze queries thoroughly and generate precise responses. Once an **agent** is selected, the system displays the **associated primary and fallback models along with current version.** Additionally, you can navigate to the Prompt Playground by clicking the **redirect** icon, allowing them to view the agent details and make necessary modifications. The image below shows the recipe using the **Agent node**, including its version and the redirect icon for **Agent prompts.**
To switch to **a specific version**, click on the d**isplayed version**. This will generate a list of all associated versions. Select the required version, and the system will load the complete agent details corresponding to the selected version. The following image displays the associated versions.
{% hint style="info" %} **Only published agent prompt are visible within the recipe.** {% endhint %} Refer sample agent prompt. ``` You are a vehicle guide expert specializing in automotive manuals and answering queries based on the provided context. The context includes details on Keys, Doors, and Windows; Seats and Restraints; Storage; Instruments and Controls; Lighting; Infotainment System; Climate Controls; Driving and Operating; Vehicle Care; Service and Maintenance; Technical Data; Customer Information; Reporting Safety Defects; OnStar; Connected Services; Only use the tools to answer the user's query. Your primary role is to assist users with vehicle-related inquiries, provide accurate information from the context, and send email notifications if requested. Key Responsibilities: {key_responsibilities} Important Instructions: {important_rules} Guidelines for User Interactions: {interaction_guidelines} ``` **Invoke Retries** specifies the number of retry attempts when an execution fails, ensuring improved reliability and fault tolerance in processing. The [**State settings**](#state-settings) are detailed in the preceding section; refer to it for comprehensive information. For a comprehensive overview of the online evaluation process, please refer to the[ online evaluation metrics](/karini-ai-documentation/recipes/workflow-recipe/set-up-workflow-recipe.md#online-evaluation) outlined in the section above. #### Metadata: Metadata refers to supplementary information that provides context, details, or additional insights about a specific process, event, or interaction. Within the framework of agents, metadata enables the dynamic exchange of external data, facilitating more informed and context-aware decision-making. {% hint style="info" %} For metadata to be utilized, the agent must explicitly reference the {metadata} variable within its body {% endhint %} Metadata includes user-specific data, enabling the agent to personalize interactions and enhance user experience. Metadata can be reviewed in the copilot history. Ensure that the scratchpad is enabled and that the agent prompt includes Scratchpad as a variable. Refer [Online evaluation ](#online-evaluation)for comprehensive information. {% hint style="info" %} Connect **Agent** node to **End** node in recipe. {% endhint %} ### End Marks the conclusion of a workflow, signaling that no further actions are required. ### Sink Integrating the Sink node into a workflow facilitates the secure storage, export, or transmission of processed data from upstream nodes to a designated destination. This ensures that the final output is efficiently managed, preserved, and made available for further use or analysis. The Sink node is designed to seamlessly store structured or unstructured data in cloud storage solutions like Amazon S3 or database. There are four output types: * **Connector** Sends the output to an external service through a configured connector, enabling seamless integration with systems outside the platform. This is particularly useful for storing, forwarding, or processing data externally. When using a connector such as **Amazon S3**, the following configuration details are required: * S3 Bucket Path : Provide the target Amazon S3 bucket path where the data will be stored. * File Name Pattern : Allows you to define a structured naming pattern for saved files. {% hint style="success" %} Filename Pattern Guide: Define custom filenames for saved files using dynamic placeholders. Use the following placeholders to define your filename: \ 1 - {filename\_prefix} → File name prefix. \ 2 - {current\_datetime} → The timestamp when the file is saved. \ 3 - {metadata.KEY} → Metadata value (replace KEY with an actual key) Examples: * "processed/{filename\_prefix}{metadata.project}{current\_datetime}.json" * "processed/{filename\_prefix}{current\_datetime}.json" {% endhint %} {% hint style="success" %} Note: Files are saved in .json format by default. {% endhint %} * **Save raw** * When enabled, the raw output from the previous node is saved as a .txt file. When disabled, the first valid JSON from the output is extracted and saved as a .json file. * **Lambda :** Data will be pre-processed within an **AWS Lambda function** before transmission to the output , enabling dynamic transformation such as formatting, filtering, enrichment, and other rule-based modifications to ensure data integrity and compliance with business logic. * **Lambda ARN** : Enter the Amazon Resource Name (ARN) of the Lambda function you want to invoke. This uniquely identifies the Lambda function within AWS. * **Input Variables:** * Defines the parameters or variables to be passed to the Lambda function. * These variables allow for dynamic data handling and contextual processing. * **Input test payload**: This is a sample payload that will be used to test the Lambda function. This helps ensure the function behaves as expected with the provided input. * **Test button** : Enables you to validate the function by executing the test payload. * **Knowledge Base :** When the **Output Type** is set to **Knowledge Base**. In this case, the output from the recipe is directed to a knowledge base for storage or future retrieval. * Key configuration settings for this setup include: * **Dataset**: You are required to select the dataset where the output will be stored within the knowledge base. * **Type**: The type selection defines how the data will be processed or indexed within the knowledge base. In this case, **OpenSearch** is present, which integrates with OpenSearch for indexing and querying stored data. This setup ensures that data is searchable and can be easily retrieved when needed. * **Knowledge Graph:** In this setup, the final output of the recipe is directed into a knowledge graph system for structured data storage, enabling complex relationships and querying between entities. * Key elements in the configuration are: * **Type**: The **Neptune** refers to **Amazon Neptune**, a managed graph database service optimized for storing and querying highly connected data. This configuration ensures that the data is structured and stored in a graph format, making it suitable for complex querying, analytics, and relationship mapping. * **Dataset**: The **Dataset** field requires you to select a dataset where the data will be stored within the graph database. This helps categorize and organize the data appropriately. **State Settings**: Configures data access within the workflow, controlling what information nodes can retrieve and process.. * **Message :** Retrieves only the most recent message(Last message) in the conversation, making it ideal for nodes that require processing the latest user input. {% hint style="info" %} Connect the Prompt, Agent, Custom Function, and Transform nodes to the Sink node as required based on workflow needs. {% endhint %} ### Transform The Transform module provides a **Split** and **Merge** node that enables users to manipulate data by either splitting it into smaller chunks or merging multiple segments. This functionality is particularly useful in data processing workflows where structured transformation of information is required. There are two available node methods, listed as follows: * **Split :** The **Split Node** divides input data into multiple segments and routes each segment through downstream workflow nodes for independent processing. The data is split based on a user-defined strategy, ensuring each segment meets specific criteria. This approach improves data processing, retrieval, and transformation efficiency and supports multiple segmentation strategies, including: * **Character**- Flattens the input into a single text block and splits it into segments of a fixed number of characters. * **Words**- Flattens the input into a single text block and splits it into segments of a fixed number of words. Preserves whole words at segment boundaries, making it a good default for natural-language content. * **Lambda**- Delegates the splitting logic to Lambda function. The function receives the input data and must return a list of segments. * **Transform Type** : Lambda function receives an event payload where the entire input is wrapped under the key input. Ensure your function extracts and processes data accordingly. * **Lambda ARN** : Enter the Amazon Resource Name (ARN) for the AWS Lambda function that will process and split the data. * **Input Test Payload:** Enter test data to validate the Lambda function’s behavior before deployment. * **Test Button** : Allows you to execute a test run of the configured Lambda function for validation. * **Overwrite Credentials (Optional)** → Allows you to override existing authentication settings with new credentials. * **Pages** - Groups document pages into segments of a specified number of pages. Use this when your input is a multi-page document and you want to process a set number of pages together. * **Item List:** Parses the input as a JSON array and creates one segment per array element. Each element (string, object, or nested array) becomes its own segment. * **Split size**: Split size controls how large each segment is when the split node divides the input. Its meaning depends on the selected splitting strategy: | Strategy | What split size represents | Example | | --------- | -------------------------------- | ------------------------------------------------------------------- | | Character | Number of characters per segment | Split size 1000 → each segment contains up to 1,000 characters | | Words | Number of words per segment | Split size 500 → each segment contains up to 500 words | | Pages | Number of pages per segment | Split size 2 → each segment contains 2 consecutive pages | | JSONL | Not applicable | Each array element becomes its own segment regardless of split size | | Lambda | Not applicable | The Lambda function determines segment boundaries | * **Sequential Processing:** By default, the Split Node processes all segments concurrently. When **S**equential Processing is enabled, segments are processed one at a time, in order, ensuring that each segment completes before the next one begins. * When to use sequential processing: | Scenario | Recommended Setting | | ----------------------------------------------------- | ---------------------------- | | Segments are independent; maximize throughput | Sequential **off** (default) | | Downstream services enforce rate limits or throttling | Sequential **on** | | Output order must exactly match input order | Sequential **on** | | Segments have ordering dependencies | Sequential **on** | | Processing a small number of segments | Sequential **on** | | Processing many independent segments quickly | Sequential **off** | * **Scratchpad:**. It serves as a temporary storage or intermediary space for processing and managing data within the workflow. The Split operation utilizes an input method, meaning it receives the output from the preceding Scratchpad as its input for the current Split process. * * **Merge:** The Merge node in the Transform module is used to combine multiple data segments into a unified structure. This is particularly useful when working with split data that needs to be reconstructed or when consolidating multiple data sources into a single format. * Merge Strategy :The Merge strategy determines the format in which the data will be merged. The available options include: * JSON → Combines data in a structured JSON format. * Lambda → Utilizes a custom AWS Lambda function to programmatically merge data. * Lambda ARN: Provide AWS Lambda function’s Amazon Resource Name (ARN). * Input Test Payload: Sample input data to test the transformation logic. * Test Button: Allows you to validate the function’s processing behavior. * Overwrite Credentials (Optional) → Allows you to override existing authentication settings with new credentials. * **Text** : Merges content into a plain text format.
* **Merge method** -The Merge method determines how the merging process handles existing data. Two options are available: * Overwrite- Replaces any existing data with the newly merged data. This ensures that only the most recent merged version is retained. * Append- Instead of replacing, new data is added to an existing array or list within the JSON structure. * **Scratchpad** - Temporary storage is used to retain intermediate data before final output. * **Output**: The merged data is written to an output location. * **Method** Selection: * **Overwrite**: Replaces the existing data. * **Extend**: Allows appending new data instead of replacing it. ### Pass Through The **Pass Through** node is a utility node that forwards the incoming payload to the output without modification. It performs no transformation, validation, or decision logic, and does not alter the structure or content of the data. At runtime, it routes the payload to the appropriate output based on the current state settings. Use it to preserve data flow continuity, connect nodes, or act as a placeholder in a workflow. ### Deep agent Select an **deep agent** prompt from the available options. Each prompt encompasses pre-configured tools and settings essential for processing inquiries and responding with effectiveness. Once you've selected the prompt, the canvas will reveal the tools and configurations integrated into the agent prompt. You can update the configurations for the preset tools. You cannot add or delete an agent tool from the recipe canvas. In order to edit tool types, or add or delete tools, you need to edit the deep agent prompt in the prompt playground. These tools empower the agent to analyze queries thoroughly and generate precise responses. Once an **deep agent** is selected, the system displays the **associated primary and fallback models along with current version.** Additionally, you can navigate to the Prompt Playground by clicking the **redirect** icon, allowing them to view the agent details and make necessary modifications. The image below shows the recipe using the **Deep Agent node**, including its version and the redirect icon for **Deep Agent prompts.**
{% hint style="info" %} **Only published deep agent prompt are visible within the recipe.** {% endhint %} ### Guardrail The Guardrail Node is a dedicated workflow node that acts as a safety checkpoint within an agentic workflow. It evaluates user input against configurable safety policies before the request reaches downstream nodes. If the input violates any policy, the Guardrail Node blocks the request, returns a configurable blocked message, and terminates the workflow early by routing to the END node. If the input passes all checks, the workflow continues to the next connected node. Use the Guardrail Node when you need to enforce content safety, PII protection, or topic restrictions at a specific point in your workflow independent of any LLM or agent node.
A recipe that includes a Guardrail node is shown below.
#### Guardrail Node and Inline Guardrails The platform supports guardrail enforcement through two implementation models: a dedicated **Guardrail Node** and **inline guardrail configuration** within supported LLM-driven nodes. The appropriate model depends on whether guardrail validation is required at the workflow level or at an individual node level. | Feature | Guardrail Node | Inline Guardrails (Prompt / Agent / Router Nodes) | | ---------------------- | --------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- | | **Placement** | Implemented as a standalone node within the workflow graph | Configured directly within an existing LLM-powered node | | **Input Validation** | Supported | Supported | | **Output Validation** | Not supported; designed for input validation only | Supported when `guardrail_output` is enabled | | **Block Handling** | Routes execution to `END`, terminating the workflow | Replaces the node response with the configured blocked message | | **Primary Use Case** | Enforces validation at the beginning of the workflow before LLM processing occurs | Applies targeted input and/or output protection at the individual node level | | **Streaming Behavior** | Not applicable | For terminal nodes, output guardrail validation suppresses streaming until the validation step is completed | ### Semantic Cache The Semantic Cache node is a vector-similarity caching layer that stores and retrieves LLM-generated responses on the basis of *semantic* equivalence rather than exact string matching. When an incoming query is sufficiently close to one that has been answered before, the previously generated response is returned directly, bypassing downstream retrieval and inference. The result is lower latency, reduced inference cost, and higher throughput for repeated or near-duplicate queries, with no measurable degradation in answer quality when the node is configured appropriately. The node operates in one of two modes: * **Query Cache (Lookup):** Place this node near the beginning of the workflow to check whether a valid cached response already exists. * **Create Cache (Storage):** Place this node after the response has been generated to store the result for subsequent requests. *Both nodes must reference the same **Cache Name** to ensure they interact with the same logical cache instance.* #### Query Cache (Lookup) The Query Cache node evaluates an incoming question against the cache. When a stored answer meets the similarity threshold and passes the feedback and permission filters, it is returned immediately and the downstream pipeline is bypassed (***cache hit***). Otherwise, execution continues along the normal path (***cache miss***).
Refer to the following table to configure the **Query Cache** node. **Lookup Configuration** | Field | Type | Default | Description | | ------------------------------ | ------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Cache name | Text | — |

1.Unique identifier for the cache.

2.Must begin with a letter and contain only letters, digits, and underscores. 3.Must match the corresponding Create Cache node.

| | Similarity threshold | Decimal (0–1) | `0.92` |

1.Minimum cosine similarity between the incoming question and a stored entry required for a hit.

2.Higher values enforce closer semantic matches. See Similarity Threshold section.

| | User feedback strategy | Option | Exclude negative | Governs how historical thumbs-up / thumbs-down ratings influence retrieval. See [User Feedback Strategy ](#user-feedback-strategy)section. | | Negative feedback invalidation | Toggle | Off | When enabled, a single thumbs-down permanently removes the affected answer from reuse for all users. | | Include user permission | Toggle | Off | When enabled, incorporates the requesting user's permission groups into the lookup, producing permission-aware hits. | #### Similarity Threshold The **Similarity threshold** defines the minimum cosine similarity, in `[0, 1]`, required for a stored answer to qualify as a hit. It directly governs the precision–recall trade-off of the cache. * A **higher** threshold increases precision: hits are closer semantic matches, but fewer questions qualify, reducing the hit rate. * A **lower** threshold increases recall: more questions produce hits, at the cost of occasionally returning a less relevant answer. | Threshold | Characteristic | | ----------------------------------- | ----------------------------------------------------------------------------- | | `0.92` (recommended starting value) | High-precision matches; conservative reuse. | | `0.85` | Broader coverage with acceptable semantic drift. | | Below `0.80` | May surface only loosely related answers; not recommended without validation. | Begin at `0.92` and adjust incrementally, measuring hit rate against answer relevance in your domain. Thresholds should be tuned per use case rather than applied uniformly, since acceptable semantic drift varies with subject matter. #### User Feedback Strategy The cache incorporates thumbs-up and thumbs-down ratings collected from chat interactions to keep low-quality answers from being reused. The **User feedback strategy** sets the eligibility rule applied at every lookup, and the separate **Negative feedback invalidation** toggle provides an additional, permanent safeguard on top of it. #### Rating States Every cached answer occupies one of three rating states, determined by the ratings it has accumulated across its original generation and all subsequent reuses: * **Unrated** : no user has rated the answer. * **Positive** : the answer has received at least one thumbs-up and no thumbs-down. * **Negative** : the answer has received at least one thumbs-down from any user. Each strategy decides which of these states is eligible for reuse. Because a single thumbs-down from any user places an answer in the negative state, that answer is treated as negative for every subsequent request, not only for the user who rated it. #### Available Strategies | Strategy | Behavior | | ------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Exclude negative** *(recommended)* |

| | **Positive only** |

| | **All feedback** |

| #### Negative Feedback Invalidation **Negative feedback invalidation** is an independent toggle that applies on top of whichever strategy is selected; it is not a fourth strategy. When enabled, a single thumbs-down from any user permanently retires the affected answer for all users, rather than merely filtering it out at lookup time. The distinction is one of scope and permanence. A strategy filters eligibility at each lookup and leaves the stored answer intact; invalidation permanently removes the affected entry so that it can never be served again. It may be combined with any of the three strategies. > **Note:** `Exclude negative` is the recommended default for most production deployments. Reserve `Positive only` for cases where the cost of a wrong answer is high, and `All feedback` for early deployments where ratings have not yet accumulated. *** #### Include user permission When **Include user permission** is enabled on a Query Cache node, the requesting user's permission groups are factored into the lookup. An answer stored under a set of permission groups is served only when the requesting user belongs to *every* group associated with that answer - the caller's groups must be a superset of the answer's groups. This guarantees that: * Users at one access level never receive an answer cached for a different level. * Content cached for privileged users is never served to restricted users. Enable permission-aware caching for multi-tenant deployments containing permission-scoped documents, and for any scenario in which the same question may produce different answers depending on the caller's access rights. #### **Create Cache (Storage)** The Create Cache node persists a question–answer pair, together with its metadata -citation references, time-to-live, and, where enabled, permission tags. It is normally positioned after the answer has been generated and validated. | Field | Type | Default | Description | | ----------------------------- | --------- | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Cache name | Text | — | Must match the Query Cache node so both operate against the same cache. | | Enable classification | Toggle | Off |

1.When enabled, an LLM determines whether each answer is eligible for caching, excluding errors, refusals, and ephemeral or highly personalized responses.

2.Adds one LLM call to the storage path.

| | Model | Selection | — | Model used for classification. Displayed only when classification is enabled. | | Customize citation extraction | Toggle | Off | Enables custom rules for extracting source references from answers. | | Citation regex patterns | Tags | — |

1.One or more custom extraction patterns.

2.Displayed only when citation extraction is enabled. 3.Each pattern must expose a named capture group (?P\...).

| | TTL (hours) | Integer | 720(30days) |

1.Duration for which a stored answer remains valid before it expires.

2.Set to 0 to disable expiration.

| | Cache key | Option | Start |

1.The node whose output is used as the question for the stored entry.

2.Defaults to the user's original input; select a downstream node only when the query is transformed before it becomes meaningful for caching.

| #### Response Classification When **Enable classification** is active on the Create Cache node, an LLM evaluates each answer's suitability for caching before it is stored. This adds one LLM call to the storage path and should therefore be reserved for pipelines where caching eligibility is non-trivial. The following categories are classified as *not cacheable* and are skipped: 1. Refusals or scope limitations (for example, "I can't help with that"). 2. Answers expressing uncertainty without substantive content ("I don't know"). 3. System or retrieval failures ("No documents found", timeouts). 4. Clarifying questions or deflections containing no answer. 5. Empty or trivially short responses. Only substantive, complete answers are stored. Where classification itself cannot complete, the system conservatively skips the write rather than caching an unverified answer. Enable classification for recipes in which errors or refusals are common, so that transient or low-value responses are never persisted. #### Citation Tracking and Invalidation When answers cite source documents, the cache can extract and store those references so that a change to a source document automatically invalidates every cached answer derived from it. This prevents stale answers from being served after a document is updated or removed. #### Testing Custom Citation Extraction Custom citation extraction uses regular expressions to identify citation references within unstructured text. Testing verifies that each expression is syntactically valid, detects the intended citation formats, captures the required source information, and avoids matching unrelated content.
Follow these steps to test and verify citation extraction patterns. **Steps:** 1. Enable custom citation extraction. 2. Add the required regular expression patterns. 3. Enter sample text containing valid citations and non-citation content. 4. Run the test. 5. Review the results: * **Status:** Confirms whether the test completed successfully. * **Valid:** Indicates whether the regular expression syntax is correct. * **Matches:** Shows the number of citations detected. * **Sources:** Displays extracted source values, when applicable. * **Error:** Shows any validation or processing issues. 6. Update patterns that return incorrect or zero matches. 7. Repeat the test until the expected citations are detected without false matches. #### Built-in Formats With no custom patterns supplied, the following standard formats are recognized automatically: ``` [1] [2] [3] ``` The extracted `source_ref` values link each cached answer to its source documents.
#### 12.2 Custom Formats For non-standard citation formats, add one or more regex patterns. Each pattern must expose the source reference through a named capture group `(?P...)`: ``` ]*\bdocument_path\s*=\s*"(?P[^"]+)"[^>]*> ``` Validate the patterns using the built-in Regex Tester before saving the configuration.
#### Invalidation Behavior * Cached responses maintain references to their source documents. * Updates to a referenced document automatically invalidate all dependent cached responses. * Invalidated responses are excluded from future cache retrievals. * TTL enforces a maximum cache lifetime, while citation-based invalidation provides immediate, document-level freshness. #### Time-to-Live and Expiry The **TTL (hours)** field sets how long a stored answer remains valid. On expiry the answer is no longer eligible for reuse and is subsequently removed. The default is `720` hours (30 days); a value of `0` disables expiration entirely. TTL should reflect the volatility of the underlying content: * **Volatile content** (frequently updated data, live metrics): 24-72 hours. * **Stable content** (reference documentation, policy, product facts): 720 hours or longer. Setting TTL too long relative to content churn is a common cause of stale answers; where content changes are driven by document updates, pair TTL with citation-based invalidation for tighter freshness control. The simple recipe configured with semantic cache nodes is shown below.
### Save and publish recipe Saving the recipe preserves all configurations and connections made in the workflow for future reference or deployment. After a recipe is created and saved, it must be **published and deployed** before it can be used. For end-to-end guidance on publishing and deployment, refer to the [**Recipe Management** ](/karini-ai-documentation/recipe-management.md)section. Refer the following video to create **workflow recipe** with **chat** node. {% embed url="" %} Refer the following video to create **workflow recipe** with **webhook node**. {% embed url="" %}