Prompt Task Types

Karini AI supports following prompt task types thorough it's prompt playground.

Classification

Text classification prompt can be used to instruct the LLM to classify the user input into different classes based on the content of the input text.

Example:

Here is an example classification prompt. Karini's prompt playground gives you the flexibility to add prompt instructions, variables and few-shot-examples as relevant to the task. Pay attention to the use of special variables if you want to dynamically replace the values during production use.

Prompt:

Classify the summarized context into the appropriate category:

<summarized_context>
 {summarized_context} 
</summarized_context>

<categories>
 {categories}
</categories>

Provide the response in JSON format.

Variables:

1. summarized_context

The automotive industry began in the 1860s with hundreds of manufacturers that pioneered the horseless carriage. For many decades, the United States led the world in total automobile production. In 1929, before the Great Depression, the world had 32,028,500 automobiles in use, and the U.S. automobile industry produced over 90% of them. At that time, the U.S. had one car per 4.87 persons. After 1945, the U.S. produced about 75 percent of world's auto production. In 1980, the U.S. was overtaken by Japan and then became a world leader again in 1994. In 2006, Japan narrowly passed the U.S. in production and held this rank until 2009, when China took the top spot with 13.8 million units. With 19.3 million units manufactured in 2012, China almost doubled the U.S. production of 10.3 million units, while Japan was in third place with 9.9 million units. From 1970 (140 models) over 1998 (260 models) to 2012 (684 models), the number of automobile models in the U.S. has grown exponentially.

1. categories

Technology
Finance
Travel
Sports
HealthCare
Education

Output:

{
"category": "Technology"
}

Summarization

Summarization prompt can be used to instruct the LLM to summarize the user input text .

Example:

Here is an example summarization prompt. Karini's prompt playground gives you the flexibility to add prompt instructions, variables and few-shot-examples as relevant to the task. Pay attention to the use of special variables if you want to dynamically replace the values during production use.

Prompt:

Summarize the following text in {number_of_words} words:

{context}

Variables:

1. number_of_words

40

1. context

The automotive industry comprises a wide range of companies and organizations involved in the design, development, manufacturing, marketing, selling, repairing, and modification of motor vehicles. It is one of the world's largest industries by revenue (from 16 % such as in France up to 40 % to countries like Slovakia).The word automotive comes from the Greek autos (self), and Latin motivus (of motion), referring to any form of self-powered vehicle. This term, as proposed by Elmer Sperry
(1860–1930), first came into use with reference to automobiles in 1898.


== History ==

The automotive industry began in the 1860s with hundreds of manufacturers that pioneered the horseless carriage. For many decades, the United States led the world in total automobile production. In 1929, before the Great Depression, the world had 32,028,500 automobiles in use, and the U.S. automobile industry produced over 90% of them. At that time, the U.S. had one car per 4.87 persons. After 1945, the U.S. produced about 75 percent of world's auto production. In 1980, the U.S. was overtaken by Japan and then became a world leader again in 1994. In 2006, Japan narrowly passed the U.S. in production and held this rank until 2009, when China took the top spot with 13.8 million units. With 19.3 million units manufactured in 2012, China almost doubled the U.S. production of 10.3 million units, while Japan was in third place with 9.9 million units. From 1970 (140 models) over 1998 (260 models) to 2012 (684 models), the number of automobile models in the U.S. has grown exponentially.Early car manufacturing involved manual assembly by a human worker. The process evolved from engineers working on a stationary car, to a conveyor belt system where the car passed through multiple stations of more specialized engineers. Starting in the 1960s, robotic equipment was introduced to the process, and today most cars are produced largely with automated machinery.


== Safety ==

Safety is a state that implies being protected from any risk, danger, damage, or cause of injury. In the automotive industry, safety means that users, operators, or manufacturers do not face any risk or danger coming from the motor vehicle or its spare parts. Safety for the automobiles themselves implies that there is no risk of damage.
Safety in the automotive industry is particularly important and therefore highly regulated. Automobiles and other motor vehicles have to comply with a certain number of regulations, whether local or international, in order to be accepted on the market. The standard ISO 26262, is considered one of the best practice frameworks for achieving automotive functional safety.In case of safety issues, danger, product defect, or faulty procedure during the manufacturing of the motor vehicle, the maker can request to return either a batch or the entire production run. This procedure is called product recall. Product recalls happen in every industry and can be production-related or stem from raw materials.
Product and operation tests and inspections at different stages of the value chain are made to avoid these product recalls by ensuring end-user security and safety and compliance with the automotive industry requirements. However, the automotive industry is still particularly concerned about product recalls, which cause considerable financial consequences.

Output:

The automotive industry involves companies engaged in designing, manufacturing, marketing, and servicing motor vehicles. It is one of the largest industries globally, with revenue contributions ranging from 16% to 40% in different countries. The industry originated in the 1860s and has evolved from manual assembly to automated production lines. Safety is a critical aspect, governed by stringent regulations and standards to ensure user protection and compliance.

QnA (Deprecated)

This prompt help to obtain answers from the provided context.

Example:

Here is an example QNA prompt. Karini's prompt playground gives you the flexibility to add prompt instructions, variables and few-shot-examples as relevant to the task. Pay attention to the use of special variables if you want to dynamically replace the values during production use.

Prompt:

Considering the information presented in the context, your task is to analyze and answer the following questions in a concise and structured manner. Ensure that your responses directly address the specific details outlined in the context provided.

Context:{context}

Question:{question}

Variables:

1. context

A travel document is an identity document issued by a government or international entity pursuant to international agreements to enable individuals to clear border control measures. Travel documents usually assure other governments that the bearer may return to the issuing country, and are often issued in booklet form to allow other governments to place visas as well as entry and exit stamps into them.

1. question

What is travel document?

Output:

A travel document is an identity document issued by a government or international entity, designed in accordance with international agreements to facilitate the movement of individuals across borders. It serves to verify the identity and nationality of the bearer, ensuring that they can pass through border control measures. Additionally, it provides assurance to foreign governments that the holder has the right to return to the country that issued the document. Typically issued in booklet form, travel documents allow for the inclusion of visas, as well as entry and exit stamps, by other countries.

Evaluation

This prompt assess answers against ground truth answers, offering metric scores based on specified criteria.

Example:

Here is an example Evaluation prompt. Karini's prompt playground gives you the flexibility to add prompt instructions, variables and few-shot-examples as relevant to the task. Pay attention to the use of special variables if you want to dynamically replace the values during production use.

Prompt:

Task: Evaluate Relevance

Objective:
Assess the alignment of the Model Output with the Evaluation Input and, crucially, the Ground Truth. The evaluation should rigorously follow the defined metric and grading criteria, with a special focus on the Ground Truth's role in determining relevance.

Metric: {evaluation_metric_name}
Description: {evaluation_metric_description}

Grading Criteria: {evaluation_grading_criteria}

Parameters:
- Input: {evaluation_input}
- Model Output: {evaluation_output}
- Ground Truth: {evaluation_ground_truth} (Note: This parameter is pivotal. If the output significantly diverges from the Ground Truth, the relevance score should be adjusted downwards accordingly.)

Instructions:
Critically analyze the Model Output in the context of the Evaluation Input and especially the Ground Truth. The presence, accuracy, and alignment of the Ground Truth with the Model Output are paramount in this evaluation. Assign a relevance score that reflects this emphasis, with a lowered score for outputs that do not align with a well-defined Ground Truth. Provide a concise justification, limited to 10 words, for the assigned score.

Output Format (JSON string) example with key-values for each metric:
[{{'metric_name': 'metric-1-name', 'score': x1, 'justification': 'Provide justification for metric 1 here.'}}, {{'metric_name': 'metric-2-name', 'score': x2, 'justification': 'Provide justification for metric 2 here.'}}]

Note: Ensure that each entry in the output array corresponds to one of the metrics listed. The 'score' should reflect the assessment based on the specific grading criteria for that metric, and the 'justification' should concisely capture the rationale behind the assigned score. The number of objects in the output array should match the number of metrics being evaluated.

Variables:

1. evaluation_metric_name

Relevance

1. evaluation_metric_description

Relevance evaluates the extent to which the provided answer aligns with the question and context. A fully relevant answer should directly and comprehensively address the query, avoiding extraneous content. Conversely, an irrelevant answer diverges from the query, offering incomplete responses or unrelated details.

1. evaluation_grading_criteria

- Score 5: Direct, complete, and fully relevant answer.
- Score 4: Mostly relevant, with minor irrelevant details.
- Score 3: Partial relevance, noticeable deviation or incompleteness.
- Score 2: Limited relevance, significant deviation or incorrectness.
- Score 1: Completely irrelevant, lacking any pertinent content.

1. evaluation_input

What is AWS?

1. evaluation_output

AWS (Amazon Web Services) is a comprehensive cloud computing platform provided by Amazon. It offers a broad set of global cloud-based products and services for businesses and developers to build, deploy, and manage applications and services cost-effectively and at scale.

Some key areas where AWS provides services include:

Compute: Services like EC2 (Elastic Compute Cloud) for virtual servers, Lambda for serverless computing, and more.
Storage: Services like S3 (Simple Storage Service) for object storage, EBS (Elastic Block Store) for block storage, and more.
Databases: Managed database services like RDS (Relational Database Service), DynamoDB (NoSQL database), and more.
Networking: Services like VPC (Virtual Private Cloud), Direct Connect, and more for secure networking.
Analytics: Services like Athena, Redshift, EMR (Elastic MapReduce), and more for big data processing and analytics.
Machine Learning: Services like SageMaker, Rekognition, Lex, and more for building and deploying ML models.
Internet of Things (IoT): Services like IoT Core, IoT Greengrass, and more for building IoT applications.
And many more services across various domains like security, management tools, mobile, AR/VR, and more.

1. evaluation_ground_truth

Amazon Web Services (AWS) is a comprehensive cloud computing platform provided by Amazon.com. AWS offers a wide range of services that cater to computing power, storage, databases, machine learning, analytics, content delivery, Internet of Things (IoT), security, and more. These services are delivered over the internet on a pay-as-you-go basis, allowing businesses and individuals to scale and utilize computing resources without the need to invest in and maintain physical infrastructure.

Key components and services of AWS include:

Compute Services:

Amazon Elastic Compute Cloud (EC2): Provides resizable compute capacity in the cloud.
AWS Lambda: Lets you run code without provisioning or managing servers.
Storage Services:

Amazon Simple Storage Service (S3): Scalable object storage for data backup, archiving, and analytics.
Amazon Elastic Block Store (EBS): Provides block-level storage volumes for use with EC2 instances.
Database Services:

Amazon RDS: Managed relational database service.
Amazon DynamoDB: Managed NoSQL database service.
Networking:

Amazon Virtual Private Cloud (VPC): Enables you to launch Amazon Web Services resources into a virtual network.
Amazon Route 53: Scalable domain name system (DNS) web service.
Machine Learning and AI:

Amazon SageMaker: Fully managed service for building, training, and deploying machine learning models.
Amazon Polly: Text-to-speech service.
Amazon Rekognition: Image and video analysis service.
Analytics and Big Data:

Amazon Redshift: Fully managed, petabyte-scale data warehouse service.
Amazon Athena: Query data in Amazon S3 using SQL.
Security and Identity:

AWS Identity and Access Management (IAM): Securely control access to AWS services and resources.
AWS Key Management Service (KMS): Managed service that makes it easy for you to create and control the keys used to encrypt your data.
Internet of Things (IoT):

AWS IoT Core: Connects devices to the cloud and enables them to interact with applications.
AWS has become a dominant force in the cloud computing industry and is widely used by businesses, startups, and individuals globally for building and deploying applications and services.

Output:

[{'metric_name': 'Relevance', 'score': 4, 'justification': 'Mostly relevant, minor irrelevant details.'}]

Agent

This prompt has the ability to take actionable steps, creating a more sophisticated system that can understand and process information, evaluate situations, take appropriate actions, communicate responses, and track ongoing situations.

Example:

Here is an example Agent prompt. Karini's prompt playground gives you the flexibility to add prompt instructions, variables and few-shot-examples as relevant to the task. Pay attention to the use of special variables if you want to dynamically replace the values during production use.

Prompt:

You are a virtual insurance agent. Answer questions as best as you can.

Rules for Email Notification:

1.For specific information requests, such as policy details or policy status, ask the user if they would like an email notification with the information.
2.If the user confirms they would like an email notification, ask them to provide their email address. Follow this instruction strictly.
3.Do not send an email on your own; simply confirm receipt of the email address and proceed to the next step.
4.When the email address is provided, acknowledge with "Final Answer" and confirm that the user's request for email notification is noted.Follow this instruction craefully and strictly.
5.Always provide a complete and detailed answer to the user's question as your "Final Answer" before asking if they need an email notification.

Question

Give me details on Alice's policy. Give structured and easy to read output, explaining impact.

Output:

The details of Alice's policy have been provided, including her personal information, policy number, expiry date, status, premium amount, and coverage amount. The policy is currently pending, and Alice needs to ensure it becomes active to benefit from the $100,000 coverage.

Deep Agent

Deep Agent represents an advanced evolution in agentic workflows, providing developers with fine-grained control over AI agent behavior, tool execution, and decision-making. Unlike standard agents that execute autonomously, Deep Agents support interruption points, approval workflows, and hierarchical agent structures.

Here is an example Deep Agent prompt. Karini's prompt playground gives you the flexibility to add prompt instructions, variables and few-shot-examples as relevant to the task. Pay attention to the use of special variables if you want to dynamically replace the values during production use.

Prompt:

# Task: Answer Questions Using Available Tools

You are an AI assistant equipped with various tools to help answer questions accurately and effectively.

## Objective
Provide the best possible answer to the user's question by leveraging the tools at your disposal.

## Instructions

1. **Analyze the Question**: Carefully read and understand what the user is asking.

2. **Identify Relevant Tools**: Determine which of your available tools would be most helpful in answering the question.

3. **Use Tools Strategically**: 
   - Select the most appropriate tool(s) for the task
   - Use tools in a logical sequence if multiple tools are needed
   - Interpret tool outputs accurately

4. **Formulate Your Answer**:
   - Synthesize information gathered from the tools
   - Provide a clear, comprehensive response
   - Ensure your answer directly addresses the question asked

5. **Handle Limitations**:
   - If no suitable tool is available, use your general knowledge
   - If you cannot answer with confidence, acknowledge the limitation
   - Explain your reasoning process when appropriate

Question

How can Amazon SageMaker Clarify help streamline the evaluation process?

Output:

Let me use the Knowledge Base Tool to gather information about Amazon SageMaker Clarify and its evaluation capabilities.. Workflow interrupted: {'interrupted': True, 'decisions_required': [{'node_name': 'DeepAgent', 'tool_name': 'Knowledge_Base_Tool', 'args': {'query': 'How does Amazon SageMaker Clarify make evaluation easier? What are its key features for model evaluation?'}, 'allowed_decisions': ['approve', 'edit', 'reject']}]} Based on the information retrieved, let me provide a comprehensive response about how Amazon SageMaker Clarify makes evaluation easier. Amazon SageMaker Clarify makes model evaluation easier in several key ways: 1. Unified Evaluation Platform: - Provides a single place to evaluate and compare any LLM based on predefined criteria - Supports evaluation during model selection and throughout the model customization workflow 2. Comprehensive Evaluation Methods: - Offers automatic evaluation capabilities - Includes human-in-the-loop capabilities for subjective criteria assessment - Allows evaluation of metrics like helpfulness, creative intent, and style 3. Flexible Evaluation Options: - Provides curated prompt datasets for common LLM tasks: * Open-ended text generation * Text summarization * Question answering (Q&A) * Classification - Supports custom prompt datasets and metrics for specific use cases 4. Workforce Options: - Allows using your own workforce - Provides access to managed workforce through SageMaker Ground Truth 5. Detailed Reporting: - Generates evaluation reports with natural language summaries - Includes visualizations and examples - Provides downloadable metrics and reports - Integrates with SageMaker MLOps workflows 6. Easy Access and Setup: - Accessible through SageMaker Studio under "Jobs" menu - Direct evaluation option from model details page in SageMaker JumpStart - Guided UI wizard for setting up evaluation jobs 7. Broad Model Support: - Evaluates LLMs from SageMaker JumpStart - Supports models trained and hosted outside of AWS - Removes the complexity of adopting academic benchmarks for generative AI use cases These features significantly simplify the process of evaluating and selecting foundation models while providing comprehensive assessment capabilities.