> For the complete documentation index, see [llms.txt](https://karini-ai.gitbook.io/karini-ai-documentation/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://karini-ai.gitbook.io/karini-ai-documentation/prompt-management/prompt-and-agent-evaluation/evaluation-steps.md).

# Evaluation steps

Karini AI provides a structured evaluation pipeline designed to test prompts and agents against standardized datasets. The system automates dataset validation, variable mapping, output generation, and metric scoring to ensure consistent, repeatable, and data-driven assessments. By following the evaluation workflow, teams can measure performance across large input sets, detect regressions between versions, and make informed improvements before deployment.

#### Follow the steps below to run and review your evaluation.

#### **Step 1: Select What You Want to Evaluate**

1. Open the **Prompt** or **Agent** you want to test in the prompt playground.
2. Ensure the version you want to evaluate is **published**.
3. Navigate to the **Evaluation** tab for that prompt or agent.\[Refer screenshot]

<figure><img src="/files/XOJYgvaHrNJBIImfnXxU" alt=""><figcaption></figcaption></figure>

#### **Step 2: Upload Your Dataset**

1. Click **“Upload dataset”** in the Evaluation tab.

<figure><img src="/files/TGaOXQ0dT7XKix8ARpjI" alt=""><figcaption></figcaption></figure>

2. Drag-and-drop or select your **CSV file** from your system.
3. The system automatically validates:
   * File type (`.csv` or supported `.txt`)
   * Presence of required columns (`input` or `agent_input`, and `ground_truth`)
   * A **header row** in the first line and at least one **data row+**
   * Column names not exceeding **60 characters**

After the dataset is uploaded, it is displayed in the interface as shown below.

<figure><img src="/files/aU5ozms204HxGNGD8yVi" alt=""><figcaption></figcaption></figure>

> Each row in this dataset becomes a single evaluation test case.

#### **Step 3: Map Dataset Variables**

1. After upload, the system detects **all column names** in the CSV.
2. Columns such as `context`, `instructions`, or any custom fields are automatically mapped to corresponding **prompt variables** (e.g., `{context}`, `{instructions}`).
3. You can review and confirm these mappings in the interface.

> This ensures that your prompt or agent receives the right inputs and contextual data for each test case.

#### **Step 4: Choose Evaluation Strategy**

1. In the Evaluation panel, pick one or both of:
   * [**Default Evaluation**](/karini-ai-documentation/prompt-management/prompt-and-agent-evaluation/evaluation-metrics-and-scoring.md#default-evaluation-metrics) – uses built-in metrics (relevancy, faithfulness, similarity, tool accuracy, etc.).
   * [**Custom Evaluation** ](/karini-ai-documentation/prompt-management/prompt-and-agent-evaluation/evaluation-metrics-and-scoring.md#creating-custom-metrics)– uses your own evaluation prompt and scoring logic.
2. If using custom evaluation:
   * Select a **custom evaluation prompt** from your library.
   * Choose the **LLM model** to act as the evaluator.

<figure><img src="/files/AMyT0rE4LEBLixhZQpj5" alt=""><figcaption></figcaption></figure>

> This step defines *how* the system will judge each answer and which metrics will be applied.

#### **Step 5: Run the Evaluation**

1. Review the configuration:
   * Selected prompt/agent version
   * Uploaded dataset
   * Chosen evaluation strategy (default, custom, or both)
2. Click **“Run”** to start the evaluation job.

<figure><img src="/files/dScfOur2J4kcbGPHmym7" alt=""><figcaption></figcaption></figure>

3. The system:
   1. Iterates over each dataset row
   2. Generates an output using the selected prompt or agent
   3. Applies the chosen metrics to score each output
   4. Stores scores, confidence, and reasoning per metric per test case

> Progress and status (e.g., processing, completed, failed) are shown in real time.

#### **Step 6: Review Aggregate Results**

1. When evaluation completes, click **“**[**View Results**](/karini-ai-documentation/prompt-management/prompt-and-agent-evaluation/analyzing-results.md)**”**.
2. In the **Results Dashboard**, you can view:
   * Overall performance summary (per metric)
   * Average scores, distributions, min/max, and standard deviation

> This gives you a high-level picture of how well your prompt or agent is performing across the whole dataset.

#### **Step 7: Inspect Individual Test Cases**

1. Drill down into **individual test cases** to see:
   * Input (`input` or `agent_input`)
   * Ground truth
   * Variables (e.g., context)
   * Model/agent response
   * Tool usage and execution trace (for agents)
   * Metric scores, confidence, and detailed reasoning
2. Focus especially on **low-scoring cases** to identify failure modes.

> This granular view helps you understand *why* certain cases failed and what to improve.

#### **Step 8: Compare Versions and Iterate**

1. Make changes to your prompt or agent based on insights (e.g., adjust instructions, improve tools, refine context retrieval).
2. Publish a **new version**.
3. Re-run evaluation using the **same dataset** and compare:
   * Metric deltas
   * Improvements or regressions
4. Choose the best-performing version for deployment.

> This creates a tight feedback loop and turns evaluation into a continuous optimization process.