# Test & Compare

Karini AI's prompt playground allows you to test your prompt against different models and model parameters simultaneously in real-time.&#x20;

### A/B Testing

You can select up to 3 models to test your prompts. The models available for selection must be registered in Karini's [Model Hub](../../model-hub). You can update the model parameters such as **Temperature** and **Max Tokens** to tune the  performance of your prompt.&#x20;

The "**Test**" button will trigger LLM invocation simultaneously for all the selected models for the prompt.  You can review the responses generated by the models in real-time and continue to fine-tune and re-test the prompt as required by tweaking the prompt instructions or modifying the model and parameter configurations. The real-time, side-by-side comparison of prompt responses for various models and parameters gives you the ability to view and analyze the variations in responses generated by each model and guides you select the best combination of model and parameters for your prompt task.&#x20;

### Prompt Responses and Statistics

Upon clicking "**Test**",  you can see real-time response from each of the selected model in the prompt test. If the selected model supports streaming,  you will see a streaming response.  Once the response generation is complete, you can also review the statistics displayed for each prompt test run. They include:&#x20;

* **Input Tokens**: Total number of input tokens in the LLM request. This includes the prompt instructions, system prompt, context and user query.&#x20;
* **Output Tokens:** Total number of output tokens generated by the LLM in response to the prompt request. This number does not exceed the **Max Tokens** value setup during the prompt testing.&#x20;
* **LLM Response Time:** The amount of time in milliseconds taken by the LLM to generate complete response for the given prompt request.
* **Time to First Token:** The time that it takes for the model to produce the first token of the response after receiving the prompt. TTFT is particularly relevant for applications utilizing streaming, where providing immediate feedback is crucial.

These statistics provide additional guidance when testing the performance of the prompt and the LLM and can be used to decide if the prompt output is satisfactory or if the prompt needs fine-tuning to obtain more precise results.  You can save the prompt experiments by clicking "[**Save prompt runs**](prompt-runs)" button.&#x20;

### Selecting the Best Answer

Based on the live response and the statistics of the model, you can select the best answer for your prompt requests. Additionally, you can also view the model [tracing details](prompt-observability) to get an in-depth understanding of request execution process by the LLM.&#x20;

Follow these steps to finalize model selection endpoints and manage prompt runs efficiently.

### **Configuring Model Endpoints**

#### **Assigning the Primary Model**

If prompt is tested with one model, then that model can only be assigned as the **Primary Model**.&#x20;

Below are the steps to assign a model as the Primary Model.

* Click "**Select as best answer**" to assign a model as the Primary model.
* Once selected, the Select as best answer will turn green.
* After selecting the **Primary model**, you must save the prompt run.
* Upon saving, you will be **redirected** to the **Edit** page to save the prompt.

<figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXc8_QR1yewoO8zt8RK79t8gvtzISia7FJ1IMB7TN1HDhmznVff4mJK3OiemV-b6QEYiUaDHNMyf8G9qaFC3L2BbShD40PpQfVyGwczII3BbrnmrFwaK9VODA4L2hJHp1GxJWYw2rA?key=sd7ID78MwYC7TUnxCWmg1dC8" alt=""><figcaption></figcaption></figure>

#### **Assigning Primary and Fallback Models**

When prompt is tested with multiple models, designate a **Primary Model** and a **Fallback Model** to optimize response accuracy and reliability. The **Primary Model** serves as the default AI system, selected based on its performance in terms of accuracy, response time, and cost-effectiveness. The **Fallback Model** acts as a backup to ensure system robustness, automatically taking over when the primary model fails to generate a response due to latency issues, errors, or unavailability. This dual-model configuration enhances system resilience, minimizes downtime, and ensures uninterrupted workflow in AI-driven decision-making environments.

Below are the steps to assign a model as the Primary Model and Fallback model.

* Select appropriate model endpoints as **Primary model** and **Fallback model** based on performance.
* Click on “**Select as best answe**r” for the preferred model.
* A pop-up window will appear, prompting you to choose between **Primary** or **Fallback model** assignment.

{% embed url="https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7ZrVuiAUMyuYVvrK5KaB%2Fuploads%2FL6YsMxxenNI01Qtuf2kr%2FFallback%20Model%20Annoucement.mp4?alt=media&token=91051bf1-e7fd-45d9-bd79-36faebcb0ea6" %}

* Selecting **Primary** changes the "**Select as best answer"** button to green.
* Selecting **Fallback** changes the button to an orange background.
* Once selections are made, **save the prompt run** to confirm your choices. This action will redirect you to the **Edit page** to save the prompt. Once selections are made, save the prompt run to confirm your choices. This action will redirect you to the Edit Page to save the prompt.

Refer to the video below for selecting the **primary** and **fallback** models in prompts and **publishing** the prompt.

{% embed url="https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7ZrVuiAUMyuYVvrK5KaB%2Fuploads%2FAbxZVCnqL1HbKZ15vsGV%2FPrimary%20and%20fallback%20model.mp4?alt=media&token=84a51f52-0647-4db1-b3e2-9dda62094e60" %}

{% hint style="info" %}


* If you attempt to save the prompt run before assigning models as Primary and Fallback, a pop-up message will appear stating:\
  "Please select a primary model before saving the prompt run".
* If you navigate directly to the Edit Prompt page without saving the prompt run, a pop-up notification will appear, prompting you to “Please save your changes before existing”.
{% endhint %}

The models designated as **Primary** and **Fallback** will be displayed on the **Edit p**age as shown below.

<figure><img src="https://415930246-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7ZrVuiAUMyuYVvrK5KaB%2Fuploads%2FpxAiIMFyllvAo9kgJWZC%2Fimage.png?alt=media&#x26;token=0ccf46a4-e1ee-4339-a018-0cb4d69ca64c" alt=""><figcaption></figcaption></figure>

The prompt can now be saved and **published** with a relevant **tag** to ensure proper categorization. Upon publication, the version will be displayed. For more details, refer to the [prompt versions](../prompt-versions).

\
\
