> For the complete documentation index, see [llms.txt](https://karini-ai.gitbook.io/karini-ai-documentation/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://karini-ai.gitbook.io/karini-ai-documentation/installation/karini-ai-on-premise-deployment.md).

# Karini AI On Premise Deployment

Karini AI Platform Deployment Guide

This guide covers hardware requirements, infrastructure pre-requisites, and step-by-step deployment instructions for the Karini AI platform on Amazon Elastic Kubernetes Service (Amazon EKS).

<figure><img src="https://415930246-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7ZrVuiAUMyuYVvrK5KaB%2Fuploads%2FLPh7t4m1XJUvnuammWgi%2FKariniAIV3.drawio.png?alt=media&amp;token=56663b96-ef09-41b7-979f-5d13aded3af5" alt=""><figcaption></figcaption></figure>

### Hardware Requirements

Karini AI is optimized for the following node configuration on Amazon EKS.

| Resource           | Minimum          | Recommended |
| ------------------ | ---------------- | ----------- |
| vCPUs per node     | 4                | **8**       |
| Memory per node    | 16 GB            | **32 GB**   |
| Node count         | 2                | 3 – 5       |
| OS Disk            | 50 GB            | 100 GB      |
| Persistent Storage | 100 GB EBS / EFS | 200 GB+     |

**Recommended Instance Type:** `m5.2xlarge` (8 vCPU / 32 GB RAM). Smaller instances may cause scheduling failures for memory-intensive services such as the Observability Datastore and Ray workers.

#### Instance Type Reference

| Instance Type     | vCPU | Memory | Use Case                          |
| ----------------- | ---- | ------ | --------------------------------- |
| `m5.xlarge`       | 4    | 16 GB  | Development / Evaluation          |
| `m5.2xlarge`      | 8    | 32 GB  | **Production (recommended)**      |
| `m5.4xlarge`      | 16   | 64 GB  | High-concurrency / Large datasets |
| `m7i-flex.xlarge` | 4    | 16 GB  | Cost-optimized (latest gen)       |

### Pre-requisites

Ensure the following tooling and infrastructure are in place before deploying.

#### Local Tools

| Tool      | Min. Version | Install Guide                                                                         |
| --------- | ------------ | ------------------------------------------------------------------------------------- |
| Terraform | `>= 1.7.5`   | [hashicorp.com](https://developer.hashicorp.com/terraform/downloads)                  |
| AWS CLI   | `>= 2.x`     | [aws.amazon.com](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) |
| kubectl   | `>= 1.28`    | [kubernetes.io](https://kubernetes.io/docs/tasks/tools/)                              |
| Helm      | `>= 3.12`    | [helm.sh](https://helm.sh/docs/intro/install/)                                        |

Configure the AWS CLI with a profile that has sufficient IAM permissions:

```bash
aws configure --profile karini
export AWS_PROFILE=karini
```

#### AWS Infrastructure

The following AWS resources must exist before running Terraform:

* **VPC** : with at least two private subnets and two public subnets, each in different Availability Zones
* **Subnet Requirements** : Each private subnet must be sized at **/24 or larger** (e.g., `10.x.x.0/24`), providing sufficient usable IPs per Availability Zone. This accounts for AWS VPC CNI pod IP pre-allocation per EKS node, EFS mount targets, internal Network Load Balancers, OpenSearch nodes, and Lambda Hyperplane ENIs
* **EKS Cluster :** Kubernetes version `1.31` recommended, with an OIDC provider configured
* **ACM Certificate :** a valid SSL/TLS certificate for your domain (or one will be created by the module)
* **ECR Registry :** containing the Karini AI container images (CoreML, Sentinel, Tesseract, Ray, etc.) provided by Karini AI

#### DNS & Domain

* A registered domain name (e.g., `karini.yourdomain.com`)
* DNS managed via Route 53 or an external DNS provider
* Ability to create/update A or CNAME records pointing to the provisioned AWS load balancer

**Identity Provider (Optional, for SSO)**

If you plan to enable Single Sign-On, register an OAuth2 client with one of the supported identity providers before deployment:

* **Google Workspace (Gmail)**
* **Okta**
* **Microsoft Azure AD (Entra ID)**
* **PingFederate**

Use `https://karini.yourdomain.com/api/auth/callback` (replace with your domain) as the authorized redirect URI. See Configure Single Sign-On for the credentials required by each provider.

#### Secrets & Credentials

Prepare the following before deployment. These will be stored in AWS Secrets Manager:

| Secret                              | Description                                                                                                                    |
| ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| MongoDB password                    | Required if using self-managed MongoDB                                                                                         |
| OpenSearch credentials              | Master username and password                                                                                                   |
| Observability Datastore credentials | Username and password                                                                                                          |
| Email / SMTP credentials            | For platform notifications                                                                                                     |
| Third-party API keys                | As required by your use case (e.g., LLM provider keys)                                                                         |
| SSO OAuth2 credentials              | Client ID, Client Secret, and issuer/tenant details for Google, Okta, Azure AD, or PingFederate. Required only if enabling SSO |

#### Karini AI Container Images

Contact [Karini AI](https://www.karini.ai/) to obtain:

* ECR image URIs for CoreML, Sentinel, Tesseract, Ray and  Semantic Chunking
* Application version tags for each service
* Helm chart repository credentials

### Deployment Steps

#### Step 1 : Unzip the Terraform Module

Contact Karini AI to recieve the Terraform module

```bash
unzip eks-terraform-module.zip
cd eks-terraform-module
```

#### Step 2 : Configure Your Variables

Copy the example variables file and populate your environment-specific values:

```bash
cp examples/existing-eks/terraform.tfvars.example terraform.tfvars
```

At minimum, set the following required variables in `terraform.tfvars`:

```hcl
# AWS & Networking
aws_region         = "us-east-1"
vpc_id             = "vpc-xxxxxxxxxx"
private_subnet_ids = ["subnet-aaa", "subnet-bbb"]
public_subnet_ids  = ["subnet-ccc", "subnet-ddd"]

# EKS Cluster
eks_cluster_name                 = "my-karini-cluster"
cluster_endpoint                 = "https://<your-eks-endpoint>"
cluster_identity_oidc_issuer_arn = "arn:aws:iam::<account-id>:oidc-provider/..."

# Domain & TLS Certificate
domain_name         = "karini.yourdomain.com"
acm_certificate_arn = "arn:aws:acm:us-east-1:<account-id>:certificate/..."

# Application images (provided by Karini AI)
image_repository = "<account-id>.dkr.ecr.us-east-1.amazonaws.com"
coreml_version   = "<version>"
sentinel_version = "<version>"

# Lambda image URIs (provided by Karini AI)
semantic_chunking_image_repo_uri = "<account-id>.dkr.ecr.us-east-1.amazonaws.com/semantic-chunking"
semantic_chunking_image_tag      = "<version>"
tesseract_image_repo_uri         = "<account-id>.dkr.ecr.us-east-1.amazonaws.com/tesseract"
tesseract_image_tag              = "<version>"
event_bridge_image_repo_uri      = "<account-id>.dkr.ecr.us-east-1.amazonaws.com/event-bridge"
event_bridge_image_tag           = "<version>"
ray_image_uri                    = "<account-id>.dkr.ecr.us-east-1.amazonaws.com/ray"
ray_image_tag                    = "<version>"
ray_version                      = "<version>"

# Secrets
secrets = {
  mongodb = {
    username = "admin"
    password = "your-secure-password"
  }
  opensearch = {
    username = "admin"
    password = "your-secure-password"
  }
  observability_datastore = {
    username = "default"
    password = "your-secure-password"
  }
}
```

#### Step 3 : Initialize Terraform

```bash
terraform init
```

This downloads all required providers and modules. Expected providers:

* `hashicorp/aws` `>= 5.41.0`
* `hashicorp/helm` `>= 2.12.1`
* `gavinbunney/kubectl` `>= 2.0.3`

#### Step 4 : Preview the Deployment Plan

```bash
terraform plan -var-file=terraform.tfvars
```

Review the output carefully to confirm the resources that will be created. A typical deployment creates **80–120 AWS resources**.

#### Step 5 : Apply the Deployment

```bash
terraform apply -var-file=terraform.tfvars
```

Type `yes` when prompted. The full deployment typically takes **15–25 minutes**.

#### Step 6 : Configure kubectl Access

```bash
aws eks update-kubeconfig \
  --region us-east-1 \
  --name my-karini-cluster
```

Verify connectivity:

```bash
kubectl cluster-info
kubectl get nodes
```

#### Step 7 : Verify All Services Are Running

```bash
kubectl get pods --all-namespaces
```

All pods should reach `Running` or `Completed` status within 5–10 minutes of `terraform apply` completing.

| Namespace       | Service                      | Expected Status           |
| --------------- | ---------------------------- | ------------------------- |
| `default`       | `sentinel-*`                 | Running                   |
| `default`       | `coreml-*`                   | Running                   |
| `default`       | `redis-*`                    | Running                   |
| `default`       | `observability-datastore-*`  | Running                   |
| `default`       | `mongodb-*`                  | Running (if self-managed) |
| `default`       | `kuberay-operator-*`         | Running                   |
| `default`       | `ray-cluster-head-*`         | Running                   |
| `ingress-nginx` | `ingress-nginx-controller-*` | Running                   |
| `kube-system`   | `cluster-autoscaler-*`       | Running                   |
| `kube-system`   | `efs-csi-*`                  | Running                   |

#### Step 8 : Run Initial Platform Setup

Once all pods are healthy, run the setup script to create your initial organization and admin user:

```bash
bash setup.sh
```

You will be prompted for:

* **Admin email address**
* **Admin password**

The script connects to the running Sentinel pod via `kubectl exec` and initializes your Karini AI workspace.

#### Step 9 : Access the Platform

Navigate to your configured domain in a browser:

```
https://karini.yourdomain.com
```

&#x20;

#### Configure Single Sign-On (SSO)

Karini AI supports SSO via **OAuth2** with the following identity providers:

| Provider                          | `terraform.tfvars` block | Notes                                                               |
| --------------------------------- | ------------------------ | ------------------------------------------------------------------- |
| **Google Workspace (Gmail)**      | `google_secrets`         | OAuth2 client plus a service account for Workspace directory access |
| **Okta**                          | `okta_secrets`           | OIDC Web Application plus an API token for directory access         |
| **Microsoft Azure AD (Entra ID)** | `azure_secrets`          | Single-tenant app registration                                      |
| **PingFederate**                  | `ping_secrets`           | OAuth client with the Authorization Code grant                      |

SSO is optional. Configure only the blocks for the providers you use; leave the others empty. If no provider is configured, users authenticate with the email/password accounts created in Karini AI.

**Step 1 : Register an OAuth2 Client with Your Identity Provider**

Create an OAuth2 / OIDC client in your identity provider with:

* **Grant type:** Authorization Code
* **Redirect / Callback URI:** `https://karini.yourdomain.com/api/auth/callback`
* **Scopes:** `openid`, `email`, `profile`

{% hint style="info" %}
Provider setup guides: [Google](https://developers.google.com/identity/openid-connect/openid-connect) · [Okta](https://developer.okta.com/docs/guides/implement-grant-type/authcode/main/) · [Azure AD](https://learn.microsoft.com/en-us/entra/identity-platform/quickstart-register-app) · [PingFederate](https://docs.pingidentity.com/pingfederate/latest/administrators_reference_guide/pf_configuring_oauth_clients.html)
{% endhint %}

**Step 2 : Add the Credentials to `terraform.tfvars`**

The values are written to AWS Secrets Manager and injected into the Sentinel service as environment variables.

{% tabs %}
{% tab title="Google Workspace" %}

```hcl
google_secrets = {
  GOOGLE_ID                    = "<oauth-client-id>.apps.googleusercontent.com"
  GOOGLE_CLIENT_SECRET         = "<oauth-client-secret>"
  GOOGLE_SERVICE_ACCOUNT_EMAIL = "<service-account>@<project>.iam.gserviceaccount.com"
  GOOGLE_PRIVATE_KEY           = "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n"
  GOOGLE_PRIVATE_KEY_ID        = "<private-key-id>"
  GOOGLE_ADMIN_EMAIL           = "admin@yourdomain.com"
}
```

| Variable                       | Where to find it                                                                 |
| ------------------------------ | -------------------------------------------------------------------------------- |
| `GOOGLE_ID`                    | Google Cloud Console → APIs & Services → Credentials → OAuth 2.0 Client ID       |
| `GOOGLE_CLIENT_SECRET`         | Same OAuth client                                                                |
| `GOOGLE_SERVICE_ACCOUNT_EMAIL` | Service account created in the same project, with domain-wide delegation enabled |
| `GOOGLE_PRIVATE_KEY`           | `private_key` field of the service account JSON key (keep `\n` line breaks)      |
| `GOOGLE_PRIVATE_KEY_ID`        | `private_key_id` field of the service account JSON key                           |
| `GOOGLE_ADMIN_EMAIL`           | A Google Workspace super-admin the service account impersonates                  |
| {% endtab %}                   |                                                                                  |

{% tab title="Okta" %}

```hcl
okta_secrets = {
  OKTA_CLIENT_ID     = "<client-id>"
  OKTA_CLIENT_SECRET = "<client-secret>"
  OKTA_ISSUER        = "https://<your-org>.okta.com/oauth2/default"
  OKTA_API_KEY_TOKEN = "<okta-api-token>"
}
```

| Variable             | Where to find it                                                 |
| -------------------- | ---------------------------------------------------------------- |
| `OKTA_CLIENT_ID`     | Okta Admin → Applications → your OIDC app → General              |
| `OKTA_CLIENT_SECRET` | Same application                                                 |
| `OKTA_ISSUER`        | Okta Admin → Security → API → Authorization Servers → Issuer URI |
| `OKTA_API_KEY_TOKEN` | Okta Admin → Security → API → Tokens → Create token              |
| {% endtab %}         |                                                                  |

{% tab title="Azure AD" %}

```hcl
azure_secrets = {
  AZURE_AD_CLIENT_ID     = "<application-client-id>"
  AZURE_AD_CLIENT_SECRET = "<client-secret-value>"
  AZURE_AD_TENANT_ID     = "<directory-tenant-id>"
}
```

| Variable                 | Where to find it                                                           |
| ------------------------ | -------------------------------------------------------------------------- |
| `AZURE_AD_CLIENT_ID`     | Azure Portal → App registrations → your app → Application (client) ID      |
| `AZURE_AD_CLIENT_SECRET` | App registration → Certificates & secrets → Client secrets (use the Value) |
| `AZURE_AD_TENANT_ID`     | App registration → Directory (tenant) ID                                   |
| {% endtab %}             |                                                                            |

{% tab title="PingFederate" %}

```hcl
ping_secrets = {
  PING_CLIENT_ID     = "<client-id>"
  PING_CLIENT_SECRET = "<client-secret>"
  PING_ISSUER        = "https://<your-pingfederate-host>"
}
```

| Variable             | Where to find it                                                               |
| -------------------- | ------------------------------------------------------------------------------ |
| `PING_CLIENT_ID`     | PingFederate Admin → Applications → OAuth → Clients                            |
| `PING_CLIENT_SECRET` | Same client, under Client Authentication                                       |
| `PING_ISSUER`        | Base URL of your PingFederate runtime (the `issuer` in its discovery document) |
| {% endtab %}         |                                                                                |
| {% endtabs %}        |                                                                                |

{% hint style="warning" %}
Never commit `terraform.tfvars` containing these values to source control. Client secrets and private keys should be stored only in AWS Secrets Manager after deployment.
{% endhint %}

**Step 3 : Apply the Configuration**

```bash
terraform apply -var-file=terraform.tfvars
```

Sentinel restarts and picks up the new secrets. Confirm the pod is healthy:

```bash
kubectl get pods -l app=sentinel -n default
```

**Step 4 : Verify SSO Login**

1. Open `https://karini.yourdomain.com` in a private browser window.
2. Confirm the **Sign in with Google / Okta / Microsoft / Ping** option appears on the login page.
3. Sign in with a user from your identity provider.

{% hint style="info" %}
Keep at least one local admin account active so you retain access if the identity provider becomes unavailable.
{% endhint %}

**Troubleshooting SSO**

| Symptom                                 | Likely Cause                                                                             |
| --------------------------------------- | ---------------------------------------------------------------------------------------- |
| `redirect_uri_mismatch`                 | Callback URI registered in the IdP does not exactly match your Karini domain             |
| `invalid_client`                        | Client ID / Secret mismatch in `terraform.tfvars`                                        |
| `invalid_issuer` or discovery failure   | `OKTA_ISSUER` / `PING_ISSUER` / `AZURE_AD_TENANT_ID` is incorrect                        |
| Google login fails after consent        | `GOOGLE_PRIVATE_KEY` lost its `\n` line breaks, or domain-wide delegation is not granted |
| User authenticates but is denied access | User's email is not provisioned in the Karini AI organization                            |

View the full OAuth2 error in the Sentinel logs:

```bash
kubectl logs -l app=sentinel -n default --tail=100 -f | grep -i -E "oauth|sso|auth"
```

**Deployment Complete.** Log in with the admin credentials created in Step 8. You should see the Karini AI dashboard.

***

### Component Overview

| Component                   | Purpose                                                    | Storage    | Default Resources       |
| --------------------------- | ---------------------------------------------------------- | ---------- | ----------------------- |
| **Sentinel**                | Core API, orchestration, and WebSocket service             | —          | 2 vCPU / 4 GB           |
| **CoreML**                  | LLM Gateway and Agent Runtime                              | —          | 2 vCPU / 4 GB           |
| **Observability Datastore** | Analytics and vector store (hot on EFS, cold on Amazon S3) | 30 GB+     | 4 vCPU / 16 GB          |
| **Redis**                   | Caching and session management                             | 10 GB EBS  | 0.5 vCPU / 3 GB         |
| **MongoDB**                 | Document store for platform metadata                       | 20 GB EFS  | 1 vCPU / 2 GB           |
| **OpenSearch**              | Vector search and Document Cache                           | 100 GB EBS | `r8g.medium.search` × 2 |
| **Ray (KubeRay)**           | Distributed Knowlegebase processing and async workloads    | —          | Configurable            |
| **AWS Batch**               | Serverless async job processing (Fargate)                  | —          | 1 vCPU / 2 GB per job   |
| AWS **Lambda**              | Semantic chunking, OCR (Tesseract), event processing       | —          | Serverless              |

***

### Network Architecture

#### Traffic Flow

1. End users connect to the platform via HTTPS on your domain
2. DNS resolves to the AWS Network Load Balancer (NLB) in the public subnets
3. NLB forwards traffic to the NGINX Ingress Controller running in the EKS cluster
4. NGINX routes requests to Sentinel (API) or CoreML (inference) based on path rules
5. All inter-service communication stays within the VPC private subnets
6. AWS managed services (S3, Secrets Manager) are accessed via VPC endpoints or NAT Gateway

### IAM Permissions

The AWS credentials used during deployment require the following IAM permissions. It is recommended to create a dedicated deployment IAM role.

#### Required IAM Actions

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "EKS",
      "Effect": "Allow",
      "Action": [
        "eks:Describe*",
        "eks:List*",
        "eks:CreateAddon",
        "eks:DeleteAddon",
        "eks:UpdateAddon"
      ],
      "Resource": "*"
    },
    {
      "Sid": "IAM",
      "Effect": "Allow",
      "Action": [
        "iam:CreateRole",
        "iam:DeleteRole",
        "iam:AttachRolePolicy",
        "iam:DetachRolePolicy",
        "iam:CreatePolicy",
        "iam:DeletePolicy",
        "iam:GetRole",
        "iam:GetPolicy",
        "iam:ListRolePolicies",
        "iam:PassRole",
        "iam:CreateOpenIDConnectProvider",
        "iam:DeleteOpenIDConnectProvider"
      ],
      "Resource": "*"
    },
    {
      "Sid": "Storage",
      "Effect": "Allow",
      "Action": [
        "elasticfilesystem:*",
        "s3:*",
        "kms:*"
      ],
      "Resource": "*"
    },
    {
      "Sid": "Compute",
      "Effect": "Allow",
      "Action": [
        "ec2:Describe*",
        "ec2:CreateSecurityGroup",
        "ec2:AuthorizeSecurityGroupIngress",
        "ec2:CreateTags",
        "batch:*",
        "lambda:*"
      ],
      "Resource": "*"
    },
    {
      "Sid": "Observability",
      "Effect": "Allow",
      "Action": [
        "logs:*",
        "cloudwatch:*",
        "es:*"
      ],
      "Resource": "*"
    },
    {
      "Sid": "Messaging",
      "Effect": "Allow",
      "Action": [
        "sqs:*",
        "events:*",
        "secretsmanager:*"
      ],
      "Resource": "*"
    }
  ]
}
```

#### IRSA Roles (Created by Terraform)

The module automatically creates the following IAM Roles for Service Accounts (IRSA):

| Role                      | Service Account         | Permissions                           |
| ------------------------- | ----------------------- | ------------------------------------- |
| `coreml-irsa-role`        | `coreml`                | S3, Bedrock, Secrets Manager          |
| `sentinel-irsa-role`      | `sentinel`              | S3, Secrets Manager, Lambda           |
| `efs-csi-role`            | `efs-csi-controller-sa` | EFS full access                       |
| `cluster-autoscaler-role` | `cluster-autoscaler`    | EC2 AutoScaling describe/set capacity |
| `kuberay-role`            | `kuberay-operator`      | Batch, S3                             |

### Scaling Recommendations

#### Horizontal Node Scaling

Cluster Autoscaler is pre-configured and will scale EKS node groups automatically:

```hcl
node_count_min     = 2   # Always-on nodes for high availability
node_count_desired = 3   # Starting node count
node_count_max     = 10  # Maximum nodes under load
```

The autoscaler uses a **30-minute scale-down window** to prevent thrashing during intermittent workloads

#### Service-Level Tuning

| Service                        | Variable                                | Default   | Recommendation                         |
| ------------------------------ | --------------------------------------- | --------- | -------------------------------------- |
| Observability Datastore memory | `observability_datastore_pod_memory_gb` | `16`      | Increase for datasets > 50 GB          |
| Obs. Datastore CPU             | `observability_datastore_pod_cpu_limit` | `"4"`     | Increase for high query concurrency    |
| Batch job memory               | `batch_job_memory`                      | `2048` MB | Increase for large document processing |
| Log retention                  | `log_retention_days`                    | `30`      | Reduce to lower CloudWatch costs       |
| OpenSearch nodes               | `opensearch_instance_count`             | `2`       | Use `3` for production HA              |
| OpenSearch storage             | `opensearch_ebs_volume_size`            | `100` GB  | Scale with log/index volume            |

#### Workload Profiles

| Profile             | Instance Type | Nodes  | Use Case                      |
| ------------------- | ------------- | ------ | ----------------------------- |
| Development         | `m5.xlarge`   | 1 – 2  | Testing and evaluation        |
| Production (Small)  | `m5.2xlarge`  | 2 – 3  | Up to 50 concurrent users     |
| Production (Medium) | `m5.2xlarge`  | 3 – 5  | Up to 200 concurrent users    |
| Production (Large)  | `m5.4xlarge`  | 5 – 10 | Enterprise / high-concurrency |

### Multi-Region High Availability

For enterprise deployments requiring cross-region redundancy.

#### Active-Passive Setup

1. **Primary region** runs the full Karini AI stack
2. **Secondary region** maintains replicated data stores only
3. Route 53 health checks with DNS failover route traffic to the secondary region on failure

#### Data Replication

Enable EFS cross-region replication in your `terraform.tfvars`:

```hcl
efs_replication_destination_region = <target aws region>
```

For OpenSearch, enable multi-AZ with dedicated master nodes:

```hcl
opensearch_instance_count = 3
opensearch_instance_type  = "r8g.large.search"
```

#### Recovery Objectives

| Tier       | RPO        | RTO         | Configuration              |
| ---------- | ---------- | ----------- | -------------------------- |
| Standard   | 1 hour     | 2 hours     | Single region, multi-AZ    |
| Enhanced   | 15 minutes | 30 minutes  | EFS replication enabled    |
| Enterprise | Near-zero  | < 5 minutes | Active-active multi-region |

***

### Troubleshooting

#### Pods Stuck in Pending State

```bash
kubectl describe pod <pod-name> -n default
```

Common causes:

* **Insufficient CPU/Memory** — Scale up the node group or use a larger instance type
* **PVC not bound** — Check EFS/EBS CSI driver pods are running in `kube-system`
* **Image pull error** — Verify ECR image URIs and IAM permissions for the node role

#### Terraform Apply Fails on Helm Release

```bash
helm list --all-namespaces
helm status <release-name> -n <namespace>
```

If a release is stuck in `pending-install`, delete it and re-apply:

```bash
helm delete <release-name> -n <namespace>
terraform apply -var-file=terraform.tfvars
```

#### Cannot Access the Platform URL

1. Confirm the NLB is provisioned: `kubectl get svc -n ingress-nginx`
2. Check the external IP/hostname is assigned to the NLB
3. Verify your DNS record points to the NLB hostname
4. Confirm the ACM certificate status is `ISSUED` in the AWS Console
5. Check ingress rules: `kubectl get ingress --all-namespaces`

#### Observability Datastore Out of Memory

Increase the memory allocation in `terraform.tfvars`:

```hcl
observability_datastore_pod_memory_gb         = 32
observability_datastore_pod_memory_request_gb = 16
```

Then re-apply:

```bash
terraform apply -var-file=terraform.tfvars
```

#### View Service Logs

```bash
# Sentinel logs
kubectl logs -l app=sentinel -n default --tail=100 -f

# CoreML logs
kubectl logs -l app=coreml -n default --tail=100 -f

# Observability Datastore logs
kubectl logs -l app=observability-datastore -n default --tail=100 -f
```

***

### Support

For deployment assistance or to obtain Karini AI container images and Helm chart access, contact:

* **Website:** [https://www.karini.ai](https://www.karini.ai/)
* **Email:** <support@karini.ai>

This guide applies to Karini AI deployments using the `eks-terraform-module`. For SaaS or managed deployment options, contact Karini AI directly.