Table of Contents

Pulling the AI Designer API container image

1. The AI Designer API container image is hosted on https://quay.io at quay.io/joget/ai-designer-api:8.1-BETA
2. To pull the image, authentication to quay.io is required, and credentials can be requested from https://intelligence.int.joget.cloud.
3. With the credentials, use docker login in docker or imagePullSecrets in Kubernetes to pull the container image.

The secret for accessing the API

The secret is used for authentication to call the APIs.

...

1. Accessing the OpenAPI.json through the /docs endpoint with username as admin and password as the SECRET set in the env file.
2. Accessing the master API endpoint for revoking specific API keys.
3. Used to generate API keys by encoding JWT using the payload and the SECRET using the HS256 algorithm.

ENV File

Save this file as a .env file file:

SECRET=Please request from https://intelligence.int.joget.cloud/ or set your own

MODEL_TEMPERATURE=0.2

LOG_LEVEL=info

FASTAPI_APPLICATION_PORT=8000
APPLICATION_WORKERS=1
APPLICATION_THREADS=1

MAX_ATTEMPTS=2

HUGGING_FACE_TOKEN=Please add your hugging face token if using custom open source models

BACKEND_CORS_ORIGINS=*

ENVIRONMENT=server

BUILD_CELERY=false
BUILD_MONGO=false

Deployment using docker-compose

Save this file as docker-compose.yml or download it from the release package:

...

version: "3.3"
services:
 llm_workflow_api:
   image: quay.io/joget/ai-designer-api:8.1-BETA
   command: gunicorn -k uvicorn.workers.UvicornWorker --bind 0.0.0.0:8000 --workers ${APPLICATION_WORKERS} --threads ${APPLICATION_THREADS} llm_workflow.fastapi_application --timeout 400 --log-level=${LOG_LEVEL}
   env_file:
     - .env
   container_name: ai_designer_api
   restart: unless-stopped
   ports:
     - ${FASTAPI_APPLICATION_PORT}:8000
   environment:
     LOG_LEVEL: ${LOG_LEVEL}
     ENVIRONMENT: server
   volumes:
     - ./datafiles:/LLMWorkflow/llm_workflow/datafiles
   networks:
     - llm_workflow
   healthcheck:
     test: ["CMD", "curl", "-f", "-X", "POST", "http://localhost:8000/health-check"]
     interval: 30s
     timeout: 10s
     retries: 5
networks:
 llm_workflow:

...

Command

sudo docker compose -f ./docker-compose.yml --env-file .env up --build -d

Deployment on Kubernetes

Use the following deployment YAML and ensure it is placed inside the Kubernetes directory:

...

#!/bin/bash

TAG=${TAG:-"8.1-BETA"}
DOCKER_IMAGE=${DOCKER_IMAGE:-"quay.io/joget/ai-designer-api:$TAG"}
K8S_NAMESPACE=${K8S_NAMESPACE:-"ai-designer"}
INGRESS_HOST=${INGRESS_HOST:-"YOUR HOSTNAME"}

# Load environment variables from the .env file
if [ -f llm_workflow/.env ]; then
 set -a # Automatically export all variables
 source llm_workflow/.env
 set +a # Stop exporting variables automatically
else
 echo "Environment file llm_workflow/.env not found!"
 exit 1
Fi

echo Create Namespace $K8S_NAMESPACE
kubectl get namespace $K8S_NAMESPACE || 
 kubectl create namespace $K8S_NAMESPACE

echo Create ConfigMap
kubectl -n $K8S_NAMESPACE create configmap llm-wflow-config --from-env-file=./llm_workflow/.env.k8s --dry-run=client -o yaml > ./kubernetes/llm-config-configmap.yaml
kubectl -n $K8S_NAMESPACE apply -f ./kubernetes/llm-config-configmap.yaml

echo Deploy API Server
envsubst <  ./kubernetes/wflow-api-deployment.yaml | kubectl -n $K8S_NAMESPACE apply -f -

echo Wait for Deployment and Display Pods and Ingress
kubectl -n $K8S_NAMESPACE rollout status deploy ai-designer-api
kubectl -n $K8S_NAMESPACE get po

kubectl -n $K8S_NAMESPACE get ingress

Deployment using Docker

1. Create a Docker network.
   

   Code Block
   language bash
   docker network create llm_workflow

   
    
2. Source ENV variables.
   

   Code Block
   language bash
   set -a source .env set +a

   
   
3. Start the API server
   

   Code Block
   language bash
   docker run -d \ --name llm_workflow_api \ --env-file .env \ -e LOG_LEVEL=${LOG_LEVEL} \ -e ENVIRONMENT=server \ -v $(pwd)/datafiles:/LLMWorkflow/llm_workflow/datafiles \ -p ${FASTAPI_APPLICATION_PORT}:8000 \ --network llm_workflow \ --restart always \ quay.io/joget/ai-designer-api:8.1-BETA \ gunicorn -k uvicorn.workers.UvicornWorker --bind 0.0.0.0:8000 --workers ${APPLICATION_WORKERS} --threads ${APPLICATION_THREADS} llm_workflow.fastapi_application --timeout 400 --log-level=${LOG_LEVEL}

...