Cloud Deployment

Set up Helicone on cloud infrastructure. Step-by-step instructions for deploying the LLM observability platform on a single node or Kubernetes cluster in the cloud.

Running on a single node

Create your node on the cloud. For example, on AWS, you can create an EC2 instance. Make sure that the instance has enough memory and disk space to run Helicone. For example, you can use the m5.2xlarge instance type on AWS.

Recommend Requirements

• 8 vCPUs
• 32 GB RAM
• 250 GB disk space

Within your remote machine

Install Docker

Step 1: Clone the repo

git clone https://github.com/Helicone/helicone.git

Step 2: Run docker compose

cd helicone/docker
cp .env.example .env
sudo docker compose up -d

Assuming you have your ssh config setup like this…

Host helicone
    HostName 1.1.1.1
    User ubuntu
    IdentityFile ~/Desktop/secret.pem

Let’s test it by running this command on your localhost

ssh -N \
  -L 3000:localhost:3000 \
  -L 8989:localhost:8989 \
  -L 8000:localhost:8000 \
  -L 8787:localhost:8787 \
  -L 8788:localhost:8788 \
  helicone

Now you can access the Helicone UI at http://localhost:3000

You can add users by going here: http://localhost:8989/project/default/auth/users and clicking “Add User”.

After you add a user, you can connect to the dashboard at http://localhost:3000/signin

The proxy is setup at localhost:8787 for OpenAI.

You can test it works by adding a new key on the http://localhost:3000 and then running this command on your localhost

# Test command
curl localhost:8787/helicone/test

# Example log command
curl --request POST \
    --url http://localhost:8787/v1/chat/completions \
    --header 'Authorization: Bearer <<YOUR_OPENAI_API_KEY>>' \
    --header 'Content-Type: application/json' \
    --header 'Helicone-Auth: Bearer <<YOUR_HELICONE_API_KEY>>' \
    --data '{
        "model": "gpt-3.5-turbo",
        "messages": [
            {
                "role": "system",
                "content": "Say Hello!"
            }
        ],
        "temperature": 1,
        "max_tokens": 10
 }'

You should see all the data show up on Helicone.

The next steps before becoming production ready are:

1. Follow the instructions here like

   • Change the default password
   • Add a custom domain
   • Add SSL
   • Add a SMTP server
   • Generate API keys

2. Add a load balancer in front of the server

   • Add HTTPS to web and workers
   • Change public variables for our backend routes. (NEXT_PUBLIC_BASE_PATH, SUPABASE_PUBLIC_URL, API_EXTERNAL_URL)

3. Add a database backup strategy and change the volume to a persistent volume

Running in Kubertes (Beta)

We have a Helm chart that is not well maintained that you all can try. It’s not production ready yet.

https://github.com/Helicone/helicone-helm-chart