OpenAI Compatible Endpoints

If you are utilizing an OpenAI compatible endpoint, you can conveniently log your requests to Helicone by modifying the base URL from https://oai.helicone.ai/v1 to any of the following supported dedicated endpoints:
- https://together.helicone.ai/v1

Are you using another target that is OpenAI compatible?

If the target is a domain that has been approved with a dedicated domain (e.g., TogetherAI), then you can replace the base URL with the dedicated domain. If the target has not been approved, you can utilize the Helicone Gateway to log your requests to Helicone.

Example:

Instead of using the base URL

base_url=“https://oai.helicone.ai/v1”,

You can use the base URL

base_url=“https://together.helicone.ai/v1”,

• Python
• Node.js
• Node.js (w/ package)
• cURL
• Langchain

OpenAI v1+

Set HELICONE_API_KEY as an environment variable

export HELICONE_API_KEY=<your API key>

Initialize the OpenAI client. This can also be done via a global client.

client = OpenAI(
  api_key="your-api-key-here",  # Replace with your OpenAI API key
  base_url="https://oai.helicone.ai/v1",  # Set the API endpoint
  default_headers= {  # Optionally set default headers or set per request (see below)
    "Helicone-Auth": f"Bearer {HELICONE_API_KEY}",
  }
)

Send a request and attach extra headers

chat_completion = client.chat.completions.create(
  model="gpt-4-vision-preview",
  messages=[
      {"role": "user", "content": "Hello world!"}
  ],
  extra_headers={ # Can also attach headers per request
      "Helicone-Auth": f"Bearer {HELICONE_API_KEY}",
  },
)

Congratulations! Your OpenAI requests are now logging results to Helicone.

OpenAI <v1

Set HELICONE_API_KEY as an environment variable

export HELICONE_API_KEY=<your API key>

Modify the API base and add a Helicone-Auth header

import openai

client = OpenAI(
    base_url="https://oai.helicone.ai/v1",
    default_headers={
        "Helicone-Auth": f"Bearer {HELICONE_API_KEY}",
    }
)

client.chat.completions.create(
  # ...other parameters
  extra_headers={
    "Helicone-Auth": f"Bearer {HELICONE_API_KEY}",
  }
)

Congratulations! Your OpenAI requests are now logging results to Helicone.

OpenAI v1+

Set HELICONE_API_KEY as an environment variable

export HELICONE_API_KEY=<your API key>

Initialize the OpenAI client. This can also be done via a global client.

client = OpenAI(
  api_key="your-api-key-here",  # Replace with your OpenAI API key
  base_url="https://oai.helicone.ai/v1",  # Set the API endpoint
  default_headers= {  # Optionally set default headers or set per request (see below)
    "Helicone-Auth": f"Bearer {HELICONE_API_KEY}",
  }
)

Send a request and attach extra headers

chat_completion = client.chat.completions.create(
  model="gpt-4-vision-preview",
  messages=[
      {"role": "user", "content": "Hello world!"}
  ],
  extra_headers={ # Can also attach headers per request
      "Helicone-Auth": f"Bearer {HELICONE_API_KEY}",
  },
)

Congratulations! Your OpenAI requests are now logging results to Helicone.

OpenAI <v1

Set HELICONE_API_KEY as an environment variable

export HELICONE_API_KEY=<your API key>

Modify the API base and add a Helicone-Auth header

import openai

client = OpenAI(
    base_url="https://oai.helicone.ai/v1",
    default_headers={
        "Helicone-Auth": f"Bearer {HELICONE_API_KEY}",
    }
)

client.chat.completions.create(
  # ...other parameters
  extra_headers={
    "Helicone-Auth": f"Bearer {HELICONE_API_KEY}",
  }
)

Congratulations! Your OpenAI requests are now logging results to Helicone.

Set HELICONE_API_KEY as an environment variable

export HELICONE_API_KEY=<your API key>

Modify the base path and add a Helicone-Auth header

OpenAI V4+

import OpenAI from "openai";

const openai = new OpenAI({
  apiKey: request.env.OPENAI_API_KEY,
  baseURL: "https://oai.helicone.ai/v1",
  defaultHeaders: {
    "Helicone-Auth": `Bearer ${request.env.HELICONE_API_KEY}`,
  },
});

OpenAI < V4

import { Configuration, OpenAIApi } from "openai";

const configuration = new Configuration({
    apiKey: process.env.OPENAI_API_KEY,
    basePath: "https://oai.helicone.ai/v1",
    baseOptions: {
      headers: {
          "Helicone-Auth": `Bearer ${process.env.HELICONE_API_KEY}`,
      },
    },
});

const openai = new OpenAIApi(configuration);

Installation and Setup

1

To get started, install the `helicone-openai` package

npm install @helicone/helicone

2

Set `HELICONE_API_KEY` as an environment variable

export HELICONE_API_KEY=sk-<your-api-key>

You can also set the Helicone API Key in your code (See below)

3

Replace

const { ClientOptions, OpenAI } = require("openai");

with

const { HeliconeProxyOpenAI as OpenAI,
  IHeliconeProxyClientOptions as ClientOptions } = require("helicone");

4

Make a request

Chat, Completion, Embedding, etc usage is equivalent to OpenAI package.

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
  heliconeMeta: {
    apiKey: process.env.HELICONE_API_KEY, // Can be set as env variable
    // ... additional helicone meta fields
  },
});

const chatCompletion = await openai.chat.completion.create({
  model: "gpt-3.5-turbo",
  messages: [{ role: "user", content: "Hello world" }],
});

console.log(chatCompletion.data.choices[0].message);

Send feedback

Ensure you store the helicone-id header returned in the original response

const { data, response } = await openai.chat.completion.create({
  model: "gpt-3.5-turbo",
  messages: [{ role: "user", content: "Hello world" }],
}).withResponse();

const heliconeId = response.headers.get("helicone-id");

await openai.helicone.logFeedback(heliconeId, HeliconeFeedbackRating.Positive) // or Negative

HeliconeMeta options

interface IHeliconeMeta {
  apiKey?: string;
  properties?: { [key: string]: any };
  cache?: boolean;
  retry?: boolean | { [key: string]: any };
  rateLimitPolicy?: string | { [key: string]: any };
  user?: string;
  baseUrl?: string;
  onFeedback?: OnHeliconeFeedback; // Callback after feedback was processed
}

type OnHeliconeLog = (response: Response) => Promise<void>;
type OnHeliconeFeedback = (result: Response) => Promise<void>;

Replace the OpenAI Base URL with Helicone’s

- POST https://api.openai.com/v1

- POST https://oai.helicone.ai/v1

Add a Helicone-Auth header into the requests

"Helicone-Auth": "Bearer HELICONE_API_KEY"

Here is an example cURL command:

Please ensure to replace API keys with your own

      curl --request POST \
          --url https://oai.helicone.ai/v1/chat/completions \
          --header 'Authorization: Bearer <<YOUR_OPENAI_API_KEY>>' \
          --header 'Content-Type: application/json' \
          --header 'Helicone-Auth: Bearer <<YOUR_HELICONE_API_KEY>>' \
          --data '{
              "model": "gpt-3.5-turbo",
              "messages": [
                  {
                      "role": "system",
                      "content": "Say Hello!"
                  }
              ],
              "temperature": 1,
              "max_tokens": 10
      }'

Python

Modify the API base and add a Helicone-Auth header

#  Option 1
openai.api_base = "https://oai.helicone.ai/v1"
llm = ChatOpenAI(
    openai_api_key='<>',
    model_kwargs={
      "extra_headers":{
        "Helicone-Auth": f"Bearer {HELICONE_API_KEY}"
      }
    }
)

# Option 2
llm = ChatOpenAI(
    openai_api_key='<>',
    model_kwargs={
      "extra_headers":{
        "Helicone-Auth": f"Bearer {HELICONE_API_KEY}"
      }
    },
    openai_api_base="https://oai.helicone.ai/v1",
)

Typescript

const llm = new OpenAI({
  modelName: "gpt-3.5-turbo",
  configuration: {
    basePath: "https://oai.helicone.ai/v1",
    defaultHeaders: {
      "Helicone-Auth": `Bearer HELICONE_API_KEY`,
    },
  },
});