OpenLLMetry Async Integration

Log LLM traces directly to Helicone, bypassing our proxy, with OpenLLMetry. Supports OpenAI, Anthropic, Azure OpenAI, Cohere, Bedrock, Google AI Platform, and more.

Overview

Async Integration let’s you log events and calls without placing Helicone in your app’s critical path. This ensures that an issue with Helicone will not cause an outage to your app.

• Node.js
• Python

1

Install Helicone Async

npm install @helicone/async

2

Initialize Logger

import { HeliconeAsyncLogger } from "@helicone/async";
import OpenAI from "openai";

const logger = new HeliconeAsyncLogger({
  apiKey: process.env.HELICONE_API_KEY,
  // pass in the providers you want logged
  providers: {
    openAI: OpenAI,
    //anthropic: Anthropic,
    //cohere: Cohere
    // ...
  }
});
logger.init();

const openai = new OpenAI();

async function main() {
  const completion = await openai.chat.completions.create({
    messages: [{"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Who won the world series in 2020?"},
      {"role": "assistant", "content": "The Los Angeles Dodgers won the World Series in 2020."},
      {"role": "user", "content": "Where was it played?"}],
    model: "gpt-3.5-turbo",
  });

  console.log(completion.choices[0]);
}

main();

3

Properties

You can set properties on the logger to be used in Helicone using the withProperties method. (These can be used for Sessions, User Metrics, and more.)

const sessionId = randomUUID();

logger.withProperties({
    "Helicone-Session-Id": sessionId,
    "Helicone-Session-Path": "/abstract",
    "Helicone-Session-Name": "Course Plan",
}, () => {
    const completion = await openai.chat.completions.create({
        ...
    })
})

Supported Providers

• OpenAI
• Anthropic
• Azure OpenAI
• Cohere
• Bedrock
• Google AI Platform

Other Integrations

• Comparing Proxy vs Async Integration
• Gateway Integration