Proxy vs Async Integration

Compare Helicone’s Proxy and Async integration methods. Understand the features, benefits, and use cases for each approach to choose the best fit for your LLM application.

Quick Compare

There are two ways to interface with Helicone - Proxy and Async. We will help you decide which one is right for you, and the pros and cons with each option.

	Proxy	Async
Easy setup	✅	❌
Prompts	✅	✅
Prompts Auto Formatting (easier)	✅	❌
Custom Properties	✅	✅
Bucket Cache	✅	❌
User Metrics	✅	✅
Retries	✅	❌
Custom rate limiting	✅	❌
Open-source	✅	✅
Not on critical path	❌	✅
0 Propagation Delay	❌	✅
Negligible Logging Delay	✅	✅
Streaming Support	✅	✅

Proxy

The primary reason Helicone users choose to integrate with Helicone using Proxy is its simple integration.

It’s as easy as changing the base URL to point to Helicone, and we’ll forward the request to the LLM and return the response to you.

Proxy: flow of data.

Since the proxy sits on the edge and is the gatekeeper of the requests, you get access to a suite of Gateway tools such as caching, rate limiting, API key management, threat detection, moderations and more.

Here's a simple example

Instead of calling the OpenAI API with api.openai.com, you will change the URL to a Helicone dedicated domain oai.helicone.ai.

You can also use the general Gateway URL gateway.helicone.ai if Helicone doesn’t have a dedicated domain for the provider yet.

For a detailed documentation, check out Gateway Integration.

Async

Helicone Async allows for a more flexible workflow where the actual logging of the event is not on the critical path. This gives some users more confidence that if we are going down or if there is a network issue that it will not affect their application.

Get started with OpenLLMetry.

Async: flow of data.

The downside is that we cannot offer the same suite of tools as we can with the proxy.

Summary

When to Use Proxy

• When you need a quick and easy setup.
• If you require Gateway features like custom rate limiting, caching, and retries.
• When you want to use tools that can be instrumented directly into the proxy.

When to Use Async

• If you prefer the logging of events to be off the critical path, ensuring that network issues do not affect your application.
• When you need zero propagation delay.

Integrate with Helicone

Choose your LLM provider and get started with Helicone.