User Feedback

When building AI applications, you need real-world signals about response quality to improve prompts, catch regressions, and understand what users find helpful. User Feedback lets you collect positive/negative ratings on LLM responses, enabling data-driven improvements to your AI systems based on actual user satisfaction.

Why use User Feedback

• Improve response quality: Identify patterns in poorly-rated responses to refine prompts and model selection
• Catch regressions early: Monitor feedback trends to detect when changes negatively impact user experience
• Build training datasets: Use highly-rated responses as examples for fine-tuning or few-shot prompting

Quick Start

1. Make a request and capture the ID

   Capture the Helicone request ID from your LLM response:

   import OpenAI from "openai";
   
   const openai = new OpenAI({
     apiKey: process.env.OPENAI_API_KEY,
     baseURL: "https://oai.helicone.ai/v1",
     defaultHeaders: {
       "Helicone-Auth": `Bearer ${process.env.HELICONE_API_KEY}`,
     },
   });
   
   // Use a custom request ID for feedback tracking
   const customId = crypto.randomUUID();
   
   const response = await openai.chat.completions.create({
     model: "gpt-4o-mini",
     messages: [{ role: "user", content: "Explain quantum computing" }]
   }, {
     headers: {
       "Helicone-Request-Id": customId
     }
   });
   
   // Use your custom ID for feedback
   const heliconeId = customId;
   

   Alternative: Getting request ID from response

   You can also try to get the Helicone ID from response headers, though this may not always be available:

   const response = await openai.chat.completions.create({
     model: "gpt-4o-mini",
     messages: [{ role: "user", content: "Explain quantum computing" }]
   });
   
   // Try to get the Helicone request ID from response headers
   const heliconeId = response.response?.headers?.get("helicone-id");
   
   // If not available, you'll need to use a custom ID approach
   if (!heliconeId) {
     console.log("Helicone ID not found in headers, use custom ID approach instead");
   }
   

2. Submit feedback rating

   Send a positive or negative rating for the response:

   const feedback = await fetch(
     `https://api.helicone.ai/v1/request/${heliconeId}/feedback`,
     {
       method: "POST",
       headers: {
         "Authorization": `Bearer ${process.env.HELICONE_API_KEY}`,
         "Content-Type": "application/json",
       },
       body: JSON.stringify({
         rating: true  // true = positive, false = negative
       }),
     }
   );
   

3. View feedback analytics

   Access feedback metrics in your Helicone dashboard to analyze response quality trends and identify areas for improvement.

Configuration Options

Feedback collection requires minimal configuration:

Parameter	Type	Description	Default	Example
rating	boolean	User’s feedback on the response	N/A	true (positive) or false (negative)
helicone-id	string	Request ID to attach feedback to	N/A	UUID

Processing multiple feedback ratings

When you need to submit feedback for multiple requests, use parallel API calls:

// Note: There is no bulk feedback endpoint - each rating requires a separate API call
const feedbackBatch = [
  { requestId: "f47ac10b-58cc-4372-a567-0e02b2c3d479", rating: true },
  { requestId: "6ba7b810-9dad-11d1-80b4-00c04fd430c8", rating: false },
  { requestId: "6ba7b811-9dad-11d1-80b4-00c04fd430c8", rating: true }
];

// Submit feedback in parallel for better performance
const feedbackPromises = feedbackBatch.map(({ requestId, rating }) =>
  fetch(`https://api.helicone.ai/v1/request/${requestId}/feedback`, {
    method: "POST",
    headers: {
      "Authorization": `Bearer ${process.env.HELICONE_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ rating }),
  })
);

// Wait for all feedback submissions to complete
const results = await Promise.all(feedbackPromises);

// Check for any failed submissions
results.forEach((result, index) => {
  if (!result.ok) {
    console.error(`Failed to submit feedback for request ${feedbackBatch[index].requestId}`);
  }
});

Use Cases

Chat Application Quality

Track user satisfaction with AI assistant responses:

Node.js

import OpenAI from "openai";

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
  baseURL: "https://oai.helicone.ai/v1",
  defaultHeaders: {
    "Helicone-Auth": `Bearer ${process.env.HELICONE_API_KEY}`,
  },
});

// In your chat handler
async function handleChatMessage(userId: string, message: string) {
  const requestId = crypto.randomUUID();
  
  const response = await openai.chat.completions.create(
    {
      model: "gpt-4o-mini",
      messages: [
        { role: "system", content: "You are a helpful assistant." },
        { role: "user", content: message }
      ]
    },
    {
      headers: {
        "Helicone-Request-Id": requestId,
        "Helicone-User-Id": userId,
        "Helicone-Property-Feature": "chat"
      }
    }
  );

  // Store request ID for later feedback
  await storeRequestMapping(userId, requestId, response.id);
  
  return response;
}

// When user clicks thumbs up/down
async function handleUserFeedback(userId: string, responseId: string, isPositive: boolean) {
  const requestId = await getRequestId(userId, responseId);
  
  await fetch(
    `https://api.helicone.ai/v1/request/${requestId}/feedback`,
    {
      method: "POST",
      headers: {
        "Authorization": `Bearer ${process.env.HELICONE_API_KEY}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({ rating: isPositive }),
    }
  );
}

Understanding User Feedback

How it works

User feedback creates a continuous improvement loop for your AI application:

• Each LLM request gets a unique Helicone ID
• Users rate responses as positive (helpful) or negative (not helpful)
• Feedback is linked to the original request for analysis
• Dashboard aggregates feedback to show quality trends

Explicit vs Implicit Feedback

Explicit feedback is when users directly rate responses (thumbs up/down, star ratings). While valuable, it has low response rates since users must take deliberate action.

Implicit feedback is derived from user behavior and is much more valuable since it reflects actual usage patterns:

Track user actions that indicate response quality:

// Code completion acceptance (like Cursor)
async function trackCodeCompletion(requestId: string, suggestion: string) {
  // Monitor if user accepts the completion
  const accepted = await waitForUserAction(suggestion);
  
  await fetch(`https://api.helicone.ai/v1/request/${requestId}/feedback`, {
    method: "POST",
    headers: {
      "Authorization": `Bearer ${process.env.HELICONE_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ 
      rating: accepted // true if accepted, false if rejected/ignored
    }),
  });
}

// Chat engagement patterns
async function trackChatEngagement(requestId: string, response: string) {
  // Track user behavior after response
  const userActions = await monitorUserBehavior(60000); // 1 minute
  
  const implicitRating = 
    userActions.continuedConversation || // User asked follow-up
    userActions.copiedResponse ||        // User copied the answer
    userActions.sharedResponse ||        // User shared/saved
    userActions.timeSpent > 30;          // User read for >30 seconds
  
  await submitFeedback(requestId, implicitRating);
}

// Search/recommendation clicks
async function trackSearchResult(requestId: string, results: string[]) {
  // Monitor if user clicks on suggested results
  const clicked = await trackClicks(results, 300000); // 5 minutes
  
  // High click-through rate = good recommendations
  const rating = clicked.length > 0;
  await submitFeedback(requestId, rating);
}

Related Features

Custom Properties

Segment feedback by feature, user type, or experiment for deeper insights

User Metrics

Combine feedback with usage data to understand user satisfaction trends

Sessions

Track feedback across multi-turn conversations and workflows

Alerts

Set up notifications when feedback rates drop below thresholds