holden

I previously had Helicone working as a proxy to rate limit requests, using an older version of the vercel ai sdk and openai-edge.

I just upgraded to the latest version and moved to openai v4, updating helicone as described here: https://docs.helicone.ai/getting-started/integration-method/openai-proxy

I only get 500s now, and have narrowed it to the Helicone-RateLimit-Policy header (it works if I omit that header).

Plain Text

export const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
  baseURL: "https://oai.hconeai.com/v1",
  defaultHeaders: {
    "Helicone-Auth": `Bearer ${heliconeKey}`,
    "Helicone-Retry-Enabled": "true",
    "Helicone-Retry-Num": "3",
    // I get a 500 if I uncomment the next line
    "Helicone-RateLimit-Policy": "1000;w=60;s=ip",
  },
});

I don't see the request/error in the Helicone dashboard, and the thrown error only has this:

Plain Text

[Error: Unknown error occurred] {
  status: 500,
  headers: {
  cf-ray: '8072fdfa795e1f32-DEN',
  connection: 'keep-alive',
  content-length: '220',
  content-type: 'application/json;charset=UTF-8',
  date: 'Fri, 15 Sep 2023 18:39:23 GMT',
  helicone-error: 'true',
  server: 'cloudflare',
  vary: 'Accept-Encoding'
},
  error: undefined,
  code: undefined,
  param: undefined,
  type: undefined
}

How should I fix or debug this?

Helicone Community Page

Rate limit header causing 500s