Helicone Community Page

Updated 3 months ago

Cache returns wrong results

Hi, I'm seeing the cache returns wrong responses, for example if I send "orange" what I see on the Cache logs is helicone thinks I sent "burger". I think it's likely to be using the Helicone-Cache-Seed header (because it returns previously returned entries for the same user).

My current request header looks like this:

const headers = {
"Authorization": "Bearer OPENAI_KEY",
"OpenAI-Organization": "OPENAI_ORG",
"Content-Type": "application/json",
"Helicone-Auth": "Bearer HELICONE_KEY",
// Cache: https://docs.helicone.ai/features/advanced-usage/caching
"Cache-Control": "max-age=31104000", // 360 days == 31104000 seconds
"Helicone-Cache-Enabled": "true",
"Helicone-Cache-Bucket-Max-Size": "20", // different responses saved, 20 is max for free accounts
"Helicone-Cache-Seed": userID, // returns the same response to the same user
// Retry: https://docs.helicone.ai/features/advanced-usage/retries
"Helicone-Retry-Enabled": "true",
"Helicone-Retry-Num": "2",
// Rate limit: https://docs.helicone.ai/features/advanced-usage/custom-rate-limits
"Helicone-RateLimit-Policy": "20;w=600;s=user", // limited by user, +20 requests within 10 min
};

Or am I missing something / doing something wrong here?

Thanks so much for the help!
Luis.
J
L
4 comments
Yep! Great find - sorry about this!! Throwing in a fix now
wow that was fast! thank you Justin
Adding a regression test now so this wont be an issue in the future. Great edge case! Sorry about that
Add a reply
Sign up and join the conversation on Discord