Helicone

Log in to community

Helicone Community Page

C

K

m

H

E

Join on Discord

Tokens calculation is wrong

Tokens calculation is wrong

Join on Discord

Hey guys, found the issue with tokens calculation which affects cost projection. Instead of 2-4k we get ~22 tokens 😀. Noticed that for all OpenAI models almost a month ago (streaming is on). Have you met the same before 😀 ? We're using proxy integration (https://docs.helicone.ai/integrations/openai/python) with the following headers

headers = {
"Helicone-Auth": f"Bearer AUTHKEY",
"Helicone-Cache-Enabled": "true",
"Helicone-Property-CUSTOMPROPERTY": "custom_prop_value",
"Helicone-User-Id": user_id
}

Attachments

1

N

J

O

33 comments

Tested with disabled cache - got the same problem. Request IDs:
f8f66937-ba2c-4f58-8a9f-303171437e39
dbd193e8-9757-4773-9d07-b5a6ee15136d
c90556fc-e0c2-46f1-9b91-92692a5f980a
b34e6482-d751-4eef-8d35-a51ae396cb86

Attachment

Hi are you using streaming? If you are you might need to add the following options
https://docs.helicone.ai/use-cases/enable-stream-usage

Hey , yep we're using streaming. It helped, thanks a lot! 😄

Hi , we've been noticing the same issues with cost estimation of OpenAI's models since yesterday (streaming is on). Have smth changed in your API? 🙂

Request ID: 77051de5-4200-406f-8039-5942f378029c

Attachment

Hi Thanks for flagging I am taking a look now

Thanks a lot!

Am I on your org?

Nope 🙂

can you add me please? justin@helicone.ai?

and send me the org name?

Oh, turned out I don't have rights to add members :D. Jsonify

can you send me the org id?

ohh

okay

I see

25d70dfb-342e-45fb-8632-56b3ab0f13ac

Hi Jsonify

❤️ love your product btw

🤩

We love helicone as well ^_^

ahh I see the issue :/

Thanks you for flagging!

We fixed this and we are rolling out a fix. We are going to try to restore the previous logs..

Awesome, ty! ❤️

Hi we were able to restore these logs! thanks again for flagging

Hey , glad to hear it! 😀

OOlivier Balais

Hi there, we also have a problem with token computation on Helicone with latest OpenAI models. Is there something we can do?

Attachment

Hi is this still an issue?

If it is - can you please share your organization ID and I can take a look?

OOlivier Balais

Hello Justin, yes, it is still a problem. Here is our organization ID: b4f3ca3a-9749-4231-82f7-4ac53003e4ba. Thanks for your help.

Hi @Olivier Balais you will need to include this flag:

Plain Text

    stream_options={
        "include_usage": True # set to true
    }

https://docs.helicone.ai/use-cases/enable-stream-usage

@Justin - If I'm using the Vercel AI library, do you know if there is any way to pass in that stream_options value?

Add a reply

Sign up and join the conversation on Discord

Join on Discord