Helicone Community Page

Updated 9 months ago

Streaming Capabilities and Token Counting Support

also do you not support openrouter token counting? for streaming
Attachment
image.png
J
S
38 comments
We do not, but we are going to add it soon@
cool! how soon is soon? πŸ˜„
might roll my own if its more than a week or two
Ummm depends how quickly I can get a list of all the models and the cost
i think they have that on their api, no? if not its largely static, even letting me input the cost manually would be nice, tokenization is fairly standard ( i think BPE is most common right, not sure of exact differences between the models)
Is the tokens in the response at all? That is kind of surprising they dont automatically return them... or do they just not return them in streaming?
yeah its just a streaming thing
Also I found a list of all the models and costs here
https://openrouter.ai/docs#quick-start
openrouter has the model costs listed per mil tokens just dunno if its on their api
Which model are you using? And is it not on their API?
also would totally pay for like a "api normalizer" type thing, like openrouter or that open source python one
im using anthropic claude opus mostly
have you seen LITELLM?
some of our users use haiku sonnet and gpt4 but most opus
yeah thats what i was thinking of
dont super feel like hosting it but can
your using open router for GPT-4?
nah that routes to the normal openai endpoint
Which models are you using on OpenRouter?
ohh sorry I was miss understanding
I think we can get this out optimistically end of week. Realistically, end of next week.. Sorry for the delay, we are already prety over subscribed on work this week
no worries thats pretty solid
hey any word on the token count for openrouter?
also i saw yall supported images now but mine are still not being included
Sorry for the delay on open router!! It’s on track this week, what are you using for images?
Are you using OpenAI?
Hi @Sulaiman Khan Ghori sorry for the the delay, we are adding support for claude streaming and cost here on OpenRouter:
https://github.com/Helicone/helicone/pull/1782
This should be merged tomorrow
we are looking to add support for images soon, a bunch of other pressing stuff came up. Sorry about that
Ah it's okay I've moved to Keywords
ahh makes sense
Add a reply
Sign up and join the conversation on Discord