Streaming Capabilities and Token Counting Support

We do not, but we are going to add it soon@

cool! how soon is soon? 😄

might roll my own if its more than a week or two

Ummm depends how quickly I can get a list of all the models and the cost

i think they have that on their api, no? if not its largely static, even letting me input the cost manually would be nice, tokenization is fairly standard ( i think BPE is most common right, not sure of exact differences between the models)

Is the tokens in the response at all? That is kind of surprising they dont automatically return them... or do they just not return them in streaming?

yeah its just a streaming thing

Also I found a list of all the models and costs here
https://openrouter.ai/docs#quick-start

for sync they do

openrouter has the model costs listed per mil tokens just dunno if its on their api

Which model are you using? And is it not on their API?

also would totally pay for like a "api normalizer" type thing, like openrouter or that open source python one

im using anthropic claude opus mostly

have you seen LITELLM?

some of our users use haiku sonnet and gpt4 but most opus

yeah thats what i was thinking of

dont super feel like hosting it but can

your using open router for GPT-4?

nah that routes to the normal openai endpoint

Which models are you using on OpenRouter?

ohh sorry I was miss understanding

okay cool

I think we can get this out optimistically end of week. Realistically, end of next week.. Sorry for the delay, we are already prety over subscribed on work this week

no worries thats pretty solid

hey any word on the token count for openrouter?

also i saw yall supported images now but mine are still not being included

@Justin

Hey!

Sorry for the delay on open router!! It’s on track this week, what are you using for images?

Are you using OpenAI?

claude 3

via openrouter

Hi @Sulaiman Khan Ghori sorry for the the delay, we are adding support for claude streaming and cost here on OpenRouter:
https://github.com/Helicone/helicone/pull/1782

This should be merged tomorrow

we are looking to add support for images soon, a bunch of other pressing stuff came up. Sorry about that

Ah it's okay I've moved to Keywords

ahh makes sense