Hey ! Going to take a look at it asap. Can you add me to your org? Scott@helicone.ai
small bug btw: if I click on the "J" icon for my name in the top left and try to invite members from there, it sends a GET request instead of a POST and throws an error. doing it through the organization tab works as expected
Thanks for Reporting , sorry you are experiencing these issues, please check out our deal on the YC deals page too!
Glad you are liking it, would love to learn more about Needl too
Thanks guys, definitely interested in the deal at some point, want to evaluate a few different products a bit more before I commit!
Let me know if y'all have any trouble replicating the issue we're having and/or when it's fixed.
Hi , how are you using Helicone? You mentioned you're using asynchronous requests w/ the Helicone proxy. Are you using our Python or Node package? Or are you using an OpenAI package but modifying the baseUrl? Thanks
openai package - i'm changing api_base to the helicone URL and calling the API with openai.ChatCompletion.acreate(..., stream=True)
I'm pretty sure it has something to do with using streaming and function calling at the same time- when I use function calling without streaming it works fine and when I use streaming without function calling everything also logs correctly
Thanks . I attempted to recreate it today with no luck. I will try again tomorrow and report back. Are you able to send a code snippet or how you’re making the request? Would appreciate it, thanks.
Hi sorry for the delay here. Is this for a streamed request?
yes, its a streamed request
and it happens when gpt responds with a function call
I spoke with and we don't support streaming functions at the moment. This is something we want to support and I will add a ticket to track this. Do you have a need to stream functions?
Dang that sucks - yeah we need to stream them - we’re generating responses to phone calls so latency is very important for us
if its not something you guys can prioritize, could you maybe point me to where changes would be needed in the codebase? i’d be willing to contribute if i get a chance
Oh cool sure! I think it would be best if we hopped on a call and I showed you, are you open for that?
Sorry didn't see your reply, yeah we can jump on a call - today's a bit hectic but wednesday and friday this week are good for me if you're free
Yeah Friday I’m open all day. Want to just throw something on my calendar?
justin@helicone.ai
sent you an invite, lmk if you need a different time
fwiw, I also just ran into this. A lot of the docs about OpenAI functions use streaming by default, e.g.
https://sdk.vercel.ai/docs/guides/providers/openai-functionsIf so, args and completion tokens are missing in Helicone. I've turned off streaming for now, but would interested in seeing Helicone support it!
Hi and
Just fixed this
This is with streams! However, the TokenCount is still quite messed up, and should not be relied on when using streams with functions. We still dont know what the internals of this API look like so it is really hard to calculate the tokens here
