Can I see your full code with LLM request?

At a glance

The post asks for the full code with a LiteLLM request. The comments discuss the issue of logging not working for async streaming with Anthropic's Sonnet model, but working for non-streaming and OpenAI. Community members suggest passing the correct Helicone API key and metadata, but the issue persists. They also discuss testing OpenAI with streaming, and one community member requests to be added to the organization to investigate further. The issue with Anthropic's async streaming logs not working remains unresolved.

sst_bokarev

Oh, can I see your full code with litellm request?

18 comments

CCaco

hey!

this is the code:

sst_bokarev

Hi @Caco ! Instead of extra_headers you need to provide metadata and pass helicone headers the same way

Attachment

sst_bokarev

Like this

response = completion(
    model="gpt-3.5-turbo",
    messages=[{"role": "user", "content": "Hi 👋 - i'm openai"}],
    metadata={
        "Helicone-Property-Hello": "World"
    }
)

CCaco

thanks!

that works for async non-streaming gpt-4o-mini
But async streaming sonnet doens't log

sst_bokarev

I guess it's because LiteLLM doesn't support async streaming for Anthropic, let me double check it

CCaco

it works for me! just the logs don't work

sst_bokarev

Hm, do you pass correct helicone api key?

CCaco

yep, passing them as

litellm.headers = {
"Helicone-Auth": f"Bearer {helicone_api_key}",
}

and in the metadata:

metadata = {
"Helicone-Auth": f"Bearer {helicone_api_key}",
}

CCaco

tried removing litellm.drop_params = True and doesn't work either

sst_bokarev

Are the logs not appearing only for Anthropic or also for OpenaAI?

CCaco

OpenAI without streaming works. Let me test OpenAI with streaming.

CCaco

This is the result of OpenAI with streaming 😅

Attachment