We have just starting using Helicone Pro version. Our main use case is to add monitoring and a better control of keys. Vault was the reason for us to test Helicone in prod. We enable the same on a prod app and we are getting latency increase that is not acceptable. Without Vaulted keys latency incrase vs OpenAI is on <10's ms. with that enable increases to > 2-4 Seconds! For stable requests in a range of 700ms on raw OpenAI hits. Is this something "normal"? We are using python directly with hostname and manual headers for the vaulted keys.
Also linked to that we don't see latency numbers matching with raw numbers on new relic. It's really weird. Your dashboard shows a stable >2seconds now (Without vault) and New Relic responses from your API endpoints are back to 700-800ms. I don't really know how to explain all of this. 😫
Hi ! Thank you for reporting this. We will be moving our vault reads to be cached on the edge soon. This is on our roadmap. Sorry you are experiencing this delay. We just moved this up on our priority list and will try to fix this tomorrow. Is there a good email for you all that I can ping when this is implemented?
