Storing Qualitative User Feedback and Evaluations

Hi Deen! I'd love to learn more about what you are trying to measure, can we hop on a call?

sure scheduled

happy to chat regarding my specific use case over the call regardless, but i take it it's not possible today then?

Sorry - can we push back 3 hours?

lets say i have someone manually grading model outputs. Lets say we prompted "2+2" and the model output "5". is there a way in helicone to store what would be the correct model output e.g. "4"? either as "request scoring/grading" or as some alternative metadata (although it feels like this should be a first class citizen)?

the thought is that later this info could ne potentially used in things such as evaluations/finetuning

Hi Deen, sorry for the delay here

lets say i have someone manually grading model outputs. Lets say we prompted "2+2" and the model output "5". is there a way in helicone to store what would be the correct model output e.g. "4"? either as "request scoring/grading" or as some alternative metadata (although it feels like this should be a first class citizen)?

yes you might be looking for this
https://docs.helicone.ai/features/advanced-usage/custom-properties#option-1-pre-define-the-request-id