Repository navigation
#7609 AI Evaluation Redesign bugs #7736
Description
Description
After updating to 10.9 I have noticed some issues with @grafanaKibana 's redesign, namely:
- StringMetrics are no longer rendered, they appear as "?".
- Ground Truth is no longer visible.
These are what I noticed immediately, will update this issue if I find more.
Reproduction Steps
- Run the AI Evaluation with custom StringMetrics
- Open the AI Evaluation Report and see that both StringMetrics and Ground Truth's are all broken.
Expected behavior
I expect the StringMetrics and Ground Truths to be rendered correctly as was the cause for <10.9.
Actual behavior
The AI Evaluation Report shows the actual StringMetric result and the Ground Truth for each case.
Regression?
This was broken with version 10.9. Specifically with @grafanaKibana 's redesign in #7609 (issue #7593 )
Known Workarounds
No response
Configuration
No response
Other information
No response
Activity
grafanaKibana commentedon Sep 3, 2026
Will look into it shortly.
grafanaKibana commentedon Sep 3, 2026
@RickS-jpg Just took a look onto your findings:
Actually, they are rendered, the pill with question mark is intended to show interpretation status, not the value. Clicking on the row, should show the string value right after "Unknown" interpretation reason. But I see why you find it confusing. I will try to play with it, to make it more clear/show the value if there is not interpretation
Indeed,
supplied searchevaluationcontextis completely missingWill fix the findings ASAP. If you find anything else, ping me here additionally please
grafanaKibana commentedon Sep 4, 2026
@RickS-jpg
Is that how you would expect it to look like?
I would suggest to move it from end of transcript to the metric that using the context, to not create visual duplication if several metrics had same value provided to the different contexts
RickS-jpg commentedon Sep 4, 2026
@grafanaKibana
StringMetrics: Yes that's exactly how I would expect/want it to look like. Fits perfectly with how I could use it previously.
Ground Truth: I agree with your suggestion. Since Ground Truths can require different input (Completeness vs Groundedness for example) its best location seems to be in each metric itself.
Change looks great! (as does the rest of your redesign, it actually looks like a professional report now!)
I haven't encountered anything else that I'm missing compared to the old report.