You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When running the evaluation for a query set containing one query only no metrics are calculated and stored in the metrics index. As soon as a second query is in the query set the metrics are calculated and stored in the metrics index.
This appears to be independent from the judgements available in the judgments index.
Example for creating a query set with only one query:
POST search_quality_eval_query_sets/_doc/1
{
"sampling": "none",
"name": "30 inch ceiling fan without light",
"description": "only 30 inch ceiling fan without light",
"queries": [
{
"30 inch ceiling fan without light": 11
}
],
"timestamp": "2024-12-12T11:30:20.916Z"
}
The log output indicates that no lookups happen in the judgments index as no frog number (ratio of missing judgements) is reported.
Update
It appears that for a query set with two queries, only metrics for one query are calculated, and for a query set of three only two queries do get metrics calculated and indexed.
The issue may lie in the handling of query sets in general. Query sets of one query seem to only be a special case of these.
The text was updated successfully, but these errors were encountered:
When running the evaluation for a query set containing one query only no metrics are calculated and stored in the metrics index. As soon as a second query is in the query set the metrics are calculated and stored in the metrics index.
This appears to be independent from the judgements available in the
judgments
index.Example for creating a query set with only one query:
The log output indicates that no lookups happen in the
judgments
index as no frog number (ratio of missing judgements) is reported.Update
It appears that for a query set with two queries, only metrics for one query are calculated, and for a query set of three only two queries do get metrics calculated and indexed.
The issue may lie in the handling of query sets in general. Query sets of one query seem to only be a special case of these.
The text was updated successfully, but these errors were encountered: