-
Notifications
You must be signed in to change notification settings - Fork 689
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MB-60029: Accommodate vector_index_optimized_for into vector field mapping #1959
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove the draft tags so github workflows can run.
477ef28
to
348d132
Compare
mapping/mapping_vectors.go
Outdated
@@ -202,6 +208,12 @@ func validateVectorFieldAlias(field *FieldMapping, parentName string, | |||
reflect.ValueOf(index.SupportedSimilarityMetrics).MapKeys()) | |||
} | |||
|
|||
if _, ok := index.SupportedVectorIndexOptimizations[field.VectorIndexOptimizedFor]; !ok { | |||
return fmt.Errorf("field: '%s', invalid optimization "+ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's not error out, with future in mind if we add more options.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought updating index.SupportedVectorIndexOptimizations in bleve_index_api and its version in bleve's go.mod would take care of this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do you mean!? I'm talking about the case in the future - when we add more optimizations and there's a mixed cluster scenario where some nodes won't support it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah right.
Removed this snippet so there's no check at this point for valid optimisation metrics.
This PR adds the functionality needed to introduce "vector_index_optimized_for" into the vector field mapping to allow each vector fields' index to be optimised for latency/recall.