-
Notifications
You must be signed in to change notification settings - Fork 689
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MB-59421: add validation for vector field aliases #1903
Conversation
i believe i follow the changes your trying to make over here. basically, the argument there is a cache object being passed in the documentMapping.Validate(cache) [basically the signature before the PR] which is also maintained throughout the parsing of the document mapping. Now, my suggestion over here is if this cache object can be leveraged to do the validations. roughly speaking, my thinking revolves around possiblity of introducing a new field in the registry.Cache say "VectorCache" which maps field.Name -> &vecFieldConfig{simlarity, dims ... } and during parsing perhaps what can be done is
and inside the cache, you check that if there is an entry corresponding to the field.Name in the VectorCache, and then validate the passed config value with what's there in the cache entry. if the entry is not present already, just set the value. the main reason why i'm suggesting this is because, i feel like the changes would be isolated to the VectorCache component and avoid change all the function signatures over here. please let me know what you all think about this. |
- If there are vector field Aliases (fields with same name and type as vector) across defaultMapping and typeMapping, Then, their dimensions and similarity value must be same
3e6ef06
to
4023370
Compare
@Thejas-bhat , For example, If in the index mapping, we specify "en" analyzer. And that way, after the completion of Validation, we will populate our cache with all the relevant analysis related objects (analyzers, tokenizers, .... ) Now,
During index and Search operations, we will call AnalyzerNamed method of the indexMapping to get the cached Analyzer object. That's the whole use of this Registry.Cache. I don't think fieldAliasCtx should go into this, because, we don't want to keep this object after Validation is over. @abhinavdangeti also agreed to this. |
vector field Aliases (fields with same name and type as vector) must have same value for dimensions and similarity, respectively.