MB-60029 - Add index types for tuning for recall/latency. #46
Conversation
vector.go
Outdated
| @@ -52,3 +52,8 @@ const IndexTypeRecall = "recall" | |||
| const IndexTypeLatency = "latency" | |||
|
|
|||
| const DefaultIndexType = IndexTypeRecall | |||
|
|
|||
| var SupportedIndexTunables = map[string]struct{}{ | |||
There was a problem hiding this comment.
Poor naming again - very vague. Change to SupportedIndexConfigOptimizations or some.
|
Thinking a little more here - how about offering a numeric setting as opposed to these 2 strings, it'd be more flexible and configurable. So one config - OptimizeForRecall - can take a numeric value ranging from 0 through 10. The growth could be a factor - so you'll need to apply settings based on your experiments. |
vector.go
Outdated
| @@ -45,3 +47,13 @@ var SupportedSimilarityMetrics = map[string]struct{}{ | |||
| EuclideanDistance: {}, | |||
| CosineSimilarity: {}, | |||
| } | |||
|
|
|||
| const IndexOptimizationForRecall = "recall" | |||
| const IndexOptimizationForLatency = "latency" | |||
There was a problem hiding this comment.
Do you not foresee us having to accommodate anything else here in the future?
vector.go
Outdated
|
|
||
| const DefaultIndexOptimization = IndexOptimizationForRecall | ||
|
|
||
| var SupportedIndexConfigOptimizations = map[string]struct{}{ |
There was a problem hiding this comment.
SupportedVectorIndexOptimizations*
vector.go
Outdated
| @@ -23,6 +23,8 @@ type VectorField interface { | |||
| Dims() int | |||
| // Similarity metric to be used for scoring the vectors | |||
| Similarity() string | |||
| // index type - tuned for either latency/recall | |||
| IndexOptimization() string | |||
There was a problem hiding this comment.
IndexOptimizedFor() string
abhinavdangeti
requested review from
Thejas-bhat,
CascadingRadium,
Likith101 and
moshaad7
metonymic-smokey
force-pushed
the
ivf-tuning
branch
from
8ab771f to
2fad192
Compare
Extends VectorField interface for optimizing for either recall/latency.