Class Lucene99ScalarQuantizedVectorsFormat
- java.lang.Object
-
- org.apache.lucene.codecs.KnnVectorsFormat
-
- org.apache.lucene.codecs.hnsw.FlatVectorsFormat
-
- org.apache.lucene.codecs.lucene99.Lucene99ScalarQuantizedVectorsFormat
-
- All Implemented Interfaces:
NamedSPILoader.NamedSPI
public class Lucene99ScalarQuantizedVectorsFormat extends FlatVectorsFormat
Format supporting vector quantization, storage, and retrieval- WARNING: This API is experimental and might change in incompatible ways in the next release.
-
-
Field Summary
Fields Modifier and Type Field Description static float
DYNAMIC_CONFIDENCE_INTERVAL
Dynamic confidence intervalstatic String
NAME
static String
QUANTIZED_VECTOR_COMPONENT
-
Fields inherited from class org.apache.lucene.codecs.KnnVectorsFormat
DEFAULT_MAX_DIMENSIONS, EMPTY
-
-
Constructor Summary
Constructors Constructor Description Lucene99ScalarQuantizedVectorsFormat()
Constructs a format using default graph construction parametersLucene99ScalarQuantizedVectorsFormat(Float confidenceInterval, int bits, boolean compress)
Constructs a format using the given graph construction parameters.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description static float
calculateDefaultConfidenceInterval(int vectorDimension)
FlatVectorsReader
fieldsReader(SegmentReadState state)
Returns aKnnVectorsReader
to read the vectors from the index.FlatVectorsWriter
fieldsWriter(SegmentWriteState state)
Returns aFlatVectorsWriter
to write the vectors to the index.String
toString()
-
Methods inherited from class org.apache.lucene.codecs.hnsw.FlatVectorsFormat
getMaxDimensions
-
Methods inherited from class org.apache.lucene.codecs.KnnVectorsFormat
availableKnnVectorsFormats, forName, getName, reloadKnnVectorsFormat
-
-
-
-
Field Detail
-
QUANTIZED_VECTOR_COMPONENT
public static final String QUANTIZED_VECTOR_COMPONENT
- See Also:
- Constant Field Values
-
NAME
public static final String NAME
- See Also:
- Constant Field Values
-
DYNAMIC_CONFIDENCE_INTERVAL
public static final float DYNAMIC_CONFIDENCE_INTERVAL
Dynamic confidence interval- See Also:
- Constant Field Values
-
-
Constructor Detail
-
Lucene99ScalarQuantizedVectorsFormat
public Lucene99ScalarQuantizedVectorsFormat()
Constructs a format using default graph construction parameters
-
Lucene99ScalarQuantizedVectorsFormat
public Lucene99ScalarQuantizedVectorsFormat(Float confidenceInterval, int bits, boolean compress)
Constructs a format using the given graph construction parameters.- Parameters:
confidenceInterval
- the confidenceInterval for scalar quantizing the vectors, when `null` it is calculated based on the vector dimension. When `0`, the quantiles are dynamically determined by sampling many confidence intervals and determining the most accurate pair.bits
- the number of bits to use for scalar quantization (must be between 1 and 8, inclusive)compress
- whether to compress the quantized vectors by another 50% when bits=4. If `true`, pairs of (4 bit quantized) dimensions are packed into a single byte. This must be `false` when bits=7. This provides a trade-off of 50% reduction in hot vector memory usage during searching, at some decode speed penalty.
-
-
Method Detail
-
calculateDefaultConfidenceInterval
public static float calculateDefaultConfidenceInterval(int vectorDimension)
-
fieldsWriter
public FlatVectorsWriter fieldsWriter(SegmentWriteState state) throws IOException
Description copied from class:FlatVectorsFormat
Returns aFlatVectorsWriter
to write the vectors to the index.- Specified by:
fieldsWriter
in classFlatVectorsFormat
- Throws:
IOException
-
fieldsReader
public FlatVectorsReader fieldsReader(SegmentReadState state) throws IOException
Description copied from class:FlatVectorsFormat
Returns aKnnVectorsReader
to read the vectors from the index.- Specified by:
fieldsReader
in classFlatVectorsFormat
- Throws:
IOException
-
-