Add From<&[f32]> conversions for bulk upload optimization#244
Merged
Conversation
Member
|
Thank you for the detailed description. This makes a lot of sense. I'll try to review this tomorrow. And I'll make sure to merge this as part of the upcoming release. |
timvisee
requested changes
Nov 12, 2025
Enables creating vectors from borrowed slices to optimize bulk uploads from contiguous memory sources (Arrow, NumPy). Reduces per-vector allocations by deferring copy to serialization time. Adds From implementations for: - Vector from &[f32] - Vectors from &[f32] - Vectors from HashMap<String, &[f32]> Particularly useful for ETL pipelines processing embeddings from Arrow FixedSizeListArray or NumPy arrays with named vector fields.
4bac9a6 to
8627b2a
Compare
timvisee
approved these changes
Nov 12, 2025
Contributor
Author
|
@timvisee Seems you got to the rebase before I did. Many thanks |
timvisee
pushed a commit
that referenced
this pull request
Nov 12, 2025
Enables creating vectors from borrowed slices to optimize bulk uploads from contiguous memory sources (Arrow, NumPy). Reduces per-vector allocations by deferring copy to serialization time. Adds From implementations for: - Vector from &[f32] - Vectors from &[f32] - Vectors from HashMap<String, &[f32]> Particularly useful for ETL pipelines processing embeddings from Arrow FixedSizeListArray or NumPy arrays with named vector fields.
Member
|
@nazq Good news. We've just released a new version of this client, which means your changes are now available in the stable release: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Adds
From<&[f32]>implementations forVectorandVectorstypes to optimize bulk uploads from contiguous memory sources like Apache Arrow and NumPy.Performance Impact
Benchmarks show 4-17% throughput improvement with significantly lower variance for bulk uploads:
Key findings:
Benchmark setup: 384-dim vectors, 5K batch size, 5 iterations, localhost Qdrant 1.12, HNSW indexing disabled (
m=0)Changes
impl From<&[f32]> for Vector- Create dense vector from borrowed sliceimpl From<&[f32]> for Vectors- Create single vector from borrowed sliceimpl From<HashMap<String, &[f32]>> for Vectors- Create named vectors from borrowed slicesMotivation
When bulk uploading vectors from contiguous memory (Arrow
FixedSizeListArray, NumPy arrays), the current API requires per-vector allocations:With this change:
The copy still happens during protobuf serialization but with better cache locality and reduced allocator contention.
Use Cases
Benefits
&[f32]slices directlyTesting
Backward Compatibility
✅ Fully backward compatible - no breaking changes. Existing
From<Vec<f32>>implementations unchanged.Benchmark Methodology
The performance numbers above were generated using a real-world ETL scenario: