close
Skip to content

Add From<&[f32]> conversions for bulk upload optimization#244

Merged
timvisee merged 1 commit into
qdrant:devfrom
nazq:feat/vector-slice-conversions
Nov 12, 2025
Merged

Add From<&[f32]> conversions for bulk upload optimization#244
timvisee merged 1 commit into
qdrant:devfrom
nazq:feat/vector-slice-conversions

Conversation

@nazq
Copy link
Copy Markdown
Contributor

@nazq nazq commented Nov 5, 2025

Summary

Adds From<&[f32]> implementations for Vector and Vectors types to optimize bulk uploads from contiguous memory sources like Apache Arrow and NumPy.

Performance Impact

Benchmarks show 4-17% throughput improvement with significantly lower variance for bulk uploads:

Dataset Size Vec (current) Slice (optimized) Improvement Notes
10K points 93,321 pts/sec (±27K) 97,437 pts/sec (±10K) +4.4% More stable
50K points 75,906 pts/sec (±60K) 88,744 pts/sec (±4K) +16.9% Much more stable
100K points 73,579 pts/sec (±46K) 84,956 pts/sec (±16K) +15.5% Consistent gain

Key findings:

  • ✅ Throughput improvements scale with dataset size
  • ✅ Significantly lower variance (more predictable performance)
  • ✅ 99,999 fewer allocations per 100K points

Benchmark setup: 384-dim vectors, 5K batch size, 5 iterations, localhost Qdrant 1.12, HNSW indexing disabled (m=0)

Changes

  • impl From<&[f32]> for Vector - Create dense vector from borrowed slice
  • impl From<&[f32]> for Vectors - Create single vector from borrowed slice
  • impl From<HashMap<String, &[f32]>> for Vectors - Create named vectors from borrowed slices

Motivation

When bulk uploading vectors from contiguous memory (Arrow FixedSizeListArray, NumPy arrays), the current API requires per-vector allocations:

// Current: N allocations
for row_idx in 0..num_rows {
    let vector = slice[start..end].to_vec(); // Allocation per vector
    points.push(PointStruct::new(id, vector, payload));
}

With this change:

// Optimized: Copy deferred to serialization
for row_idx in 0..num_rows {
    let vector_slice = &slice[start..end]; // Just a slice reference
    points.push(PointStruct::new(id, vector_slice, payload));
}

The copy still happens during protobuf serialization but with better cache locality and reduced allocator contention.

Use Cases

  • ETL pipelines processing Parquet files with embedding columns
  • Bulk uploads with multiple named vector fields (text + image embeddings)
  • High-throughput scenarios with large dimensions (768+)
  • Memory-constrained environments where allocation overhead matters

Benefits

  1. Performance: 4-17% faster bulk uploads with lower variance
  2. Memory efficiency: Eliminates per-vector allocations (N → ~1 during serialization)
  3. API ergonomics: More idiomatic Rust - pass &[f32] slices directly
  4. Cache locality: Sequential memory access during serialization
  5. Reduced allocator pressure: Especially valuable under high concurrency

Testing

  • Added comprehensive unit tests for all three implementations
  • Backward compatibility verified - existing code unchanged
  • All existing tests pass
  • Performance validated with real-world Arrow → Qdrant benchmark

Backward Compatibility

✅ Fully backward compatible - no breaking changes. Existing From<Vec<f32>> implementations unchanged.

Benchmark Methodology

The performance numbers above were generated using a real-world ETL scenario:

  1. Generate Parquet file with FixedSizeList embeddings (384 dims)
  2. Read with Apache Arrow and extract contiguous buffer
  3. Upload to Qdrant with indexing disabled (bulk load best practice)
  4. Measure throughput across 5 iterations
  5. Compare Vec (current) vs Slice (optimized) approaches

@timvisee
Copy link
Copy Markdown
Member

Thank you for the detailed description. This makes a lot of sense.

I'll try to review this tomorrow. And I'll make sure to merge this as part of the upcoming release.

Copy link
Copy Markdown
Member

@timvisee timvisee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I'm happy to see the added tests as well.

Enables creating vectors from borrowed slices to optimize bulk uploads
from contiguous memory sources (Arrow, NumPy). Reduces per-vector
allocations by deferring copy to serialization time.

Adds From implementations for:
- Vector from &[f32]
- Vectors from &[f32]
- Vectors from HashMap<String, &[f32]>

Particularly useful for ETL pipelines processing embeddings from
Arrow FixedSizeListArray or NumPy arrays with named vector fields.
@timvisee timvisee force-pushed the feat/vector-slice-conversions branch from 4bac9a6 to 8627b2a Compare November 12, 2025 11:31
@timvisee timvisee changed the base branch from master to dev November 12, 2025 11:31
@timvisee timvisee merged commit cf041ef into qdrant:dev Nov 12, 2025
@nazq
Copy link
Copy Markdown
Contributor Author

nazq commented Nov 12, 2025

@timvisee Seems you got to the rebase before I did. Many thanks

timvisee pushed a commit that referenced this pull request Nov 12, 2025
Enables creating vectors from borrowed slices to optimize bulk uploads
from contiguous memory sources (Arrow, NumPy). Reduces per-vector
allocations by deferring copy to serialization time.

Adds From implementations for:
- Vector from &[f32]
- Vectors from &[f32]
- Vectors from HashMap<String, &[f32]>

Particularly useful for ETL pipelines processing embeddings from
Arrow FixedSizeListArray or NumPy arrays with named vector fields.
@timvisee
Copy link
Copy Markdown
Member

@nazq Good news. We've just released a new version of this client, which means your changes are now available in the stable release:

https://github.com/qdrant/rust-client/releases/tag/v1.16.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants