CommunityToolkit.VectorData.Chroma 1.1.0-preview

CommunityToolkit.VectorData.Chroma

Chroma provider for Microsoft.Extensions.VectorData, by the .NET Community Toolkit.

Chroma is an open-source vector database for AI applications.

The provider is built on ChromaDotNet.Client, a .NET client for Chroma and Chroma Cloud.

Quick start

1. Run Chroma with Docker:

docker run -d --name chroma -p 8000:8000 chromadb/chroma

2. Install the NuGet package:

dotnet add package CommunityToolkit.VectorData.Chroma

For more information, see the Microsoft.Extensions.VectorData documentation.

Limitations

• Hybrid search needs BM25 indexes, which distributed Chroma, the one Chroma Cloud runs on, and the servers that declare sparse vector indexes have. For distributed Chroma installed elsewhere, like with the Helm chart of Chroma or an operator, set Distributed=true in the connection string, or WithDistributedChroma() in the options of the client. On a single Chroma server, a collection does not offer IKeywordHybridSearchable, and a hybrid search throws an InvalidOperationException that says why.
• On Chroma Cloud, a vector search returns at most 300 results, Skip included, the default quota of Chroma Cloud: beyond that, Chroma Cloud answers with a quota error.
• Chroma does not store empty lists, so an empty array or list comes back as null.
• A data property whose storage name is empty, or starts with #, $ or chroma:, throws a NotSupportedException when the collection object is created. Chroma rejects the first ones in a write, and drops the chroma: keys from every read, so such a property would always come back as null.
• Array and list properties, like the namespaces of the TextSearchStore of Semantic Kernel, need Chroma 1.5.0 or later.
• A collection created by another client without a space has the space l2. A search on it with a distance function of another space throws, except on Chroma 1.0.0 to 1.0.5: there the provider cannot see the space, and the search returns wrong scores without an error. Use Chroma 1.0.6 or later.
• On self-hosted Chroma 1.0.21 to 1.5.9, a vector search can miss a record after an upsert, while a get still finds it. It is KD-49 in the compatibility notes of ChromaDotNet.Client. The environment variable RAYON_NUM_THREADS=1 on the server makes it much rarer. In the client, ChromaUpsertStrategy.SkipUnchangedEmbeddings writes a record whose vector does not change without its vector, so that write does not touch the vector index.
• On self-hosted Chroma 1.5.9, while upserts of the same record run at the same time, a read can return a mix of two of them: the keys that one sets to null and the other writes. Once the writes end, the record is one of them.
• On Chroma 1.5.9, with the cosine space and vectors that are nearly parallel, a vector search over all the records can miss some of them, and pages can repeat records. It comes from the approximate index of Chroma; with the l2 space the same records are all found.
• On Chroma Cloud, a record has at most 32 metadata keys, and a null value counts as one. The provider writes every data property, null or not. A write with null values that would take a record beyond 32 keys throws an ArgumentException. With WithReadsBeyondKeyQuota(), the client reads such a record first and sends only the null values of the keys it has. Two writes of the same record at the same time can then mix.
• On Chroma Cloud, a metadata key has at most 36 bytes in UTF-8. With a data property whose storage name is longer, the collection is created, but every upsert throws a VectorStoreException with the quota error of Chroma Cloud.
• With two or more full-text properties, each has a BM25 index of its own, whose key is its storage name with _bm25. On Chroma Cloud, a storage name of more than 31 bytes makes that key longer than 36 bytes: the collection is created without that index, and a hybrid search on that property throws an InvalidOperationException that says why.
• On distributed Chroma, the one Chroma Cloud runs on, Contains on the full-text property throws an ArgumentException for a text of fewer than 3 characters: on Chroma Cloud, and elsewhere with Distributed=true in the connection string or WithDistributedChroma() in the options of the client. Once the server has indexed the records, such a filter would find no record, and its negation every record.

The connection string does not set WithUpsertStrategy and WithReadsBeyondKeyQuota(). Set them on the options of a ChromaClient created in code, and pass it to the vector store:

var options = new ChromaConfigurationOptions("http://localhost:8000")
    .WithUpsertStrategy(ChromaUpsertStrategy.SkipUnchangedEmbeddings);

builder.Services.AddChromaVectorStore(_ => new ChromaClient(options));