Benchmarks · November 2026

Tzivia benchmark results

How Tzivia compares with Mem0 and common baselines on the LoCoMo long-term memory benchmark, plus measured speed. All numbers were measured on the same server size Tzivia Cloud runs on (8 vCPU / 8 GB).

Accuracy

LoCoMo: 10 conversations, 1,540 questions

SystemAccuracyContext per question
Tzivia50.1%~490 tokens
Mem0 (open-source mem0ai)44.6%~330 tokens
Naive RAG (vector search over raw turns)35.7%~480 tokens
BM25 keyword search29.9%~480 tokens
Whole conversation in the prompt53.3%~26,400 tokens

Tzivia comes within about 3 points of handing the model the entire conversation while using about 50 times fewer tokens.

By question type

Where Tzivia wins and where it doesn't

Question typeTziviaNaive RAGBM25Whole conversation
Single-hop60.7%45.9%43.0%76.9%
Temporal (dates, timing)57.6%31.8%19.3%18.7%
Multi-hop21.3%17.0%10.3%33.7%
Open-domain16.7%14.6%7.3%19.8%

Tzivia stamps each note with resolved dates, so questions about when things happened score about 3 times higher than reading the raw conversation, which only says things like "last Tuesday". Multi-hop and open-domain questions are hard for every system here.

Head to head

Tzivia vs Mem0, per conversation

ConversationQuestionsTziviaMem0
115244.1%44.1%
28159.3%45.7%
315257.6%48.7%
419945.2%40.2%
517850.0%39.9%
612350.4%48.0%
715053.3%43.3%
819155.0%44.5%
915645.5%46.2%
1015845.6%48.7%
All1,54050.1%44.6%

Tzivia led on 7 conversations, tied 1 and trailed 2.

Cost

Saving memories

TziviaMem0
LLM calls to saveNoneOne or more per message
LLM cost to save$0About $0.20–0.25 per conversation
Time to save one conversationUnder 1 minute65–90 minutes
Speed

Large memories

Memory sizeStored and searchableThroughputRecall avg / p95
180,000 tokens (0.7 MB)52 s3,478 tokens/s37 / 51 ms
1,000,000 tokens (4 MB)272 s3,681 tokens/s50 / 65 ms

Everyday use by one person: recall about 38 ms, saving a chat message about 50 ms.

Method

How we tested, and the caveats

LoCoMo question categories 1–4 (the adversarial category 5 is excluded, as is common). gpt-4o-mini answered every question and an LLM judge marked each answer, with identical prompts for every system. Mem0 is the open-source mem0ai library run by us, with gpt-4o-mini for memory extraction, local Qdrant hybrid search and MiniLM embeddings — not Mem0's hosted platform. Mem0 publishes higher LoCoMo scores from its own setup; these results compare both systems on identical terms. LLM-judge scores vary by about 1–2 points between runs. The 1M-token speed corpus is the LoCoMo text plus shuffled copies of it.