How Tzivia compares with Mem0 and common baselines on the LoCoMo long-term memory benchmark, plus measured speed. All numbers were measured on the same server size Tzivia Cloud runs on (8 vCPU / 8 GB).
| System | Accuracy | Context per question |
|---|---|---|
| Tzivia | 50.1% | ~490 tokens |
| Mem0 (open-source mem0ai) | 44.6% | ~330 tokens |
| Naive RAG (vector search over raw turns) | 35.7% | ~480 tokens |
| BM25 keyword search | 29.9% | ~480 tokens |
| Whole conversation in the prompt | 53.3% | ~26,400 tokens |
Tzivia comes within about 3 points of handing the model the entire conversation while using about 50 times fewer tokens.
| Question type | Tzivia | Naive RAG | BM25 | Whole conversation |
|---|---|---|---|---|
| Single-hop | 60.7% | 45.9% | 43.0% | 76.9% |
| Temporal (dates, timing) | 57.6% | 31.8% | 19.3% | 18.7% |
| Multi-hop | 21.3% | 17.0% | 10.3% | 33.7% |
| Open-domain | 16.7% | 14.6% | 7.3% | 19.8% |
Tzivia stamps each note with resolved dates, so questions about when things happened score about 3 times higher than reading the raw conversation, which only says things like "last Tuesday". Multi-hop and open-domain questions are hard for every system here.
| Conversation | Questions | Tzivia | Mem0 |
|---|---|---|---|
| 1 | 152 | 44.1% | 44.1% |
| 2 | 81 | 59.3% | 45.7% |
| 3 | 152 | 57.6% | 48.7% |
| 4 | 199 | 45.2% | 40.2% |
| 5 | 178 | 50.0% | 39.9% |
| 6 | 123 | 50.4% | 48.0% |
| 7 | 150 | 53.3% | 43.3% |
| 8 | 191 | 55.0% | 44.5% |
| 9 | 156 | 45.5% | 46.2% |
| 10 | 158 | 45.6% | 48.7% |
| All | 1,540 | 50.1% | 44.6% |
Tzivia led on 7 conversations, tied 1 and trailed 2.
| Tzivia | Mem0 | |
|---|---|---|
| LLM calls to save | None | One or more per message |
| LLM cost to save | $0 | About $0.20–0.25 per conversation |
| Time to save one conversation | Under 1 minute | 65–90 minutes |
| Memory size | Stored and searchable | Throughput | Recall avg / p95 |
|---|---|---|---|
| 180,000 tokens (0.7 MB) | 52 s | 3,478 tokens/s | 37 / 51 ms |
| 1,000,000 tokens (4 MB) | 272 s | 3,681 tokens/s | 50 / 65 ms |
Everyday use by one person: recall about 38 ms, saving a chat message about 50 ms.
LoCoMo question categories 1–4 (the adversarial category 5 is excluded, as is common). gpt-4o-mini answered every question and an LLM judge marked each answer, with identical prompts for every system. Mem0 is the open-source mem0ai library run by us, with gpt-4o-mini for memory extraction, local Qdrant hybrid search and MiniLM embeddings — not Mem0's hosted platform. Mem0 publishes higher LoCoMo scores from its own setup; these results compare both systems on identical terms. LLM-judge scores vary by about 1–2 points between runs. The 1M-token speed corpus is the LoCoMo text plus shuffled copies of it.