Ingest Data

Store memories, conversations, and documents into Hindsight.

Prerequisites

Make sure you've completed the Quick Start to install the client and start the server.

Store a Single Memory

Python

from hindsight_client import Hindsight

client = Hindsight(base_url="http://localhost:8888")

client.retain(
    bank_id="my-bank",
    content="Alice works at Google as a software engineer"
)

Node.js

import { HindsightClient } from '@vectorize-io/hindsight-client';

const client = new HindsightClient({ baseUrl: 'http://localhost:8888' });

await client.retain('my-bank', 'Alice works at Google as a software engineer');

CLI

hindsight memory put my-bank "Alice works at Google as a software engineer"

Store with Context and Date

Add context and event dates for better retrieval:

Python

client.retain(
    bank_id="my-bank",
    content="Alice got promoted to senior engineer",
    context="career update",
    timestamp="2024-03-15T10:00:00Z"
)

Node.js

await client.retain('my-bank', 'Alice got promoted to senior engineer', {
    context: 'career update',
    timestamp: '2024-03-15T10:00:00Z'
});

CLI

hindsight memory put my-bank "Alice got promoted" \
    --context "career update" \
    --event-date "2024-03-15"

The timestamp enables temporal queries like "What happened last spring?"

Batch Ingestion

Store multiple memories in a single request:

Python

client.retain_batch(
    bank_id="my-bank",
    items=[
        {"content": "Alice works at Google", "context": "career"},
        {"content": "Bob is a data scientist at Meta", "context": "career"},
        {"content": "Alice and Bob are friends", "context": "relationship"}
    ],
    document_id="conversation_001"
)

Node.js

await client.retainBatch('my-bank', [
    { content: 'Alice works at Google', context: 'career' },
    { content: 'Bob is a data scientist at Meta', context: 'career' },
    { content: 'Alice and Bob are friends', context: 'relationship' }
], { documentId: 'conversation_001' });

The document_id groups related memories for later management.

Store from Files

CLI

# Single file
hindsight memory put-files my-bank document.txt

# Multiple files
hindsight memory put-files my-bank doc1.txt doc2.md notes.txt

# With document ID
hindsight memory put-files my-bank report.pdf --document-id "q4-report"

How Retain Works

Learn about fact extraction, entity resolution, and graph construction in the Retain Architecture guide.

Async Ingestion

For large batches, use async ingestion:

Python

# Start async ingestion
result = client.retain_batch(
    bank_id="my-bank",
    items=[...large batch...],
    document_id="large-doc",
    async_=True
)

# Result contains operation_id for tracking
print(result["operation_id"])

Node.js

// Start async ingestion
const result = await client.retainBatch('my-bank', largeItems, {
    documentId: 'large-doc',
    async: true
});

console.log(result.operation_id);

Best Practices

Do	Don't
Include context for better retrieval	Store raw unstructured dumps
Use document_id to group related content	Mix unrelated content in one batch
Add timestamp for temporal queries	Omit dates if time matters
Store conversations as they happen	Wait to batch everything