Ingesting Your Data
Send messages, documents, tickets, and files into Honeycomb with the ingest and upload APIs — including batching, deduplication, updates, and deletion.
Honeycomb turns your organization's raw activity — messages, emails, tickets, meeting notes, CRM records, files — into durable, queryable memory. This page covers everything about getting data in: the ingest API field by field, batch ingestion, file uploads, what happens after Honeycomb accepts a record, and how to update or delete content you've already sent.
At a glance
- One endpoint for text:
POST /api/v1/ingestaccepts any text content plus metadata. It returns201in well under a second, and the record is searchable immediately. - Enrichment is asynchronous: entities, memories, facts, and graph relationships are extracted in the background after the
201— the response counts are always0. - Batching built in:
POST /api/v1/ingest/batchtakes up to 100 records per call. - Files too:
POST /api/v1/uploadextracts text from PDF, DOCX, TXT, Markdown, CSV, and JSON files up to 10 MB. externalRefis your idempotency key: re-sending the same reference deduplicates; sending changed content withonConflict: "supersede"versions it.- Deletion is real:
POST /api/v1/entries/tombstonehard-deletes a record, its version history, and everything derived from it.
All examples use $HONEYCOMB_URL for your API base URL and an hck_ API key in the Authorization header. You can create keys and find your base URL on the Admin screen of the console — see the Quickstart if you haven't set that up yet.
The lifecycle of a record
- Send. Your system POSTs content plus metadata.
- Accepted. Honeycomb chunks, embeds, and durably stores the record, assigns it to a Space, and returns
201. At this point the record is already retrievable by search. - Enriched. In the background, Honeycomb extracts entities, atomic memories, temporal facts, and relationships, and links them into your organization's knowledge graph.
- Queryable. Once enriched, the record contributes to Ask answers, recall, the graph, and Insights.
Note: The
201means accepted and stored, not fully processed. Enrichment typically completes within seconds to a few minutes depending on load. You can watch per-source ingest counts and the enrichment backlog on the Sources screen in the console.
Your first ingest
curl -X POST "$HONEYCOMB_URL/api/v1/ingest" \
-H "Authorization: Bearer hck_live_51Hx9mK2p" \
-H "Content-Type: application/json" \
-d '{
"content": "Renewal call with Acme Corp. Dana Reyes confirmed they will renew the Enterprise plan for 2027 at $180k, pending a security review of our SSO integration. She flagged that their new CISO, Marcus Webb, wants SOC 2 evidence by November 15.",
"metadata": {
"namespace": "sales",
"contentType": "meeting_notes",
"sourceAgent": "crm-sync",
"source": "salesforce",
"title": "Acme Corp renewal call — 2026-07-07",
"externalRef": "sfdc:meeting:00T5f000004QzXy",
"audience": ["[email protected]", "[email protected]"],
"validFrom": "2026-07-07T21:00:00Z",
"tags": ["renewal", "enterprise"]
}
}'A successful response:
{
"id": "e7c1a2f0-9b4d-4c6e-8a1f-2d3b5c7e9f01",
"chunksCreated": 1,
"entitiesExtracted": 0,
"factsRecorded": 0,
"memoriesExtracted": 0,
"extractionTier": 1,
"deduplicated": false
}Keep the id if you want to reference the record later. The zero counts are expected — extraction happens asynchronously, and Honeycomb will discover Acme Corp, Dana Reyes, Marcus Webb, the $180k renewal, and the November deadline in the background.
The ingest payload
The body has three parts: content, metadata (required), and options (optional).
content
A string, or an array of strings that Honeycomb joins with a blank line. Limits: 1,000,000 characters per string, up to 100 strings. Empty or whitespace-only content is rejected.
metadata
| Field | Required | What it does |
|---|---|---|
namespace | yes | A soft partition within your organization — typically a department or domain like sales, engineering, support. Usable as a search filter later. |
contentType | yes | The semantic type of the record. See the full list below. |
sourceAgent | yes | A stable identifier for the system or agent writing the record, e.g. crm-sync or support-bot. Recorded for attribution and auditing. |
source | no | The platform the record came from — slack, github, jira, hubspot, etc. Freeform. Helps Honeycomb weight and contextualize extraction. |
sourceChannel | no | The channel or context within the source, e.g. #engineering, acme/api-repo, Sales Board. Groups related records and drives channel-level access routing. |
sourceChannelPublic | no | Set true only when you know the channel is visible to your whole organization (a public Slack channel, a public repo). Absent or false keeps the record private. |
audience | no | The participants — the people who could already see this content at its source (email from/to/cc, channel members). Each participant is granted read access to the record. Omitted means no per-person restriction within the namespace. |
orgWide | no | Set true only for genuinely company-wide content: an all-hands doc, a public wiki page, a company-wide announcement. This is the explicit opt-in to org-wide visibility. |
spaceId | no | Route the record into a specific pre-existing Space by id — the strongest routing signal. Ignored (with a fallback to normal routing) if the Space doesn't exist in your organization. |
externalRef | no | Your external reference id — a ticket number, document URL, or message id. This is the deduplication and versioning key, unique within your organization. Strongly recommended for anything you might re-send. |
title | no | Human-readable title. Derived from the first line of content when omitted. |
tags | no | Array of categorization tags, usable as search filters. |
speakerRole | no | Who authored the entire content: user, agent, or system. Mark agent-generated text (assistant replies, generated digests) as agent — it will be stored and searchable but never mined for memories as if a person said it. |
validFrom | no | ISO timestamp for when this content happened or became true (defaults to now). Anchors memory dating and versioning order — set it when backfilling historical data. |
validUntil | no | ISO timestamp for when this content expires or stops being true. |
Content types. Pick the closest match — it tunes chunking, extraction, and access routing:
- Communication:
message,thread,email,announcement - Work tracking:
ticket,incident_report - Engineering:
commit,pull_request,code_review,changelog - Documents:
meeting_notes,report,product_spec,knowledge_article,policy,procedure,contract,onboarding_doc - Sales and CRM:
crm_record,competitive_intel,deal_note,customer_feedback - External:
news_article,social_post - Meta:
conversation_summary,calendar_event,other
Where your record lands: audience and Space hints
Every record is isolated to your organization and assigned to exactly one Space that controls who can retrieve it. Routing follows the strongest signal you provide:
| Priority | You send | The record lands in |
|---|---|---|
| 1 | spaceId | That exact Space. |
| 2 | audience participants | A private Space shared with exactly those participants. |
| 3 | orgWide: true | Your organization's shared, org-visible Space. |
| 4 | sourceChannel + sourceChannelPublic: true | An org-visible Space for that channel. |
| 5 | sourceChannel only | A private Space for that channel. |
| 6 | Nothing | A private quarantine Space owned by the ingesting agent. |
Warning: Routing fails closed. Ambiguous provenance never defaults to org-wide visibility — if you omit audience and Space hints, the record is stored but visible only to its ingesting agent until it's shared or re-routed. If teammates can't find data you ingested, missing hints are the first thing to check. See Spaces and Permissions.
Honeycomb also classifies each record's sensitivity at ingest (credentials, PII, confidential markers) and uses it to prevent sensitive content from ever being promoted org-wide. See Security.
options
| Field | Default | What it does |
|---|---|---|
extractionTier | 1 | Extraction depth. 0 = store and index only, no extraction (fastest, cheapest — good for reference material). 1 = standard background enrichment. 2 = full enrichment including deep graph extraction. |
chunkStrategy | "auto" | How content is split for indexing. auto detects the right strategy (conversation turns, email threads, pages, sections, semantic windows). Leave it on auto unless you have a specific reason not to. |
onConflict | "skip" | What to do when externalRef already exists with different content: skip keeps the existing record; supersede versions it. See below. |
deferReflection | — | Bulk-load hint: set true during large backfills to defer per-entity summarization to background maintenance and keep throughput high. |
entities | — | Bring your own entities: name + type pairs for already-structured data (CRM records, API responses). Skips LLM entity extraction for these. |
relationships | — | Bring your own relationships: from + to + type triples, e.g. a works_at edge between a person and a company. |
facts | — | Bring your own temporal facts: entity + fact + value, recorded as of validFrom, e.g. deal stage or ARR from your CRM. |
memories | — | Pre-extracted memories to seed alongside (not instead of) Honeycomb's own extraction. Each needs content; optional memoryType, eventDate, priority, persistence. |
Note: If you're forwarding native platform payloads (Slack events, email JSON, CRM webhooks) rather than plain text, Honeycomb can parse them directly — including thread structure and participant lists — via Connectors. Agents can also write memory through the
honeycomb_ingestandhoneycomb_uploadMCP tools; see Agents and MCP.
Batch ingest
POST /api/v1/ingest/batch accepts 1–100 items per call, each a complete ingest payload. Items are processed with bounded concurrency (concurrency: 1–20, default 5), and one item's failure never aborts the rest.
curl -X POST "$HONEYCOMB_URL/api/v1/ingest/batch" \
-H "Authorization: Bearer hck_live_51Hx9mK2p" \
-H "Content-Type: application/json" \
-d '{
"concurrency": 5,
"items": [
{
"content": "Ticket ACME-4312: SSO login loops back to the sign-in page for Okta users on Safari.",
"metadata": { "namespace": "support", "contentType": "ticket", "sourceAgent": "helpdesk-sync", "source": "zendesk", "externalRef": "zendesk:ticket:4312" }
},
{
"content": "Ticket ACME-4313: Export to CSV times out for reports over 50k rows.",
"metadata": { "namespace": "support", "contentType": "ticket", "sourceAgent": "helpdesk-sync", "source": "zendesk", "externalRef": "zendesk:ticket:4313" }
}
]
}'The response reports per-item outcomes, in order:
{
"total": 2,
"succeeded": 2,
"failed": 0,
"results": [
{ "index": 0, "success": true, "result": { "id": "..." } },
{ "index": 1, "success": true, "result": { "id": "..." } }
],
"durationMs": 1840
}Status is 201 when every item succeeded and 207 on partial failure — always check results for the failed indexes and their error messages. There is no batch-level transaction: successful items stay ingested even if others fail. For very large backfills, loop batches of 100 and set deferReflection: true on each item.
Uploading files
POST /api/v1/upload takes a multipart form, extracts the text, and pipes it through the same pipeline as /api/v1/ingest.
| Upload | Details |
|---|---|
| Form field for the file | file (required) |
| Size limit | 10 MB |
| Accepted types | PDF, DOCX, DOC (best effort), TXT and any plain-text type, Markdown, CSV, JSON |
| Required fields | namespace, contentType, sourceAgent |
| Optional fields | title (defaults to the filename), source (defaults to upload), sourceChannel, tags (comma-separated string), extractionTier |
curl -X POST "$HONEYCOMB_URL/api/v1/upload" \
-H "Authorization: Bearer hck_live_51Hx9mK2p" \
-F "[email protected]" \
-F "namespace=security" \
-F "contentType=report" \
-F "sourceAgent=docs-uploader" \
-F "title=Q3 Security Review" \
-F "tags=audit,soc2"Success returns 201 with the standard ingest result plus filename and textLength. Unsupported file types, empty extractions, and missing required fields return 400.
Warning: The upload endpoint doesn't accept
audience,orgWide, orspaceId— uploaded files always land in a private Space (the channel's, if you setsourceChannel, otherwise the uploader's own). Share or re-route them afterwards via Spaces and Permissions if a wider audience needs them.
Deduplication and updating content
externalRef is unique within your organization, and it's how Honeycomb decides whether an incoming record is new, a duplicate, or an update.
Duplicates are free. Re-sending an externalRef that already exists returns 201 immediately with "deduplicated": true and the existing record's id — no reprocessing, no duplicate data. This makes retries and webhook redeliveries safe by default.
Duplicates still widen access. If the same record reaches Honeycomb from more than one connection — two teammates' connectors each syncing a shared ticket under the same externalRef — the duplicate isn't wasted: each delivery is treated as evidence that its owner can see the record, so their read access is unioned onto the single stored record. You get one record per ticket, readable by everyone whose connection delivered it, instead of one private copy per connection. Access can only ever widen this way, and an explicit revocation is never undone by a redelivery.
Updates are versions. When the content behind a reference changes — a ticket gets a new status, a CRM record gets a new owner — send it again with onConflict: "supersede":
curl -X POST "$HONEYCOMB_URL/api/v1/ingest" \
-H "Authorization: Bearer hck_live_51Hx9mK2p" \
-H "Content-Type: application/json" \
-d '{
"content": "Ticket ACME-4312 resolved: SSO loop was caused by a stale relay-state cookie. Fix deployed in 4.19.2.",
"metadata": {
"namespace": "support",
"contentType": "ticket",
"sourceAgent": "helpdesk-sync",
"source": "zendesk",
"externalRef": "zendesk:ticket:4312",
"validFrom": "2026-07-07T09:30:00Z"
},
"options": { "onConflict": "supersede" }
}'What supersede does:
- The new version becomes current: search, recall, and Ask see the updated content from now on.
- The old version is closed, not erased: its knowledge is preserved with an end date, so time-scoped questions ("what was the status of this ticket last Tuesday?") still answer correctly.
- Identical content always dedup-skips — you can re-send unchanged records with
supersedeset and nothing churns. - Out-of-order snapshots are ignored: an incoming version whose
validFromis older than the stored one won't overwrite newer data, so backfills can't clobber the present.
Note: Without an
externalRefthere is no record-level deduplication — the same content sent twice creates two records. Give every record you might re-send a stable reference, and use a namespaced scheme likezendesk:ticket:4312so related records share a prefix. Lifecycle versioning is enabled per deployment; ifsupersedebehaves likeskipfor you, contact your Lua representative.
Deleting content
POST /api/v1/entries/tombstone permanently deletes a record by its externalRef — the current version, its entire version history, and everything derived from it: extracted memories, facts, graph relationships attested only by that record, and all search vectors.
curl -X POST "$HONEYCOMB_URL/api/v1/entries/tombstone" \
-H "Authorization: Bearer hck_live_51Hx9mK2p" \
-H "Content-Type: application/json" \
-d '{ "externalRef": "zendesk:ticket:4312" }'The response is a content-free receipt of what was removed:
{
"entriesDeleted": 2,
"memoriesDeleted": 7,
"factsDeleted": 3,
"edgesDeleted": 1,
"edgesDetached": 2,
"synthesisDeleted": 0
}Semantics worth knowing:
- It's a hard delete, designed for right-to-be-forgotten and data-hygiene workflows — not an archive. There is no undelete.
- Knowledge corroborated elsewhere survives: a graph relationship attested by other records is kept (that's
edgesDetached); one attested only by the deleted record is removed. - It's idempotent — tombstoning a reference that's already gone succeeds with zero counts.
- It doesn't block the future: if the source system sends the record again later, it's re-ingested normally.
- When you disconnect an entire data source, Honeycomb forgets that source's corpus in bulk — you don't need to tombstone record by record. See Connectors.
Like versioning, deletion is part of the lifecycle feature set enabled per deployment; a 404 from this endpoint means it isn't enabled for yours yet.
Next steps
- Connectors — let Honeycomb pull from Slack, email, GitHub, and your CRM instead of pushing yourself.
- Spaces and Permissions — how audience routing and sharing actually govern who sees what.
- Ask and Search and Recall — start querying what you've ingested.
- API Reference — the complete endpoint catalog.
Company Connections
Connect your company's CRM, Linear, and Slack from the Lua app so their records feed org-wide memory every member can query — and, optionally, so any agent can use them as a tool.
Connectors & Webhooks
Wire GitHub, Slack, CRMs, ticket trackers, and your own systems into Honeycomb with signed webhooks and authenticated push endpoints — then watch source health live.