Lua/Honeycomb

Ingesting Your Data

Send messages, documents, tickets, and files into Honeycomb with the ingest and upload APIs — including batching, deduplication, updates, and deletion.

Honeycomb turns your organization's raw activity — messages, emails, tickets, meeting notes, CRM records, files — into durable, queryable memory. This page covers everything about getting data in: the ingest API field by field, batch ingestion, file uploads, what happens after Honeycomb accepts a record, and how to update or delete content you've already sent.

At a glance

  • One endpoint for text: POST /api/v1/ingest accepts any text content plus metadata. It returns 201 in well under a second, and the record is searchable immediately.
  • Enrichment is asynchronous: entities, memories, facts, and graph relationships are extracted in the background after the 201 — the response counts are always 0.
  • Batching built in: POST /api/v1/ingest/batch takes up to 100 records per call.
  • Files too: POST /api/v1/upload extracts text from PDF, DOCX, TXT, Markdown, CSV, and JSON files up to 10 MB.
  • externalRef is your idempotency key: re-sending the same reference deduplicates; sending changed content with onConflict: "supersede" versions it.
  • Deletion is real: POST /api/v1/entries/tombstone hard-deletes a record, its version history, and everything derived from it.

All examples use $HONEYCOMB_URL for your API base URL and an hck_ API key in the Authorization header. You can create keys and find your base URL on the Admin screen of the console — see the Quickstart if you haven't set that up yet.

The lifecycle of a record

  1. Send. Your system POSTs content plus metadata.
  2. Accepted. Honeycomb chunks, embeds, and durably stores the record, assigns it to a Space, and returns 201. At this point the record is already retrievable by search.
  3. Enriched. In the background, Honeycomb extracts entities, atomic memories, temporal facts, and relationships, and links them into your organization's knowledge graph.
  4. Queryable. Once enriched, the record contributes to Ask answers, recall, the graph, and Insights.

Note: The 201 means accepted and stored, not fully processed. Enrichment typically completes within seconds to a few minutes depending on load. You can watch per-source ingest counts and the enrichment backlog on the Sources screen in the console.

Your first ingest

curl -X POST "$HONEYCOMB_URL/api/v1/ingest" \
  -H "Authorization: Bearer hck_live_51Hx9mK2p" \
  -H "Content-Type: application/json" \
  -d '{
    "content": "Renewal call with Acme Corp. Dana Reyes confirmed they will renew the Enterprise plan for 2027 at $180k, pending a security review of our SSO integration. She flagged that their new CISO, Marcus Webb, wants SOC 2 evidence by November 15.",
    "metadata": {
      "namespace": "sales",
      "contentType": "meeting_notes",
      "sourceAgent": "crm-sync",
      "source": "salesforce",
      "title": "Acme Corp renewal call — 2026-07-07",
      "externalRef": "sfdc:meeting:00T5f000004QzXy",
      "audience": ["[email protected]", "[email protected]"],
      "validFrom": "2026-07-07T21:00:00Z",
      "tags": ["renewal", "enterprise"]
    }
  }'

A successful response:

{
  "id": "e7c1a2f0-9b4d-4c6e-8a1f-2d3b5c7e9f01",
  "chunksCreated": 1,
  "entitiesExtracted": 0,
  "factsRecorded": 0,
  "memoriesExtracted": 0,
  "extractionTier": 1,
  "deduplicated": false
}

Keep the id if you want to reference the record later. The zero counts are expected — extraction happens asynchronously, and Honeycomb will discover Acme Corp, Dana Reyes, Marcus Webb, the $180k renewal, and the November deadline in the background.

The ingest payload

The body has three parts: content, metadata (required), and options (optional).

content

A string, or an array of strings that Honeycomb joins with a blank line. Limits: 1,000,000 characters per string, up to 100 strings. Empty or whitespace-only content is rejected.

metadata

FieldRequiredWhat it does
namespaceyesA soft partition within your organization — typically a department or domain like sales, engineering, support. Usable as a search filter later.
contentTypeyesThe semantic type of the record. See the full list below.
sourceAgentyesA stable identifier for the system or agent writing the record, e.g. crm-sync or support-bot. Recorded for attribution and auditing.
sourcenoThe platform the record came from — slack, github, jira, hubspot, etc. Freeform. Helps Honeycomb weight and contextualize extraction.
sourceChannelnoThe channel or context within the source, e.g. #engineering, acme/api-repo, Sales Board. Groups related records and drives channel-level access routing.
sourceChannelPublicnoSet true only when you know the channel is visible to your whole organization (a public Slack channel, a public repo). Absent or false keeps the record private.
audiencenoThe participants — the people who could already see this content at its source (email from/to/cc, channel members). Each participant is granted read access to the record. Omitted means no per-person restriction within the namespace.
orgWidenoSet true only for genuinely company-wide content: an all-hands doc, a public wiki page, a company-wide announcement. This is the explicit opt-in to org-wide visibility.
spaceIdnoRoute the record into a specific pre-existing Space by id — the strongest routing signal. Ignored (with a fallback to normal routing) if the Space doesn't exist in your organization.
externalRefnoYour external reference id — a ticket number, document URL, or message id. This is the deduplication and versioning key, unique within your organization. Strongly recommended for anything you might re-send.
titlenoHuman-readable title. Derived from the first line of content when omitted.
tagsnoArray of categorization tags, usable as search filters.
speakerRolenoWho authored the entire content: user, agent, or system. Mark agent-generated text (assistant replies, generated digests) as agent — it will be stored and searchable but never mined for memories as if a person said it.
validFromnoISO timestamp for when this content happened or became true (defaults to now). Anchors memory dating and versioning order — set it when backfilling historical data.
validUntilnoISO timestamp for when this content expires or stops being true.

Content types. Pick the closest match — it tunes chunking, extraction, and access routing:

  • Communication: message, thread, email, announcement
  • Work tracking: ticket, incident_report
  • Engineering: commit, pull_request, code_review, changelog
  • Documents: meeting_notes, report, product_spec, knowledge_article, policy, procedure, contract, onboarding_doc
  • Sales and CRM: crm_record, competitive_intel, deal_note, customer_feedback
  • External: news_article, social_post
  • Meta: conversation_summary, calendar_event, other

Where your record lands: audience and Space hints

Every record is isolated to your organization and assigned to exactly one Space that controls who can retrieve it. Routing follows the strongest signal you provide:

PriorityYou sendThe record lands in
1spaceIdThat exact Space.
2audience participantsA private Space shared with exactly those participants.
3orgWide: trueYour organization's shared, org-visible Space.
4sourceChannel + sourceChannelPublic: trueAn org-visible Space for that channel.
5sourceChannel onlyA private Space for that channel.
6NothingA private quarantine Space owned by the ingesting agent.

Warning: Routing fails closed. Ambiguous provenance never defaults to org-wide visibility — if you omit audience and Space hints, the record is stored but visible only to its ingesting agent until it's shared or re-routed. If teammates can't find data you ingested, missing hints are the first thing to check. See Spaces and Permissions.

Honeycomb also classifies each record's sensitivity at ingest (credentials, PII, confidential markers) and uses it to prevent sensitive content from ever being promoted org-wide. See Security.

options

FieldDefaultWhat it does
extractionTier1Extraction depth. 0 = store and index only, no extraction (fastest, cheapest — good for reference material). 1 = standard background enrichment. 2 = full enrichment including deep graph extraction.
chunkStrategy"auto"How content is split for indexing. auto detects the right strategy (conversation turns, email threads, pages, sections, semantic windows). Leave it on auto unless you have a specific reason not to.
onConflict"skip"What to do when externalRef already exists with different content: skip keeps the existing record; supersede versions it. See below.
deferReflectionBulk-load hint: set true during large backfills to defer per-entity summarization to background maintenance and keep throughput high.
entitiesBring your own entities: name + type pairs for already-structured data (CRM records, API responses). Skips LLM entity extraction for these.
relationshipsBring your own relationships: from + to + type triples, e.g. a works_at edge between a person and a company.
factsBring your own temporal facts: entity + fact + value, recorded as of validFrom, e.g. deal stage or ARR from your CRM.
memoriesPre-extracted memories to seed alongside (not instead of) Honeycomb's own extraction. Each needs content; optional memoryType, eventDate, priority, persistence.

Note: If you're forwarding native platform payloads (Slack events, email JSON, CRM webhooks) rather than plain text, Honeycomb can parse them directly — including thread structure and participant lists — via Connectors. Agents can also write memory through the honeycomb_ingest and honeycomb_upload MCP tools; see Agents and MCP.

Batch ingest

POST /api/v1/ingest/batch accepts 1–100 items per call, each a complete ingest payload. Items are processed with bounded concurrency (concurrency: 1–20, default 5), and one item's failure never aborts the rest.

curl -X POST "$HONEYCOMB_URL/api/v1/ingest/batch" \
  -H "Authorization: Bearer hck_live_51Hx9mK2p" \
  -H "Content-Type: application/json" \
  -d '{
    "concurrency": 5,
    "items": [
      {
        "content": "Ticket ACME-4312: SSO login loops back to the sign-in page for Okta users on Safari.",
        "metadata": { "namespace": "support", "contentType": "ticket", "sourceAgent": "helpdesk-sync", "source": "zendesk", "externalRef": "zendesk:ticket:4312" }
      },
      {
        "content": "Ticket ACME-4313: Export to CSV times out for reports over 50k rows.",
        "metadata": { "namespace": "support", "contentType": "ticket", "sourceAgent": "helpdesk-sync", "source": "zendesk", "externalRef": "zendesk:ticket:4313" }
      }
    ]
  }'

The response reports per-item outcomes, in order:

{
  "total": 2,
  "succeeded": 2,
  "failed": 0,
  "results": [
    { "index": 0, "success": true, "result": { "id": "..." } },
    { "index": 1, "success": true, "result": { "id": "..." } }
  ],
  "durationMs": 1840
}

Status is 201 when every item succeeded and 207 on partial failure — always check results for the failed indexes and their error messages. There is no batch-level transaction: successful items stay ingested even if others fail. For very large backfills, loop batches of 100 and set deferReflection: true on each item.

Uploading files

POST /api/v1/upload takes a multipart form, extracts the text, and pipes it through the same pipeline as /api/v1/ingest.

UploadDetails
Form field for the filefile (required)
Size limit10 MB
Accepted typesPDF, DOCX, DOC (best effort), TXT and any plain-text type, Markdown, CSV, JSON
Required fieldsnamespace, contentType, sourceAgent
Optional fieldstitle (defaults to the filename), source (defaults to upload), sourceChannel, tags (comma-separated string), extractionTier
curl -X POST "$HONEYCOMB_URL/api/v1/upload" \
  -H "Authorization: Bearer hck_live_51Hx9mK2p" \
  -F "[email protected]" \
  -F "namespace=security" \
  -F "contentType=report" \
  -F "sourceAgent=docs-uploader" \
  -F "title=Q3 Security Review" \
  -F "tags=audit,soc2"

Success returns 201 with the standard ingest result plus filename and textLength. Unsupported file types, empty extractions, and missing required fields return 400.

Warning: The upload endpoint doesn't accept audience, orgWide, or spaceId — uploaded files always land in a private Space (the channel's, if you set sourceChannel, otherwise the uploader's own). Share or re-route them afterwards via Spaces and Permissions if a wider audience needs them.

Deduplication and updating content

externalRef is unique within your organization, and it's how Honeycomb decides whether an incoming record is new, a duplicate, or an update.

Duplicates are free. Re-sending an externalRef that already exists returns 201 immediately with "deduplicated": true and the existing record's id — no reprocessing, no duplicate data. This makes retries and webhook redeliveries safe by default.

Duplicates still widen access. If the same record reaches Honeycomb from more than one connection — two teammates' connectors each syncing a shared ticket under the same externalRef — the duplicate isn't wasted: each delivery is treated as evidence that its owner can see the record, so their read access is unioned onto the single stored record. You get one record per ticket, readable by everyone whose connection delivered it, instead of one private copy per connection. Access can only ever widen this way, and an explicit revocation is never undone by a redelivery.

Updates are versions. When the content behind a reference changes — a ticket gets a new status, a CRM record gets a new owner — send it again with onConflict: "supersede":

curl -X POST "$HONEYCOMB_URL/api/v1/ingest" \
  -H "Authorization: Bearer hck_live_51Hx9mK2p" \
  -H "Content-Type: application/json" \
  -d '{
    "content": "Ticket ACME-4312 resolved: SSO loop was caused by a stale relay-state cookie. Fix deployed in 4.19.2.",
    "metadata": {
      "namespace": "support",
      "contentType": "ticket",
      "sourceAgent": "helpdesk-sync",
      "source": "zendesk",
      "externalRef": "zendesk:ticket:4312",
      "validFrom": "2026-07-07T09:30:00Z"
    },
    "options": { "onConflict": "supersede" }
  }'

What supersede does:

  • The new version becomes current: search, recall, and Ask see the updated content from now on.
  • The old version is closed, not erased: its knowledge is preserved with an end date, so time-scoped questions ("what was the status of this ticket last Tuesday?") still answer correctly.
  • Identical content always dedup-skips — you can re-send unchanged records with supersede set and nothing churns.
  • Out-of-order snapshots are ignored: an incoming version whose validFrom is older than the stored one won't overwrite newer data, so backfills can't clobber the present.

Note: Without an externalRef there is no record-level deduplication — the same content sent twice creates two records. Give every record you might re-send a stable reference, and use a namespaced scheme like zendesk:ticket:4312 so related records share a prefix. Lifecycle versioning is enabled per deployment; if supersede behaves like skip for you, contact your Lua representative.

Deleting content

POST /api/v1/entries/tombstone permanently deletes a record by its externalRef — the current version, its entire version history, and everything derived from it: extracted memories, facts, graph relationships attested only by that record, and all search vectors.

curl -X POST "$HONEYCOMB_URL/api/v1/entries/tombstone" \
  -H "Authorization: Bearer hck_live_51Hx9mK2p" \
  -H "Content-Type: application/json" \
  -d '{ "externalRef": "zendesk:ticket:4312" }'

The response is a content-free receipt of what was removed:

{
  "entriesDeleted": 2,
  "memoriesDeleted": 7,
  "factsDeleted": 3,
  "edgesDeleted": 1,
  "edgesDetached": 2,
  "synthesisDeleted": 0
}

Semantics worth knowing:

  • It's a hard delete, designed for right-to-be-forgotten and data-hygiene workflows — not an archive. There is no undelete.
  • Knowledge corroborated elsewhere survives: a graph relationship attested by other records is kept (that's edgesDetached); one attested only by the deleted record is removed.
  • It's idempotent — tombstoning a reference that's already gone succeeds with zero counts.
  • It doesn't block the future: if the source system sends the record again later, it's re-ingested normally.
  • When you disconnect an entire data source, Honeycomb forgets that source's corpus in bulk — you don't need to tombstone record by record. See Connectors.

Like versioning, deletion is part of the lifecycle feature set enabled per deployment; a 404 from this endpoint means it isn't enabled for yours yet.

Next steps