Get Started with the HTTP Ingest API🔗

This document defines the request and response semantics of the HTTP Ingest endpoint and provides guidance for building reliable senders against it. UI-driven configuration is covered in the Configure HTTP Ingest setup guide.

Introduction🔗

The Secureworks® Taegis™ XDR HTTP Ingest API is a push-based HTTPS interface for delivering security-relevant logs to Taegis in near real time. Each integration is provisioned with its own URL and integration key. The customer's own systems, or a third-party system configured with the URL and key, initiate every request to ingest data. XDR does not initiate connections to, or pull data from, customer systems through this integration.

HTTP Ingest is a transport mechanism. Some optimized integrations use HTTP Ingest internally to deliver data from supported third-party sources, in which case the platform applies source-specific normalization and detections to the delivered data. When HTTP Ingest is used directly to deliver data from a data source that is not part of an existing optimized integration, it functions as a custom integration. Transport into Taegis is guaranteed for any sender that posts well-formed requests, but downstream outcomes—such as normalization into a typed schema, search relevance, and source-specific detections—depend on whether the source is recognized. See Downstream Processing for details.

Audience🔗

This reference is intended for engineers building or operating systems that deliver telemetry to XDR over HTTPS, including custom in-house forwarders, webhook bridges, and integrations with third-party producers capable of issuing HTTPS POST requests.

Use Cases🔗

Some use cases of the HTTP Ingest API include:

Streaming security telemetry from applications or services capable of issuing HTTPS POST requests.
Forwarding logs from on-host agents, scripts, or scheduled jobs that read from an upstream API or local source.
Bridging third-party SaaS data sources that support outbound HTTP webhooks.

Conventions🔗

The key words MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL in this document are to be interpreted as described in BCP 14 (RFC 2119, RFC 8174)¹ when, and only when, they appear in all capitals.

Terminology🔗

Platform-wide terms used throughout this document, such as tenant, event, data source, ingest, normalization, schema, custom integrations, and optimized integrations, are defined in the Secureworks® Taegis™ Glossary and are used here in accordance with those definitions.

The following terms are specific to the HTTP Ingest API:

Integration: A configured HTTP Ingest instance, provisioned in the XDR UI, that is scoped to exactly one tenant and exposes a unique URL and integration key. The general platform term is defined in the glossary as integration.
Integration key: The opaque bearer credential issued at integration creation time and used to authenticate requests to the integration's URL.
log record: An individual unit of input within an HTTP Ingest request body. For example, one newline-delimited line of text/plain or one JSON object in a JSON Lines body. After successful processing, a log record becomes one or more events in the platform.

Versioning🔗

The URL path component /v1/ identifies the current major version of the HTTP Ingest API. New optional request headers, additional supported content types, additional supported response codes for transient conditions, and additional supported data sources MAY be introduced into v1 over time without prior notice; senders SHOULD be implemented to ignore unrecognized optional response headers and to treat unrecognized response codes per the broader class semantics defined in RFC 9110² (for example, an unrecognized 5xx status SHOULD be treated as retryable).

Changes that alter or remove existing behavior—such as renaming or removing request headers, narrowing accepted content types, or changing the meaning of an existing response code—will be introduced under a new version path (for example, /v2/, etc.) rather than modified in place, so that existing senders continue to operate against v1.

Endpoint🔗

The HTTP Ingest endpoint URL is presented to the operator in the XDR UI at the time of integration creation. The host portion of the URL is specific to the customer's region. The path uniquely identifies the integration:

POST https://<regional-ingest-host>/http-endpoint/v1/<integration-id>

Operators SHOULD retrieve the exact URL from the XDR UI rather than constructing it manually.

All requests MUST be made over HTTPS. The transport layer is TLS (RFC 8446)³. Plaintext HTTP is not accepted.

Authentication🔗

The HTTP Ingest API authenticates requests using a bearer token presented in the Authorization HTTP header (RFC 9110, Section 11)². The integration key serves as the bearer token.

Clients SHOULD use the Bearer scheme as defined in RFC 6750⁴:

Authorization: Bearer <Integration Key>

Clients designed to interoperate with the Splunk HTTP Event Collector (HEC) MAY alternately present the credential using the Splunk scheme:

Authorization: Splunk <Integration Key>

Clients MUST NOT rely on any other authentication scheme.

Integration Key Lifecycle🔗

The integration key is generated once, at integration creation time, and is displayed exactly once in the XDR UI. Operators MUST capture and store the key at that time; the platform does not display or recover keys after the creation dialog is closed.
Integration keys do not expire automatically.
The platform does not currently support in-place rotation of an existing integration's key. To rotate, operators SHOULD create a new HTTP Ingest integration, migrate the sender to the new URL and Key, verify ingestion on the new integration, then delete the previous integration.
Integrations may be deleted at any time from the XDR UI. Requests presenting a deleted integration's key will be rejected with 401 Unauthorized or 403 Forbidden.

Credential Handling🔗

Integration keys are full bearer credentials and MUST be treated with the same operational care as any long-lived secret (for example, storage in a secret manager or platform keystore, redaction from logs, and restricted distribution).
Provision one integration per logical data source. A single integration's URL and key MAY be shared across multiple senders, but throughput, error rates, and health status are reported per integration in the XDR UI. Co-mingling multiple data sources behind a single integration aggregates these signals and makes it materially harder to detect, attribute, or remediate issues with any individual source. Per-source integrations also allow independent rotation and revocation of credentials.

Request Format🔗

Content Types🔗

The following request Content-Type values are supported:

text/plain: Line-delimited log records, one log record per newline-terminated line.
application/json: Either a single JSON object per request body, or one JSON object per newline-delimited line (JSON Lines / NDJSON). JSON content MUST conform to RFC 8259⁵.

Request bodies MUST be UTF-8 encoded for both content types. For application/json this requirement is also imposed by RFC 8259, Section 8.1⁵.

Content Encoding🔗

Request bodies MAY be sent uncompressed or compressed with gzip (RFC 1952)⁶. Compressed requests MUST declare the encoding via the Content-Encoding request header:

Content-Encoding: gzip

Other content encodings are not supported.

Record Framing🔗

Plain text: Log records are delimited by the line-feed character (\n). Records that do not terminate with a line-feed MAY be combined with the following record.
JSON: Clients SHOULD submit either one JSON object per request body, or one JSON object per line (JSON Lines / NDJSON; see https://jsonlines.org/⁷). Pretty-printed JSON spanning multiple lines is not recommended for line-delimited submissions, because intermediate line breaks will be treated as record boundaries.

Size Limits🔗

Limit	Value	Behavior
Recommended request body size	≈ 100 KB	Optimized for the lowest end-to-end ingest latency.
Maximum request body size	100 MB	Hard ceiling. Requests exceeding this size are rejected with `413 Payload Too Large`.
Recommended maximum individual log record size	≈ 500 KB	Individual records larger than this MAY be split or truncated by downstream processing.

Clients SHOULD batch many small log records into appropriately sized requests rather than emitting either a single oversized record or a high volume of very small requests.

The API does not enforce a separate cap on the number of log records per request; the byte-size limits above are the only binding constraint. A request that fits within the size limits is accepted regardless of how many log records it contains.

Submission Rate🔗

The platform applies rate protections to ensure stability for all tenants. Clients SHOULD tune senders for steady, evenly distributed traffic and avoid bursts that compress many minutes of data into a few seconds. Rate-limited requests are signalled with 429 Too Many Requests (RFC 9110, Section 15.5.29)².

Response Semantics🔗

Status Codes🔗

Status codes follow RFC 9110². Per-code semantics for this API are:

Status	Meaning	Recommended Client Action
`200 OK`	The request body was accepted for asynchronous processing.	Continue.
`400 Bad Request`	The request was malformed, empty, or otherwise unparseable.	MUST NOT retry without changes. Inspect the payload, headers, and encoding.
`401 Unauthorized`	The `Authorization` header was missing or its credential was not recognized.	MUST NOT retry. Verify the integration key and URL pair.
`403 Forbidden`	The credential was recognized but is no longer authorized (for example, the integration has been deleted).	MUST NOT retry. Generate a replacement integration if necessary.
`408 Request Timeout`	The request did not complete in time.	MAY be retried with backoff.
`413 Payload Too Large`	The request body exceeded the maximum accepted size.	MUST NOT retry. Split the payload into smaller requests.
`429 Too Many Requests`	The request was rate-limited by platform protections.	SHOULD be retried with backoff and reduced concurrency or batch size.
`500`, `502`, `503`, `504`	A transient service-side error occurred.	SHOULD be retried with backoff.

A 200 OK response confirms only that the request body has been accepted for ingest. Parsing, normalization, search indexing, and detection generation occur asynchronously and are not reflected in the HTTP response.

Retry Strategy🔗

Clients SHOULD implement an exponential-backoff-with-jitter retry policy for retryable responses (408, 429, 5xx), starting at a short initial interval (for example, one second) and capping at a few minutes.

The HTTP Ingest API provides at-least-once delivery semantics:

The API does not currently support client-supplied idempotency keys, and does not deduplicate log records across retried requests.
A log record that is delivered more than once will appear more than once in downstream search and detection results unless the source-specific parser performs its own deduplication.

Clients SHOULD persist undelivered batches locally so that extended outages do not result in data loss.

Tenant Scoping🔗

Each integration is bound to a single tenant, as determined by the integration that owns the presented integration key. To deliver data into multiple tenants, operators MUST provision one HTTP Ingest integration per tenant and route from the sender to the corresponding URL and key.

Downstream Processing🔗

A successful 200 OK confirms transport into Taegis. The depth of downstream processing depends on whether the submitted data source is supported by an optimized integration; HTTP Ingest can serve as the transport for either an optimized integration maintained by XDR or a standalone custom integration wired up directly by the customer.

Data sources that are supported by an optimized integration are processed by source-specific normalization and produce events mapped to one or more typed schemas. These events are eligible for source-specific detections out of the box.
Data sources that are not supported by an optimized integration are preserved using the generic schema. Log records stored using the generic schema are searchable and retained for the tenant's configured retention period but are not mapped to source-specific schemas and are not eligible for source-specific detections without additional configuration.
Even when the submitted data source is supported by an optimized integration, individual log records that cannot be parsed are preserved using the generic schema.

The current catalog of optimized integrations is maintained in the Capabilities at a Glance reference. To extend processing for a data source that is not currently supported by an optimized integration, operators MAY author Custom Parsers and Custom Detection Rules for the tenant.

Security Considerations🔗

Transport: All requests MUST be made over HTTPS (RFC 8446)³. Plaintext HTTP is not accepted.
Credential exposure: Integration keys are long-lived bearer credentials. They MUST NOT be embedded in client-side code that is distributed to untrusted parties, logged in plaintext, or transmitted over insecure channels.
Per-source isolation: Sharing a single integration's key across many independent senders increases the impact radius of a compromise. Where independent attribution, rotation, or revocation matters, operators SHOULD provision a separate integration per logical source.
Rotation hygiene: Because in-place key rotation is not supported, operators SHOULD treat the create-new / migrate / delete-old workflow as a routine operational procedure and SHOULD rotate keys proactively on a defined cadence and in response to any suspected compromise.
Revocation: Deleting an integration in the XDR UI is the supported mechanism for immediate key revocation; subsequent requests presenting the deleted integration's key are rejected at the authentication layer.

Alternative Push-Based Transports🔗

HTTP Ingest is one of several push-based transports for delivering data to XDR without granting XDR access to your environment. See Custom Transport Methods for the full list, including:

XDR Collector for syslog forwarding
File Upload API for batched file submission
S3 Ingest — Secureworks-Managed for delivery to a Secureworks-managed bucket
Azure Event Hubs and Azure Storage Accounts for Azure-native sources

Examples🔗

The following examples send the contents of a local file to a configured HTTP Ingest integration. Set the HTTP_INGEST_URL and HTTP_INGEST_KEY environment variables before running:

export HTTP_INGEST_URL="https://<regional-ingest-host>/http-endpoint/v1/<integration-id>"
export HTTP_INGEST_KEY="your_integration_key"

cURL🔗

Verify connectivity to the integration with a single literal record:

curl --request POST "$HTTP_INGEST_URL" \
     --header "Authorization: Bearer $HTTP_INGEST_KEY" \
     --header "Content-Type: text/plain" \
     --data "hello world"

A successful call returns 200 OK. The submitted record is searchable in the XDR UI shortly afterward.

Post a plain-text log file:

curl --request POST "$HTTP_INGEST_URL" \
     --header "Authorization: Bearer $HTTP_INGEST_KEY" \
     --header "Content-Type: text/plain" \
     --data-binary @logs.txt

Send a JSON payload compressed with gzip:

gzip --to-stdout payload.json | \
  curl --request POST "$HTTP_INGEST_URL" \
       --header "Authorization: Bearer $HTTP_INGEST_KEY" \
       --header "Content-Type: application/json" \
       --header "Content-Encoding: gzip" \
       --data-binary @-

Python🔗

This example demonstrates the recommended response-code handling and exponential-backoff-with-jitter retry strategy described in Response Semantics:

#!/usr/bin/env python3
"""Send a local file to HTTP Ingest with retry handling."""

import os
import random
import sys
import time

import requests

API_URL = os.environ["HTTP_INGEST_URL"]
API_KEY = os.environ["HTTP_INGEST_KEY"]

# Status codes the HTTP Ingest API contract considers retryable.
RETRYABLE_STATUS = {408, 429, 500, 502, 503, 504}


def post(body: bytes, content_type: str, max_attempts: int = 5) -> None:
    headers = {
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": content_type,
    }

    backoff_cap = 60.0  # seconds
    backoff = 1.0
    for attempt in range(1, max_attempts + 1):
        response = requests.post(API_URL, data=body, headers=headers, timeout=30)
        status = response.status_code

        if status == 200:
            return

        # Non-retryable per the documented contract: stop immediately.
        if status == 400:
            sys.exit("400 Bad Request — inspect payload, headers, and encoding.")
        if status == 401:
            sys.exit("401 Unauthorized — verify the integration key and URL.")
        if status == 403:
            sys.exit("403 Forbidden — Integration may have been deleted; provision a new one.")
        if status == 413:
            sys.exit("413 Payload Too Large — split the body into smaller batches.")

        # Retryable: sleep with full-jitter backoff, then try again.
        if status in RETRYABLE_STATUS and attempt < max_attempts:
            sleep_for = random.uniform(0, min(backoff, backoff_cap))
            time.sleep(sleep_for)
            backoff *= 2
            continue

        sys.exit(f"Unexpected status {status}: {response.text}")

    sys.exit(f"Failed after {max_attempts} attempts.")


if __name__ == "__main__":
    if len(sys.argv) != 2:
        sys.exit(f"usage: {sys.argv[0]} <filename>")

    with open(sys.argv[1], "rb") as fd:
        post(fd.read(), content_type="text/plain")

To submit JSON Lines (NDJSON) instead, set content_type="application/json" and ensure the file contains one JSON object per newline-delimited line.

Configure HTTP Ingest: UI-driven setup guide.
HTTP Ingest Transport Method Overview: High-level description and reference architecture.
Custom Transport Methods: Overview of all custom transports supported by XDR.
Custom Parsers: Extending normalization to additional data sources.
Custom Detection Rules : Authoring detections over ingested data.

References🔗

BCP 14 — Best Current Practice 14: RFC 2119, RFC 8174. https://www.rfc-editor.org/info/bcp14/. RFC 2119 — Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. https://www.rfc-editor.org/info/rfc2119. RFC 8174 — Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, May 2017. https://www.rfc-editor.org/info/rfc8174. ↩
RFC 9110 — Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke, Ed., "HTTP Semantics", STD 97, RFC 9110, June 2022. https://www.rfc-editor.org/info/rfc9110 ↩↩↩↩
RFC 8446 — Rescorla, E., "The Transport Layer Security (TLS) Protocol Version 1.3", RFC 8446, August 2018. https://www.rfc-editor.org/info/rfc8446 ↩↩
RFC 6750 — Jones, M. and D. Hardt, "The OAuth 2.0 Authorization Framework: Bearer Token Usage", RFC 6750, October 2012. https://www.rfc-editor.org/info/rfc6750 ↩
RFC 8259 — Bray, T., Ed., "The JavaScript Object Notation (JSON) Data Interchange Format", STD 90, RFC 8259, December 2017. https://www.rfc-editor.org/info/rfc8259 ↩↩
RFC 1952 — Deutsch, P., "GZIP file format specification version 4.3", RFC 1952, May 1996. https://www.rfc-editor.org/info/rfc1952 ↩
JSON Lines. https://jsonlines.org/ ↩