Netflow Schema

Note

Schema docs show the fields available for normalization. For a schema field to be populated in XDR, its corresponding field defined in the parser must exist in the original data. Normalized data shows in the Normalized Data tab of events and is searchable in XDR only if the corresponding field exists in the original data. The Schema Library in Data Lake Search shows only searchable fields.

Normalized Field	Type	Parser Field	Description
resource_id	string	resourceId$	Full resource string identifying the record
tenant_id	string	tenantId$	The ID of the tenant that owns this specific to CTPX ID
sensor_type	string	sensorType$	Type of device that generated this event. Ex: redcloak
sensor_event_id	string	sensorEventId$	Event ID of original_data assigned by the sensor
sensor_tenant	string	sensorTenant$	A customer ID supplied by the application that originated the data. Ex: redloak-domain, ctp-client-id
sensor_id	string	sensorId$	An ID for the data supplied by the application that originated it. Ex: redcloak-agent-id
sensor_cpe	string	sensorCpe$	CPE of the platform producing the alert. Example: cpe:2.3:a:secureworks:redcloak:*:*:*:*:*:*:*
original_data	string	originalData$	Original, unadulterated data prior to any transformation.
event_time_usec	uint64	eventTimeUsec$	Event time in microseconds (µs)
ingest_time_usec	uint64	ingestTimeUsec$	Ingest time in microseconds (µs).
event_time_fidelity	TimeFidelity	eventTimeFidelity$	Specifies the original precision of the time used to populate event_time_usec
host_id	string	hostId$	Host ID -- uniquely identifies the host where the event originated. e.g. IPv(4/6) address; Device Mac Address
sensor_version	string	sensorVersion$	The agent version as string.
source_mac	string	sourceMac$	Source MAC Address in text canonical format
destination_mac	string	destinationMac$	Destination MAC Address in text canonical format
source_address	string	sourceAddress$	IP Address of the source
destination_address	string	destinationAddress$	IP Address of the destination
source_port	uint32	sourcePort$	Port of the source
destination_port	uint32	destinationPort$	Port of the destination
protocol	uint32	protocol$	Transfer protocol
tx_packet_count	uint64	txPacketCount$	Number of packets transferred
tx_byte_count	uint64	txByteCount$	Number of bytes transferred
rx_packet_count	uint64	rxPacketCount$	Number of packets received
rx_byte_count	uint64	rxByteCount$	Number of bytes received
direction	Netflow.Direction	direction$	Direction of the network traffic between the source and destination from the perspective of the sensor. Ex: INBOUND, OUTBOUND
start_timestamp_usec	uint64	startTimestampUsec$	microseconds of the flow start
end_timestamp_usec	uint64	endTimestampUsec$	microseconds of the flow end
src_ipblacklists	repeated string	srcIpblacklists$	Provides the names of blacklists matched by the source
dest_ipblacklists	repeated string	destIpblacklists$	Provides the names of blacklists matched by the destination
src_ipgeo_summary	GeoSummary	srcIpgeoSummary$	The geographic location of the source IP. Not writable from a Custom Parser — populated automatically by Taegis IP enrichment from source_address. See note below.
dest_ipgeo_summary	GeoSummary	destIpgeoSummary$	The geographic location of the destination IP. Not writable from a Custom Parser — populated automatically by Taegis IP enrichment from destination_address. See note below.
source_nat_address	string	sourceNatAddress$	If Network Address translation is done, what is the source IP used
destination_nat_address	string	destinationNatAddress$	If Network Address translation is done, what is the destination IP used
source_nat_port	uint32	sourceNatPort$	If Network Address translation is done, what is the source port used
destination_nat_port	uint32	destinationNatPort$	If Network Address translation is done, what is the destination port used
application_name	string	applicationName$	Application detected by Deep Packet Inspection engine such as PaloAlto's APP-ID
flow_action	Netflow.fw_action	flowAction$	Provides an enum to define what the firewall might have done to the flow, if free form text is needed, see connection_end_reason = 206; 0 - FW_UNKNOWN - Not used. 1 - FW_BLOCKED - Flow dropped (firewall blocked the TCP handshake or the UDP packet) 2 - FW_ALLOWED - Flow was not denied by the firewall. 3 - FW_RESET_CLIENT_TO_SERVER - Firewall sent a RST packet to the server. (TCP probe). 4 - FW_RESET_SERVER_TO_CLIENT - Firewall sent a RST packet to the client. (TCP probe). 5 - FW_RESET_BOTH - Firewall sent a RST packet to both client and server. (TCP probe). 6 - FW_ICMP_CLIENT - ICMP destination host unreachable sent to client. (UDP probe).
connection_end_reason	string	connectionEndReason$	Provides details on why the session ended if flow_action was not enough example: tcp-rst-from-client (Paloalto), TCP Reset - I (ASA)
community_id_hash	string	communityIdHash$	Allows us to relate netflow record to other records like IDS alerts: https://github.com/corelight/community-id-spec{: target="_blank"}
event_metadata	KeyValuePairsIndexed	eventMetadata$	event_metadata can be provided by the data source to add context, such firewall rule name or source interface.
processCorrelationID	ProcessCorrelationID	processCorrelationId$	ProcessCorrelationID of the process creating this netflow session
process_image_path	string	processImagePath$	Image patch of the process initiating this netflow

Geographic IP fields (src_ipgeo_summary, dest_ipgeo_summary) are not writable from Custom Parsers

The src_ipgeo_summary and dest_ipgeo_summary fields are populated by an automatic Taegis IP enrichment step that runs after Custom Parser normalization. Any value a Custom Parser writes to these fields will be replaced before the event reaches queryable storage.

To get geographic data populated, ensure your parser writes the source and destination IP addresses to source_address$ and destination_address$. Enrichment uses those fields to look up the geographic data and fill in src_ipgeo_summary / dest_ipgeo_summary automatically.

Enrichment only applies to publicly routable IP addresses; private and loopback addresses are skipped.

Netflow.Direction

Name	Number	Description
UNKNOWN	0	internal: unused but required for proto3
INBOUND	1	The flow is inbound to the source.
OUTBOUND	2	The flow is outbound to the source.
CLIENT_TO_SERVER	3	When reported by a firewall, the control will only know who is the initiator (client) of the connection and who is the receiver (server).
SERVER_TO_CLIENT	4	When reported by a firewall, the control will only know who is the initiator (client) of the connection and who is the receiver (server).

Netflow.fw_action

Did the firewall appliance allow or deny the traffic, if deny, how?

Name	Number	Description
FW_UNKNOWN	0	internal: unused but required for proto3
FW_BLOCKED	1	Flow dropped (firewall blocked the TCP handshake or the UDP packet)
FW_ALLOWED	2	Flow was not denied by the firewall.
FW_RESET_CLIENT_TO_SERVER	3	Firewall sent a RST packet to the server. (TCP probe)
FW_RESET_SERVER_TO_CLIENT	4	Firewall sent a RST packet to the client. (TCP probe)
FW_RESET_BOTH	5	Firewall sent a RST packet to both client and server. (TCP probe)
FW_ICMP_CLIENT	6	ICMP destination host unreachable sent to client. (UDP probe)