omazuredce batches JSON records and sends them to the Azure Monitor Logs Ingestion API by using a Data Collection Endpoint (DCE), Data Collection Rule (DCR), and Microsoft Entra client-credentials authentication.

omazuredce: Azure Monitor Logs Ingestion Output Module#

Module Name:	omazuredce
Author:	Jan Kängsepp
Available since:	v8.2604

Purpose#

This module provides native support for forwarding log data to the Azure Monitor Logs Ingestion API.

Each message rendered by the configured template must be a single JSON object. omazuredce collects those objects into a JSON array, requests an OAuth access token from Microsoft Entra ID, and posts the resulting batch to the configured Azure Data Collection Endpoint.

Notable Features#

Size-aware batching with a configurable max_batch_bytes limit
Timed flushing for partially filled batches via flush_timeout_ms
Automatic OAuth token acquisition for the https://monitor.azure.com/.default scope
Retry-friendly error handling that cooperates with rsyslog action suspension

Requirements#

To use omazuredce, you need the following:

libcurl support at build time
A valid Azure Data Collection Endpoint URL
A Data Collection Rule ID
A target stream or table name accepted by the DCR
A Microsoft Entra application with client_id, client_secret, and tenant_id values that can request tokens for https://monitor.azure.com/.default
A template that renders one valid JSON object per message, or the built-in StdJSONFmt default

The module is built only when ./configure is invoked with --enable-omazuredce=yes.

Configuration Parameters#

Note

Parameter names are case-insensitive; snake_case is used in the examples because the module’s parameter names contain underscores.

Note

This module supports action parameters only.

Action Parameters#

Parameter	Summary
template	Selects the rsyslog template used to render each message before it is added to the Azure ingestion batch.
client_id	Specifies the Microsoft Entra application client ID used for OAuth token requests.
client_secret	Supplies the client secret paired with `client_id` for OAuth token requests.
tenant_id	Sets the Microsoft Entra tenant used when requesting OAuth access tokens.
dce_url	Defines the Azure Data Collection Endpoint base URL used for batch submission.
dcr_id	Specifies the Azure Data Collection Rule immutable ID used in the ingestion URL.
table_name	Sets the stream or table name appended to the Azure ingestion request path.
max_batch_bytes	Limits the estimated total size of one Azure ingestion request, including the payload and HTTP overhead.
flush_timeout_ms	Controls how long a partially filled batch may stay idle before it is flushed.

Batching Behavior#

omazuredce builds one JSON array per worker and flushes it when one of the following happens:

Adding the next record would exceed max_batch_bytes
The action queue transaction ends and flush_timeout_ms is set to 0
The batch has been idle for at least flush_timeout_ms milliseconds

The internal size check is conservative. It includes both the JSON payload size and an estimate for HTTP headers before sending the request.

Error Handling#

The module obtains an OAuth access token before sending data. If Azure returns 401 Unauthorized for a batch request, the module refreshes the token and returns RS_RET_SUSPENDED. The failed batch is then retried by rsyslog’s central action engine according to the configured action backoff and retry settings. Other retryable HTTP failures follow the same RS_RET_SUSPENDED path.

If a rendered message is too large to fit into an empty batch under the current max_batch_bytes setting, the module logs an error and drops that record.

Example#

The following example renders each event as one JSON object and forwards it to Azure Monitor Logs Ingestion:

module(load="omazuredce")

template(name="tplAzureDce" type="list" option.jsonf="on") {
   property(outname="TimeGenerated" name="timereported" dateFormat="rfc3339" format="jsonf")
   property(outname="Host" name="hostname" format="jsonf")
   property(outname="AppName" name="app-name" format="jsonf")
   property(outname="Message" name="msg" format="jsonf")
}

action(
   type="omazuredce"
   template="tplAzureDce"
   client_id="<application-id>"
   client_secret="<client-secret>"
   tenant_id="<tenant-id>"
   dce_url="https://<dce-name>.<region>.ingest.monitor.azure.com"
   dcr_id="<dcr-id>"
   table_name="Custom-MyTable_CL"
   max_batch_bytes="1048576"
   flush_timeout_ms="2000"
)

Support: rsyslog Assistant | GitHub Discussions | GitHub Issues: rsyslog source project

Contributing: Source & docs: rsyslog source project