mmpstrucdata: RFC5424 structured data parsing module#

Module Name:

mmpstrucdata

Author:

Rainer Gerhards <rgerhards@adiscon.com>

Available since:

7.5.4

Purpose#

The mmpstrucdata parses the structured data of RFC5424 into the message json variable tree. The data parsed, if available, is stored under “jsonRoot!container!…”. By default, container is rfc5424-sd. Please note that only RFC5424 messages will be processed.

The difference of RFC5424 is in the message layout: the SYSLOG-MSG part only contains the structured-data part instead of the normal message part. Further down you can find a example of a structured-data part.

Configuration Parameters#

Note

Parameter names are case-insensitive; camelCase is recommended for readability.

Action Parameters#

Parameter

Summary

jsonRoot

Sets the JSON root container where parsed structured data is stored.

container

Sets the JSON object member name that receives parsed structured data.

maxStructuredDataSize

Sets the largest RFC5424 structured-data field, in bytes, that mmpstrucdata parses.

sd_name.lowercase

Controls whether structured data element names (SDIDs) are lowercased.

See Also#

Caveats/Known Bugs#

  • this module is currently experimental; feedback is appreciated

  • property names are treated case-insensitive in rsyslog. As such, RFC5424 names are treated case-insensitive as well. If such names only differ in case (what is not recommended anyways), problems will occur.

  • structured data with duplicate SD-IDs and SD-PARAMS is not properly processed

Examples#

Below you can find a structured data part of a random message which has three parameters.

[exampleSDID@32473 iut="3" eventSource="Application"eventID="1011"]

In this snippet, we parse the message and emit all json variable to a file with the message anonymized. Note that once mmpstrucdata has run, access to the original message is no longer possible (except if stored in user variables before anonymization).

module(load="mmpstrucdata") action(type="mmpstrucdata")
template(name="jsondump" type="string" string="%msg%: %$!%\\n")
action(type="omfile" file="/path/to/log" template="jsondump")

The same setup in YAML:

modules:
  - load: mmpstrucdata

actions:
  - type: mmpstrucdata

A more practical one:

Take this example message (inspired by RFC5424 sample;)):

<34>1 2003-10-11T22:14:15.003Z mymachine.example.com su - ID47 [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"][id@2 test="test"] BOM'su root' failed for lonvick on /dev/pts/8

We apply this configuration:

module(load="mmpstrucdata") action(type="mmpstrucdata")
template(name="sample2" type="string"
  string="ALL: %$!%\\nSD: %$!RFC5424-SD%\\nIUT:%$!rfc5424-sd!exampleSDID@32473!iut%\\nRAWMSG: %rawmsg%\\n\\n")
action(type="omfile" file="/path/to/log" template="sample2")

The structured-data container name can be changed while keeping the same JSON root:

action(type="mmpstrucdata" jsonRoot="$!structured-data" container="sd"
       maxStructuredDataSize="64k")

actions:
  - type: mmpstrucdata
    jsonRoot: "$!structured-data"
    container: sd
    maxStructuredDataSize: 64k

This will output:

ALL: { "rfc5424-sd": { "examplesdid@32473": { "iut": "3", "eventsource": "Application", "eventid": "1011" }, "id@2": { "test": "test" } } }
SD: { "examplesdid@32473": { "iut": "3", "eventsource": "Application", "eventid": "1011" }, "id@2": { "test": "test" } }
IUT:3
RAWMSG: <34>1 2003-10-11T22:14:15.003Z mymachine.example.com su - ID47 [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"][id@2 test="test"] BOM'su root' failed for lonvick on /dev/pts/8

As you can seem, you can address each of the individual items. Note that the case of the RFC5424 parameter names has been converted to lower case.

Support: rsyslog Assistant | GitHub Discussions | GitHub Issues: rsyslog source project

Contributing: Source & docs: rsyslog source project

© 2008–2026 Rainer Gerhards and others. Licensed under the Apache License 2.0.