Stay organized with collections
Save and categorize content based on your preferences.
Create a parser
This guide explains how to create a parser. Parsers perform data
transformation and mapping. The exact transformation steps are defined on the
whistle script documentation
for Manufacturing Data Engine (MDE).
Create a parser
Parsers perform source-to-target mappings by means of a Whistle script. When
creating a parser, you must define three core elements:
The message class to whose message stream a parser subscribes (input).
Type version of the proto record stream that the parser emits (output).
The Whistle script to transform source messages from the specified
message class (input) to proto records of the defined type version (output).
The Whistle script is applied to every message in the source message class
stream, and it outputs proto records of a specific type version. We
recommend that you take time to model your source message classes so they
share a common semantic and schematic structure. Well-defined source message
classes help minimize complex conditional logic in parsers. See the section on
modelling source message classes
for more guidance.
You can create a parser using the configuration API or the Console:
SOURCE_MESSAGE_CLASS_NAME: the name of the Source Message Class to
whose message stream this parser subscribes.
TYPE_NAME: the name of type that is shared by the proto records
emitted by this parser.
TYPE_VERSION: the version of type that is shared by the proto
records emitted by this parser.
WHISTLE_SCRIPT: the Whistle script that defines the transformation.
Console
To create a new parser using the console, select the PARSERS
section on the top menu. A list of the available
parsers is displayed:
For each available parser the following information is shown in the
list:
Name:Name of the parser.
Message Class Name: Name of the message class the parser is
feeding from.
Type: Destination type that the parser emits.
Type Version: Destination type version that the parser emits.
Enabled: Status of the parser (enabled or disabled).
Actions: Available actions for the parser:
'View/Edit': Opens the edit menu for a given parser.
'Disable/Enable: Allows disabling an enabled parser or enabling
a disabled parser.
'Test parser': Opens a dedicated interface to test the parser using a
given JSON file.
'Delete': Deletes the parser from MDE.
To create a new parser click ADD NEW PARSER.
To open the edit section of a given parser click View/Edit
in the Actions icon. The side menu exposing all required parameters to
create a new parser is displayed on the right side of the screen:
To define the new parser the following parameters must be supplied:
Name: Name of the parser. Can't be edited once created.
Message Class: Name of the Message Class the parser feeds
from. Can't be modified once the parser is created. Select the Message
Class from the list of available Message Classes.
Parser Code: Whistle file that defines the mapping
associated to the parser. It can be modified at any time. Modifying the
Whistle code won't generate a new version of the parsers. Parsers are
indeed not versioned.
Type: Output Type emitted by the parser. It can't be
modified after the parser is created. Select the type from the list of
available types available. For new parsers the latest type version is
selected by default.
To create the parser modify the parser parameters and click CREATE
at the bottom. A confirmation message is displayed if the new parser
has been created successfully.
Test a parser
You can test a parser by providing a Whistle script and a sample input message:
REST
POST /configuration/v1/parsers:test
{"script":"SCRIPT","testMessage":TEST_MESSAGE}
Replace the following:
SCRIPT: Whistle script to be tested (formatted as string).
TEST_MESSAGE: Input test message (formatted as JSON object).
Console
You can test the parser directly from the Actions menu as well as in
the Edit Parser menu.
To test a parser click TEST. A testing screen opens:
Provide a sample JSON Message to test the parser.
Introduce the message in the Input box and click the RUN TEST
button.
If the transformation is successful, the output message is displayed in
the Result box.
If the transformation is unsuccessful, an error message is displayed
with the likely cause of the error.
Use a valid JSON file as test message that has a structure that can be
processed by the Whistle file of the parser. For example, this is a sample
of a numerical payload generated by Manufactueing Connect edge that can be
tested in the default-numeric-value-to-default-numeric-records parser:
{"datatype":"int","description":"","deviceID":"0619E715-D1B8-438F-A1AB-E4D65D27EE83","deviceName":"MicroLogix1100","metadata":{"location":"ES-BCN-GRA","manufacturer":"AllenBradley","model":"MicroLogix1100","os_revision":"Series B FRN 12.0","source":"mce"},"registerId":"0997D2ED-B2AB-434C-9754-C1A3C2E9C165","success":true,"tagName":"Manufactueing Connect edge_July_test_tag","timestamp":1691163012045,"value":8004}
Edit a parser
You can update an existing parser, including updating the Whistle script:
TYPE_NAME: Name of type that is shared by the proto records
emitted by this parser.
TYPE_VERSION: Version of type that is shared by the proto
records emitted by this parser.
WHISTLE_SCRIPT: Whistle script that defines the transformation.
Console
To open the edit section of a given parser click View/Edit
in the Actions icon:
The Edit Parser side menu opens exposing the required parameters to
configure the parser:
Name: Name of the parser. Can't be edited once created.
Message Class: Name of the Message Class the parser feeds
from. Can't be modified once the parser is created.
Parser Code: Whistle file that defines the mapping
associated to the parser. It can be modified at any time. Modifying the
Whistle code won't generate a new version of the parsers. Parsers are
not versioned.
Type: Output Type emitted by the parser. It can't be
modified after the parser is created.
Version: Output type version emitted by the parser. Can be
modified at any time for higher versions.
To edit a parser, modify the parser parameters and click SAVE.
A confirmation messages is displayed if the parser has
been created successfully.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-25 UTC."],[],[],null,["# Create a parser\n===============\n\nThis guide explains how to create a parser. Parsers perform data\ntransformation and mapping. The exact transformation steps are defined on the\n[*whistle* script documentation](/manufacturing-data-engine/docs/guides/parse/how-to-write-a-whiste-script)\nfor Manufacturing Data Engine (MDE).\n\nCreate a parser\n---------------\n\nParsers perform source-to-target mappings by means of a Whistle script. When\ncreating a parser, you must define three core elements:\n\n1. The *message class* to whose message stream a parser subscribes (input).\n2. *Type version* of the proto record stream that the parser emits (output).\n3. The Whistle script to transform source messages from the specified *message class* (input) to proto records of the defined *type version* (output).\n\nThe Whistle script is applied to every message in the source message class\nstream, and it outputs *proto records* of a specific *type* version. We\nrecommend that you take time to model your source message classes so they\nshare a common semantic and schematic structure. Well-defined source message\nclasses help minimize complex conditional logic in *parsers* . See the section on\n[modelling source message classes](/manufacturing-data-engine/docs/guides/ingest/how-to-create-a-source-message-class#modeling-a-source-message-class)\nfor more guidance.\n| **Important:** As of Manufacturing Data Engine (MDE) release 1.4.1, the Pub/Sub message headers are now stored in a reserved attribute called `mdeHeaders`. If your input messages contain that attribute, then an expression like `embeddedMetadata: $root[where ($.field !=...)]` will overwrite the reserved attribute and result in the loss of the Pub/Sub header data that was provided with the edge message. Make sure that you use the generic Whistle filter expression with caution so that it won't result in data loss.\n\nYou can create a parser using the configuration API or the Console: \n\n### REST\n\n`POST /configuration/v1/parsers` \n\n {\n \"name\": \"\u003cvar translate=\"no\"\u003ePARSER_NAME\u003c/var\u003e\",\n \"messageClassName\": \"\u003cvar translate=\"no\"\u003eSOURCE_MESSAGE_CLASS_NAME\u003c/var\u003e\",\n \"typeReference\": {\n \"name\": \"\u003cvar translate=\"no\"\u003eTYPE_NAME\u003c/var\u003e\",\n \"version\": \u003cvar translate=\"no\"\u003e\u003cspan class=\"devsite-syntax-err\"\u003eTYPE_VERSION\u003c/span\u003e\u003c/var\u003e\n },\n \"script\": \"\u003cvar translate=\"no\"\u003eWHISTLE_SCRIPT\u003c/var\u003e\"\n }\n\nReplace the following:\n\n- `PARSER_NAME`: the name of the parser.\n- `SOURCE_MESSAGE_CLASS_NAME`: the name of the *Source Message Class* to whose message stream this parser subscribes.\n- `TYPE_NAME`: the name of *type* that is shared by the proto records emitted by this parser.\n- `TYPE_VERSION`: the *version* of *type* that is shared by the proto records emitted by this parser.\n- `WHISTLE_SCRIPT`: the Whistle script that defines the transformation.\n\n### Console\n\n1. To create a new **parser** using the console, select the **PARSERS**\n section on the top menu. A list of the available\n parsers is displayed:\n\n2. For each available parser the following information is shown in the\n list:\n\n - **`Name`**:Name of the parser.\n - **`Message Class Name`**: Name of the message class the parser is feeding from.\n - **`Type`**: Destination type that the parser emits.\n - **`Type Version`**: Destination type version that the parser emits.\n - **`Enabled`**: Status of the parser (enabled or disabled).\n - **`Actions`** : Available actions for the parser:\n - **'View/Edit'**: Opens the edit menu for a given parser.\n - **'Disable/Enable**: Allows disabling an enabled parser or enabling a disabled parser.\n - **'Test parser'** : Opens a dedicated interface to test the parser using a given `JSON` file.\n - **'Delete'**: Deletes the parser from MDE.\n3. To create a new parser click **ADD NEW PARSER**.\n\n4. To open the edit section of a given parser click **View/Edit**\n in the **Actions** icon. The side menu exposing all required parameters to\n create a new parser is displayed on the right side of the screen:\n\n5. To define the new parser the following parameters must be supplied:\n\n - **`Name`**: Name of the parser. Can't be edited once created.\n - **`Message Class`** : Name of the **Message Class** the parser feeds from. Can't be modified once the parser is created. Select the Message Class from the list of available Message Classes.\n - **`Parser Code`** : **Whistle** file that defines the mapping associated to the parser. It can be modified at any time. Modifying the Whistle code won't generate a new version of the parsers. Parsers are indeed not versioned.\n - **`Type`** : Output **Type** emitted by the parser. It can't be modified after the parser is created. Select the type from the list of available types available. For new parsers the latest type version is selected by default.\n\nTo create the parser modify the parser parameters and click **CREATE**\nat the bottom. A confirmation message is displayed if the new parser\nhas been created successfully.\n\nTest a parser\n-------------\n\nYou can test a parser by providing a Whistle script and a sample input message: \n\n### REST\n\n`POST /configuration/v1/parsers:test` \n\n {\n \"script\": \"\u003cvar translate=\"no\"\u003eSCRIPT\u003c/var\u003e\",\n \"testMessage\": \u003cvar translate=\"no\"\u003e\u003cspan class=\"devsite-syntax-err\"\u003eTEST_MESSAGE\u003c/span\u003e\u003c/var\u003e\n }\n\nReplace the following:\n\n- `SCRIPT`: Whistle script to be tested (formatted as string).\n- `TEST_MESSAGE`: Input test message (formatted as JSON object).\n\n### Console\n\nYou can test the parser directly from the **Actions** menu as well as in\nthe **Edit Parser** menu.\n\n1. To test a parser click **TEST**. A testing screen opens:\n\n2. Provide a sample **JSON Message** to test the parser.\n\n3. Introduce the message in the **Input** box and click the **RUN TEST**\n button.\n\n- If the transformation is successful, the output message is displayed in the **Result** box.\n- If the transformation is unsuccessful, an error message is displayed with the likely cause of the error.\n\n1. Use a valid `JSON` file as test message that has a structure that can be processed by the Whistle file of the parser. For example, this is a sample of a numerical payload generated by Manufactueing Connect edge that can be tested in the **`default-numeric-value-to-default-numeric-records`** parser:\n\n {\n \"datatype\": \"int\",\n \"description\": \"\",\n \"deviceID\": \"0619E715-D1B8-438F-A1AB-E4D65D27EE83\",\n \"deviceName\": \"MicroLogix1100\",\n \"metadata\": {\n \"location\": \"ES-BCN-GRA\",\n \"manufacturer\": \"AllenBradley\",\n \"model\": \"MicroLogix1100\",\n \"os_revision\": \"Series B FRN 12.0\",\n \"source\": \"mce\"\n },\n \"registerId\": \"0997D2ED-B2AB-434C-9754-C1A3C2E9C165\",\n \"success\": true,\n \"tagName\": \"Manufactueing Connect edge_July_test_tag\",\n \"timestamp\": 1691163012045,\n \"value\": 8004\n }\n\nEdit a parser\n-------------\n\nYou can update an existing parser, including updating the Whistle script: \n\n### REST\n\n`PATCH /configuration/v1/parsers/`\u003cvar translate=\"no\"\u003eNAME\u003c/var\u003e \n\n {\n \"disabled\": \u003cvar translate=\"no\"\u003e\u003cspan class=\"devsite-syntax-err\"\u003eDISABLED\u003c/span\u003e\u003c/var\u003e,\n \"script\": \"\u003cvar translate=\"no\"\u003eSCRIPT\u003c/var\u003e\",\n \"typeReference\": {\n \"name\": \"\u003cvar translate=\"no\"\u003eTYPE_NAME\u003c/var\u003e\",\n \"version\": \u003cvar translate=\"no\"\u003e\u003cspan class=\"devsite-syntax-err\"\u003eTYPE_VERSION\u003c/span\u003e\u003c/var\u003e\n },\n \"script\": \"\u003cvar translate=\"no\"\u003eWHISTLE_SCRIPT\u003c/var\u003e\"\n }\n\nReplace the following:\n\n- `DISABLED`: State of the parser (true or false).\n- `NAME`: Name of the parser.\n- `TYPE_NAME`: Name of *type* that is shared by the proto records emitted by this parser.\n- `TYPE_VERSION`: *Version* of *type* that is shared by the proto records emitted by this parser.\n- `WHISTLE_SCRIPT`: Whistle script that defines the transformation.\n\n### Console\n\n1. To open the edit section of a given parser click **View/Edit**\n in the **Actions** icon:\n\n2. The **Edit Parser** side menu opens exposing the required parameters to\n configure the parser:\n\n- **`Name`**: Name of the parser. Can't be edited once created.\n- **`Message Class`** : Name of the **Message Class** the parser feeds from. Can't be modified once the parser is created.\n- **`Parser Code`** : **Whistle** file that defines the mapping associated to the parser. It can be modified at any time. Modifying the Whistle code won't generate a new version of the parsers. Parsers are not versioned.\n- **`Type`** : Output **Type** emitted by the parser. It can't be modified after the parser is created.\n- **`Version`** : Output type **version** emitted by the parser. Can be modified at any time for higher versions.\n\n1. To edit a parser, modify the parser parameters and click **SAVE**.\n2. A confirmation messages is displayed if the parser has been created successfully."]]