Source Handling
General Source Configurations
These configs are available for all sources.
| Key | Default | Type | Description |
| sourceType | NoSource | String | Set the type of the Source (available options: DefaultSource, CSVSource, SenseSource, MQTTSource, BinarySource, KafkaSource, OPCSource) |
| inputFormat | “JSON” | String | input data format, available types: JSON, CSV |
| sourceFrequency | 1 | uint32_t | Set the sampling frequency of the source |
| skipHeader | false | bool | Skip first line of the file |
| numberOfBuffersToProduce | 1 | uint32_t | Set the number of buffers to produce |
| numberOfTuplesToProducePerBuffer | 1 | uint32_t | Set the number of tuples to produce per buffer |
| physicalSourceName | default_physical | String | Set the physical name of the source |
| logicalSourceName | default_logical | String | Set the logical name of the source |
Default Source
A simple source with default data created inside NES, useful for testing. No further configurations needed.
SenseSource
| Key | Default | Type | Description |
| SenseSourceUdfs | “” | String | Set user defined function |
CSVSource
This source reads data from a csv file, numberOfBuffersToProduce defines how often the csv file is repeated.
| Key | Default | Type | Description |
| CSVSourceFilePath | “” | String | file path for source |
| CSVSourceSkipHeader | false | bool | Skip first line of the file |
BinarySource
Reads from a binary source file.
| Key | Default | Type | Description |
| BinarySourceFilePath | “” | String | file path for source |
MQTTSource
Connect to an MQTT broker and read data from there.
| Key | Default | Type | Description |
| MQTTSourceUrl | “” | String | url to connect to |
| MQTTSourceClientId | “” | String | client Id to connect to MQTT broker |
| MQTTSourceUserName | “” | String | user name |
| MQTTSourceTopic | “” | String | Topic to listen to |
| MQTTSourceQos | 2 | uint32_t | quality of service - needs to be at-least-once or exactly-once for persistent sessions (cleanSession=false) to work |
| MQTTSourceCleanSession | true | bool | cleanSession true = clean up session after client loses connection, false = keep data for client after connection loss (persistent session) → requires the correct qos(see qos) |
| MQTTSourceFlushIntervalMS | -1 | float | tuple Buffer flush interval in milliseconds |
KafkaSource
Connect to a kafka broker and read data form there.
| Key | Default | Type | Description |
| KafkaSourceUrl | “” | String | Set kafka broker string |
| KafkaSourceAutoCommit | 1 | uint32_t | auto commit, boolean value where 1 equals true, and 0 equals false |
| KafkaSourceGroupId | “” | String | Set groupId |
| KafkaSourceTopic | “” | String | Topic to listen to |
| KafkaSourceConnectionTimeout | 10 | uint32_t | connection time out for source |
OPCSource
Connect to an OPC server and read data from there.
| Key | Default | Type | Description |
| OPCSourceNamespaceIndex | | uint32_t | namespace index for node |
| OPCSourceNodeIdentifier | “” | String | node identifier |
| OPCSourceUserName | “” | String | user name |
| OPCSourcePassword | “” | String | password |
