Configure Sources

Source Handling

General Source Configurations

These configs are available for all sources.

Key Default Type Description
sourceType NoSource String Set the type of the Source (available options: DefaultSource, CSVSource, SenseSource, MQTTSource, BinarySource, KafkaSource, OPCSource)
inputFormat “JSON” String input data format, available types: JSON, CSV
sourceFrequency 1 uint32_t Set the sampling frequency of the source
skipHeader false bool Skip first line of the file
numberOfBuffersToProduce 1 uint32_t Set the number of buffers to produce
numberOfTuplesToProducePerBuffer 1 uint32_t Set the number of tuples to produce per buffer
physicalSourceName default_physical String Set the physical name of the source
logicalSourceName default_logical String Set the logical name of the source
Default Source

A simple source with default data created inside NES, useful for testing. No further configurations needed.

SenseSource
Key Default Type Description
SenseSourceUdfs “” String Set user defined function
CSVSource

This source reads data from a csv file, numberOfBuffersToProduce defines how often the csv file is repeated.

Key Default Type Description
CSVSourceFilePath “” String file path for source
CSVSourceSkipHeader false bool Skip first line of the file
BinarySource

Reads from a binary source file.

Key Default Type Description
BinarySourceFilePath “” String file path for source
MQTTSource

Connect to an MQTT broker and read data from there.

Key Default Type Description
MQTTSourceUrl “” String url to connect to
MQTTSourceClientId “” String client Id to connect to MQTT broker
MQTTSourceUserName “” String user name
MQTTSourceTopic “” String Topic to listen to
MQTTSourceQos 2 uint32_t quality of service - needs to be at-least-once or exactly-once for persistent sessions (cleanSession=false) to work
MQTTSourceCleanSession true bool cleanSession true = clean up session after client loses connection, false = keep data for client after connection loss (persistent session) → requires the correct qos(see qos)
MQTTSourceFlushIntervalMS -1 float tuple Buffer flush interval in milliseconds
KafkaSource

Connect to a kafka broker and read data form there.

Key Default Type Description
KafkaSourceUrl “” String Set kafka broker string
KafkaSourceAutoCommit 1 uint32_t auto commit, boolean value where 1 equals true, and 0 equals false
KafkaSourceGroupId “” String Set groupId
KafkaSourceTopic “” String Topic to listen to
KafkaSourceConnectionTimeout 10 uint32_t connection time out for source
OPCSource

Connect to an OPC server and read data from there.

Key Default Type Description
OPCSourceNamespaceIndex uint32_t namespace index for node
OPCSourceNodeIdentifier “” String node identifier
OPCSourceUserName “” String user name
OPCSourcePassword “” String password
 
sources-configurations.txt · Last modified: 2022/03/04 17:28 by 134.96.191.189
 
Recent changes RSS feed Creative Commons License Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki