kafka-sink-plugin is a Kafka egress plugin for MTConnect Agents. The plugin streams data from the agent to the Apache Kafka broker server(s).
Please visit the Apache Kafka website for setup info.
Devices = _metaagent_.xml
Port = 7979
Adapters
{
Mazak
{
Host = demo.metalogi.io
Port = 7879
}
}
Sinks
{
KAFKA {
#
# Kafka broker list
#
KafkaBrokers = localhost:9093
# various publish topics
# topic for assets, default is none (disabled)
KafkaTopicDeviceAsset = MTConnect.NewDeviceAsset
# topic for probe info, default is none (disabled)
KafkaTopicDeviceProbe = MTConnect.NewDeviceProbe
# topic for item info, default is none (disabled)
KafkaTopicDeviceItemInfo = MTConnect.NewDeviceItemInfo
# topic for node tree, default is none (disabled)
KafkaTopicDeviceNodeTree = MTConnect.NewDeviceNodeTree
# topic for device data, default is MTConnect.DeviceData
KafkaTopicDeviceData = MTConnect.NewDeviceData
# batchSize = <# of records to be processed in a batch, default is 100>
batchSize = 200
# in demo mode, each data may only get updated every 20 seconds
# licenseKey = <your license key - default is demo mode if not provided>
# override logger config
logger_config {
output = file kafka-sink.log
level = info
}
}
}
KafkaBrokers : One or more Kafka Brokers can be specified as a list of <host>:<port #> and separated by a comma.KafkaTopicDeviceProbe : The topic name on which the metadata of the Agent's machine is published, default is None (disabled).KafkaTopicDeviceNodeTree : The topic name for publishing the unified data selection tree constructed from the metadata, default is None (disabled).KafkaTopicDeviceItemInfo : The topic name for publishing the type info for each data item, default is None (disabled).KafkaTopicDeviceAsset : The topic name for publishing the asset info in the Agent's data stream, default is None (disabled).KafkaTopicDeviceData : The topic name for publishing the Samples, Events, and Condition real-time data, default is MTConnect.DeviceData.batchSize : Specify how many data items are to be batched for each publication, default is 100.
C:\Program Files\MetaAgent\examples\kafka-sink-plugin>agent run
MTConnect Agent Version 2.2.0.592 - built on Mon Oct 9 22:38:46 2023
[2023-10-29 05:09:23.664879] [0x000007b8] [trace] Found file 'agent.cfg' in path "C:\\Program Files\\MetaAgent\\examples\\kafka-sink-plugin"
[2023-10-29 05:09:23.664879] [0x000007b8] [info] Loading configuration from: "C:\\Program Files\\MetaAgent\\examples\\kafka-sink-plugin\\agent.cfg"
Loading configuration from:"C:\\Program Files\\MetaAgent\\examples\\kafka-sink-plugin\\agent.cfg"