opcua-dataminer is an OPC UA DataConnect Gateway process. It streams data from any OPC UA services to data services for RethinkDB, InfluxDB, ElasticSearch, or Kafak.
# OPCUA Server information
OpcUAServer:
name: d1
url: opc.tcp://demo.metalogi.io:4840/
sampleInterval: 250
namespaceFilter: ns=3
#
# applicationUri: <application name, for some OPC UA server, they may require it to match the client's certificate and/or the server's application name>
# default is urn:open62541.server.application
#
# Additional optional connection info
# cert: <location of the certificate file. Format: DER>
# key: <location of the key file. Format: PEM>
# user: <user's id>
# password: <user's password>
#
# certificateCheck: [true, false]
# default is false
#
# messageSecurityMode: [None, sign, sign&encrypt]
# securityPolicy: [None, Basic128Rsa15, Basic256, Basic256Sha256, Aes128_Sha256_RsaOaep]
#
# messageSecurityMode and SecurityPolicy are only required if the server does not provide discovery service
#
#cert: ../certs/public.cert.der
#key: ../certs/key.pem
#user: demo
#password: test
#certifcateCheck: false
# =================== RethinkDB Defaults ==========================
#
# this section should be removed if RethinkDB is not required
#
RethinkDBServer: localhost:28015
#RethinkDBAuthKey: <your authKey if needed>
RethinkDBDatabase: testing # default is mtconnect
RethinkDBNodeTreeTable: sourceinfo # default is none (disabled)
RethinkDBDataTable: mtdata # default is mtdata
RethinkDBQueueSize: 20000 # default is 10000, max # of queue items.
RethinkDBBatchSize: 100 # default is 200, number of records to be processed together
# =================== InfluxDB Defaults ==========================
#
# this section should be removed if InfluxDB is not required
#
InfluxDBUrl: http://localhost:8086
InfluxDBOrganization: YourOrganization # InfluxDB Organization where the bucket is located
InfluxDBBucket: YourBucket # InfluxDB Bucket
#InfluxDBAuthKey: <your authKey if needed>
InfluxDBOrganization: testing
InfluxDBQueueSize: 20000 # default is 10000, max # of queue items.
InfluxDBBatchSize: 100 # default is 200, number of records to be processed together
# =================== ElasticSearch Defaults ==========================
#
# this section should be removed if ElasticSearch is not required
#
ElasticSearchUrl: http://localhost:9200
ElasticSearchUser: YourUserId
ElasticSearchPassword: YourPassword
ElasticSearchIndexName: mtconnect # default is mtconnect, Index name associated with the data.
ElasticSearchQueueSize: 20000 # default is 10000, max # of queue items.
ElasticSearchBatchSize: 100 # default is 200, number of records to be processed together
# =================== Kafka Defaults ==========================
#
# this section should be removed if Kafka is not required
#
KafkaBrokers: localhost:9092
KafkaTopicDeviceNodeTree: MTConnect.NewDeviceNodeTree # default is none (disabled)
KafkaTopicDeviceData: MTConnect.NewDeviceData # default is MTConnect.DeviceData
KafkaQueueSize: 20000 # default is 10000, max # of queue items.
KafkaBatchSize: 100 # default is 200, number of records to be processed together
# license
#licenseKey: <your Metalogi license key>
# serverity can be debug, info, warn, error or off
logging:
severity: info
messageOnly: false
OpcUAServer
name: The name of the device. This is used to form the data IDs in the data services.url : The OPC UA server's connection URLapplicationUrl : Specify the application URL, for some OPC UA servers, they may require it to match the client's certificate and the server's application URL, the default is urn:open62541.server.application.namespaceFilter: comma-separated list of node's id using regex matching. stream data only from this node or below.sampleInterval: the fastest rate at which the OPC UA server's monitored items should be accessed and evaluated (ms), the default is 0, indicating that it should use the fastest practical rate.user / password : the OPC UA server's login credential if required.key /cert : the location of the client's certificate and its key file.certificateCheck : Validate the OPC UA server's certificate, the default is false.messageSecurityMode : None, sign, or sign&encrypt.securityPolicy : None, Basic128Rsa15, Basic256, Basic256Sha256, or Aes128_Sha256_RsaOaep.RethinkDBServer : RethinkDB client driver connection info in <host name>:<port #>RethinkDBDatabase : Target Rethink DB database nameRethinkDBAuthKey : RethinkDB connection authorization keyRethinkDBProbeTable : Specify the table to store the metadata of the Agent's machine, the default is None (Disabled).RethinkDBNodeTreeTable : Specify the table to store the unified data selection tree from the metadata, the default is None (Disabled).RethinkDBItemInfoTable : Specify the table to store the type info for each data item, the default is None (Disabled).RethinkDBAssetInfoTable : Specify the table to store the asset info in the Agent's data stream, the default is None (Disabled)RethinkDBDataTable : Specify the table to store the Samples, Events, and Condition real-time data, the default is mtconnect.RethinkDBQueueSize : max # of queue items, the default is 10000.RethinkDBBatchSize : Specify # of records to be processed in a batch, the default is 200.InfluxDBUrl : The InfluxDB HTTP service's URLInfluxDBOrganization : Organization name - default is mtconnect.InfluxDBBucket : Bucket name - default is mtconnect.InfluxDBAuthKey : InfluxDB's authorization key info.InfluxDBQueueSize : max # of queue items, the default is 10000.InfluxDBBatchSize : Specify # of records to be processed in a batch, the default is 200.ElasticSearchUrl : Elasticsearch API service's URL.ElasticSearchUser/ElasticSearchPassword : Elasticsearch service login credential.ElasticSearchIndexName : The index name for the bulk insert data, default is mtconnect.ElasticSearchBatchSize : Specify # of records for each bulk insert, default is 200.KafkaBrokers : One or more Kafka Brokers can be specified as a list of <host>:<port #> and separated by a comma.KafkaTopicDeviceProbe : The topic name on which the metadata of the Agent's machine is published, default is None (disabled).KafkaTopicDeviceNodeTree : The topic name for publishing the unified data selection tree constructed from the metadata, default is None (disabled).KafkaTopicDeviceItemInfo : The topic name for publishing the type info for each data item, default is None (disabled).KafkaTopicDeviceAsset : The topic name for publishing the asset info in the Agent's data stream, default is None (disabled).KafkaTopicDeviceData : The topic name for publishing the Samples, Events, and Condition real-time data, default is MTConnect.DeviceData.KafkaQueueSize : max # of queue items, the default is 10000.KafkaBatchSize : Specify how many data items are to be batched for each publication, default is 200.
C:\Program Files\MetaAgent\examples\opcua-dataminer>opcua-dataminer opcua-dataminer.cfg
Metalogi MetaAgent - OPCUA Dataminer 2.2.0.600 - built on Wed Oct 11 22:28:27 2023
[2023-10-30T08:06:38] info Settings are loaded from opcua-dataminer.cfg
[2023-10-30T08:06:38] info Product RethinkDB Sink is verified
[2023-10-30T08:06:38] info Product InfluxDB Sink is verified
[2023-10-30T08:06:38] info Product ElasticSearch Sink is verified
[2023-10-30T08:06:38] info Product Kafka Sink is verified
[2023-10-30T08:06:38] info Product OPC UA <=> MTConnect is verified
[2023-10-30T08:06:38] warn AcceptAll Certificate Verification. Any remote certificate will be accepted.
[2023-10-30T08:06:42] info RethinkDB Connect Info: localhost:28015 database:testing
[2023-10-30T08:06:46] info InfluxDB connection info: http://localhost:8086 Organization: YourOrganization Bucket: YourBucket
[2023-10-30T08:06:50] info ElasticSearch connection info: http://localhost:9200 index: mtconnect
[2023-10-30T08:06:54] info Kakfa connection info: brokers localhost:9092
[2023-10-30T08:06:55] info Top branch selected: OKUMA [ns=3;s=OKUMA.123456]