Azure Event Hubs data source discovery
- Last UpdatedOct 08, 2024
- 4 minute read
When running a discovery against an adapter namespace defined within your Configure Data source, you can specify query parameters to narrow the scope of the discovery, targeting specific event hubs within the namespace. When the discovery completes, you can add the discovered items to your data selection configuration.
Azure Event Hubs query string
Azure Event Hubs query strings must include one or more EventHubName, along with a WaitTime that specifies how long the discovery runs.
EventHubNames=<EVENT_HUB_1>,<EVENT_HUB_2>;WaitTime=<DAYS>.<HOURS>:<MINUTES>:<SECONDS>
Within the discovery request JSON payload, enter the Azure Event Hubs query using the query parameter, as shown in the query examples below. For more information on using the query parameter during API requests to the api/v1/configuration/<componentId>/discoveries endpoint, see Discovery.
Query parameters
When including a query within your discovery, you can add parameters from the table that follows. Separate the EventHubNames and WaitTime parameters with a semicolon (;).
|
String item |
Required |
Description |
|---|---|---|
|
EventHubNames |
Required |
Specifies one or more event hub as query targets during discovery. Discovery looks for the specified event hubs within the namespace set in the Configure Data source using the EventHubNamespaceConnectionString parameter.
|
|
WaitTime |
Optional |
Specifies the duration of the discovery. If you omit the WaitTime parameter, discovery defaults to one minute: 0.00:01:00.
|
Query examples
The following section contains several examples of Azure Event Hub queries. When creating a discovery request, include the Azure Event Hub query within the request payload as shown in Discovery.
Example 1: Single event hub
This Azure Event Hubs query example discovers a single event hub.
{
"query": "EventHubNames=event-hub-1;WaitTime=0.00:00:30"
}
Example 2: Multiple event hubs
This Azure Event Hubs query example discovers multiple event hubs.
{
"query": "EventHubNames=event-hub-1,event-hub-2;WaitTime=0.00:00:30"
}
Example 3: No WaitTime
This Azure Event Hubs query example discovers multiple event hubs, but omits the WaitTime parameter. Without a specified WaitTime, the discovery defaults to a WaitTime of one minute.
{
"query": "EventHubNames=event-hub-1,event-hub-2"
}
Query results
After you submit a discovery request and the discovery completes, you can view its results and use them to create a Configure Data selection. To view results for your discovery and query, perform a GET request against the api/v1/configuration/componentId/discoveries/<discoveryId>/result endpoint. For more information, see Discovery.
The discovery results includes a data selection configuration for each stream it finds for an event hub. For example, the following discovery results include a configuration for two streams found for "eventHubName":"event-hub-1": "streamId": "event-hub-1.$.PumpTemperature" and "streamId": "event-hub-1.$.TimeStamp".
[
{
"eventHubName": "event-hub-1",
"deviceId": null,
"valueField": "$.PumpTemperature",
"IndexField": null,
"IndexFormat": null,
"dataType": "Single",
"selected": false,
"name": null,
"streamId": "event-hub-1.$.PumpTemperature",
"dataFilterId": null
},
{
"eventHubName": "event-hub-1",
"deviceId": null,
"valueField": "$.TimeStamp",
"IndexField": null,
"IndexFormat": null,
"dataType": "DateTime",
"selected": false,
"name": null,
"streamId": "event-hub-1.$.TimeStamp",
"dataFilterId": null
}
]
StreamId special character replacement
In query results for streamId, the text parser replaces any automatically generated special characters with a replacement character to make identifiers more meaningful. You can manually override these replacement characters by manually editing the data selection configuration.
-
For more information on character replacement, see Text parser.
-
For more information on overriding replacement characters within a streamId, see Configure Data selection.
Combining query results into a valid data selection configuration
After viewing the results of a discovery, you can manually combine them into a single valid data selection configuration. In the example below, the two data selection configurations from the example above are combined into single configuration. Notice that IndexField is updated to $.TimeStamp.
After creating a valid data selection configuration, you can activate it by performing a PUT request against the api/v1/configuration/<ComponentId>/DataSelection endpoint. For more information, see Configure Data selection.
[
{
"eventHubName": "event-hub-1",
"deviceId": null,
"valueField": "$.PumpTemperature",
"IndexField": "$.TimeStamp",
"IndexFormat": null,
"dataType": "Single",
"selected": true,
"name": null,
"streamId": "event-hub-1.$.PumpTemperature",
"dataFilterId": null
}
]
Additional considerations for query and autoselect
When you use the query parameter within a discovery request, AVEVA recommends setting the Discovery parameter to false for most use cases. AVEVA makes this recommendation because the data selection configuration that discovery returns includes only values and not indexes. Therefore, the configuration is not suitable for a production environment without manually editing its settings. "autoSelect": true should only be used for data sources with simple data structures, as it automatically applies the discovery results as the active Configure Data selection.