On this Page

MetricsHub
MetricsHub Community Connectors 1.0.22

Integration Testing

This page assumes your connector already works when run interactively — the CLI workflow, including --record and the emulation options used below, is covered in Run and Debug Locally[1].

To create a new integration test, follow these steps:

Reference: MetricsHub Recording and Emulation Guide[2]

1. Prepare IT Resources Structure

Create the connector-specific resources folder under src/it/resources/ e.g., src/it/resources/Linux/.

Under this folder, create the following subfolders:

Folder Description
config Contains metricshub.yaml used for replay
emulation Contains recorded protocol data and emulation files
expected Contains expected.json, the expected IT output
Important

You must name the connector-specific resources folder exactly as the connector identifier configured in the metricshub.yaml (e.g., for +Linux in metricshub.yaml create src/it/resources/Linux/).

2. Record and Organize Emulation Data

Use the MetricsHub CLI with the --record option to capture protocol exchanges (HTTP requests/responses, SSH commands, WMI queries, etc.):

metricshub <hostname> -t <type> --ssh-username userName --ssh-password userPassword -c +Linux --record

Recorded protocol files are generated in protocol-specific folders under the MetricsHub logs directory:

  • Linux: /opt/metricshub/logs/<protocol>/
  • Windows: C:\Program Files\MetricsHub\logs\<protocol>\

Each protocol folder contains an image.yaml index file plus response/data files.

Supported protocol folders include: http, snmp, ssh, wbem, jdbc, ipmi, jmx, wmi

Special Case for SNMP Connectors

For SNMP connectors, use the snmpcli command to record walk files:

snmpcli dev-01 --walk 1.3.6.1 --community public --version v1 --port 161 --timeout 60 > /opt/metricshub/logs/snmp/1.3.6.1.walk

You may generate multiple .walk files (e.g., for different OIDs or components).

Organize Emulation Files

Copy the recorded protocol folders (e.g., http, ssh, snmp, etc.) into the connector's src/it/resources/<MyConnectorId>/emulation folder, organizing them by protocol:

src/it/resources/<MyConnectorId>/emulation/
├── http/
│   ├── image.yaml
│   └── response files...
├── ssh/
│   ├── image.yaml
│   └── response files...
└── snmp/
    ├── 1.3.6.1.walk
    └── other .walk files...

3. Prepare the config Folder

Add a minimal metricshub.yaml config under src/it/resources/<MyConnectorId>/config (e.g. src/it/resources/Linux/config). Example for Linux:

otel:
  otel.exporter.otlp.metrics.protocol: noop
patchDirectory: src/main/connector
resources:
  localhost:
    attributes:
      host.name: localhost
      host.type: linux
    protocols:
      emulation:
        ssh:
          directory: src/it/resources/Linux/emulation/ssh
    connectors: [ +Linux ]

For protocol-specific emulation directories, configure each protocol under the emulation section:

  • http: for HTTP/HTTPS API calls
  • ssh: for SSH command execution
  • snmp: for SNMP queries
  • wbem: for WBEM/CIM operations
  • jdbc: for database connections
  • ipmi: for IPMI commands
  • jmx: for JMX queries
  • wmi: for WMI queries
Important

Make sure to set the patchDirectory to the location of your connector source code (e.g., src/main/connector) so that the connector code is used during the IT.

4. Generate the Expected Output

The ConnectorReplayIT class provides a helper method writeExpectedJson to generate the expected JSON output for your connector. Follow these steps:

  • Add a temporary test method in ConnectorReplayIT.java that calls writeExpectedJson with your connector identifier:
@Test
void generateExpectedMyConnector() throws Exception {
  writeExpectedJson("MyConnector");
}
  • Run that test via Maven:
mvn clean verify -Dtest=ConnectorReplayIT#generateExpectedMyConnector

This will generate expected-gen.json in src/it/resources/<MyConnectorId>/expected/.

  • Remove the temporary generation test method once expected.json has been updated.

  • Review the generated file to ensure it captures the correct telemetry data.

  • Rename the generated expected-gen.json to expected.json:

mv src/it/resources/<MyConnectorId>/expected/expected-gen.json src/it/resources/<MyConnectorId>/expected/expected.json
  • Remove dynamic attributes such as agent.host.name from the expected.json to avoid test failures due to environment differences.
Important

The generate-then-rename workflow above is for the initial creation of expected.json. When later modifying an existing connector, avoid regenerating the file with the writeExpectedJson helper. Instead, manually update the expected file to reflect the specific connector changes. This keeps the expected file stable and ensures it changes only for intentional connector behavior updates. It also allows reviewers to clearly identify and validate the exact expected changes, without noise introduced by the recording process.

5. Add the Connector to the IT Tests

In ConnectorReplayIT.java, add a new test method for your connector, replacing MyConnector with your connector identifier:

@Test
void testMyConnector() throws Exception {
	testConnectorReplay("MyConnector");
}

Each test method corresponds to a single connector to minimize merge conflicts and provide clear JUnit output per connector.

For connectors that have specific service criteria (e.g., Windows-only), add the appropriate condition annotation:

@Test
@EnabledOnOs(WINDOWS)
void testMyWindowsConnector() throws Exception {
	testConnectorReplay("MyWindowsConnector");
}

6. Debugging and Troubleshooting

Tip

To debug the integration test, you can run the ConnectorReplayIT class in debug mode from your IDE. If you want to generate log files during the test execution, set the loggerLevel system property to debug, and specify a log directory using outputDirectory:

otel:
  otel.exporter.otlp.metrics.protocol: noop
patchDirectory: src/main/connector
loggerLevel: debug            # Available logger levels: trace, debug, info, warn, error
outputDirectory: src/it/logs  # Directory where logs will be saved
resources:
  localhost:
    attributes:
      host.name: localhost
      host.type: linux
    protocols:
      emulation:
        ssh:
          directory: src/it/resources/Linux/emulation/ssh
    connectors: [ +Linux ]
Note

Depending on the connector behavior and how MetricsHub executes requests in parallel, you may end up with a recording where the same request is executed twice but returns different results — for example, a temperature value changing between the first and second execution. In such cases, the expected output may randomly differ from one execution to another because requests are not executed in the exact same order that they were recorded. If this is the case for now, keep only unique requests in the image. We plan to enhance the framework to add an identifier (optional) to the recorded requests in order to be able to distinguish them during the replay and match them with the expected output, even if they are executed in a different order or multiple times.

Note

Another scenario can also happen due to race conditions: a connector may reference sources that are only available during the second collection cycle. Since MetricsHub runs monitor jobs in parallel, a referenced source might not yet be available during the very first collection. In that case, the integration test may fail even if the expected output was generated successfully at a given execution time T. When this happens, the connector design itself should really be reconsidered to avoid such behavior. A source being unavailable for another dependent source is not considered normal, even during the first collection cycle.

integration testing replay tests emulation connector validation metricshub community connector hardware system
Links:
  • [1] run-and-debug.html
  • [2] https://github.com/MetricsHub/metricshub-community/blob/main/EMULATION.md
Searching...
No results.