added

Observability over queue latency

In addition to automatically tracing E2E flows that include handling message queues such as Kafka or Amazon SQS - Helios now provides observability & monitoring over queue latency of the messages handled.

Queue latency is calculated at the instrumentation level and thus can be handled as all other span properties are, through the labels, alerts and notifications mechanism in Helios.

Alert to label dashboard indicates queue latency

From notification to the label dashboard in the Helios Sandbox with complete E2E context of the flow with the long queue latency

Also, each API that is essentially the service consuming a message - now includes the queue latency distribution widget as part of the API dashboard. Clicking on each bar takes you to the relevant traces with the corresponding queue latency time.

The queue latency distribution widget appears for each API involving consuming a message from a queue (and specifically, Kafka)

The queue latency distribution widget appears for each API involving consuming a message from a queue

Read more about Kafka OpenTelemetry instrumentation and Amazon SQS OpenTelemetry instrumentation in Helios.

Queue latency calculation for Kafka is supported in the Helios OpenTelemetry SDK for Node.js version 1.0.84 or newer, and Python version 1.0.100 or newer. For SQS it's supported in the Helios OpenTelemetry SDK for Node.js version 1.0.89 or newer.