Observe one task across queues and tools

Logs are easier to use when every component emits structured events with the same identifiers. Include task_id, run_id, component, event type, timestamp, deployment version and relevant tool or dependency name. Add tool_call_id, action_id and provider request identifiers at their boundaries. Keep user content and credentials out of ordinary operational fields.

Distributed traces can connect request acceptance, queue wait, worker execution, memory retrieval, model calls and tool operations. OpenTelemetry provides common trace, metric and log conventions without tying the installation to one storage backend. Trace context must be carried in queued messages and resumed tasks. Otherwise the trace ends at dispatch and the slow part appears unrelated.

Do not rely on traces alone. Sampling may omit the task under investigation, and a task can remain queued without producing another span. Durable task events provide the complete business sequence. Traces explain timing inside an attempt. Metrics show whether a fault is affecting more than one task.

Choose metrics that reveal blocked work:

• task counts by current status and task type;
• age of the oldest ready task;
• queue-to-claim delay and active lease count;
• expired leases and repeated attempts;
• dependency errors by stable reason code;
• memory retrieval latency, empty results and timeouts;
• credential expiry windows and authentication failures;
• model and tool call latency, rate limits and uncertain outcomes.

Set alerts against service expectations that operators have agreed to support. Queue depth by itself can be misleading because a burst of quick jobs may be harmless. Oldest task age, stalled status transitions and a growing set of unresolved actions usually carry more context.

The operator view should reconstruct a task without searching several systems manually. Show its current status, last durable event, dependency state at that time, active or expired lease, pending approval and external action evidence. Link to restricted detail rather than copying full prompts, tool payloads or memory content into a broadly accessible dashboard.

Redaction needs tests. Provider errors and tool exceptions often include headers, file paths or request fragments that the success path never returns. Feed representative failure objects through the logging pipeline and confirm that secrets and private fields are removed before they reach log storage.