Description
Privacy-Preserving AI Tracing adds operational visibility to five provider workloads without copying sensitive inputs or outputs into telemetry. The tracer opens request or job traces, carries the active handle through asynchronous work, and wraps LLM completion, embedding, reranking, vision description, and transcription providers. Generations record model identity, latency, typed usage units, estimated cost when available, counts, and redaction metadata appropriate to the workload.
Raw prompts, responses, documents, queries, image bytes, audio, and transcripts are intentionally excluded. If tracing credentials are absent, the factory returns the same no-op interface; if the telemetry vendor throws, the wrapper logs and swallows that failure while preserving the provider result or original provider error. Standalone job calls still receive a trace, and graceful shutdown can flush buffered events.
Selling Points
- Gives engineering and operations teams cost and latency visibility across five AI workload types without exporting customer content.
- Preserves application reliability because tracing failures never change a successful provider result.
- Keeps provider error taxonomy intact, avoiding telemetry wrappers that obscure the real failure.
- Works in open-source and local environments with an identical no-op interface when vendor keys are absent.
- Supports request-correlated and standalone background traces through one asynchronous context model.
User Story
An operator investigates an expensive AI workflow. The trace shows an LLM generation, a batch embedding call, and a rerank step with models, duration, token counts, document counts, and estimated cost. No prompt, document, or generated answer appears in the telemetry. When the tracing vendor briefly fails, the user workflow still completes. The operator walks away with useful performance evidence without a second copy of sensitive knowledge.
Extension Surface
This capability is extensible. packages/observability/src/tracing.ts defines local structural provider contracts such as traced LLM, embedder, reranker, vision, transcription, and LangFuse-like seams. In-repo provider adapters can satisfy those shapes and receive the wrapper without importing a shared provider implementation. It is not a public marketplace plugin axis, so extensible is the most-open proven rating.
Capabilities & Limits
- Wraps five AI provider shapes and records workload-appropriate usage, latency, cost, counts, and model metadata.
- Maintains active trace context across asynchronous calls and supports standalone traces.
- Degrades to no-op without credentials and contains telemetry-vendor failures.
- Deliberately cannot answer questions requiring raw prompt or response inspection.
- Cost is present only when the provider or wrapper supplies an estimation function.
Implementation Map
packages/observability/src/tracing.tspackages/observability/src/index.tspackages/observability/src/tracing.test.ts