README
¶
xopotel - Golang connector between Open Telemetry and XOP (Cross Observability Platform)
Package xopotel provides gateways/connections between xop and OpenTelemetry.
SpanToLog
SpanToLog() allows xop to be used as a logger to add "Events" to an
existing OTEL span.
SeedModifier
SeedModifier allows using an OTEL Tracer as an xopbase.Logger
BaseLogger
BaseLogger() creates a xopbase.Logger that can be used as a
to gateway xop logs and spans into OTEL.
Compatibility
Data types
OTEL supports far fewer data types than xop. Mostly, xop types can be converted cleanly, but links are a special case: links can only be added to OTEL spans when the span is created. Since xop allows links to be made at any time, links will be added as ephemeral sub-spans. Distinct, Multiple, and Locked attributes will be ignored for links.
OTEL does not support unsigned ints. uint64 will be converted to a
string and smaller unsigned ints will convert to int64.
OTEL provides no way to record links (trace_id/span_id) except as part of the initial set of "Links" associated with a span. Those links are values only (no key, no name). Xop supports arbitrary numbers of links on a per-logline basis. There is no way to record those as links using the OTEL structures so if there is more than one of them, then they'll be recorded as strings. Most of the time, a log line won't have more than one trace reference.
For both events with links, and links as span metadata, an extra OTEL span will be created just to hold the link. The extra span will be marked with span.is-link-attribute:true for span metadata and span.is-link-event for events.
Documentation
¶
Overview ¶
Package xopotel provides a gateways between XOP and Open Telemetry.
There is a mismatch of features between Open Telemetry and XOP. Open Telemetry supports only a very limited set of attribute types. When gatewaying from XOP into Open Telemetry the richer set of types are almost always converted to string slices.
There are several integration points.
BaseLogger ¶
The BaseLogger() function returns a xopbase.Logger that can be used like any other base logger to configure XOP output. In this case, the XOP logs and traces will be output through the Open Telemtry system using the primary interfaces of TracerProvider, Tracer, Span, etc. There is a restriction though: to use this you MUST create the TracerProvider with the xopotel IDGenerator:
import ( "github.com/xoplog/xop-go/xopotel" sdktrace "go.opentelemetry.io/otel/sdk/trace" ) tracerProvider := NewTraceProvider(xopotel.IDGenerator(), sdktrace.WithBatcher(...))
This allows the TraceIDs and SpanIDs created by XOP to be used by Open Telemetry.
SeedModifier ¶
If for some reason, you do not have control over the creation of your TracerProvider, you can use SeedModifer() modify your xop.Seed so that it delgates SpanID and TraceID creation to Open Telemetry.
SpanToLog ¶
If you don't have access to a TracerProvider at all and instead have a "go.opentelemetry.io/otel/trace".Span, you can use that as the basis for generating logs with XOP by converting it directly to a *xop.Logger.
BufferedReplayLogger ¶
BufferedReplayLogger creates a fresh TracerProvider and Tracer for each XOP Request. It offeres the higher quality translation from XOP into OTEL but at a cost: all data relating to each Request is fully buffered in memory before the TracerProvider and Tracer are crated. There is no output until the Request is complete.
BufferedReplayLogger is meant for the situation where another xopbase.Logger is being replayed into xopotel. It is also the only way to losslessly round trip OTEL logs to XOP and then back to OTEL.
BufferedReplayExporterWrapper ¶
BufferedReplayExporterWrapper augments BufferedReplayLogger by passing information around the OTEL TracerProvider, Tracer, and Span. When not using it, Scope.Name, and all the counters that ReadOnlySpan provides are lost.
ExportToXOP ¶
Integration can go the other direction. You can flow traces from Open Telemetry to XOP base loggers. Use ExportToXOP() to wrap a xopbase.Logger so that it can be used as a SpanExporter.
Limitations ¶
Ideally, it should be possible to run data in a round trip from XOP to OTEL back to XOP and have it unchanged and also run data from OTEL to XOP and back to OTEL and have it unchanged.
The former (XOP -> OTEL -> XOP) works. Unfortunately, OTEL -> XOP -> OTEL is difficult to get working and only fully works when using the BufferedReplayLogger and the BufferedReplayExporterWrapper. This complexity could be avoided if it were possible for others to implement ReadOnlySpan.
Index ¶
- Variables
- func BaseLogger(traceProvider oteltrace.TracerProvider) xopbase.Logger
- func BufferedReplayLogger(tracerProviderOpts ...sdktrace.TracerProviderOption) xopbase.Logger
- func ExportToXOP(base xopbase.Logger) sdktrace.SpanExporter
- func IDGenerator() sdktrace.TracerProviderOption
- func NewUnhacker(exporter sdktrace.SpanExporter) sdktrace.SpanExporter
- func SeedModifier(ctx context.Context, traceProvider oteltrace.TracerProvider) xop.SeedModifier
- func SpanToLog(ctx context.Context, name string, extraModifiers ...xop.SeedModifier) *xop.Logger
- type BufferedReplayExporterWrapper
Constants ¶
This section is empty.
Variables ¶
var ErrShutdown = fmt.Errorf("Shutdown called")
Functions ¶
func BaseLogger ¶
func BaseLogger(traceProvider oteltrace.TracerProvider) xopbase.Logger
BaseLogger provides SeedModifiers to set up an OTEL Tracer as a xopbase.Logger so that xop logs are output through the OTEL Tracer.
As of the writing of this comment, the Open Telemetry Go library does not support logging so to use it for logging purposes, log lines are sent as span "Events".
The recommended way to create a TracerProvider includes using WithBatcher to control the flow of data to SpanExporters. The default configuration for the Batcher limits spans to 128 Events each. It imposes other limits too but the default event limit is the one that is likely to be hit with even modest usage.
The TracerProvider MUST be created with IDGenerator(). Without that, the SpanID created by Xop will be ignored and that will cause problems with propagation.
func BufferedReplayLogger ¶
func BufferedReplayLogger(tracerProviderOpts ...sdktrace.TracerProviderOption) xopbase.Logger
BufferedReplayLogger creates a Logger that can be used when replaying from other xopbase.Logger implementations into xopotel. It buffers all the logged data until Done() is called on a per-request basis. Additional logging after Done() is discarded.
A TracerProvider and Tracer are constructed for each Request and discarded afterwards.
For improved fideltity of OTEL -> XOP -> OTEL replay, use BufferedReplayExporterWrapper.BufferedReplayLogger instead.
func ExportToXOP ¶
func ExportToXOP(base xopbase.Logger) sdktrace.SpanExporter
ExportToXOP allows open telementry spans to be exported through a xopbase.Logger. If the open telementry spans were generated originally using xoputil, then the exported data should almost exactly match the original inputs.
func IDGenerator ¶
func IDGenerator() sdktrace.TracerProviderOption
IDGenerator generates an override that must be used with https://pkg.go.dev/go.opentelemetry.io/otel/sdk/trace#NewTracerProvider when creating a TracerProvider. This override causes the TracerProvider to respect the TraceIDs and SpanIDs generated by Xop. For accurate replay via xopotel, this is required. For general log generation, if this ID generator is not used, then the Span IDs and TraceIDs created by by the TracerProvider will be used for Xop logging.
func NewUnhacker ¶
func NewUnhacker(exporter sdktrace.SpanExporter) sdktrace.SpanExporter
NewUnhacker wraps a SpanExporter and if the input is from BaseLogger or SpanLog, then it "fixes" the data hack in the output that puts inter-span links in sub-spans rather than in the span that defined them.
func SeedModifier ¶
func SeedModifier(ctx context.Context, traceProvider oteltrace.TracerProvider) xop.SeedModifier
SeedModifier provides a xop.SeedModifier to set up an OTEL Tracer as a xopbase.Logger so that xop logs are output through the OTEL Tracer.
As of the writing of this comment, the Open Telemetry Go library does not support logging so to use it for logging purposes, log lines are sent as span "Events".
The recommended way to create a TracerProvider includes using WithBatcher to control the flow of data to SpanExporters. The default configuration for the Batcher limits spans to 128 Events each. It imposes other limits too but the default event limit is the one that is likely to be hit with even modest usage.
Using SeedModifier, the TraceProvider does not have to have been created using IDGenerator().
Types ¶
type BufferedReplayExporterWrapper ¶
type BufferedReplayExporterWrapper interface {
// WrapExporter augments the data that the wrapped exporter recevies so that if
// the ReadOnlySpan came from a BufferedReplayLogger that is replaying data that
// originally came from OTEL, then it can fill data that otherwise has no way
// to be propagated through the OTEL APIs. For example, instrumentation.Scope.Name.
WrapExporter(sdktrace.SpanExporter) sdktrace.SpanExporter
// BufferedReplayLogger creates a Logger that can be used when replaying from other
// xopbase.Logger implementations into xopotel. It buffers all the logged data until
// Done() is called on a per-request basis. Additional logging after Done() is discarded.
//
// A TracerProvider and Tracer are constructed for each Request and discarded afterwards.
//
// VERY IMPORTANT: every exporter wrapped with WrapExporter must be
// passed to BufferedReplayLogger. If not, memory will leak.
//
// Also import: if the exporter's ExportSpans() isn't called with all
// spans, memory will leak. The amount of leaked memory is not large, maybe
// 100 bytes per span, but it isn't zero.
BufferedReplayLogger(...sdktrace.TracerProviderOption) xopbase.Logger
}
func NewBufferedReplayExporterWrapper ¶
func NewBufferedReplayExporterWrapper() BufferedReplayExporterWrapper
NewBufferedReplayExportWrapper creates an export wrapper that can be used to improve the accuracy of data exported after replay. This comes into play when data is run from OTEL to XOP and then back to OTEL. When it comes back to OTEL, the export wrapper improves what a SpanExporter exports.
Use the BufferedReplayExporterWrapper to both wrap SpanExporter and to create the BufferedReplayLogger that is used to export from XOP to OTEL.
Source Files
Directories