telemetry_tracing

Tracing

• Spring Cloud Sleuth
  • Enables distributed tracing
  • Simple facade for distributed tracing systems
• Micrometer Tracing is successor of Spring Cloud Sleuth

Setting up Tracing

• Using micrometer-tracing, OpenTelemetry and Zipkin server
• https://opentelemetry.io/docs/languages/java/automatic/
• Gradle Config:

compileOnly 'io.micrometer:micrometer-tracing'  
implementation 'io.micrometer:micrometer-tracing-bridge-otel'

• application.yml config

spring:
  zipkin.enabled: true
  zipkin:
    base-url: ${ZIPKIN_BASE_URL:http://localhost:9411}
    sender:
      type: web
  # not sure if sleuth config is required or not
  sleuth:
    sampler:
      probability: "1.0"

• Launch Zipkin server:

# Navigate to http://localhost:9411/zipkin on browser
docker run -d -p 9411:9411 openzipkin/zipkin

• Download opentelemetry-javaagent.jar
• Launch app:

# OTEL supports:
# OTEL_LOGS_EXPORTER, OTEL_METRICS_EXPORTER, OTEL_TRACES_EXPORTER
export OTEL_METRICS_EXPORTER="otlp" # Is this required?
export OTEL_SERVICE_NAME=bff-mobile
export OTEL_TRACES_EXPORTER=zipkin
 
# Use this to check if OTEL javaagent is really working
export OTEL_JAVAAGENT_DEBUG=true
 
# Build the jar
./gradlew clean build -x test
 
java -javaagent:./service/build/docker/opentelemetry-javaagent-1.31.0.jar -jar ./service/build/libs/bff-mobile-0.536.0-SNAPSHOT-application.jar

• The following screenshot is from zipkin capturing info
• There are multiple spans of service bff-mobile denoting:
  • method call (example: itemcontroller.searchdpItems)
  • API call (example: post)

Tracing Terms (Dapper)

• Span: Basic unit of work.
  • They contain metadata
  • Can be started and then stopped
  • For example:
    • Sending RPC
    • Sending RPC response
• Trace: Set of spans forming tree structure
• Tracer: a library that handles span lifecycle
• Tracing Context: Identifier (Trace identifier/span identifier etc.) that must be propagated through the process and over network
• Spans are collected from all the places and then unified using tracing context (traceId) which will be unique for all the requests between microservices

Trace ID

• Trace ID needs to be propagated and consumed. so that when collected they can be unified in distributed tracing software and show information
• you can configure it in application.properties:

management.tracing.propagation.consume=W3C, B3, B3_MULTI
management.tracing.propagation.produce=W3C
management.tracing.propagation.type

• W3C provides TraceContext standard distributed tracing to work. Headers include:
  • traceparent
  • tracestate
• other popular header formats:
  • Zipkin (B3-*)
  • Jaeger (uber-*)
  • OpenTracing ‘sample’ headers (ot-*)
• OpenTelemetry has adopted W3C TraceContext and Zipkin’s B3 format

traceparent

• https://medium.com/@mesutatasoy/understanding-traceparent-and-microservices-in-opentelemetry-notepad-series-7-de5c16bf6462#:~:text=Using%20traceparent%20with%20Messaging%20Systems&text=Here’s%20how%20it%20works%3A,message%20to%20the%20messaging%20system

traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01

• Version (2): 00
  • version of traceparent specification
• Trace ID (32): 4bf92f3577b34da6a3ce929d0e0e4736
  • unique identifier for the entire trace.
• Parent ID (16): 00f067aa0ba902b7
  • identifier for the parent span
• Trace Flags: 01
  • extra information about the trace
  • 01 means trace is sampled and should be recorded

traceparent vs correlationId