Static Newsabout
meysamazad | 19 comments

nicholasjarnold|next|

I can confirm that this is a pretty good way. Building out a basic distributed tracing solution with OTEL, jaeger and the relevant Spring Boot configuration and dependencies was quite a pleasant experience once you figure out the relevant-for-your-use-cases set of dependencies. It's one of those nice things that Just Works™, at least for Java 17 and 21 and Spring Boot 3.2 (iirc) or greater.

There appeared to be wide array of library and framework support across various stacks, but I can only attest personally to the quality of the above setup (Java, Boot, etc).


ljm|parent|next|

> once you figure out the relevant-for-your-use-cases set of dependencies

> It's one of those nice things that Just Works™

did it Just Work™ or did you have to do work to make it Just Work™?


barake|root|parent|next|

Java has really good OTel coverage across tons of libraries. It should mostly Just Work™, though you'll still need to consider sampling strategies, what metrics you actually want to collect, etc.

Would say .NET isn't too far behind. Especially since there are built-in observability primitives and Microsoft is big on OTel. ASP.NET Core and other first party libraries already emit OTel compliant metrics and traces out of the box. Instrumenting an application is pretty straightforward.

I have less experience with the other languages. Can say there is plenty of opportunity to contribute upstream in a meaningful way. The OpenTelemetry SIGs are very welcoming and all the meetings are open.

Full disclosure: work at Grafana Labs on OpenTelemetry instrumentation


roshbhatia|root|parent|prev|next|

In my experience, it just worked -- I was at an org that ran 3rd party java services alongside our normal array of microservices (that all used our internal instrumentation library that wrapped OTEL) and using the OTEL autoinstrumentation for those 3rd party services was pretty trivial to get setup and running (just wrap the command to run the app with the OTEL wrapper, hand it a collector url.) Granted -- we already had invested in OTEL elsewhere and were familiar with many of the situations in which it didn't just work.

azthecx|root|parent|prev|next|

I had a very similar experience with a Quarks REST API where it's supported very well out of the box, we just had to point it to the appropriate otel collector endpoint and traces are created or propagated automatically.

demurgos|prev|next|

OpenTelemetry is nice overall as there are library for multiple platforms. I introduced it this year for a web game platform with servers in Node, Java, PHP and Rust and it all worked roughly similarly which made it good for consistency.

I like how OpenTelemetry decouples the signal sink from the context, compared to other structured logging libs where you wrap your sink in layers. The main thing that I dislike is the auto-instrumentation of third-party libraries: it works great most of the time, but when it doesn't it's hard to debug. Maintainers of the different OpenTelemetry repos are fairly active and respond quickly.

It's still relatively recent, but I would recommend OpenTelemetry if you're looking for an observability framework.


candiddevmike|parent|next|

LLM slop

demurgos|root|parent|next|

Why are you accusing me of posting an LLM reply?!

I just shared that I enjoyed using and contributing to OpenTelemetry. I never used an LLM. Do I really need to prove that I'm human?

- a couple PRs I posted to the Rust impl: https://github.com/open-telemetry/opentelemetry-rust/pulls?q... I also participate to issue discussions in this repo and others.

- OpenTelemetry tracer config in the web game platform I'm working on: https://gitlab.com/eternaltwin/eternaltwin/-/blob/main/crate...

- A somewhat cool thing: an OpenTelemetry proxy impl handling authentication of traces: https://gitlab.com/eternaltwin/eternaltwin/-/blob/main/crate...

- Node usage: https://gitlab.com/eternaltwin/labrute/labrute-react/-/blob/...

- Java usage: https://gitlab.com/eternaltwin/dinoparc/dinoparc/-/merge_req...

- PHP usage: https://gitlab.com/eternaltwin/mush/mush/-/merge_requests/20...

This is all 100% handcrafted, as all my messages ever.

Now that I proved that I actually used and contributed to OpenTelemetry, may I express that I like it overall but regret that the auto-instrumentation is brittle and hard to debug? I can expand on the particular issues I've hit or why I feel that it's still not mature enough.


kevindamm|root|parent|next|

FWIW, you gave that commenter way more than they deserved for the amount of effort they put into their comment. Also, I wouldn't have suspected your earlier comment was generated by an LLM.

aramattamara|prev|next|

The problem with OpenTelemetry is that it really only good for tracing. Metrics and logs are kinda bungee strapped later: very inefficient and clunky to use.

PS: And devs (Lightspeed?) seem to really like "Open" prefix: OpenTracing + OpenCensus = OpenTelemetry.


kbouck|prev|next|

For anyone that has built more complex collector pipelines, I'm curious to know the tech stack:

  - otel collector?
  - kafka (or other mq)?
  - cribl?
  - vector?
  - other?

azthecx|parent|next|

What sort of complexity do you need? I've used them on my previous job and am implementing it on the current one. I have never heard of the last three you mention.

Otel collector is very useful for gathering multiple different sources, eg I am at a big corporation and we both have department level Grafana stack (Prometheus Loki etc) and we need to also send the data to Dynatrace. With otel collector these things are a minor configuration away.

For Kafka if you mean tracing through Kafka messages previously we did it by propagating it in message headers. Done at a shared team level library the effort was minimal.


PeterZaitsev|prev|next|

Looking for target for your OTEL data checkout Coroot too - https://coroot.com/ Additionally to OTEL visualization it can use eBPF to generate traces for applications where OpenTelemetry installation can't be done.

candiddevmike|prev|next|

This is hypocritical content marketing from a company that doesn't want you to be vendor neutral. As seen by the laughable use of hyperlinks to their products but no links when mentioning Prometheus or elasticsearch.

OTEL is great, I just wish the CNCF had better alternatives to Grafana labs.


najork|parent|next|

Check out Perses: https://github.com/perses/perses

Less mature than Grafana but recently accepted by the CNCF as a sandbox project, hopefully a positive leading indicator of success.


physicles|root|parent|next|

Didn't know about Perses. Looks promising! I've had one foot out the door with Grafana for a couple years -- always felt heavy and not-quite-there (especially the Loki log explorer), and IMHO they made alerts far less usable with the redesign in version 8.

scubbo|parent|prev|next|

Well, no, they don't _want_ you to be vendor neutral. But they allow and support you to do so - unlike DataDog.

exabrial|prev|

JMX -> Jolokia -> Telegraf -> the-older-TICK-stack-before-influxdb-rewrote-it-for-the-3rd-time-progressively-making-it-worse-each-time