At QuintoAndar we have dozens of engineering teams maintaining over a hundred services. Honeycomb became the go-to solution when we need to dive in and understand what is happening with them. Our biggest use case is by far troubleshooting incidents. Thanks to Honeycomb's blazing fast query times, once we know something is wrong, we are often able to quickly pinpoint exactly what is going on and fix it faster than before. But we also started to use it to keep track of Service Level Objectives and even help onboarding new engineers, for example by showing some ways user requests map into our infrastructure. Finally, the customer support was great. Since we started the trial we had a direct channel with both their sales and engineering teams who were always very quick to answer any questions and act on any feedback.

Honeycomb has made it very easy to ingest any kind of structured data and "slice and dice it" to figure out what is really happening with our systems. We have extracted a lot of value from database query logs, load balancer logs, CDN logs, even Kubernetes events! However, it's at analyzing application traces where it really shines. The game changer is the "bubble up" feature: it tells you which dimension of an event is more likely to be responsible for an anomaly, and that is super useful when you are troubleshooting an incident or want to answer tricky questions such as "which API calls are causing us to not meet our Service Level Objectives?". Finally, I have to add: Honeycomb's pricing model (event-based instead of machine/CPU/license based) is probably the most cloud-friendly option out there.

Some aspects of the UI/UX could be improved. It is easy to jump on and start asking meaningful questions about your data, but it can also be a bit overwhelming to new users who are more accustomed to more traditional "point-and-click" interfaces. The online interactive tutorials are very well made and do help a lot though.

We used the FREE tier to test the product, and now we just started with PRO and are looking for getting all moving parts instrumented. Already showed benefits by not having to jump between multiple tools, but we expect to get even more in terms of the knowledge sharing and culture around observability and on-call in our org.

Honeycomb helps engineers deal with complexity. It rewards curiosity and makes lowers the unnecesary cognitive load for engineering modern systems. Ultimately, improves quality of life for engineers.

The instrumentation and the decisions about sampling can make or break the value users get from HC - and currently (especially for PRO subscriptions) the focus is more on the end result and not that much on how to get there.

Honeycomb does exactly what it says it does. The flexible data model means we can freely instrument our code without worrying about massaging it into a form that it will accept. In fact, our codebase was already instrumented using OpenTracing before we adopted Honeycomb. We were able to point our OT client to Honeycomb (via a proxy they provide) and it just worked. The query engine is flexible and fast, which makes it easy to explore our data without demanding intimate familiarity -- this is critical when debugging issues in a distributed service where I don't always know exactly how the events are structured, or what exactly I'm looking for. Honeycomb's event-based pricing model means we can shove whatever information we think might be useful into our events without worrying about blowing the budget. This is critical to maintaining a well-instrumented codebase, as we don't want our developers having to make value judgements about what to report during development. Combined, these attributes mean that any question I can think to ask can almost certainly be answered by Honeycomb. It does not take long to become dependent on this ability. In fact, when our trial period was about to expire, a member of my team said, "you can pry Honeycomb from my cold, dead hands!"

Honeycomb is excellent for exploring the system to get to the bottom of an incident. However, it is less useful for more classic monitoring use-cases. It does have dashboard and trigger features, but they are fairly rudimentary at this time, and insufficient for proactively monitoring the health of a complex service. In Honeycomb's defense, they do not position themselves as a monitoring solution, so this isn't particularly surprising, but it does mean that a separate monitoring solution is also required.

I am now requiring Honeycomb integration on all software I build and help maintain. It is the highest praise I can give.

It opens up new avenues in software engineering: what if you could ask your system how it's doing? Instrumenting everything, sampling sanely, and all the relevant data is in Honeycomb at a reasonable price. From then on forward, all the querying options are available. Bubble Up is there to help massively during incidents to find the outliers and where the problem lies. Once this is implemented properly( not an easy feast), it feels like living in the future.

Coming from a classic( and wrong) observability=logs+metrics+traces mindset it is rather hard to get accustomed to Honeycomb. Once the mental hurdle is passed it's all sunshine and rainbows, but it takes a whole lot to get there. Fortunately, their Solution Engineers are very patient and helpful.

Very easy to set up and get data into Honeycomb. We leverage AWS Lambda extensively and send all of our Cloudwatch log groups to a Kinesis stream, which another Lambda consumes & publishes to Honeycomb. We've been able to find patterns that would have been difficult to surface in other tools and the SLO functionality provides a good target for your application.

The ability to explore your application data in ways you did not know you needed, but once you get a taste you're hooked.

I can't think of anything about Honeycomb that I do not like. One thing to keep in mind when implementing it is to ensure your log data is well-structured with standardized field names, otherwise you won't get the most from it.