The log management company LogDNA has released a new report, “The Observability Challenge,” which reveals that 74% of companies are still struggling to achieve true observability despite substantial investments in tools.
The report findings were based on a survey of more than 200 senior engineering professionals who are responsible for observability and log data management at companies across the United States.
The study showed that two-thirds of organizations are currently spending $100,000 or more annually on observability tools, with 38% spending $300,000 or more annually. While many organizations have four or more tools in their arsenal, they’re often dissatisfied; in fact, more than half are unable to implement the tools they want because of vendor lock-in.
“It’s clear that organizations need to drive more value from their observability data, but they are struggling to find tools that make it easy for multiple stakeholders to extract actionable insights,” said Tucker Callaway, CEO, LogDNA.
The report highlights the critical importance of log data to observability use cases, such as tracking performance of applications and capacity resources, informing product improvements, and identifying threats and anomalous activity. However, fewer than half of respondents are very satisfied with their ability to use log data for any one of these purposes. And when outcomes fall short of expectations, respondents said frustration stems from three key areas — tools are not easy to use (66%), collaboration with colleagues on multiple teams is difficult (67%), and they have challenges with routing security events (58%). In each situation, teams report spending more time than they should trying to resolve issues.
“The Harris Poll found that the issues with cross-team collaboration appear to exist across organizations, with all levels of seniority, tenure, and company size consistently agreeing on this as a challenge,” said Callaway. “Development teams, IT operations, SRE, and security teams all need access to the same data, but they need it for different reasons, and routing that data to the right people at the right time is often cumbersome and expensive.”
Still, results show that sentiment toward observability is generally positive, with 85% of participants responding they believe true observability is possible. This supports a need for new innovation that will improve ease of use and facilitate stronger cross-team collaboration. Results also support increasing demand for observability data pipeline solutions that enable enterprises to ingest all of their data to a single platform, normalize it, and seamlessly route it to the appropriate teams, so they can take meaningful action quickly within their workflows.