Kubernetes operations teams need more than dashboards displaying CPU and memory consumption and error rates. The complexity of cloud-native environments means that a single request may pass through an ingress gateway, services, queues, storage, and background processes, while workloads move and versions change continuously. Therefore, a deployment may appear healthy at the Deployment level, while a downstream dependency, a retry loop, or pressure on a path in the control plane causes latency to rise.
In an article published on the CNCF blog, Neel Shah of Stackgen explains that traditional monitoring answers specific, predefined questions, such as: Has CPU usage exceeded a certain threshold? Are errors increasing? Observability, according to the article, aims to help the team investigate an issue it did not expect and move from noticing the symptom to understanding the cause and scope.
Metrics Open the Investigation but Do Not End It
Metrics remain the natural starting point because they are numeric and efficient to store and query, and they are suitable for alerts and trend analysis. In Kubernetes, they can show node pressure, container restarts, increased request latency, a slowing API server, or accumulating queue items.
The article suggests using the RED patterns for services—request rate, errors, and duration—and the USE pattern for infrastructure—utilization, saturation, and errors. These indicators help draw the initial picture of the incident: an increase in request rate with stable latency differs from an increase in duration and saturation with traffic remaining unchanged.
However, turning every detail into a metric label creates a cardinality problem, as the large number of unique combinations increases costs and slows queries. The article specifically warns against placing request or user identifiers, or nearly unique values, in metrics; these details are better suited to logs or traces.
Each Signal Answers a Different Question
Logs add local context that graphs usually do not preserve. When they are structured and use consistent fields such as time, severity level, service name, namespace, container identity, request path, and trace context, it becomes easier to link a specific event to the service or process that produced it. A metric may indicate that the payment service is affected, while the log reveals an expired timeout, an exception, or a failure in a dependency.
Distributed traces answer a different question: How did a single request move through the system, and where was the time consumed? Their importance is evident in Kubernetes because a failure may be distributed across multiple services, retries, queue limits, or database calls. Propagating trace context makes it possible to link different spans within a single request context, while shared semantic conventions help standardize field and attribute names across metrics, logs, and traces.
The article also places profiling within the picture. After metrics identify the slow service, tracing identifies the affected request path, and logs clarify the local event, profiling data can help identify the function or code path consuming CPU or memory.
What Changes in Practice During an Incident?
The article presents an example in which a checkout service begins exceeding its latency target after a new deployment, while CPU and memory indicators remain normal. Metrics reveal that there is a problem, then a trace for a slow request shows that most of the delay occurs in the payment authorization step. The logs then show repeated timeout messages associated with the same request context.
This sequence moves the team from a general question—why has the payment service become slow?—to more specific operational options, such as rolling back a change in a dependency, reducing retry amplification, or temporarily diverting traffic during the investigation. The article also emphasizes that the best alert reflects a risk to service quality or its reliability objective, rather than merely raw infrastructure discomfort; it gives the example of an alert associated with an increase in p99 response latency above one second for ten minutes.
Applicable Design Rules
- Start with the available metrics and logs, then gradually expand coverage instead of collecting everything without a purpose.
- Prefer service- and workload-related dimensions over highly unique labels within metrics.
- Use a consistent metadata structure across metrics, logs, and traces.
- Add request or trace identifiers to logs to facilitate moving between signals.
- Build alerts around reliability risks and service quality, not resource pressure alone.
- Consider observability part of application and platform design, not a later addition after deployment.
Editorial perspective from certi.news: The core value here is not a call to buy a tool or adopt a single implementation, but to redefine the goal of observability. What actually changes is how data is used: the metric captures the deviation, the trace identifies the path, the log explains the event, and profiling may identify the cause at the code level. Clear practical limitations remain; collecting more signals does not guarantee better understanding, and the absence of unified names and fields may make correlation fragile, while unique details may increase metric costs and weaken query performance. Therefore, the benefit depends on the quality of the design and the links between signals, not on the number of dashboards.