Kubernetes operations teams need more than dashboards displaying CPU and memory consumption and error rates. The complexity of cloud-native environments means that a single request may pass through an ingress gateway, services, queues, storage, and background processes, while workloads move and versions change continuously. Therefore, a deployment may appear healthy at the Deployment level while a downstream dependency, a retry loop, or pressure on a control-plane path causes latency to rise.
In an article published on the CNCF blog, Neel Shah of Stackgen explains that traditional monitoring answers predefined questions, such as: Has CPU usage exceeded a certain threshold? Are errors increasing? Observability, according to the article’s argument, aims to help the team investigate a problem it did not anticipate and move from noticing the symptom to understanding the cause and scope.
Metrics Open the Investigation but Do Not End It
Metrics remain the natural starting point because they are numeric, efficient to store and query, and suitable for alerts and trend analysis. In Kubernetes, they can show node pressure, container restarts, increased request latency, a slowdown in the API server, or a buildup of queue items.
The article suggests using the RED method for services—request rate, errors, and duration—and the USE method for infrastructure—utilization, saturation, and errors. These indicators help establish the initial picture of an incident: an increase in request rate with stable latency differs from increased duration and saturation with traffic remaining constant.
However, turning every detail into a metric label creates a cardinality problem, as the large number of unique combinations increases costs and slows queries. The article specifically warns against placing request or user IDs, or near-unique values, in metrics; such details are better suited to logs or traces.
Each Signal Answers a Different Question
Logs add local context that graphs usually do not preserve. When they are structured and use consistent fields such as time, severity level, service name, namespace, container identity, request path, and trace context, it becomes easier to associate a specific event with the service or process that produced it. A metric may indicate that the payment service is affected, while a log reveals an expired timeout, an exception, or a dependency failure.
Distributed traces answer a different question: How did a single request move through the system, and where was the time spent? They are particularly important in Kubernetes because a failure may be distributed across multiple services, retries, queue limits, or database calls. Passing trace context makes it possible to link different spans within the context of a single request, while shared semantic conventions help standardize field and attribute names across metrics, logs, and traces.
The article also includes profiling in the picture. After metrics identify the slow service, tracing identifies the affected request path, and logs clarify the local event, profiling data can help identify the function or code path consuming CPU or memory.
What Changes in Practice During an Incident?
The article presents an example in which a checkout service begins exceeding its latency target after a new deployment, while CPU and memory indicators remain normal. Metrics reveal that a problem exists, then a trace for a slow request shows that most of the delay occurs in the payment authorization step. The logs subsequently show repeated timeout messages associated with the same request context.
This sequence moves the team from a general question—why has the payment service become slow?—to more specific operational options, such as rolling back a change in a dependency, reducing retry amplification, or temporarily diverting traffic during the investigation. The article also emphasizes that the best alert reflects a risk to service quality or its reliability objective, rather than merely raw infrastructure-resource distress; it gives the example of an alert tied to the p99 response latency rising above one second for ten minutes.
Actionable Design Rules
- Start with the available metrics and logs, then gradually expand coverage instead of collecting everything without a goal.
- Prefer dimensions related to the service and workload over highly unique labels within metrics.
- Use a consistent metadata structure across metrics, logs, and traces.
- Add request or trace IDs to logs to make it easier to move between signals.
- Build alerts around reliability risks and service quality, not resource pressure alone.
- Treat observability as part of application and platform design, not as a later addition after deployment.
Editorial reading from certi.news: The core value here is not a call to purchase a tool or adopt a single implementation, but a redefinition of the objective of observability. What actually changes is how the data is used: the metric captures the deviation, the trace identifies the path, the log explains the event, and profiling may identify the cause at the code level. Clear practical constraints remain; collecting more signals does not guarantee better understanding, and the absence of standardized names and fields can make correlation fragile, while highly unique details may increase metric costs and weaken query performance. Therefore, the benefit depends on the quality of the design and the links between signals, not on the number of dashboards.