CVE-2023-45142 in Prometheus
Summary
by MITRE • 10/25/2023
OpenTelemetry-Go Contrib is a collection of third-party packages for OpenTelemetry-Go. A handler wrapper out of the box adds labels `http.user_agent` and `http.method` that have unbound cardinality. It leads to the server's potential memory exhaustion when many malicious requests are sent to it. HTTP header User-Agent or HTTP method for requests can be easily set by an attacker to be random and long. The library internally uses `httpconv.ServerRequest` that records every value for HTTP `method` and `User-Agent`. In order to be affected, a program has to use the `otelhttp.NewHandler` wrapper and not filter any unknown HTTP methods or User agents on the level of CDN, LB, previous middleware, etc. Version 0.44.0 fixed this issue when the values collected for attribute `http.request.method` were changed to be restricted to a set of well-known values and other high cardinality attributes were removed. As a workaround to stop being affected, `otelhttp.WithFilter()` can be used, but it requires manual careful configuration to not log certain requests entirely. For convenience and safe usage of this library, it should by default mark with the label `unknown` non-standard HTTP methods and User agents to show that such requests were made but do not increase cardinality. In case someone wants to stay with the current behavior, library API should allow to enable it.
VulDB is the best source for vulnerability data and more expert information about this specific topic.
Analysis
by VulDB Data Team • 12/17/2024
The vulnerability described in CVE-2023-45142 affects the OpenTelemetry-Go Contrib library, specifically within the httpconv package that handles server request processing. This issue stems from the default behavior of the otelhttp.NewHandler wrapper which automatically captures HTTP headers including User-Agent and method attributes without proper cardinality controls. The problem manifests when the library records every unique value for these attributes internally, creating a path for memory exhaustion attacks through the accumulation of high cardinality data points. The vulnerability is classified under CWE-1243 which deals with the improper handling of high cardinality data in monitoring systems, and aligns with ATT&CK technique T1499.001 which involves resource exhaustion attacks targeting application memory.
The technical flaw occurs because the httpconv.ServerRequest function records every HTTP method and User-Agent value without restriction, allowing attackers to craft malicious requests with arbitrarily long or random values in these headers. This creates an unbounded growth scenario where each unique combination of method and User-Agent values consumes memory resources proportional to the number of distinct inputs. The vulnerability is particularly dangerous in high-traffic environments where attackers can systematically send requests with varying User-Agent strings or HTTP methods to exhaust server memory resources. The issue affects applications that directly use the otelhttp.NewHandler wrapper without additional filtering mechanisms at earlier layers of the network stack.
The operational impact of this vulnerability can be severe, leading to potential application crashes, service unavailability, and denial of service conditions. When exploited, malicious actors can cause progressive memory consumption that eventually leads to system instability or complete service failure. The vulnerability is particularly concerning in cloud environments or containerized applications where memory limits are strict and resource exhaustion can trigger automatic restarts or termination of processes. Organizations using OpenTelemetry-Go Contrib in production environments face significant risk of operational disruption if this vulnerability is not addressed through proper mitigation strategies.
The fix implemented in version 0.44.0 addresses the core issue by restricting the values collected for the http.request.method attribute to well-known HTTP methods only, while removing other high cardinality attributes from automatic collection. This approach follows security best practices by implementing principle of least privilege in attribute collection and aligns with industry standards for secure logging and monitoring. Organizations can also implement workarounds using otelhttp.WithFilter() middleware to manually control which requests are processed, though this requires careful configuration to avoid inadvertently filtering legitimate traffic. The recommended approach for safe usage involves defaulting to marking non-standard HTTP methods and User-agents as "unknown" while maintaining API flexibility for those who require the previous behavior through explicit configuration options.