Bug: RPC method names are used unsanitized as Prometheus metric names, breaking scrapes
Summary
When rpc_enable_metrics=true, the RPC metrics middleware registers a timer per JSON-RPC method using the raw method name straight from the request, with no character sanitization before it's written out as a Prometheus metric name. Any method name containing characters outside [a-zA-Z0-9_:] (e.g. a .) produces an invalid metric line that Prometheus's exposition-format parser rejects — and since Prometheus's parser is strict, this single bad line fails the entire scrape, not just that one metric.
Root cause
crates/rpc/rpc-middlewares/src/metrics.rs: register_timer_with_group("async_rpc", name.as_str()) uses req.method_name() verbatim.
crates/util/metrics/src/report_prometheus.rs: format!("{}_{}", group, name) concatenates group + method name with no validation/escaping before emitting # TYPE <name> counter.
Steps to reproduce
-
Start a node with:
metrics_enabled=true
rpc_enable_metrics=true
metrics_prometheus_listen_addr="127.0.0.1:9777"
-
Send any RPC request whose method name contains a . — it doesn't even need to be a real method, since the metrics middleware runs before method dispatch:
curl -X POST http://<node>:8545 \
-H "Content-Type: application/json" \
-d '{"jsonrpc":"2.0","id":1,"method":"totally.made.up","params":[]}'
The node correctly rejects it:
{"jsonrpc":"2.0","id":1,"error":{"code":-32601,"message":"Method not found"}}
-
Scrape the metrics endpoint:
curl http://127.0.0.1:9777/metrics | grep made.up
Expected behavior
Metric names are sanitized (e.g. invalid characters replaced with _) so the exposition output is always valid Prometheus format, regardless of what method name a client sends.
Actual behavior
Output includes an invalid line such as:
# TYPE async_rpc_totally.made.up_counter_total counter
async_rpc_totally.made.up_counter_total 1
despite the method never existing. Prometheus fails to parse this and errors the whole scrape (e.g. invalid metric type "made.up_counter_total counter"), so no metrics at all are collected from the node until the offending series ages out or the node is restarted.
Since any client-supplied method name, valid or not, is instrumented unconditionally, this also means an unauthenticated caller can grow the metrics registry indefinitely by sending requests with unique junk method names, which is an unbounded-cardinality / memory-growth concern independent of the Prometheus formatting issue.
Suggested fix
Sanitize name/group in report_prometheus.rs (or at registration time in the RPC metrics middleware) by replacing any character outside [a-zA-Z0-9_:] with _ before building the metric name.
Bug: RPC method names are used unsanitized as Prometheus metric names, breaking scrapes
Summary
When
rpc_enable_metrics=true, the RPC metrics middleware registers a timer per JSON-RPC method using the raw method name straight from the request, with no character sanitization before it's written out as a Prometheus metric name. Any method name containing characters outside[a-zA-Z0-9_:](e.g. a.) produces an invalid metric line that Prometheus's exposition-format parser rejects — and since Prometheus's parser is strict, this single bad line fails the entire scrape, not just that one metric.Root cause
crates/rpc/rpc-middlewares/src/metrics.rs:register_timer_with_group("async_rpc", name.as_str())usesreq.method_name()verbatim.crates/util/metrics/src/report_prometheus.rs:format!("{}_{}", group, name)concatenates group + method name with no validation/escaping before emitting# TYPE <name> counter.Steps to reproduce
Start a node with:
Send any RPC request whose method name contains a
.— it doesn't even need to be a real method, since the metrics middleware runs before method dispatch:The node correctly rejects it:
{"jsonrpc":"2.0","id":1,"error":{"code":-32601,"message":"Method not found"}}Scrape the metrics endpoint:
curl http://127.0.0.1:9777/metrics | grep made.upExpected behavior
Metric names are sanitized (e.g. invalid characters replaced with
_) so the exposition output is always valid Prometheus format, regardless of what method name a client sends.Actual behavior
Output includes an invalid line such as:
despite the method never existing. Prometheus fails to parse this and errors the whole scrape (e.g.
invalid metric type "made.up_counter_total counter"), so no metrics at all are collected from the node until the offending series ages out or the node is restarted.Since any client-supplied method name, valid or not, is instrumented unconditionally, this also means an unauthenticated caller can grow the metrics registry indefinitely by sending requests with unique junk method names, which is an unbounded-cardinality / memory-growth concern independent of the Prometheus formatting issue.
Suggested fix
Sanitize
name/groupinreport_prometheus.rs(or at registration time in the RPC metrics middleware) by replacing any character outside[a-zA-Z0-9_:]with_before building the metric name.