Repository navigation
Conversation
SimpleMetrics only implemented Timer. histogram(String) and counter(String) returned null, so any caller using those metric types hit a NullPointerException, and neither type was reported by all(), so MetricsPrinter never saw them either. Counter is backed by an AtomicInteger, which matches the int-typed Metrics.Counter interface. Histogram keeps a bounded reservoir of 1028 samples -- the size Go-OPA takes from rcrowley/go-metrics -- and reports the same statistics as the Go implementation: population standard deviation, and linearly interpolated percentiles at 75%, 90%, 95%, 99%, 99.9% and 99.99%. Histogram.Values keeps its existing int fields, so the row shapes MetricsPrinterTest asserts are unchanged. Fixes open-policy-agent#60 Signed-off-by: c879873067877881111 <c879873067877881111@gmail.com>
0b9ef5b to
ec9976d
Compare
| return new TreeMap<>(timers); | ||
| Map<String, Metric> all = new TreeMap<>(timers); | ||
| all.putAll(counters); | ||
| all.putAll(histograms); |
There was a problem hiding this comment.
Now that counters and histograms are in all(), DecisionLogPlugin.buildDecisionEvent falls into its else branch and calls MAPPER.valueToTree(...) on SimpleCounter/SimpleHistogram — neither has a Jackson-visible property, so it throws No serializer found ... and no properties discovered to create BeanSerializer. DecisionLogs.logDecision swallows that as "Failed to log decision", so the entire decision event is dropped, which is the opposite of the CHANGELOG's "so they reach ... decision logs".
MetricsModule already solves this for Timer; the same mixin treatment covers the other two:
setMixInAnnotation(Metrics.Counter.class, CounterMixin.class);
setMixInAnnotation(Metrics.Histogram.class, HistogramMixin.class);
abstract static class CounterMixin {
@JsonValue
abstract int value();
}
abstract static class HistogramMixin {
@JsonValue
abstract Metrics.Histogram.Values value();
}There was a problem hiding this comment.
Mixins added. Testing them turned up a second path: DecisionLogs.MAPPER is a bare ObjectMapper + JavaTimeModule, and opa-jackson is only runtimeOnly for opa-services, so the mixins don't reach it — with them in place the repro still failed identically. Timers avoid it only because of the explicit timer_<name>_ns branch.
So buildDecisionEvent now writes counters and histograms itself as well, under formatKey's names, counter_<name> and histogram_<name>; unknown Metrics keep the Jackson fallback. Covered by decisionLogs_includesEveryMetricType.
| } | ||
|
|
||
| private static int round(double value) { | ||
| return (int) Math.round(value); |
There was a problem hiding this comment.
This can overflow. The largest value an int holds is about 2.1 billion, which is only 2.15 seconds in nanoseconds — and nanosecond timings are what Go feeds histograms in topdown/instrumentation.go. Anything bigger turns negative: update(3e9) reports -1294967296 for every stat. The line above does Math.min(count, Integer.MAX_VALUE) to avoid this; the same idea works here:
| return (int) Math.round(value); | |
| return (int) Math.max(Integer.MIN_VALUE, Math.min(Integer.MAX_VALUE, Math.round(value))); |
There was a problem hiding this comment.
The suggestion was adopted without modification.
Overflow occurs in the negative direction as well—update(-3e9) results in +1294967296; his original comment only mentioned the positive-to-negative case.
Both directions were tested; the test names are histogram_saturatesStatsAboveIntRange and histogram_saturatesStatsBelowIntRange.
I left Values as int rather than widening to Go's int64/float64.
| public Map<String, Metric> all() { | ||
| return new TreeMap<>(timers); | ||
| Map<String, Metric> all = new TreeMap<>(timers); | ||
| all.putAll(counters); |
There was a problem hiding this comment.
Since all three maps share one key space, reusing a name across types silently drops one. After metrics.timer("x") and metrics.counter("x"), all() returns just the counter — the timer is gone from the printer, the CLI table and decision logs. Go's All() prefixes each key by type (timer_x, counter_x), so both survive there.
Matching that means moving the prefixing out of MetricsPrinter and into all(), which is bigger than this PR — fine as a follow-up, but worth noting the gap here.
There was a problem hiding this comment.
Agreed with all three types sharing a name, only one survives all().
Confirmed formatKey behaves as you describe. Leaving it out of this PR
Do you want to open an issue to track it?
Now that all() reports counters and histograms, buildDecisionEvent falls into its else branch and calls MAPPER.valueToTree on them. Neither SimpleCounter nor SimpleHistogram has a Jackson-visible property, so Jackson throws "No serializer found ... and no properties discovered to create BeanSerializer". logDecision swallows that as "Failed to log decision", so the entire decision event is dropped, not just the metric. Counters and histograms are now written out field by field under the names OPA Go publishes them as (see metrics.formatKey): counter_<name> and histogram_<name>, alongside the existing timer_<name>_ns. Timers never hit this because they already had an explicit branch. Any other Metric implementation still falls back to Jackson under its bare name, as Go's default branch does. Registering MetricsModule is not an alternative fix here: the mapper in DecisionLogs is a bare ObjectMapper plus JavaTimeModule, and opa-jackson is only a runtimeOnly dependency of opa-services, so the module's mixins never reach it. MetricsModule still needs the same treatment for its own consumers, though, so Counter and Histogram get the mixin that Timer already had. All three mixins are now interfaces, matching the interfaces they are applied to. Signed-off-by: c879873067877881111 <c879873067877881111@gmail.com>
Histogram.Values holds ints, but Go feeds histograms nanosecond timings (topdown/instrumentation.go) and Integer.MAX_VALUE nanoseconds is only 2.15 seconds. Anything past that wrapped: update(3e9) reported -1294967296 for every stat, and the sign flips both ways, so update(-3e9) reported +1294967296. round() now clamps to the int range the way the count field above it already did, rather than letting the cast wrap. Widening Values to long/double would match Go more closely, but it is a public type this change does not otherwise touch. Signed-off-by: c879873067877881111 <c879873067877881111@gmail.com>
Integer.MAX_VALUE nanoseconds is 2.147s, not 2.15s, so the wording implied a safe window that already overflowed. Also note the sign flips in both directions, which update(-3e9) does. Signed-off-by: c879873067877881111 <c879873067877881111@gmail.com>
addMetric's javadoc claims any Metric that is not a Timer, Counter or Histogram still reaches the event under its bare name, matching Go's formatKey default branch. Nothing tested that: deleting the else branch left every test green. Metrics is a public interface, so an implementation reporting a metric of some other type is reachable, and silently dropping it would be a real loss. The test uses a Jackson-serializable Metric and fails when the fallback is removed. Signed-off-by: c879873067877881111 <c879873067877881111@gmail.com>
Integer.MAX_VALUE nanoseconds is 2.147s, not 2.15s, so the wording implied a safe window that already overflowed. The CHANGELOG was corrected but these two comments were missed. Signed-off-by: c879873067877881111 <c879873067877881111@gmail.com>
Fixes #60
Summary
SimpleMetricsonly implementedTimer.histogram(String)andcounter(String)returned
null, so any caller of those metric types threw an NPE, andall()returned only the timer map, so neither type ever reached
MetricsPrinter.Changes
Counterbacked by anAtomicInteger. The issue suggestsAtomicLong/LongAdder,but
Metrics.Counterdeclaresadd(int)andint value(), so a long accumulatorwould only add a lossy narrowing at the boundary.
Histogramover a bounded reservoir of 1028 samples — the size Go-OPA takes fromrcrowley/go-metrics— replacing a random slot once full, so a long-livedMetricsinstance cannot grow without bound. Statistics follow the Go implementation:
population standard deviation (
SampleVariancedivides byN) and the linearlyinterpolated
SamplePercentiles, at Go's hardcoded set (median,75%,90%,95%,99%,99.9%,99.99%).countis the number of updates, not the reservoiroccupancy, matching
snap.Count().all()andclear()now cover all three metric types.Histogram.Valueskeeps itsintfields, so the rowsMetricsPrinterTestassertsare unchanged.
Timerare untouched — that is Make SimpleMetrics thread-safe #56. The newcounter/histogram code is thread-safe as written.
Verification
./gradlew :opa-evaluator:test :opa-evaluator:checkstyleMain :opa-evaluator:checkstyleTest :opa-evaluator:pmdMain :opa-evaluator:pmdTest— no PMD findings, and no Checkstylefindings for
SimpleMetrics.java.SimpleMetricsTestpicks up the usualMethodNamewarnings for
snake_casetest names, as the existing test sources do (Relax Checkstyle MethodName for test sources #88).SimpleMetricsTest, 8 cases. Reverting onlySimpleMetrics.javafails 7 of the8; the 8th covers
clear(), which passes trivially when nothing can be registered../gradlew build: the only remaining failures arejson_match_schema/invalid documentand
:opa-proto:vendorProtoSchemas. The first reproduces unchanged with my changereverted; the second is
go mod downloadunable to reachproxy.golang.orgfrom here.Open questions
histogram with hardcoded percentiles" and CONTRIBUTING asks to keep the public
surface minimal, so I matched Go. Happy to add a constructor overload if you would
rather have it configurable.
meanandstddevarefloat64in Go butintinHistogram.Values, so precisionis lost. Widening them is a public-API change that also rewrites the
MetricsPrinterTestexpectations — worth a separate issue?