Computed metrics are something I wanted to do for a very long time, already in RHQ, but never really got around it and sort of forgot about it again.
Lately I found a post that contained a DSL to do exactly this (actually you should read that post not because of the DSL, but because of the idea behind it).
After seeing this, I got the idea on what to do and to include this in HawkFX, my pet project, which is an explorer for Hawkular.
HawkFx with the input window for formulae, that shows a formula and also a parser error.
The orange chart shows Non-Heap used, the redish one the heap usage of a JVM.
Formulae
Formulas are in a DSL that looks a bit like UPN, e.g. as in the following (I've shortened the metric ID for readability, more on them below):
(+ metric( "MI~...Heap Used" , "max") metric( "MI~...NonHeap Used", "max"))
to sum up two metrics (see also screenshot below). The 'metric' element gets two parameters, the metric id and also which of the aggregates that the server sends should be taken (in this case the max value) - this comes from the fact that we request the values to be put into 120 buckets by the server.
Or if you have the total amount of memory you could also subtract the used memory to get a graph of the remaining:
(- 1000000 metric( "MI~...NonHeap Used", "max"))
You could also get the total wait time for responses at a point in time when you multiply the average wait time with the number of visitors:
(* metric("MI~..ResponseTime","avg")
metric("MI~..NumberVisitors","sum"))
Computed total memory usage
Summing up the metrics for 'Heap Used' and 'NonHeap Used' as shown above would then give you a nice graph of the total memory consumption of a JVM:
The green chart now shows to combined memory usage of Heap and Non-Heap, which is computed from the other two series. Orange and red are as above.
On metric IDs
Metric IDs are the IDs under which a metric is stored inside of Hawkular. The example here comes from an installation of Hawkular-services in Docker. If you just feed your metrics into Hawkular metrics, the IDs will looks like the ones you are using.
ID (upper) and path fields (lower) for a selected item in the tree
I have just pushed an update to HawkFX that provides the ID and path in their own fields at the bottom of the main window, so you can copy&paste them.
Future
I will talk more about the parser in an upcoming article. For now it is a personal playground to also better understand what is doable here. If this turns out to be successful I can imagine that the DSL could directly be incorporated into Hawkular-metrics so that the rules are available to all metrics clients.
It would of course be cool to have an editor for the formulas that allows to interactively pick metric IDs etc, but I doubt that I will get to this any time soon.
