Monday, January 23, 2012

JBoss, Fedora and more from Red Hat at FOSDEM 2012

A lot of folks from Red Hat are visiting FOSDEM 2012 this year.

I've listed the speakers here - I will myself talk on Saturday on RHQ.

In addition to the talks there are also stands:

You will be able to get a printed version of this list at the Fedora stand.







Welcome to the CrossDesktop Devroom

CrossDesktop  (H.1308)

Christophe Fergeau


Welcome to the Legal Issues DevRoom

Legal Issues  (AW1.125)

Richard Fontana


BoxGrinder : Grind your appliances easily  (K.3.201)

Marek Goldmann


Welcome to the Free Java DevRoom

Free Java  (K.4.401)

Mark Wielaard, Andrew Haley, Andrew Hughes


Drools Planner: Planning optimization by example  (K.3.201)

Geoffrey De Smet


libguestfs - tools for modifying virtual machine
disk images

Virtualization & Cloud  (Chavanne)

Richard Jones


Cloud high availability with pacemaker-cloud

Virtualization & Cloud  (Chavanne)

Pádraig Brady


Openshift  (K.3.201)

Grant Shipley


The Aeolus Project

Virtualization & Cloud  (Chavanne)

Francesco Vollero


JBoss AS7 : Building JBoss AS 7 for Fedora  (K.3.201)

Carlo De Wolf


Virtualization with KVM: bottom to top, past to future

Hypervisors  (Janson)

Paolo Bonzini


JBoss Forge / Arquillian: Two Missing Links in
Enterprise Java Development  (K.3.201)

Koen Aers


Open Clouds with Deltacloud API

Virtualization & Cloud  (Chavanne)

Michal Fojtik


DMTF CIMI and Apache Deltacloud

Virtualization & Cloud  (Chavanne)

Marios Andreou


Infinispan: where open source, Java and in-memory
data grids converge  (K.3.201)

Manik Surtani


Crossdesktop group picture

CrossDesktop  (H.1308)

Christophe Fergeau


The (possible) decline of the GPL, and what to do
about it

Legal Issues  (AW1.125)

Richard Fontana


RHQ: Recent and future developments in the RHQ
systems monitoring and management framework  (K.3.201)

Heiko Rupp


Panel on Application Stores

Legal Issues  (AW1.125)

Richard Fontana


Guvernor/JBPM : Managing workflows and business
rules with Guvnor and the jBPM designer  (K.3.201)

Geoffrey De Smet, Marco Rietveld


Thermostat: Taking over the Java tooling world with
Open Source Software

Free Java  (K.4.401)

Jon VanAlten, Omair Majid


PMD5: What can it do for you?

Lightning Talks (Ferrer)



Tracing, Debugging and Testing With Byteman

Free Java  (K.4.401)

Andrew Dinn







Spice "Open remote computing" introduction

Virtualization & Cloud  (Chavanne)

Hans de Goede


USB redirection over the network

Virtualization & Cloud  (Chavanne)

Hans de Goede


Systems Management with Matahari

Configuration & Systems Management  (K.3.601)

Zane Bitter


Boxes, use other systems with ease

CrossDesktop  (H.1308)

Zeeshan Ali (Khattak), Marc-André Lureau


Powerful tools for Linux C/C++ developers based on

Lightning Talks (Ferrer)

Andrew Overholt


Virtualization Management the oVirt way

Virtualization & Cloud  (Chavanne)

Itamar Heim


oVirt Engine Core: Internals and Infrastructure

Virtualization & Cloud  (Chavanne)

Omer Frenkel


Can I legally do that?

Free Java  (K.4.401)

Mark Wielaard


VDSM - The oVirt Node Management Agent

Virtualization & Cloud  (Chavanne)

Federico Simoncelli


OpenJDK on ARM: Quo vadis?

Free Java  (K.4.401)

Andrew Haley


Building app sandboxes on top of LXC and KVM with

Virtualization & Cloud  (Chavanne)

Daniel Berrange


IcedTea and IcedTea-Web

Free Java  (K.4.401)

Deepak Bhole


Discussion on the Future of Free Java

Free Java  (K.4.401)

Andrew Haley

Sunday, January 15, 2012

Pushing metrics / baselines via REST interface in RHQ

A few days ago I wrote about pulling raw metrics from RHQ via the REST interface.

It is also possible to push metrics as well as read and write baselines.

Pushing metrics

There are two ways to push numeric metric values into the server:

Single metric

curl -i -u rhqadmin:rhqadmin \
http://localhost:7080/rest/1/metric/data/10022/raw/1324656971 \
-X PUT \
-d @/tmp/foo \

With /tmp/foo containing:
{"timeStamp": 132465716178, "value": 123, "scheduleId":10023}

Note that you need to give the schedule id (10022) and the timestamp in the URL. The samples-project also contains an example in Python.

Multiple metrics

If you want to push multiple metrics at once you can use this call (of course it will also work for a single one):

curl -u rhqadmin:rhqadmin \
http://localhost:7080/rest/1/metric/data/raw \
-d @/tmp/foo \

with /tmp/foo containing:

{"timeStamp": 132465716078, "value": 123, "scheduleId":10022},
{"timeStamp": 132465716079, "value": 223, "scheduleId":10022}


Baselines are an interesting feature in the sense that they mark a band in which a dynamic metric usually oscillates. When the metric goes out of those bounds, you can get an alarm. The system usually computes those baselines by taking the minimum and maximum values from the existing dataset for the last n days (n is configurable). When a baseline is computed, they are also displayed on the large metric graphs. Here the baselines could also be manually set.

It is now (master/RHQ 4.3) possible to set the computation frequency to 0 to disable the automatic calculation by the server and push baseline data via the REST interface into the server. This allows to e.g. read metrics from a system like R, compute projections for future bands (e.g. via Holt-Winters) or x% quantiles of the existing data and write the results back as baseline data so that the normal alerting workflow can pick up that data.

The first call is to obtain the baseline for schedule with id 10013:

curl -u rhqadmin:rhqadmin \
http://localhost:7080/rest/1/metric/data/10013/baseline \

This call updates the schedule 10013 with new values:

curl -u rhqadmin:rhqadmin \
http://localhost:7080/rest/1/metric/data/10013/baseline \
-HContent-Type:application/json \
-HAccept:application/json \
-X PUT \
-d '{"max":2.58304512E9, \
"min":0.119968768E9, \
"mean":1.285011894659459E9, \

Saturday, January 14, 2012

Polyglot management of a secured AS7

JBossAS7 comes with a nice management interface that tools like the built-in admin-console or the console app are using. Next to the "more binary" DMR protocol, there is also a JSON interface available that can be accessed via http. Using this interface allows to manage AS7 from any programing language.
Luckily :-) this interface is secured by default and only accessible for a valid user via http digest authentication.

Set up admin user

The first step is to enable a user on the server to use for this management interface:

$ cd /jboss-as-7.1.0
$ bin/

Enter the details of the new user to add.
Realm (ManagementRealm) : <press enter>
Username : heiko
Password : <provide password>
Re-enter Password : <provide password again>
About to add user 'user' for realm 'ManagementRealm'
Is this correct yes/no? yes
Added user 'user' to file '/jboss-as-7.1.0/standalone/configuration/'

Now we have created a user 'heiko' with password 'okieh'.

Shell with curl

The following command with shut down the server via the management interface:

$ curl --digest -u heiko http://localhost:9990/management/ -d '{"operation":"shutdown" }' -HContent-Type:application/json

Note that for option '-u' only the username is given — curl will ask for the password. One important part here is that to mark the content-type of the data sent as "application/json". Curl will, if this header is not provided, send the request as 'application/x-www-form-urlencoded' which is disallowed by AS7.

If you run curl with option '-v' you can nicely see the re-negotiation to acquire the nonce from the server in order to compute the digest:

$ curl -v --digest -u heiko http://localhost:9990/management/ -d '{"operation":"shutdown" }' -HContent-Type:application/json
Enter host password for user 'heiko': <okieh>
* About to connect() to localhost port 9990 (#0)
* Trying connected
* Connected to localhost ( port 9990 (#0)
* Server auth using Digest with user 'heiko'
> POST /management/ HTTP/1.1
> User-Agent: curl/7.21.7 (x86_64-apple-darwin10.8.0) libcurl/7.21.7 OpenSSL/1.0.0e zlib/1.2.5 libidn/1.22
> Host: localhost:9990
> Accept: */*
> Content-Type:application/json
> Content-Length: 0
< HTTP/1.1 401 Unauthorized
< Content-length: 0
< Www-authenticate: Digest realm="ManagementRealm",nonce="6089edca29aa27b064aa1db42d9651eb"
< Date: Fri, 13 Jan 2012 09:54:39 GMT

First request has been sent and the server replied with a 401 unauthorized and the nonce to use. Now curl continues:

* Connection #0 to host localhost left intact
* Issue another request to this URL: 'http://localhost:9990/management/'
* Re-using existing connection! (#0) with host localhost
* Connected to localhost ( port 9990 (#0)
* Server auth using Digest with user 'heiko'
> POST /management/ HTTP/1.1
> Authorization: Digest username="heiko", realm="ManagementRealm", nonce="6089edca29aa27b064aa1db42d9651eb", uri="/management/", response="78b9546e7485b661121e34a72d2979f1"
> User-Agent: curl/7.21.7 (x86_64-apple-darwin10.8.0) libcurl/7.21.7 OpenSSL/1.0.0e zlib/1.2.5 libidn/1.22
> Host: localhost:9990
> Accept: */*
> Content-Type:application/json
> Content-Length: 25
< HTTP/1.1 200 OK
< Transfer-encoding: chunked
< Content-type: application/json
< Date: Fri, 13 Jan 2012 09:54:39 GMT
* Connection #0 to host localhost left intact
* Closing connection #0
{"outcome" : "success"}

So we've issued the equest and the server has shutdown. Using the same technique you can also e.g. query the port, the http-connector is listening on (which has the symbolic name of 'http'):

curl --digest -u heiko http://localhost:9990/management/ -HContent-Type:application/json  --data @-<< -EOF-

In this example you also see how to pass the address of the node to inspect and the name of the attribute to the server.

Beware that if you make a typo in the json-encoding (e.g. separating key and value by comma instead of colon), the server may just respond with a 401 without telling you what went wrong


It's a long time since I did serious perl coding, so that next example may not be the most elegant. The example shows again, how to retrieve the http port via a 'read-attribute' operation. As I don't want to obsfuscate the code even more, I did just provide the password in the script.


use JSON qw(objToJson jsonToObj from_json to_json decode_json);
use LWP;

$host = "localhost";
$port = "9990";

$realm = "ManagementRealm";
$user = "heiko";
$password = "okieh";

# Construct url of management api
$url = "http://$host:$port/management";

# the command to send to the server in JSON encoding
$json_data = '
# set up a User agent
my $browser = LWP::UserAgent->new();

# Create the request
my $req = HTTP::Request->new(POST => $url);
$req->content_type( 'application/json');

# send the request to the server
$res = $browser->request($req);

# If we don't get a 200 back, we finish here
die "No success ", $res->status_line unless $res->is_success;

# Get the content from the response
my $seite_code = $res->content;
print "Received : $seite_code \n";

# decode the json retieved
my $json = JSON->new->utf8;
$obj = $json->decode($seite_code);
%pairs = %{$obj}; # json->decode returns a hash ref
# get the result
$httpPort = $pairs{"result"};
print "Http port is $httpPort \n";

The basic part to handle the digest authentication is $browser->credentials("$host:$port",$realm,$user,$password);, which makes LWP transparently handle the creation of the digest and re-sending of the request.


Unlike perl, which I was using a lot in the past, I am not yet familiar with Ruby, so there may be a much better solution -- please provide some feedback. Especially I have not found a good way to automatically handle the digest authentication, so this is done explicitly


require 'json'
require 'net/http'
require 'net/http/digest_auth'

url = URI.parse('http://localhost:9990/management/')
url.user = 'heiko'
url.password = 'okieh'

# data to send to retrieve the server name
data = { "operation" => "read-attribute",
"address" => [],
"name" => "name"}

h =, url.port

# send first request to get nonce
req = url.request_uri
res = h.request req

So far we have sent a first request to obtain the 'nonce' from the server, so we can compute the digest in the 2n step.

# compute the digest
digest_auth =
auth = digest_auth.auth_header url, res['www-authenticate'], 'POST'

# Now send the real request with the nonce
body = JSON.generate(data)

puts "Sending " + body
req = url.request_uri
req.add_field 'Content-Type', 'application/json'
req.add_field 'Authorization', auth
req.body = body

res = h.request req

print "Result " + res.body

# parse the JSON and obtain the 'result' object
data = JSON.parse(res.body)
server_name = data["result"]
print "Server name is " + server_name


I want to thank Darran Lofthouse for helping me to get going with why apparently correct requests fail with a 403 (because of the wrong content type).

Monday, January 09, 2012

Some graphing fun with D3.js

Now that the RHQ REST api can expose raw numerical metrics for the last 7 days, it is possible to create additional graphs for numerics. As before I have used D3.js to create the following graphs. The source is now in RHQ git master and will also be in RHQ 4.3.

The first graph shows the last 7 days of metrics in one go:
Last 7 days of metrics

This one shows the last 7 days where each span of 24h is represented by a colored line, which allows to compare the values directly - the darker the color, the older the data:
Bildschirmfoto 2012 01 09 um 11 06 05

This needs some more work, as the x-axis labeling does not yet take the exact start time into account.
Also time spans that have no values (e.g. because agent is down) should not show a straight line, but no line at all for that period. There should perhaps be a legend about the colors as well.

I am still very much on the learning side of JavaScript and D3.js and the above is far from the beautiful examples Mike Bostock creates, but I think one can already see the potential power here.

If anyone is interested to add tooltips for the values or fix the label for the second graph, please ping me.

Monday, January 02, 2012

Analyze your RHQ metrics with R

R plot of aggregate metrics

Plot of metric aggregates in R

You probably have seen that with RHQ 4.2 you can export the aggregate metrics for the last n (default 8) hours via the REST api by calling:


where <scheduleId> must be a valid numerical schedule.

The statistical tool R allows via "RCurl" to load data from remote URLs e.g. like this:

json_file <- getURL("http://localhost:7080/rest/1/metric/data/10013", httpheader=c(Accept = "application/json"),userpwd="rhqadmin:rhqadmin")

RCurl allows you to specify username and password unlike the simple json_file <- "http://..." calls. The next step is then to transform the received data with the help of the "rjson" library into R data structures:

## convert json to list of vectors
json_data <- fromJSON(paste(json_file, collapse=""))

## convert the embedded data points into a data frame
df <- data.frame(,json_data$dataPoints))

which you can then access and e.g. plot:

## plot the data
plot(df$timeStamp,df$value,xlab="time",ylab="Free memory (bytes)",xaxt='n',type='l')

You can find the full example as plot_metrics.r in the RHQ samples project on GitHub.

Some like it raw...

RHQ master (this will make it into RHQ 4.3) is now also able to export raw metrics via REST api


Like above you provide the schedule id and optionally a startTime and endTime or an endTime and a duration (in seconds). If nothing is provided, the data for the last 8h is exported. The following screenshot shows an example plot:

R plot of RHQ metrics

The metrics are plotted in black, the average in blue, the 5% and 95% quantils in orange and green and with the help of the TTR library, the 50 samples moving average is plotted in red. The sample code (with slightly different parameters) is also available online as plot_raw.r.

If you find other cool usages like e.g. Holt-Winters prediction on the data, please consider submitting them as example to the samples project.