Monday, January 23, 2012

JBoss, Fedora and more from Red Hat at FOSDEM 2012

A lot of folks from Red Hat are visiting FOSDEM 2012 this year.


I've listed the speakers here - I will myself talk on Saturday on RHQ.

In addition to the talks there are also stands:


You will be able to get a printed version of this list at the Fedora stand.

Saturday

WHEN

EVENT

TRACK & ROOM

SPEAKER

11:00-11:10

Welcome to the CrossDesktop Devroom

CrossDesktop  (H.1308)

Christophe Fergeau

11:00-11:05

Welcome to the Legal Issues DevRoom

Legal Issues  (AW1.125)

Richard Fontana

11:00-11:55

BoxGrinder : Grind your appliances easily

JBoss.org  (K.3.201)

Marek Goldmann

11:00-11:05

Welcome to the Free Java DevRoom

Free Java  (K.4.401)

Mark Wielaard, Andrew Haley, Andrew Hughes

12:00-12:55

Drools Planner: Planning optimization by example

JBoss.org  (K.3.201)

Geoffrey De Smet

12:30-12:55

libguestfs - tools for modifying virtual machine
disk images

Virtualization & Cloud  (Chavanne)

Richard Jones

13:00-13:25

Cloud high availability with pacemaker-cloud

Virtualization & Cloud  (Chavanne)

Pádraig Brady

13:00-13:55

Openshift

JBoss.org  (K.3.201)

Grant Shipley

14:00-14:25

The Aeolus Project

Virtualization & Cloud  (Chavanne)

Francesco Vollero

14:00-14:55

JBoss AS7 : Building JBoss AS 7 for Fedora

JBoss.org  (K.3.201)

Carlo De Wolf

15:00-15:50

Virtualization with KVM: bottom to top, past to future

Hypervisors  (Janson)

Paolo Bonzini

15:00-15:55

JBoss Forge / Arquillian: Two Missing Links in
Enterprise Java Development

JBoss.org  (K.3.201)

Koen Aers

15:00-15:25

Open Clouds with Deltacloud API

Virtualization & Cloud  (Chavanne)

Michal Fojtik

15:30-15:55

DMTF CIMI and Apache Deltacloud

Virtualization & Cloud  (Chavanne)

Marios Andreou

16:00-16:55

Infinispan: where open source, Java and in-memory
data grids converge

JBoss.org  (K.3.201)

Manik Surtani

16:15-17:00

Crossdesktop group picture

CrossDesktop  (H.1308)

Christophe Fergeau

16:30-17:00

The (possible) decline of the GPL, and what to do
about it

Legal Issues  (AW1.125)

Richard Fontana

17:00-17:55

RHQ: Recent and future developments in the RHQ
systems monitoring and management framework

JBoss.org  (K.3.201)

Heiko Rupp

17:30-18:00

Panel on Application Stores

Legal Issues  (AW1.125)

Richard Fontana

18:00-18:55

Guvernor/JBPM : Managing workflows and business
rules with Guvnor and the jBPM designer

JBoss.org  (K.3.201)

Geoffrey De Smet, Marco Rietveld

18:00-18:30

Thermostat: Taking over the Java tooling world with
Open Source Software

Free Java  (K.4.401)

Jon VanAlten, Omair Majid

18:20-18:35

PMD5: What can it do for you?

Lightning Talks (Ferrer)

Romain PELISSE

18:30-19:00

Tracing, Debugging and Testing With Byteman

Free Java  (K.4.401)

Andrew Dinn

Sunday

WHEN

EVENT

TRACK & ROOM

SPEAKER

09:00-09:25

Spice "Open remote computing" introduction

Virtualization & Cloud  (Chavanne)

Hans de Goede

09:30-09:55

USB redirection over the network

Virtualization & Cloud  (Chavanne)

Hans de Goede

10:00-10:45

Systems Management with Matahari

Configuration & Systems Management  (K.3.601)

Zane Bitter

10:45-11:15

Boxes, use other systems with ease

CrossDesktop  (H.1308)

Zeeshan Ali (Khattak), Marc-André Lureau

11:00-11:15

Powerful tools for Linux C/C++ developers based on
Eclipse

Lightning Talks (Ferrer)

Andrew Overholt

11:00-11:25

Virtualization Management the oVirt way

Virtualization & Cloud  (Chavanne)

Itamar Heim

11:30-11:55

oVirt Engine Core: Internals and Infrastructure

Virtualization & Cloud  (Chavanne)

Omer Frenkel

11:30-12:00

Can I legally do that?

Free Java  (K.4.401)

Mark Wielaard

12:00-12:25

VDSM - The oVirt Node Management Agent

Virtualization & Cloud  (Chavanne)

Federico Simoncelli

12:30-13:30

OpenJDK on ARM: Quo vadis?

Free Java  (K.4.401)

Andrew Haley

15:00-15:55

Building app sandboxes on top of LXC and KVM with
libvirt

Virtualization & Cloud  (Chavanne)

Daniel Berrange

15:30-16:00

IcedTea and IcedTea-Web

Free Java  (K.4.401)

Deepak Bhole

16:30-17:00

Discussion on the Future of Free Java

Free Java  (K.4.401)

Andrew Haley

Sunday, January 15, 2012

Pushing metrics / baselines via REST interface in RHQ

A few days ago I wrote about pulling raw metrics from RHQ via the REST interface.

It is also possible to push metrics as well as read and write baselines.

Pushing metrics


There are two ways to push numeric metric values into the server:

Single metric


curl -i -u rhqadmin:rhqadmin \
http://localhost:7080/rest/1/metric/data/10022/raw/1324656971 \
-X PUT \
-d @/tmp/foo \
-HContent-Type:application/json

With /tmp/foo containing:
 
{"timeStamp": 132465716178, "value": 123, "scheduleId":10023}

Note that you need to give the schedule id (10022) and the timestamp in the URL. The samples-project also contains an example in Python.

Multiple metrics



If you want to push multiple metrics at once you can use this call (of course it will also work for a single one):


curl -u rhqadmin:rhqadmin \
http://localhost:7080/rest/1/metric/data/raw \
-X POST \
-d @/tmp/foo \
-HContent-Type:application/json

with /tmp/foo containing:

[
{"timeStamp": 132465716078, "value": 123, "scheduleId":10022},
{"timeStamp": 132465716079, "value": 223, "scheduleId":10022}
]

Baselines



Baselines are an interesting feature in the sense that they mark a band in which a dynamic metric usually oscillates. When the metric goes out of those bounds, you can get an alarm. The system usually computes those baselines by taking the minimum and maximum values from the existing dataset for the last n days (n is configurable). When a baseline is computed, they are also displayed on the large metric graphs. Here the baselines could also be manually set.

It is now (master/RHQ 4.3) possible to set the computation frequency to 0 to disable the automatic calculation by the server and push baseline data via the REST interface into the server. This allows to e.g. read metrics from a system like R, compute projections for future bands (e.g. via Holt-Winters) or x% quantiles of the existing data and write the results back as baseline data so that the normal alerting workflow can pick up that data.

The first call is to obtain the baseline for schedule with id 10013:

curl -u rhqadmin:rhqadmin \
http://localhost:7080/rest/1/metric/data/10013/baseline \
-HContent-Type:application/json


This call updates the schedule 10013 with new values:

curl -u rhqadmin:rhqadmin \
http://localhost:7080/rest/1/metric/data/10013/baseline \
-HContent-Type:application/json \
-HAccept:application/json \
-X PUT \
-d '{"max":2.58304512E9, \
"min":0.119968768E9, \
"mean":1.285011894659459E9, \
"computeTime":1326477607296}'

Saturday, January 14, 2012

Polyglot management of a secured AS7

JBossAS7 comes with a nice management interface that tools like the built-in admin-console or the console app are using. Next to the "more binary" DMR protocol, there is also a JSON interface available that can be accessed via http. Using this interface allows to manage AS7 from any programing language.
Luckily :-) this interface is secured by default and only accessible for a valid user via http digest authentication.

Set up admin user



The first step is to enable a user on the server to use for this management interface:


$ cd /jboss-as-7.1.0
$ bin/add-user.sh

Enter the details of the new user to add.
Realm (ManagementRealm) : <press enter>
Username : heiko
Password : <provide password>
Re-enter Password : <provide password again>
About to add user 'user' for realm 'ManagementRealm'
Is this correct yes/no? yes
Added user 'user' to file '/jboss-as-7.1.0/standalone/configuration/mgmt-users.properties'


Now we have created a user 'heiko' with password 'okieh'.

Shell with curl



The following command with shut down the server via the management interface:

$ curl --digest -u heiko http://localhost:9990/management/ -d '{"operation":"shutdown" }' -HContent-Type:application/json


Note that for option '-u' only the username is given — curl will ask for the password. One important part here is that to mark the content-type of the data sent as "application/json". Curl will, if this header is not provided, send the request as 'application/x-www-form-urlencoded' which is disallowed by AS7.

If you run curl with option '-v' you can nicely see the re-negotiation to acquire the nonce from the server in order to compute the digest:

$ curl -v --digest -u heiko http://localhost:9990/management/ -d '{"operation":"shutdown" }' -HContent-Type:application/json
Enter host password for user 'heiko': <okieh>
* About to connect() to localhost port 9990 (#0)
* Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 9990 (#0)
* Server auth using Digest with user 'heiko'
> POST /management/ HTTP/1.1
> User-Agent: curl/7.21.7 (x86_64-apple-darwin10.8.0) libcurl/7.21.7 OpenSSL/1.0.0e zlib/1.2.5 libidn/1.22
> Host: localhost:9990
> Accept: */*
> Content-Type:application/json
> Content-Length: 0
>
< HTTP/1.1 401 Unauthorized
< Content-length: 0
< Www-authenticate: Digest realm="ManagementRealm",nonce="6089edca29aa27b064aa1db42d9651eb"
< Date: Fri, 13 Jan 2012 09:54:39 GMT
<


First request has been sent and the server replied with a 401 unauthorized and the nonce to use. Now curl continues:


* Connection #0 to host localhost left intact
* Issue another request to this URL: 'http://localhost:9990/management/'
* Re-using existing connection! (#0) with host localhost
* Connected to localhost (127.0.0.1) port 9990 (#0)
* Server auth using Digest with user 'heiko'
> POST /management/ HTTP/1.1
> Authorization: Digest username="heiko", realm="ManagementRealm", nonce="6089edca29aa27b064aa1db42d9651eb", uri="/management/", response="78b9546e7485b661121e34a72d2979f1"
> User-Agent: curl/7.21.7 (x86_64-apple-darwin10.8.0) libcurl/7.21.7 OpenSSL/1.0.0e zlib/1.2.5 libidn/1.22
> Host: localhost:9990
> Accept: */*
> Content-Type:application/json
> Content-Length: 25
>
< HTTP/1.1 200 OK
< Transfer-encoding: chunked
< Content-type: application/json
< Date: Fri, 13 Jan 2012 09:54:39 GMT
<
* Connection #0 to host localhost left intact
* Closing connection #0
{"outcome" : "success"}
$


So we've issued the equest and the server has shutdown. Using the same technique you can also e.g. query the port, the http-connector is listening on (which has the symbolic name of 'http'):

curl --digest -u heiko http://localhost:9990/management/ -HContent-Type:application/json  --data @-<< -EOF-
{
"operation":"read-attribute",
"address":[
{"socket-binding-group":"standard-sockets"},
{"socket-binding":"http"}
],
"name":"port"
}
-EOF-


In this example you also see how to pass the address of the node to inspect and the name of the attribute to the server.

Beware that if you make a typo in the json-encoding (e.g. separating key and value by comma instead of colon), the server may just respond with a 401 without telling you what went wrong

Perl



It's a long time since I did serious perl coding, so that next example may not be the most elegant. The example shows again, how to retrieve the http port via a 'read-attribute' operation. As I don't want to obsfuscate the code even more, I did just provide the password in the script.


#!/usr/bin/perl

use JSON qw(objToJson jsonToObj from_json to_json decode_json);
use LWP;

$host = "localhost";
$port = "9990";

$realm = "ManagementRealm";
$user = "heiko";
$password = "okieh";

# Construct url of management api
$url = "http://$host:$port/management";


# the command to send to the server in JSON encoding
$json_data = '
{
"operation":"read-attribute",
"address":[
{"socket-binding-group":"standard-sockets"},
{"socket-binding":"http"}
],
"name":"port"
}
';
# set up a User agent
my $browser = LWP::UserAgent->new();

# Create the request
my $req = HTTP::Request->new(POST => $url);
$browser->credentials("$host:$port",$realm,$user,$password);
$req->content_type( 'application/json');
$req->content($json_data);

# send the request to the server
$res = $browser->request($req);

# If we don't get a 200 back, we finish here
die "No success ", $res->status_line unless $res->is_success;

# Get the content from the response
my $seite_code = $res->content;
print "Received : $seite_code \n";

# decode the json retieved
my $json = JSON->new->utf8;
$obj = $json->decode($seite_code);
%pairs = %{$obj}; # json->decode returns a hash ref
# get the result
$httpPort = $pairs{"result"};
print "Http port is $httpPort \n";


The basic part to handle the digest authentication is $browser->credentials("$host:$port",$realm,$user,$password);, which makes LWP transparently handle the creation of the digest and re-sending of the request.

Ruby



Unlike perl, which I was using a lot in the past, I am not yet familiar with Ruby, so there may be a much better solution -- please provide some feedback. Especially I have not found a good way to automatically handle the digest authentication, so this is done explicitly


#!/opt/local/bin/ruby1.9

require 'json'
require 'net/http'
require 'net/http/digest_auth'

url = URI.parse('http://localhost:9990/management/')
url.user = 'heiko'
url.password = 'okieh'

# data to send to retrieve the server name
data = { "operation" => "read-attribute",
"address" => [],
"name" => "name"}

h = Net::HTTP.new url.host, url.port

# send first request to get nonce
req = Net::HTTP::Post.new url.request_uri
res = h.request req


So far we have sent a first request to obtain the 'nonce' from the server, so we can compute the digest in the 2n step.


# compute the digest
digest_auth = Net::HTTP::DigestAuth.new
auth = digest_auth.auth_header url, res['www-authenticate'], 'POST'

# Now send the real request with the nonce
body = JSON.generate(data)

puts "Sending " + body
req = Net::HTTP::Post.new url.request_uri
req.add_field 'Content-Type', 'application/json'
req.add_field 'Authorization', auth
req.body = body

res = h.request req

print "Result " + res.body

# parse the JSON and obtain the 'result' object
data = JSON.parse(res.body)
server_name = data["result"]
print "Server name is " + server_name

Thanks


I want to thank Darran Lofthouse for helping me to get going with why apparently correct requests fail with a 403 (because of the wrong content type).

Monday, January 09, 2012

Some graphing fun with D3.js

Now that the RHQ REST api can expose raw numerical metrics for the last 7 days, it is possible to create additional graphs for numerics. As before I have used D3.js to create the following graphs. The source is now in RHQ git master and will also be in RHQ 4.3.

The first graph shows the last 7 days of metrics in one go:
Last 7 days of metrics
raw_graph.html


This one shows the last 7 days where each span of 24h is represented by a colored line, which allows to compare the values directly - the darker the color, the older the data:
Bildschirmfoto 2012 01 09 um 11 06 05
raw_graph7.html


This needs some more work, as the x-axis labeling does not yet take the exact start time into account.
Also time spans that have no values (e.g. because agent is down) should not show a straight line, but no line at all for that period. There should perhaps be a legend about the colors as well.

I am still very much on the learning side of JavaScript and D3.js and the above is far from the beautiful examples Mike Bostock creates, but I think one can already see the potential power here.

If anyone is interested to add tooltips for the values or fix the label for the second graph, please ping me.

Monday, January 02, 2012

Analyze your RHQ metrics with R

R plot of aggregate metrics

Plot of metric aggregates in R


You probably have seen that with RHQ 4.2 you can export the aggregate metrics for the last n (default 8) hours via the REST api by calling:

http://localhost:7080/rest/1/metric/data/<scheduleId>

where <scheduleId> must be a valid numerical schedule.

The statistical tool R allows via "RCurl" to load data from remote URLs e.g. like this:

json_file <- getURL("http://localhost:7080/rest/1/metric/data/10013", httpheader=c(Accept = "application/json"),userpwd="rhqadmin:rhqadmin")


RCurl allows you to specify username and password unlike the simple json_file <- "http://..." calls. The next step is then to transform the received data with the help of the "rjson" library into R data structures:

## convert json to list of vectors
json_data <- fromJSON(paste(json_file, collapse=""))


## convert the embedded data points into a data frame
df <- data.frame(do.call(rbind,json_data$dataPoints))

which you can then access and e.g. plot:


## plot the data
plot(df$timeStamp,df$value,xlab="time",ylab="Free memory (bytes)",xaxt='n',type='l')


You can find the full example as plot_metrics.r in the RHQ samples project on GitHub.

Some like it raw...

RHQ master (this will make it into RHQ 4.3) is now also able to export raw metrics via REST api


http://localhost:7080/rest/1/metric/data/10013/raw?duration=259200


Like above you provide the schedule id and optionally a startTime and endTime or an endTime and a duration (in seconds). If nothing is provided, the data for the last 8h is exported. The following screenshot shows an example plot:

R plot of RHQ metrics


The metrics are plotted in black, the average in blue, the 5% and 95% quantils in orange and green and with the help of the TTR library, the 50 samples moving average is plotted in red. The sample code (with slightly different parameters) is also available online as plot_raw.r.

If you find other cool usages like e.g. Holt-Winters prediction on the data, please consider submitting them as example to the samples project.