If you speak Russian and want to help Denis, I am sure he will appreciate it. Check out the translation project on github for details.
and
Thank you Denis!
Wednesday, April 25, 2012
Russian translations coming to RHQ
Denis Krusko, one of the two GSoC students working on RHQ has just committed a first batch of Russian translations to RHQ. Here are two screenshots to bring you in the right mood.
If you speak Russian and want to help Denis, I am sure he will appreciate it. Check out the translation project on github for details.
and
Thank you Denis!
If you speak Russian and want to help Denis, I am sure he will appreciate it. Check out the translation project on github for details.
and
Thank you Denis!
Monday, April 23, 2012
RHQ participates in this years Google Summer of Code™
RHQ is happy to be able to participate in this years Google Summer of Code™We have as part of the JBoss.org organization seen many very good proposals and the whole JBoss.org has seen many more so that even with the generosity of Google, we have obviously obtained a lot less slots than the number of good proposals received.
So I am extremely happy that for RHQ those two proposals have been accepted:
- Replace old graphs by GWT ones - Denis Krusko: The main graphs in RHQ are still from the pre RHQ era and implemented as Servlets embedded in JSP and struts pages. The current UI is mostly written in GWT. While we were able to embed the old graphs, they still don't feel 'right'.
Denis will investigate options for replacement and then implement new GWT-compatible graphing. Denis will also look at how the grahps can become more interactive by e.g. applying formulas on the data. - Implement an RHQ agent in Python - Krzysztof Kwaśniewski: The classical RHQ agent is written in Java and probably not best suited for every purpose. With the addition of the REST api, it now became easier to implement agents in other languages.
Krzysiek will implement an agent in Python, that on one side talks REST with the RHQ server and on the other side interfaces with Matahari to take metrics from Fedora and RHEL hosts.
I am together with the other RHQ contributors very much looking forward to see Krzysztof and Denis in action.
Heiko
*) GSoC Logo is taken from http://code.google.com/p/google-summer-of-code/wiki/GsocLogos and has a CC-3.0-attrib-non-commercial-no-deratie license
Monday, February 13, 2012
IDEA: find by XPath
I like the idea of XPath since I've looked at it over 10 years ago. Today I came across a usage that is über cool. I had the need to find some pattern of data across a bunch of xml files, where a there is a list of maps where the list and maps are marked as read-only like in this example:
Luckily IDEA offers "Find by XPath"
So I could type in my expression and have it search through the whole project.
This looked good, but did not work. Then I remembered the little 'c:' and clicked on the "Edit context..." button, which allowed to set the correct namespace for 'c:':
Now with the namespace correctly set, the search ran for a few seconds and found the occurrences all across my files. /me is a happy user.
<c:list-property name="*" displayName="Installed extensions" readOnly="true" required="false">
<c:map-property name="*" displayName="Name" readOnly="true">
<c:simple-property name="module" displayName="Module name" readOnly="true"/>
</c:map-property>
</c:list-property>
Luckily IDEA offers "Find by XPath"
So I could type in my expression and have it search through the whole project.
This looked good, but did not work. Then I remembered the little 'c:' and clicked on the "Edit context..." button, which allowed to set the correct namespace for 'c:':
Now with the namespace correctly set, the search ran for a few seconds and found the occurrences all across my files. /me is a happy user.
Sunday, February 12, 2012
Small Fosdem 2012 review
On the weekend of Feb 4th, 2012 the European (and not only) Free and Open source community met in Brussels, Belgium for FOSDEM (Free and Open Source Developers Meeting).
The following video gives an impression on what FOSDEM is like - and I was only able to capture a tiny part of it.
(original size is 960x540)
I only arrived on Saturday at around noon via train. Stuttgart to Cologne was via ICE and from Cologne on I took the Thalys where I had the luck to get a a first class ticket for the price of a regular 2nd class one; the ticket included free WiFi and free breakfast with croissants, sandwiches and beverages.
I gave a talk on RHQ "Recent and future developments". In the presentation I talked about what we have achieved recently and where we may perhaps go in the future.
(original size is 960x540, here are slides as pdf)
As I had the camera with me, I also recorded a few other talks
I was lucky to meet with some of the JBoss colleagues, but was not able to make it to the virt sessions and to meet more of the various Red Hat people that were there. Fortunately some of the talks were recorded by the FOSDEM crew and will hopefully be online soon.
FOSDEM is for sure a conference to go to -- and not only because entrance is free and there is beer sold all over the place :)
Monday, January 23, 2012
JBoss, Fedora and more from Red Hat at FOSDEM 2012
A lot of folks from Red Hat are visiting FOSDEM 2012 this year.
I've listed the speakers here - I will myself talk on Saturday on RHQ.
In addition to the talks there are also stands:
You will be able to get a printed version of this list at the Fedora stand.
I've listed the speakers here - I will myself talk on Saturday on RHQ.
In addition to the talks there are also stands:
- JBoss: K building, 1st floor
- Fedora: K building, ground floor
You will be able to get a printed version of this list at the Fedora stand.
Saturday
WHEN | EVENT | TRACK & ROOM | SPEAKER |
11:00-11:10 | CrossDesktop (H.1308) | Christophe Fergeau | |
11:00-11:05 | Legal Issues (AW1.125) | Richard Fontana | |
11:00-11:55 | JBoss.org (K.3.201) | Marek Goldmann | |
11:00-11:05 | Free Java (K.4.401) | Mark Wielaard, Andrew Haley, Andrew Hughes | |
12:00-12:55 | JBoss.org (K.3.201) | Geoffrey De Smet | |
12:30-12:55 | libguestfs - tools for modifying virtual machine | Virtualization & Cloud (Chavanne) | Richard Jones |
13:00-13:25 | Virtualization & Cloud (Chavanne) | Pádraig Brady | |
13:00-13:55 | JBoss.org (K.3.201) | Grant Shipley | |
14:00-14:25 | Virtualization & Cloud (Chavanne) | Francesco Vollero | |
14:00-14:55 | JBoss.org (K.3.201) | Carlo De Wolf | |
15:00-15:50 | Hypervisors (Janson) | Paolo Bonzini | |
15:00-15:55 | JBoss Forge / Arquillian: Two Missing Links in | JBoss.org (K.3.201) | Koen Aers |
15:00-15:25 | Virtualization & Cloud (Chavanne) | Michal Fojtik | |
15:30-15:55 | Virtualization & Cloud (Chavanne) | Marios Andreou | |
16:00-16:55 | Infinispan: where open source, Java and in-memory | JBoss.org (K.3.201) | Manik Surtani |
16:15-17:00 | CrossDesktop (H.1308) | Christophe Fergeau | |
16:30-17:00 | Legal Issues (AW1.125) | Richard Fontana | |
17:00-17:55 | RHQ: Recent and future developments in the RHQ | JBoss.org (K.3.201) | Heiko Rupp |
17:30-18:00 | Legal Issues (AW1.125) | Richard Fontana | |
18:00-18:55 | Guvernor/JBPM : Managing workflows and business | JBoss.org (K.3.201) | Geoffrey De Smet, Marco Rietveld |
18:00-18:30 | Thermostat: Taking over the Java tooling world with | Free Java (K.4.401) | Jon VanAlten, Omair Majid |
18:20-18:35 | Lightning Talks (Ferrer) | Romain PELISSE | |
18:30-19:00 | Free Java (K.4.401) | Andrew Dinn |
Sunday
WHEN | EVENT | TRACK & ROOM | SPEAKER |
09:00-09:25 | Virtualization & Cloud (Chavanne) | Hans de Goede | |
09:30-09:55 | Virtualization & Cloud (Chavanne) | Hans de Goede | |
10:00-10:45 | Configuration & Systems Management (K.3.601) | Zane Bitter | |
10:45-11:15 | CrossDesktop (H.1308) | Zeeshan Ali (Khattak), Marc-André Lureau | |
11:00-11:15 | Lightning Talks (Ferrer) | Andrew Overholt | |
11:00-11:25 | Virtualization & Cloud (Chavanne) | Itamar Heim | |
11:30-11:55 | Virtualization & Cloud (Chavanne) | Omer Frenkel | |
11:30-12:00 | Free Java (K.4.401) | Mark Wielaard | |
12:00-12:25 | Virtualization & Cloud (Chavanne) | Federico Simoncelli | |
12:30-13:30 | Free Java (K.4.401) | Andrew Haley | |
15:00-15:55 | Virtualization & Cloud (Chavanne) | Daniel Berrange | |
15:30-16:00 | Free Java (K.4.401) | Deepak Bhole | |
16:30-17:00 | Free Java (K.4.401) | Andrew Haley |
Sunday, January 15, 2012
Pushing metrics / baselines via REST interface in RHQ
A few days ago I wrote about pulling raw metrics from RHQ via the REST interface.
It is also possible to push metrics as well as read and write baselines.
There are two ways to push numeric metric values into the server:
With /tmp/foo containing:
Note that you need to give the schedule id (10022) and the timestamp in the URL. The samples-project also contains an example in Python.
If you want to push multiple metrics at once you can use this call (of course it will also work for a single one):
with /tmp/foo containing:
Baselines are an interesting feature in the sense that they mark a band in which a dynamic metric usually oscillates. When the metric goes out of those bounds, you can get an alarm. The system usually computes those baselines by taking the minimum and maximum values from the existing dataset for the last n days (n is configurable). When a baseline is computed, they are also displayed on the large metric graphs. Here the baselines could also be manually set.
It is now (master/RHQ 4.3) possible to set the computation frequency to 0 to disable the automatic calculation by the server and push baseline data via the REST interface into the server. This allows to e.g. read metrics from a system like R, compute projections for future bands (e.g. via Holt-Winters) or x% quantiles of the existing data and write the results back as baseline data so that the normal alerting workflow can pick up that data.
The first call is to obtain the baseline for schedule with id 10013:
This call updates the schedule 10013 with new values:
It is also possible to push metrics as well as read and write baselines.
Pushing metrics
There are two ways to push numeric metric values into the server:
Single metric
curl -i -u rhqadmin:rhqadmin \
http://localhost:7080/rest/1/metric/data/10022/raw/1324656971 \
-X PUT \
-d @/tmp/foo \
-HContent-Type:application/json
With /tmp/foo containing:
{"timeStamp": 132465716178, "value": 123, "scheduleId":10023}
Note that you need to give the schedule id (10022) and the timestamp in the URL. The samples-project also contains an example in Python.
Multiple metrics
If you want to push multiple metrics at once you can use this call (of course it will also work for a single one):
curl -u rhqadmin:rhqadmin \
http://localhost:7080/rest/1/metric/data/raw \
-X POST \
-d @/tmp/foo \
-HContent-Type:application/json
with /tmp/foo containing:
[
{"timeStamp": 132465716078, "value": 123, "scheduleId":10022},
{"timeStamp": 132465716079, "value": 223, "scheduleId":10022}
]
Baselines
Baselines are an interesting feature in the sense that they mark a band in which a dynamic metric usually oscillates. When the metric goes out of those bounds, you can get an alarm. The system usually computes those baselines by taking the minimum and maximum values from the existing dataset for the last n days (n is configurable). When a baseline is computed, they are also displayed on the large metric graphs. Here the baselines could also be manually set.
It is now (master/RHQ 4.3) possible to set the computation frequency to 0 to disable the automatic calculation by the server and push baseline data via the REST interface into the server. This allows to e.g. read metrics from a system like R, compute projections for future bands (e.g. via Holt-Winters) or x% quantiles of the existing data and write the results back as baseline data so that the normal alerting workflow can pick up that data.
The first call is to obtain the baseline for schedule with id 10013:
curl -u rhqadmin:rhqadmin \
http://localhost:7080/rest/1/metric/data/10013/baseline \
-HContent-Type:application/json
This call updates the schedule 10013 with new values:
curl -u rhqadmin:rhqadmin \
http://localhost:7080/rest/1/metric/data/10013/baseline \
-HContent-Type:application/json \
-HAccept:application/json \
-X PUT \
-d '{"max":2.58304512E9, \
"min":0.119968768E9, \
"mean":1.285011894659459E9, \
"computeTime":1326477607296}'
Saturday, January 14, 2012
Polyglot management of a secured AS7
JBossAS7 comes with a nice management interface that tools like the built-in admin-console or the console app are using. Next to the "more binary" DMR protocol, there is also a JSON interface available that can be accessed via http. Using this interface allows to manage AS7 from any programing language.
Luckily :-) this interface is secured by default and only accessible for a valid user via http digest authentication.
The first step is to enable a user on the server to use for this management interface:
Now we have created a user 'heiko' with password 'okieh'.
The following command with shut down the server via the management interface:
Note that for option '-u' only the username is given — curl will ask for the password. One important part here is that to mark the content-type of the data sent as "application/json". Curl will, if this header is not provided, send the request as 'application/x-www-form-urlencoded' which is disallowed by AS7.
If you run curl with option '-v' you can nicely see the re-negotiation to acquire the nonce from the server in order to compute the digest:
First request has been sent and the server replied with a 401 unauthorized and the nonce to use. Now curl continues:
So we've issued the equest and the server has shutdown. Using the same technique you can also e.g. query the port, the http-connector is listening on (which has the symbolic name of 'http'):
In this example you also see how to pass the address of the node to inspect and the name of the attribute to the server.
Beware that if you make a typo in the json-encoding (e.g. separating key and value by comma instead of colon), the server may just respond with a 401 without telling you what went wrong
It's a long time since I did serious perl coding, so that next example may not be the most elegant. The example shows again, how to retrieve the http port via a 'read-attribute' operation. As I don't want to obsfuscate the code even more, I did just provide the password in the script.
The basic part to handle the digest authentication is
Unlike perl, which I was using a lot in the past, I am not yet familiar with Ruby, so there may be a much better solution -- please provide some feedback. Especially I have not found a good way to automatically handle the digest authentication, so this is done explicitly
So far we have sent a first request to obtain the 'nonce' from the server, so we can compute the digest in the 2n step.
I want to thank Darran Lofthouse for helping me to get going with why apparently correct requests fail with a 403 (because of the wrong content type).
Luckily :-) this interface is secured by default and only accessible for a valid user via http digest authentication.
Set up admin user
The first step is to enable a user on the server to use for this management interface:
$ cd /jboss-as-7.1.0
$ bin/add-user.sh
Enter the details of the new user to add.
Realm (ManagementRealm) : <press enter>
Username : heiko
Password : <provide password>
Re-enter Password : <provide password again>
About to add user 'user' for realm 'ManagementRealm'
Is this correct yes/no? yes
Added user 'user' to file '/jboss-as-7.1.0/standalone/configuration/mgmt-users.properties'
Now we have created a user 'heiko' with password 'okieh'.
Shell with curl
The following command with shut down the server via the management interface:
$ curl --digest -u heiko http://localhost:9990/management/ -d '{"operation":"shutdown" }' -HContent-Type:application/json
Note that for option '-u' only the username is given — curl will ask for the password. One important part here is that to mark the content-type of the data sent as "application/json". Curl will, if this header is not provided, send the request as 'application/x-www-form-urlencoded' which is disallowed by AS7.
If you run curl with option '-v' you can nicely see the re-negotiation to acquire the nonce from the server in order to compute the digest:
$ curl -v --digest -u heiko http://localhost:9990/management/ -d '{"operation":"shutdown" }' -HContent-Type:application/json
Enter host password for user 'heiko': <okieh>
* About to connect() to localhost port 9990 (#0)
* Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 9990 (#0)
* Server auth using Digest with user 'heiko'
> POST /management/ HTTP/1.1
> User-Agent: curl/7.21.7 (x86_64-apple-darwin10.8.0) libcurl/7.21.7 OpenSSL/1.0.0e zlib/1.2.5 libidn/1.22
> Host: localhost:9990
> Accept: */*
> Content-Type:application/json
> Content-Length: 0
>
< HTTP/1.1 401 Unauthorized
< Content-length: 0
< Www-authenticate: Digest realm="ManagementRealm",nonce="6089edca29aa27b064aa1db42d9651eb"
< Date: Fri, 13 Jan 2012 09:54:39 GMT
<
First request has been sent and the server replied with a 401 unauthorized and the nonce to use. Now curl continues:
* Connection #0 to host localhost left intact
* Issue another request to this URL: 'http://localhost:9990/management/'
* Re-using existing connection! (#0) with host localhost
* Connected to localhost (127.0.0.1) port 9990 (#0)
* Server auth using Digest with user 'heiko'
> POST /management/ HTTP/1.1
> Authorization: Digest username="heiko", realm="ManagementRealm", nonce="6089edca29aa27b064aa1db42d9651eb", uri="/management/", response="78b9546e7485b661121e34a72d2979f1"
> User-Agent: curl/7.21.7 (x86_64-apple-darwin10.8.0) libcurl/7.21.7 OpenSSL/1.0.0e zlib/1.2.5 libidn/1.22
> Host: localhost:9990
> Accept: */*
> Content-Type:application/json
> Content-Length: 25
>
< HTTP/1.1 200 OK
< Transfer-encoding: chunked
< Content-type: application/json
< Date: Fri, 13 Jan 2012 09:54:39 GMT
<
* Connection #0 to host localhost left intact
* Closing connection #0
{"outcome" : "success"}
$
So we've issued the equest and the server has shutdown. Using the same technique you can also e.g. query the port, the http-connector is listening on (which has the symbolic name of 'http'):
curl --digest -u heiko http://localhost:9990/management/ -HContent-Type:application/json --data @-<< -EOF-
{
"operation":"read-attribute",
"address":[
{"socket-binding-group":"standard-sockets"},
{"socket-binding":"http"}
],
"name":"port"
}
-EOF-
In this example you also see how to pass the address of the node to inspect and the name of the attribute to the server.
Beware that if you make a typo in the json-encoding (e.g. separating key and value by comma instead of colon), the server may just respond with a 401 without telling you what went wrong
Perl
It's a long time since I did serious perl coding, so that next example may not be the most elegant. The example shows again, how to retrieve the http port via a 'read-attribute' operation. As I don't want to obsfuscate the code even more, I did just provide the password in the script.
#!/usr/bin/perl
use JSON qw(objToJson jsonToObj from_json to_json decode_json);
use LWP;
$host = "localhost";
$port = "9990";
$realm = "ManagementRealm";
$user = "heiko";
$password = "okieh";
# Construct url of management api
$url = "http://$host:$port/management";
# the command to send to the server in JSON encoding
$json_data = '
{
"operation":"read-attribute",
"address":[
{"socket-binding-group":"standard-sockets"},
{"socket-binding":"http"}
],
"name":"port"
}
';
# set up a User agent
my $browser = LWP::UserAgent->new();
# Create the request
my $req = HTTP::Request->new(POST => $url);
$browser->credentials("$host:$port",$realm,$user,$password);
$req->content_type( 'application/json');
$req->content($json_data);
# send the request to the server
$res = $browser->request($req);
# If we don't get a 200 back, we finish here
die "No success ", $res->status_line unless $res->is_success;
# Get the content from the response
my $seite_code = $res->content;
print "Received : $seite_code \n";
# decode the json retieved
my $json = JSON->new->utf8;
$obj = $json->decode($seite_code);
%pairs = %{$obj}; # json->decode returns a hash ref
# get the result
$httpPort = $pairs{"result"};
print "Http port is $httpPort \n";
The basic part to handle the digest authentication is
$browser->credentials("$host:$port",$realm,$user,$password);, which makes LWP transparently handle the creation of the digest and re-sending of the request.Ruby
Unlike perl, which I was using a lot in the past, I am not yet familiar with Ruby, so there may be a much better solution -- please provide some feedback. Especially I have not found a good way to automatically handle the digest authentication, so this is done explicitly
#!/opt/local/bin/ruby1.9
require 'json'
require 'net/http'
require 'net/http/digest_auth'
url = URI.parse('http://localhost:9990/management/')
url.user = 'heiko'
url.password = 'okieh'
# data to send to retrieve the server name
data = { "operation" => "read-attribute",
"address" => [],
"name" => "name"}
h = Net::HTTP.new url.host, url.port
# send first request to get nonce
req = Net::HTTP::Post.new url.request_uri
res = h.request req
So far we have sent a first request to obtain the 'nonce' from the server, so we can compute the digest in the 2n step.
# compute the digest
digest_auth = Net::HTTP::DigestAuth.new
auth = digest_auth.auth_header url, res['www-authenticate'], 'POST'
# Now send the real request with the nonce
body = JSON.generate(data)
puts "Sending " + body
req = Net::HTTP::Post.new url.request_uri
req.add_field 'Content-Type', 'application/json'
req.add_field 'Authorization', auth
req.body = body
res = h.request req
print "Result " + res.body
# parse the JSON and obtain the 'result' object
data = JSON.parse(res.body)
server_name = data["result"]
print "Server name is " + server_name
Thanks
I want to thank Darran Lofthouse for helping me to get going with why apparently correct requests fail with a 403 (because of the wrong content type).
Monday, January 09, 2012
Some graphing fun with D3.js
Now that the RHQ REST api can expose raw numerical metrics for the last 7 days, it is possible to create additional graphs for numerics. As before I have used D3.js to create the following graphs. The source is now in RHQ git master and will also be in RHQ 4.3.
The first graph shows the last 7 days of metrics in one go:
raw_graph.html
This one shows the last 7 days where each span of 24h is represented by a colored line, which allows to compare the values directly - the darker the color, the older the data:
raw_graph7.html
This needs some more work, as the x-axis labeling does not yet take the exact start time into account.
Also time spans that have no values (e.g. because agent is down) should not show a straight line, but no line at all for that period. There should perhaps be a legend about the colors as well.
I am still very much on the learning side of JavaScript and D3.js and the above is far from the beautiful examples Mike Bostock creates, but I think one can already see the potential power here.
If anyone is interested to add tooltips for the values or fix the label for the second graph, please ping me.
The first graph shows the last 7 days of metrics in one go:
raw_graph.html
This one shows the last 7 days where each span of 24h is represented by a colored line, which allows to compare the values directly - the darker the color, the older the data:
raw_graph7.html
This needs some more work, as the x-axis labeling does not yet take the exact start time into account.
Also time spans that have no values (e.g. because agent is down) should not show a straight line, but no line at all for that period. There should perhaps be a legend about the colors as well.
I am still very much on the learning side of JavaScript and D3.js and the above is far from the beautiful examples Mike Bostock creates, but I think one can already see the potential power here.
If anyone is interested to add tooltips for the values or fix the label for the second graph, please ping me.
Monday, January 02, 2012
Analyze your RHQ metrics with R
Plot of metric aggregates in R
You probably have seen that with RHQ 4.2 you can export the aggregate metrics for the last n (default 8) hours via the REST api by calling:
http://localhost:7080/rest/1/metric/data/<scheduleId>where <scheduleId> must be a valid numerical schedule.
The statistical tool R allows via "RCurl" to load data from remote URLs e.g. like this:
json_file <- getURL("http://localhost:7080/rest/1/metric/data/10013", httpheader=c(Accept = "application/json"),userpwd="rhqadmin:rhqadmin")
RCurl allows you to specify username and password unlike the simple
json_file <- "http://..." calls. The next step is then to transform the received data with the help of the "rjson" library into R data structures:## convert json to list of vectors
json_data <- fromJSON(paste(json_file, collapse=""))
## convert the embedded data points into a data frame
df <- data.frame(do.call(rbind,json_data$dataPoints))
which you can then access and e.g. plot:
## plot the data
plot(df$timeStamp,df$value,xlab="time",ylab="Free memory (bytes)",xaxt='n',type='l')
You can find the full example as plot_metrics.r in the RHQ samples project on GitHub.
Some like it raw...
RHQ master (this will make it into RHQ 4.3) is now also able to export raw metrics via REST api
http://localhost:7080/rest/1/metric/data/10013/raw?duration=259200
Like above you provide the schedule id and optionally a startTime and endTime or an endTime and a duration (in seconds). If nothing is provided, the data for the last 8h is exported. The following screenshot shows an example plot:
The metrics are plotted in black, the average in blue, the 5% and 95% quantils in orange and green and with the help of the TTR library, the 50 samples moving average is plotted in red. The sample code (with slightly different parameters) is also available online as plot_raw.r.
If you find other cool usages like e.g. Holt-Winters prediction on the data, please consider submitting them as example to the samples project.
Monday, December 12, 2011
Announcing the RHQ samples project
I am proud to announce the inception of a the samples project for RHQ.
This is meant to collect samples and extensions around RHQ that could be used in RHQ and also JBoss ON. Users can go there and copy and paste the solutions to get a quicker start themselves.
While this project is started by the RHQ project team, everyone is encouraged to come and use the samples, fork the project and enhance it. To make this process easier, the project is hosted on GitHub at
https://github.com/rhq-project/samples
The initial content centers around the new REST api of RHQ, and shows how to use this API from Ruby or Python and how to build reports from JasperReports.
Wanja Pernath has also submitted his group control scripts using the CLI in RHQ and JBoss ON.
This is meant to collect samples and extensions around RHQ that could be used in RHQ and also JBoss ON. Users can go there and copy and paste the solutions to get a quicker start themselves.
While this project is started by the RHQ project team, everyone is encouraged to come and use the samples, fork the project and enhance it. To make this process easier, the project is hosted on GitHub at
https://github.com/rhq-project/samples
The initial content centers around the new REST api of RHQ, and shows how to use this API from Ruby or Python and how to build reports from JasperReports.
Wanja Pernath has also submitted his group control scripts using the CLI in RHQ and JBoss ON.
Wednesday, November 23, 2011
Join me and RHQ in London next monday
As written before, I will talk at the London JBoss User Group next monday about RHQ and JBoss ON.
You may register on the meetup page, which also lists the venue and the directions to get there.
Monday, October 31, 2011
RHQ 4.2 released
The RHQ team is proud to have released version 4.2 of the systems management and monitoring framework.
This release focuses on:
- UI enhancements / bug fixing
- Completion of drift management (this allows you to e.g. detect changes to files on your managed platform and to alert on such drift)
- Improvements in server config sync (you can basically take (parts of) the configuration of one RHQ server and apply it to a different RHQ server).
In addition to that, the release is the very first to have a (limited) REST interface. Another change is the added support for PostgreSQL 9.1.
We want to thank
- Ted Won for contributing Korean installer messages
- Rafael Torres for improvements to the Twitter plugins (OAuth support)
- All community members that provided input and bug reports
As always check the release notes for a more complete list of changes. You can get the downloads from sourceforge. And please provide us feedback; for the REST interface we are also looking for usage examples.
Heiko Rupp will showcase RHQ 4.2 at the November meeting of the London JBUG.
Thursday, October 27, 2011
Evolving interface in REST with JAX-RS (updated)
[update] In the earlier version I had if (mediaType==MediaType.xxx), which is wrong. MediaType is no enum, so we need to compare via .equals()[/update]
So in RHQ we've introduced the REST api and one of the nice things is the ease of use like this:
@GET
@Path("/customer/{id}")
public Customer getCustomer(@PathParam("id") int custId) ;
where the implementation code e.g. looks up a Customer object in the database and then returns it. Clean expressive API.
Now things start getting more interesting when you consider returning XML, JSON and HTML versions of the document. The first naive way goes like this:
@GET
@Path("/customer/{id}")
public Customer getCustomer(@PathParam("id") int custId) ;
Which fails big time for HTML as (at least RESTEasy) does not know how to serialize the Customer object into HTML. So the next version of the API could look like this:
@GET
@Path("/customer/{id}")
@Produces({"application/json","application/xml"})
public Customer getCustomer(@PathParam("id") int custId) ;
and
@GET
@Path("/customer/{id}")
@Produces("text/html")
public String getCustomerHtml(@PathParam("id") int custId) ;
Where both methods get the customer from the backend. The first one then returns the object directly (as above) whereas the second one renders the object with the help of Freemarker templates into a HTML string and returns this one. Just returning String in both cases is also not working as one may think. So this two methods that internally use almost the same code (actually the *Html one calls the other to retrieve the Customer object).
This works quite nicely, but now I want to start adding explicit return codes (and also caching information). JAX-RS offers the Response class for this, where you can e.g. say Response.ok() or Response.notFound(). So the interface changes to
@GET
@Path("/customer/{id}")
@Produces({"application/json","application/xml"})
public Response getCustomer(@PathParam("id") int custId) ;
and
@GET
@Path("/customer/{id}")
@Produces("text/html")
public Response getCustomerHtml(@PathParam("id") int custId);
Internally we have the same situation as before, but can now explicitly return the result codes we want. This is still not optimal. Luckily JAX-RS allows to inject the Request and the HttpHeaders into the called Java-method, so that we can now write
@GET
@Path("/customer/{id}")
@Produces({"application/json","application/xml","text/html"})
Response getCustomer(@PathParam("id") int custId,
@Context HttpHeaders headers);
and then in the implementation do the following:
MediaType mediaType = headers.getAcceptableMediaTypes().get(0);
ResponseBuilder builder = ...
if (mediaType.equals(MediaType.TEXT_HTML_TYPE)) {
String html = renderTemplate("customer", customer);
builder = Response.ok(html, mediaType);
}
else {
builder = Response.ok(customer);
}
So now we have one method doing all the work for us without code duplication and which can then use the same logic for caching and e.g. paging or linking.
Support for caching is now only one more step away:
@GET
@Path("/customer/{id}")
@Produces({"application/json","application/xml","text/html"})
Response getCustomer(@PathParam("id") int custId,
@Context Request request,
@Context HttpHeaders headers);
and in the implementation
// Check for conditional get
String tagString = Integer.toString(customer.hashCode());
EntityTag eTag = new EntityTag(tagString);
Date lastModifiedInDb = new Date(customer.getMtime();
Response.ResponseBuilder builder = request.
evaluatePreconditions(lastModifiedInDb,eTag);
if (builder==null ) {
// we need to send the full resource
if (mediaType.equals(MediaType.TEXT_HTML_TYPE)) {
String html = renderTemplate("customer", customer);
builder = Response.ok(html, mediaType);
} else {
builder = Response.ok(customer);
}
} builder.tag(eTag); // Set ETag on response
There is of course still a lot to do, but we have now achieved :
- uniform interface independent of the media type
- support for conditional sending and thus caching and re-validation by ETag and time stamp
The last step is to add some hints to the client how long it may cache data without the need to go out to the network to do any caching at all. Again JAX-RS has already support for that:
// Create a cache control
CacheControl cc = new CacheControl();
cc.setMaxAge(300); // Customer objects are valid for 5 mins
cc.setPrivate(false); // Proxies may cache this
builder.cacheControl(cc);
This gives the client a hint, that they can consider the customer object valid for 300s = 5min. Proxies on the way are also allowed to cache the returned object. In practice one may make the maxAge depending on e.g. some average update frequencies and also set the "you need to always verify" flag via cc.setMustRevalidate(true).I am sure, this is not yet the last version of the interface, but you can see how it can evolve over time and add new features like support for conditional get. And the best part is that so far the clients don't even have to change a single line of code.
In the future we may want to introduce our own media types like appliation/vnd.rhq-customer+json that newer clients then can make use of. The server can dispatch as seen above the media type and return the appropriate representation. The existing clients, that do not know the new media type can still be serviced by the server just sending this "old" version of it.
Monday, October 24, 2011
RHQ and JBoss ON at JBug London (updated)
I will talk on monday, Nov 28th at the London (UK) JBoss User Group about RHQ and JBoss ON. The talk will give an overview over RHQ and JBoss ON, showcase the new features in RHQ 4.2 and also talk a little bit about the new REST api.
The talk venue is the Skills matter eXchange, where the meetup page has the link to the location and also a link to the registration.
The guys from C2B2 also created a cool poster:
Friday, October 14, 2011
Small recap of JBoss One Day Talk in Munich (updated)
Yesterday I was at JBoss One Day Talk, organized by the local JBoss Users Group. The schedule consisted of 18 sessions in three parallel tracks.
One of the main topics was of course JBoss AS 7, where Heiko Braun was giving an overview, Thomas Diesler talking about OSGi on AS7 and myself talking about management of the server doing a lot of live demoing.
Other JBoss speakers included Kris Verlaenen on JBPM5 and Sashin Shinde on Open Shift. And there was Gavin King announced to talk about Ceylon. While I was looking forward to meet Gavin again, I was also afraid that I would need to talk only to myself with the Ceylon talk being in parallel. (Un :-)fortunately Gavin was not able to make it, so I had a great audience.
My buddy Pavlo Baron was talking about Big Data (slides are available on Slideshare). While this presentation was not as technical as I hoped, it nevertheless got me thinking about some stuff we could integrate in a future version of RHQ.
This conference was organized for the second time this year and I like the intimate atmosphere a lot. And of course the speakers dinner at Löwenbräukeller :-)
Update:
The organizers of One Day Talk have posted some interviews (in German) -- and especially one with Heiko Braun
Thursday, September 29, 2011
Some notes from the RHQ team meeting (and a team photo)
Last week, the whole RHQ development team met in the Red Hat office in Westford (near Boston). It was very nice for me to finally see some of the old colleagues again after two years and also to meet the new colleagues on the team.
And of course, we took a team photo
We had some very good discussions about possible future features (like e.g. supporting a REST style API into the system or on the future of alerts) - I will post more information soon. In the mornings we had show'n'tell style session where each developer was presenting some area of work to the team.
After work we did some more recreational things like:
- Seeing the Boston Red Sox win 18:9 in a 4h base ball game
- Playing minigolf at Kimball Farm (and ate huge amounts of ice cream)
- Had dinner at nice places
One of things we also did is to publish some sort or Roadmap for RHQ and discussed that we will move the source to GitHub at some time and also split up the build in an easier to build way. We will post about that when we are making more progress on this.
Ah and did I mention "Beer"?
Wednesday, September 28, 2011
REST coming to RHQ (updated)
In the past few weeks I was also working on implementing a sample REST interface for RHQ (see also here and here). At our team meeting I've showcased the work I've done in the heiko-rest branch of our git repository and we have decided to merge that into the RHQ master branch sooner or later.
UPDATE: this has been merged today (2011/10/09); if you build from source, you need to build the container again.
To stimulate your appetite I will show some samples what you could do with the REST api:
Returning resources in html representation
and as JSON
Render metric graphs in Javascript with the help of D3.js
Or an alternative resource tree in Javascript with the help of D3.js
Providing data to an Android client application (this is not very sophisticated yet, but should illustrate the idea):
List / add / delete favorite resources:
$ curl localhost:7080/rest/1/user/favorites/resource -u rhqadmin:rhqadmin
[{"resourceWithType":{"@resourceId":"10160","pluginName":"RHQServer"
,"resourceName":"Measurement Subsystem","typeId":10235
,"typeName":"RHQ Server Measurement Subsystem"}}
,{"resourceWithType":{"@resourceId":"10361","pluginName":"jboss-as-7"
,"resourceName":"\/devel\/jbas7\/jboss-as\/build\/target\/jboss-as-7.1.0.Alpha2-SNAPSHOT","typeId":10055
,"typeName":"JBossAS7-Standalone"}}]
Add resource with id 10013 to favorites:
$ curl localhost:7080/rest/1/user/favorites/resource/10013 -X PUT -u rhqadmin:rhqadmin
$ curl localhost:7080/rest/1/user/favorites/resource -u rhqadmin:rhqadmin
[{"resourceWithType":{"@resourceId":"10013","pluginName":"Platforms"
,"resourceName":"en0","typeId":10024
,"typeName":"Network Adapter"}}
,{"resourceWithType":{"@resourceId":"10160","pluginName":"RHQServer"
,"resourceName":"Measurement Subsystem","typeId":10235
,"typeName":"RHQ Server Measurement Subsystem"}}
,{"resourceWithType":{"@resourceId":"10361","pluginName":"jboss-as-7"
,"resourceName":"\/devel\/jbas7\/jboss-as\/build\/target\/jboss-as-7.1.0.Alpha2-SNAPSHOT","typeId":10055
,"typeName":"JBossAS7-Standalone"}}]
And remove it again from favorites:
$ curl localhost:7080/rest/1/user/favorites/resource/10013 -X DELETE -u rhqadmin:rhqadmin
Of course those are only some examples and should illustrate that:
- the api will be usable from different clients
- from different programming languages
- it will allow write / update access
- it will allow users to write extensions to RHQ like e.g. Über-consoles that we do not even think of
We were also thinking of the possibility for agents in other languages being able to push events and metrics over this interface.
The code is currently in the heiko-rest branch in RHQ-git (at fedorahosted), but we plan to merge it into master as soon as possible. If you want to try the code, check out the branch and either build RHQ from scratch. Or try running /etc/dev-utils/setup-rest/setup.sh to populate the server with the needed libraries (edit that file first), then build server/jar and gui/rest-war in dev-mode so that the artifacts are copied to the server. Then edit rhq.ear/META-INF/application.xml to include the rest war. Of course all this will not be needed after we have merged the code in the master, as the normal build will then have all the artifacts.
The code for the Android app is available from my repository on GitHub
An important part of this whole effort is your input:
- Provide us with your requirements e.g. on the Design - REST page of the RHQ-Wiki
- Contribute to the code
- Show us your examples you have built on top of the API (and perhaps even contribute them)
Please help us making this great so that you can create great stuff with the interface.
Monday, September 26, 2011
RHQ 4.1 released
Actually this is not new, but already three weeks old - I wanted to add it here, for completeness sake:
RHQ 4.1.0 has been released on Sept 2nd. Make sure you read the release notes and then proceed to the download.
Major changes include translations of the UI into Chinese, Japanese and Portuguese. Big thanks go to Fusayuki Minamoto, Jijun Liu, Zhongqianglee and Rafael Torres Coelho Soares. Also in this release we have started to detect changes in configurations and files ("Drift"), made enhancements in the area of Bundles (provisioning) and added a first shot at a plugin to manage JBoss AS 7. Last but not least, Elias Ross has enhanced the SNMPTrapd plugin.
As always: please try the release and give us feedback.
Thursday, August 11, 2011
RHQ 4.1.0 BETA released
The RHQ team has just released version 4.1.0.beta of the RHQ systems management and monitoring suite. This release serves as a preview of the upcoming version 4.1.0 and has a wealth of new features.
Some of the new features are work in progress and will only be finished after 4.1, which means that you can still influence the features with your ideas and feedback.
Release notes are on the RHQ wiki and also contain the link to the download at the bottom.
We want to especially thank Elias Ross, Fusayuki Minamoto, Jijun Liu, Zhongqianglee
and Rafael Torres Coelho Soares for their contributions (even if some are not in this build)
Monday, August 08, 2011
Some progress with REST for RHQ and new questions
As written some time ago on the rhq-devel mailing list, I've started implementing a RESTful interface for RHQ. A page on the RHQ wiki shows the requirement and some progress about the implemented API. Development currently happens in a personal feature branch in order to have some code to play around and gather some experience from.
One thing that I found out over the weekend is that at least with JAX-B setting up the interface and REST resource classes is far from trivial and I have been surprised a lot by interesting error messages.
Anyway, the current state of the API is already in a somewhat usable shape (mostly read-only at the moment). The implementation has been done via RESTeasy (RE). Security integration with the RHQ security model is also done and the session bean methods can just use a Subject called caller.
So, having solved some stuff rises even more questions:
- Would stuff get easier when not trying to support XML, but only Json?
- How to use much / more of the original RHQ domain model without running into LazyLoadExceptions all over the place (RE runs as a servlet to do the marshalling - at this point in time the connection to the entity manager is already closed).
- How to implement linking between related classes / concepts. E.g. a Resource can have Alerts. Now instead of embedding Alert elements when returning a Resource object, I would like to at most have a list of Links to
/alert/{id}. I feel that the answer may have to do with the next question - In RHQ we often pass just primitive ids around (e.g.
int resourceId) How can easily turn them into a link to the target Resource (given that Resource lives below/resource/{id}) - In the Atom-PUB linking I would add links to sub-resources like e.g.
/resource/1234/availabilityor/resource/1234/scheduleswithout the need to first obtain those values from the database. - Does it make sense to transfer date/times as long? Especially as Json seems to have some issues there. Complex date strings require a lot of processing on server and client.
- Is there a preferred way to expose id elements ((XML) attribute, element, not at all? With Atom-PUB links present, I think the id would not need any special treatment at all, as it is available in the URI of the resource and in the link.
A part of the solution to questions 3+4 seem to be to only use Objects in REST domain objects, as the marshalling will skip elements where the corresponding object is null, so for e.g. the parent Resource it is enough to initialize its resource with the Resource id to only include this, but not all the other fields that should then be obtained from their real URI.
But then I am not sure how much my ideas are RESTful at all :-)
I would love to get some feedback here.
Subscribe to:
Posts (Atom)