JAX-RS 2 client in RHQ (plugins)

When my fellow Red Hatter Bill Burke recently wrote about the new features in JAX-RS 2.0, I started to consider using the new Client api to use in tests and plugins. Luckily Bill provided a 3.0-beta-1 release of RESTEasy that I started with.

To get this going, I started with the plugin case and wrote my few lines of code in the editor. All red, as the classes were not known yet. After many iterations in the editor and the standalone-pc, I ended up with this list of dependencies, which all need to be included in the final plugin jar:

                <artifactItem>
                   <groupId>org.jboss.resteasy</groupId>
                   <artifactId>resteasy-client</artifactId>
                </artifactItem>
                <artifactItem>
                  <groupId>org.jboss.resteasy</groupId>
                  <artifactId>jaxrs-api</artifactId>
                </artifactItem>
                <artifactItem>
                  <groupId>org.jboss.resteasy</groupId>
                  <artifactId>resteasy-jaxrs</artifactId>
                </artifactItem>
                <artifactItem>
                  <groupId>org.apache.httpcomponents</groupId>
                  <artifactId>httpclient</artifactId>
                </artifactItem>
                <artifactItem>
                  <groupId>org.apache.httpcomponents</groupId>
                  <artifactId>httpcore</artifactId>
                </artifactItem>
                <artifactItem>
                  <groupId>org.jboss.resteasy</groupId>
                  <artifactId>resteasy-jackson-provider</artifactId>
                </artifactItem>
                <artifactItem>
                  <groupId>org.codehaus.jackson</groupId>
                  <artifactId>jackson-jaxrs</artifactId>
                </artifactItem>
                <artifactItem>
                  <groupId>org.codehaus.jackson</groupId>
                  <artifactId>jackson-core-asl</artifactId>
                </artifactItem>
                <artifactItem>
                  <groupId>org.codehaus.jackson</groupId>
                  <artifactId>jackson-mapper-asl</artifactId>
                </artifactItem>
              </artifactItems>

Still I got errors that there is no provider for a media type of application/json available. It turned out, that the RHQ plugin container's classloader does not honor the META-INF/services/javax.ws.rs.ext.Providers entry (or any of the services there, so I had to explicitly enable them in code:

javax.ws.rs.client.Client client = ClientFactory.newClient();
client.configuration().register(org.jboss.resteasy.plugins
      .providers.jackson.ResteasyJacksonProvider.class);
client.configuration().register(org.jboss.resteasy.plugins
      .providers.jackson.JacksonJsonpInterceptor.class);

Now the client works as intended. At there is quite a number of jars involved, I am thinking of providing an abstract REST plugin, that provides the jars and the basic connection functionality, so that other plugins can build on top just like we have done that with the jmx-plugin.

I was also trying to get this to work with Jersey, but failed somewhere in the maven world, as it failed for me after downloading half of the Glassfish distribution.

One interesting question to me is about the target group of this JAX-RS 2.0 client framework, as the Jersey and RESTEasy implementations seem to grow rather large (which does not matter too much for servers or on the desktop), so that Android apps are less likely to use that and will rather fall back to home grown solutions. With the huge number of mobile phones, this could perhaps the end also mean that nobody will touch the Client framework and falls back to those smaller homegrown solutions on mobile and desktop.

REST/JAX-RS documentation generation

As I may have mentioned before :-) we have added a REST api to RHQ. And one thing we have clearly found out during development and from feedback by users is that the pure linking feature of REST may not be enough to work with a RESTful api and that some. Our Mark Little posted an article on InfoQ recently that touches the same subject.

Take this example:

@PUT
@Path("/schedule/{id}")
@Consumes({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
Response updateSchedule(int scheduleId,
         MetricSchedule in,
         @Context HttpHeaders headers);

So this is a PUT request on some schedule with an id identifier where a certain MetricSchedule thing should be supplied. And then we get some Response back, which is just a generic
JAX-RS "placeholder".

Now there is a cool project called Swagger with a pretty amazing interactive UI to run REST requests in the browser against e.g. the pet store.

Still this has some limitations

• You need to deploy wagger, which not all enterprises may want
• The actual objects are not described as I've mentioned above

Metadata via annotations

The metadata in Swagger to define the semantics of the operations and the simple parameters is defined on the Java source as annotations on the restful methods like this:

@PUT
@Path("/schedule/{id}")
@Consumes({MediaType.APPLICATION_JSON,MediaType.APPLICATION_XML})
@ApiOperation(value = "Update the schedule (enabled, interval) ", 
   responseClass = "MetricSchedule")
@ApiError(code = 404, reason = NO_SCHEDULE_FOR_ID)
Response updateSchedule(
   @ApiParam("Id of the schedule to query") @PathParam("id") int scheduleId,
   @ApiParam(value = "New schedule data", required = true) MetricSchedule in,
   @Context HttpHeaders headers);

which is already better, as we now know what the operation is supposed to do, that it will return an
object of type MetricSchedule and for the two parameters that are passed we also get
a description of their semantics.

REST docs for RHQ

I was looking on how to document the stuff for some time and after finding the Swagger stuff it became clear to me that I do not need to re-invent the annotations, but should (re)use what is there. Unfortunately the annotations were deep in the swagger-core module.

So I started contributing to Swagger - first by splitting off the annotations into
their own maven module so that they do not have any dependency onto other modules, which makes
it much easier to re-use them in other projects like RHQ.

Still with the above approach the data objects like said MetricSchedule are not documented. In order to do that as well, I've now added a @ApiClass annotation to swagger-annotations, that allows to also document the data classes (a little bit like JavaDoc, but accessible from an annotation processor). So you can now do:

@ApiClass(value = "This is the Foo object",
   description = "A Foo object is a place holder for this blog post")
public class Foo {
   int id;
   @ApiProperty("Return the ID of Foo")
   public int getId() { return id; }

to describe the data classes.

The annotations themselves are defined in the following maven artifact:

    <dependency>
      <groupId>com.wordnik</groupId>
      <artifactId>swagger-annotations_2.9.1</artifactId>
      <version>1.1.1-SNAPSHOT</version>
    </dependency>

which currently (as I write this) is only available from the snapshots repo

      <!-- TODO temporary for the swagger annotations -->
      <id>sonatype-oss-snapshot</id>
      <name>Sonatype OSS Snapshot repository</name>
      <url>https://oss.sonatype.org/content/repositories/snapshots</url>
    </repository>

The generator

Based on the annotations I have written an annotation processor that analyzes the files and creates an XML document which can then be transformed via XSLT into HTML or DocBookXML (which can then be transformed into HTML or PDF with the 'normal' Docbook tool chain.

You can find the source for the rest-docs-generator in the RHQ-helpers project on GitHhub.

The pom.xml file also shows how to use the generator to create the intermediate XML file.

Examples

You can see some examples of the generated output (for some slightly older version of the generator in the RHQ 4.5.1 release files on sourceforge, as well as the in the documentation for JBoss ON, where our docs team just took the input and fed it into their DocBook tools chain almost without and change.

Postprocessing…

If you are interested to see how the toolchain is used in RHQ, you can look at the pom.xml file from the server/jar module ( search for 'REST-API' ).

Project independent

One thing I want to emphasize here is that the generator with its toolchain is completely independent from RHQ and can be reused for other projects.

Another nice JBoss OneDayTalk is over (updated)

Yesterday I was at the OneDayTalk conference organized by the Munich JBoss User Group.

And as in the last two years this was a nice conference with an interested audience and it was good
to meet with colleagues again.

This year I was talking about "RHQ and its interfaces" - and other than some other speakers, I enjoyed to just give my talk in German :)
My slides are available as PDF.

Update: A recording of my talk (which I held in German) is now available:

RHQ und seine Schnittstellen from Heiko W. Rupp on Vimeo.

Thanks for the Munich JBug to organize this nice conference and to allow me to present there.

RHQ 4.5.1 released

I am pleased to announce the immediate availability of RHQ 4.5.1.
RHQ is a system for management and monitoring of resources like application servers
or databases and can be extended by writing plugins.

Actually I wanted to announce 4.5.0 a week ago, but a first user report showed an
error in the upgrade path from a previous version, so we have pulled that release
and fixed the bug along with another one and have now created a fresh 4.5.1 release.

Notable changes are:

• Python support in the Command Line Interface (CLI)
• Support for importing of scripts in the CLI
• Enhancements in the JBossAS7 plugin
• Enhancements in the REST API
• Events tab allows to filter by date range
• Postgres 9.2 is now supported as backend database
• The Sigar library has been updated.

Special thanks goes to Elias Ross and Richard Hensman for their contributions.

Maven artifacts have been uploaded to the JBoss Nexus repo and should show up on maven central soon.

You can find the full release notes, that also contain a download link on the RHQ wiki.

This time we have included the full output from git shortlog for the commits of the release. Please tell us if this is useful for you.

Heiko on behalf of the RHQ team

Introducing the No-op plugin for RHQ

“A no-op plugin, does that make sense?” you will ask - and indeed it makes sense when you look at recent developments inside RHQ with the REST-api and also the support for jar-less plugins.

The no-op plugin is meant to support jar-less plugins that are written to define ResourceTypes on the server along with their metrics in order to be usable via the REST api, but where no resources are supposed to be found via the classic java-agent.

A concrete use case is the work our GSoC student Krzysztof is doing by bridging monitoring data from CIMmons deployed e.g. in RHEL servers to RHQ resources. Here we have the (possible) need to define resource types that will only be fed via the REST api.

How do I use this?

Basically you write a plugin descriptor (either in the classical way which you then put into a plugin jar) or with the name of *-rhq-plugin.xml and then deploy that into the server. The contents of this descriptor then refer to the No-op plugin - first in the <plugin> elements’s package declaration:

<plugin name="bar-test"
    version="4.5.1"
    displayName="Jar less plugin test"
    description="Just testing"
    package="org.rhq.plugins.noop”
    xmlns="urn:xmlns:rhq-plugin"
    xmlns:c="urn:xmlns:rhq-configuration">

The next part is then to declare a dependency on the No-op plugin to be able to use the classes:

   <depends plugin="No-op" useClasses="true"/>

And finally to use the NoopComponent for the discovery and component classes of <server>s and <service>s:

    <server name="test"
        discovery="NoopComponent"
        class="NoopComponent"
        >
     …
    </server>
</plugin>

A plugin defined like this then defines metrics etc. just like every other plugin. This is the plugin info:

And the metric definition templates:

JAR-less plugins for RHQ

Mazz has written a few times about the Custom-JMX plugin (funnily enough I did not find a newer post to link to), which is basically a plugin descriptor, that re-uses the classes from the base JMX plugin.

While this is very powerful, it has the small usability drawback of requiring to wrap the descriptor into a jar file to be recognized by the server (and the agents).

I have now implemented BZ 741682 which allows to deploy plugin descriptors only. For this to work

• the file needs to be called *-rhq-plugin.xml (e.g. foo-rhq-plugin.xml)
• the classes of the plugin need to already be present on the agent

The latter can be achieved by putting a depends directive into the descriptor:

<depends plugin="JMX" useClassed="true"/>

In the following example you see the bar-rhq-plugin.xml file picked up from the $SERVER/plugins directory
(the "drop-box") and placed into the plugins directory inside the app:

09:18:55,160 INFO  [PluginDeploymentScanner] Found plugin descriptor at 
    [/im/dev-container/plugins/bar-rhq-plugin.xml] and placed it at 
    [/im/dev-container/jbossas/server/default/deploy/rhq.ear/rhq-downloads/rhq-plugins/bar-rhq-plugin.xml]

Next is the transformation of this plugin descriptor into a jar file - if this is successful, the now obsolete plugin descriptor is removed.

09:18:55,403 INFO  [AgentPluginScanner] Found a plugin-descriptor at 
    [/im/dev-container/jbossas/server/default/deploy/rhq.ear/rhq-downloads/rhq-plugins/bar-rhq-plugin.xml],
    creating a jar from it to be deployed at the next scan
09:18:55,411 INFO  [AgentPluginScanner] Deleted the now obsolete plugin descriptor: true

At the next scan of the deployment scanner, the scanner will pick up this generated jar and deploy it like any other plugin:

09:19:26,470 INFO  [ProductPluginDeployer] Discovered agent plugin [bar-test]
09:19:26,474 INFO  [ProductPluginDeployer] Deploying [1] new or updated agent plugins: [bar-test]
09:19:26,656 INFO  [ResourceMetadataManagerBean] Persisting new ResourceType [bar-test:test(id=0)]...
09:19:29,344 INFO  [ProductPluginDeployer] Plugin metadata updates are complete for [1] plugins: [bar-test]

The last step would then be to run plugins update on the agent to get this new plugin from the server and to deploy it into the agent.

RHQPocket updated

I have worked over the last days on improving RHQPocket, an Android application that serves as an example client for RHQ. Most of the features of RHQPocket will work (best) with a current version of the RHQ server (from current master).

The following is a recording I made that shows the current state.

Update on RHQPocket from Heiko W. Rupp on Vimeo.

The video has been recorded with QuickTime player with the running emulator. I first tried this with the "classical" emulator, but this used 100% cpu (1 core) and with recording turned on, this was so slow that simple actions that are normally immediate, would take several seconds.

My next try was to film my real handset, but too much environmental light and reflexions made this a bad experience. After I heard that others had successfully installed the hardware accelerated GPU and the virtualized x86 image for the emulator I went that route and the CPU usage from the emulator went from 100% before to around 15-20% (home screen), so I went that route for recording.

You can find the full source code on GitHub - contributions and feedback are very welcome.
If you don't want to build from source, you can install the APK from here.

RHQ REST api: Support for JSONP (updated)

I have just committed support for JSONP to the RHQ REST-api.

Update: In the first version a special media type was required. This is now removed, as jQuery seems to have issues sending this type. Also the default name for the callback parameter has been renamed to jsonp.

To use it you need to pass a parameter for the callback. Lets look at an example:

$ curl -i -u rhqadmin:rhqadmin \
   http://localhost:7080/rest/1/alert?jsonp=foo \
   -Haccept:application/json
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Pragma: No-cache
Cache-Control: no-cache
Expires: Thu, 01 Jan 1970 01:00:00 CET
X-Powered-By: Servlet 2.4; JBoss-4.2.0.CR2 
Content-Type: application/javascript
Transfer-Encoding: chunked
Date: Thu, 14 Jun 2012 09:25:09 GMT

foo([{"name":"test","resource":{"typeName":null, ……..])

The name of the callback parameter is jsonp and the name of the callback function to return is
foo. In the output of the server you see how the json-data is then wrapped inside foo().
The wrapping will only happen when both the right content-type is requested and the callback parameter is present.

The content type returned is then application/javascript.

Setting the name of the callback parameter

Then name of the callback parameter (jsonp in above example) can be set in web.xml :

<filter>
  <filter-name>JsonPFilter</filter-name>
  <filter-class>org.rhq.enterprise.rest.JsonPFilter</filter-class>
  <init-param>
    <param-name>filter.jsonp.callback</param-name>
    <param-value>jsonp</param-value>
    <description>Name of the callback to use for JsonP /description>
  </init-param>
</filter>

RHQ REST api: added support for Group Definitions

I've just added some support for GroupDefinitions (aka "DynaGroups") to RHQ.

The following shows some examples:

List all definitions:

$ curl -i --user rhqadmin:rhqadmin http://localhost:7080/rest/1/group/definitions
[{"name":"group1",
  "id":10001,
  "description":"just some random test",
  "expression":["groupby resource.type.plugin","groupby resource.type.name"],
   "recursive":false,
   "recalcInterval":180000,
   "generatedGroupIds":[10082,10091]},
  {"name":"platforms",
   "id":10002,
   "description":"",
   "expression":["resource.type.category = PLATFORM","groupby resource.name"],
   "recursive":false,
   "recalcInterval":0,
   "generatedGroupIds":[10152]}
]

Get a single definition by id:

$ curl -i --user rhqadmin:rhqadmin http://localhost:7080/rest/1/group/definition/10002
{"name":"platforms",
 "id":10002,
 "description":"",
 "expression":["resource.type.category = PLATFORM","groupby resource.name"],
 "recursive":false,
 "recalcInterval":0,
 "generatedGroupIds":[10152]
}

You see in the above examples that the actual expression is encoded as a list with each line being an item in the list. The recalculation interval needs to be given in milliseconds.

Delete a definition (by id):

$ curl -i --user rhqadmin:rhqadmin http://localhost:7080/rest/1/group/definition/10031 -X DELETE

Create a new definition:

$ curl -i --user rhqadmin:rhqadmin http://localhost:7080/rest/1/group/definitions \
   -HContent-Type:application/json -HAccept:application/json -X POST \
   -d '{"name":"test1","description":"Hello","expression":["groupBy resource.name"]}'
HTTP/1.1 201 Created
Location: http://localhost:7080/rest/1/group/definition/10041

For creation a name is required. The location of the created group definition is returned in the header of the response.

And finally to update a definition:

curl -i --user rhqadmin:rhqadmin http://localhost:7080/rest/1/group/definition/10041?recalculate=true \
   -HContent-Type:application/json -HAccept:application/json -X PUT  \
   -d '{"name":"test4","description":"Hello","expression":["groupBy resource.name"]}'

By passing the query-param recalculate=true we can trigger a re-calculation of the groups defined by this group definition.

RHQ 4.4 released

I am proud to announce the immediate availability of RHQ 4.4

As before a lot of work has gone into this release:

• Availability now knows a type of "disabled". This allows you to mark resources while maintenance or non-connected network interfaces so that they do not show up as down and also don't create false alerts. This also includes the possibility that plugins request to mark a resource as enabled or disabled
• Faster availability reporting
• Faster availability checking in the agent, that is also less bursty than in the past
• Plugins can now request an availability check for a resource
• Alerting has been improved: It is now possible to react on availability being in a certain state for some period of time (see the release notes for a long explanation
• The JBoss AS 7 plugin has been massively improved
• Denis Krusko has provided some initial Russian translations of the UI
• Reports like Suspect Metrics, Recent Operations or Recent Drift can now be exported in CSV format to e.g. post-process them in Open/LibreOffice.

Of course we have fixed many small bugs too and made tweaks to the UI etc.

One very cool feature of this release is that the maven artifacts are already uploaded to the JBoss Nexus repository ;-)

Please check out the release notes, where you also find the download link. Also check out the RHQ 4.3 release notes, as 4.3 was more a slient release, which nevertheless brought enhancements in the UI (update of GWT/SmartGWT) and many bug fixes.

If you want to get a quick overview on the ways to interact with RHQ, have a look at this whitepaper (PDF)

As always: give use feedback, join us on Irc on #rhq and spread the word.

And last but not least I want to thank all our contributors for their valuable contributions.

Heiko on behalf of the RHQ team

Russian translations coming to RHQ

Denis Krusko, one of the two GSoC students working on RHQ has just committed a first batch of Russian translations to RHQ. Here are two screenshots to bring you in the right mood.

If you speak Russian and want to help Denis, I am sure he will appreciate it. Check out the translation project on github for details.

and

Thank you Denis!

RHQ participates in this years Google Summer of Code™

RHQ is happy to be able to participate in this years Google Summer of Code™

We have as part of the JBoss.org organization seen many very good proposals and the whole JBoss.org has seen many more so that even with the generosity of Google, we have obviously obtained a lot less slots than the number of good proposals received.

So I am extremely happy that for RHQ those two proposals have been accepted:

• Replace old graphs by GWT ones - Denis Krusko: The main graphs in RHQ are still from the pre RHQ era and implemented as Servlets embedded in JSP and struts pages. The current UI is mostly written in GWT. While we were able to embed the old graphs, they still don't feel 'right'.

  
  Denis will investigate options for replacement and then implement new GWT-compatible graphing. Denis will also look at how the grahps can become more interactive by e.g. applying formulas on the data.

• Implement an RHQ agent in Python - Krzysztof Kwaśniewski: The classical RHQ agent is written in Java and probably not best suited for every purpose. With the addition of the REST api, it now became easier to implement agents in other languages.

  
  Krzysiek will implement an agent in Python, that on one side talks REST with the RHQ server and on the other side interfaces with Matahari to take metrics from Fedora and RHEL hosts.
  

I am together with the other RHQ contributors very much looking forward to see Krzysztof and Denis in action.

Heiko

*) GSoC Logo is taken from http://code.google.com/p/google-summer-of-code/wiki/GsocLogos and has a CC-3.0-attrib-non-commercial-no-deratie license

IDEA: find by XPath

I like the idea of XPath since I've looked at it over 10 years ago. Today I came across a usage that is über cool. I had the need to find some pattern of data across a bunch of xml files, where a there is a list of maps where the list and maps are marked as read-only like in this example:

<c:list-property name="*" displayName="Installed extensions" readOnly="true" required="false">
     <c:map-property name="*" displayName="Name" readOnly="true">
         <c:simple-property name="module" displayName="Module name" readOnly="true"/>
     </c:map-property>
</c:list-property>

Luckily IDEA offers "Find by XPath"

So I could type in my expression and have it search through the whole project.

This looked good, but did not work. Then I remembered the little 'c:' and clicked on the "Edit context..." button, which allowed to set the correct namespace for 'c:':

Now with the namespace correctly set, the search ran for a few seconds and found the occurrences all across my files. /me is a happy user.

Small Fosdem 2012 review

On the weekend of Feb 4th, 2012 the European (and not only) Free and Open source community met in Brussels, Belgium for FOSDEM (Free and Open Source Developers Meeting).



The following video gives an impression on what FOSDEM is like - and I was only able to capture a tiny part of it.

(original size is 960x540)

I only arrived on Saturday at around noon via train. Stuttgart to Cologne was via ICE and from Cologne on I took the Thalys where I had the luck to get a a first class ticket for the price of a regular 2nd class one; the ticket included free WiFi and free breakfast with croissants, sandwiches and beverages.


I gave a talk on RHQ "Recent and future developments". In the presentation I talked about what we have achieved recently and where we may perhaps go in the future.


(original size is 960x540, here are slides as pdf)

As I had the camera with me, I also recorded a few other talks

• Byteman
• Thermostat project
• IcedTea and IcedTeaWeb

I was lucky to meet with some of the JBoss colleagues, but was not able to make it to the virt sessions and to meet more of the various Red Hat people that were there. Fortunately some of the talks were recorded by the FOSDEM crew and will hopefully be online soon.

FOSDEM is for sure a conference to go to -- and not only because entrance is free and there is beer sold all over the place :)

JBoss, Fedora and more from Red Hat at FOSDEM 2012

A lot of folks from Red Hat are visiting FOSDEM 2012 this year.


I've listed the speakers here - I will myself talk on Saturday on RHQ.

In addition to the talks there are also stands:

• JBoss: K building, 1st floor
• Fedora: K building, ground floor

You will be able to get a printed version of this list at the Fedora stand.

Saturday

WHEN	EVENT	TRACK & ROOM	SPEAKER
11:00-11:10	Welcome to the CrossDesktop Devroom	CrossDesktop  (H.1308)	Christophe Fergeau
11:00-11:05	Welcome to the Legal Issues DevRoom	Legal Issues  (AW1.125)	Richard Fontana
11:00-11:55	BoxGrinder : Grind your appliances easily	JBoss.org  (K.3.201)	Marek Goldmann
11:00-11:05	Welcome to the Free Java DevRoom	Free Java  (K.4.401)	Mark Wielaard, Andrew Haley, Andrew Hughes
12:00-12:55	Drools Planner: Planning optimization by example	JBoss.org  (K.3.201)	Geoffrey De Smet
12:30-12:55	libguestfs - tools for modifying virtual machine disk images	Virtualization & Cloud  (Chavanne)	Richard Jones
13:00-13:25	Cloud high availability with pacemaker-cloud	Virtualization & Cloud  (Chavanne)	Pádraig Brady
13:00-13:55	Openshift	JBoss.org  (K.3.201)	Grant Shipley
14:00-14:25	The Aeolus Project	Virtualization & Cloud  (Chavanne)	Francesco Vollero
14:00-14:55	JBoss AS7 : Building JBoss AS 7 for Fedora	JBoss.org  (K.3.201)	Carlo De Wolf
15:00-15:50	Virtualization with KVM: bottom to top, past to future	Hypervisors  (Janson)	Paolo Bonzini
15:00-15:55	JBoss Forge / Arquillian: Two Missing Links in Enterprise Java Development	JBoss.org  (K.3.201)	Koen Aers
15:00-15:25	Open Clouds with Deltacloud API	Virtualization & Cloud  (Chavanne)	Michal Fojtik
15:30-15:55	DMTF CIMI and Apache Deltacloud	Virtualization & Cloud  (Chavanne)	Marios Andreou
16:00-16:55	Infinispan: where open source, Java and in-memory data grids converge	JBoss.org  (K.3.201)	Manik Surtani
16:15-17:00	Crossdesktop group picture	CrossDesktop  (H.1308)	Christophe Fergeau
16:30-17:00	The (possible) decline of the GPL, and what to do about it	Legal Issues  (AW1.125)	Richard Fontana
17:00-17:55	RHQ: Recent and future developments in the RHQ systems monitoring and management framework	JBoss.org  (K.3.201)	Heiko Rupp
17:30-18:00	Panel on Application Stores	Legal Issues  (AW1.125)	Richard Fontana
18:00-18:55	Guvernor/JBPM : Managing workflows and business rules with Guvnor and the jBPM designer	JBoss.org  (K.3.201)	Geoffrey De Smet, Marco Rietveld
18:00-18:30	Thermostat: Taking over the Java tooling world with Open Source Software	Free Java  (K.4.401)	Jon VanAlten, Omair Majid
18:20-18:35	PMD5: What can it do for you?	Lightning Talks (Ferrer)	Romain PELISSE
18:30-19:00	Tracing, Debugging and Testing With Byteman	Free Java  (K.4.401)	Andrew Dinn

Sunday

WHEN	EVENT	TRACK & ROOM	SPEAKER
09:00-09:25	Spice "Open remote computing" introduction	Virtualization & Cloud  (Chavanne)	Hans de Goede
09:30-09:55	USB redirection over the network	Virtualization & Cloud  (Chavanne)	Hans de Goede
10:00-10:45	Systems Management with Matahari	Configuration & Systems Management  (K.3.601)	Zane Bitter
10:45-11:15	Boxes, use other systems with ease	CrossDesktop  (H.1308)	Zeeshan Ali (Khattak), Marc-André Lureau
11:00-11:15	Powerful tools for Linux C/C++ developers based on Eclipse	Lightning Talks (Ferrer)	Andrew Overholt
11:00-11:25	Virtualization Management the oVirt way	Virtualization & Cloud  (Chavanne)	Itamar Heim
11:30-11:55	oVirt Engine Core: Internals and Infrastructure	Virtualization & Cloud  (Chavanne)	Omer Frenkel
11:30-12:00	Can I legally do that?	Free Java  (K.4.401)	Mark Wielaard
12:00-12:25	VDSM - The oVirt Node Management Agent	Virtualization & Cloud  (Chavanne)	Federico Simoncelli
12:30-13:30	OpenJDK on ARM: Quo vadis?	Free Java  (K.4.401)	Andrew Haley
15:00-15:55	Building app sandboxes on top of LXC and KVM with libvirt	Virtualization & Cloud  (Chavanne)	Daniel Berrange
15:30-16:00	IcedTea and IcedTea-Web	Free Java  (K.4.401)	Deepak Bhole
16:30-17:00	Discussion on the Future of Free Java	Free Java  (K.4.401)	Andrew Haley

Pushing metrics / baselines via REST interface in RHQ

A few days ago I wrote about pulling raw metrics from RHQ via the REST interface.

It is also possible to push metrics as well as read and write baselines.

Pushing metrics

There are two ways to push numeric metric values into the server:

Single metric

curl -i -u rhqadmin:rhqadmin \
    http://localhost:7080/rest/1/metric/data/10022/raw/1324656971 \
    -X PUT       \
    -d @/tmp/foo \
    -HContent-Type:application/json

With /tmp/foo containing:

 
{"timeStamp": 132465716178, "value": 123, "scheduleId":10023}

Note that you need to give the schedule id (10022) and the timestamp in the URL. The samples-project also contains an example in Python.

Multiple metrics

If you want to push multiple metrics at once you can use this call (of course it will also work for a single one):

curl -u rhqadmin:rhqadmin  \
   http://localhost:7080/rest/1/metric/data/raw  \
   -X POST      \
   -d @/tmp/foo \
   -HContent-Type:application/json

with /tmp/foo containing:

[
  {"timeStamp": 132465716078, "value": 123, "scheduleId":10022},
  {"timeStamp": 132465716079, "value": 223, "scheduleId":10022}
]

Baselines

Baselines are an interesting feature in the sense that they mark a band in which a dynamic metric usually oscillates. When the metric goes out of those bounds, you can get an alarm. The system usually computes those baselines by taking the minimum and maximum values from the existing dataset for the last n days (n is configurable). When a baseline is computed, they are also displayed on the large metric graphs. Here the baselines could also be manually set.

It is now (master/RHQ 4.3) possible to set the computation frequency to 0 to disable the automatic calculation by the server and push baseline data via the REST interface into the server. This allows to e.g. read metrics from a system like R, compute projections for future bands (e.g. via Holt-Winters) or x% quantiles of the existing data and write the results back as baseline data so that the normal alerting workflow can pick up that data.

The first call is to obtain the baseline for schedule with id 10013:

curl -u rhqadmin:rhqadmin \
   http://localhost:7080/rest/1/metric/data/10013/baseline \
   -HContent-Type:application/json 

This call updates the schedule 10013 with new values:

curl -u rhqadmin:rhqadmin  \
   http://localhost:7080/rest/1/metric/data/10013/baseline \
   -HContent-Type:application/json   \
   -HAccept:application/json         \
   -X PUT                            \ 
   -d '{"max":2.58304512E9,          \
        "min":0.119968768E9,         \
        "mean":1.285011894659459E9,  \
        "computeTime":1326477607296}'

Polyglot management of a secured AS7

JBossAS7 comes with a nice management interface that tools like the built-in admin-console or the console app are using. Next to the "more binary" DMR protocol, there is also a JSON interface available that can be accessed via http. Using this interface allows to manage AS7 from any programing language.
Luckily :-) this interface is secured by default and only accessible for a valid user via http digest authentication.

Set up admin user

The first step is to enable a user on the server to use for this management interface:

$ cd /jboss-as-7.1.0
$ bin/add-user.sh 

Enter the details of the new user to add.
Realm (ManagementRealm) : <press enter>
Username : heiko
Password : <provide password>
Re-enter Password : <provide password again>
About to add user 'user' for realm 'ManagementRealm'
Is this correct yes/no? yes
Added user 'user' to file '/jboss-as-7.1.0/standalone/configuration/mgmt-users.properties'

Now we have created a user 'heiko' with password 'okieh'.

Shell with curl

The following command with shut down the server via the management interface:

$ curl --digest -u heiko http://localhost:9990/management/ -d '{"operation":"shutdown" }' -HContent-Type:application/json

Note that for option '-u' only the username is given — curl will ask for the password. One important part here is that to mark the content-type of the data sent as "application/json". Curl will, if this header is not provided, send the request as 'application/x-www-form-urlencoded' which is disallowed by AS7.

If you run curl with option '-v' you can nicely see the re-negotiation to acquire the nonce from the server in order to compute the digest:

$ curl -v --digest -u heiko http://localhost:9990/management/ -d '{"operation":"shutdown" }' -HContent-Type:application/json
Enter host password for user 'heiko': <okieh>
* About to connect() to localhost port 9990 (#0)
*   Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 9990 (#0)
* Server auth using Digest with user 'heiko'
> POST /management/ HTTP/1.1
> User-Agent: curl/7.21.7 (x86_64-apple-darwin10.8.0) libcurl/7.21.7 OpenSSL/1.0.0e zlib/1.2.5 libidn/1.22
> Host: localhost:9990
> Accept: */*
> Content-Type:application/json
> Content-Length: 0
> 
< HTTP/1.1 401 Unauthorized
< Content-length: 0
< Www-authenticate: Digest realm="ManagementRealm",nonce="6089edca29aa27b064aa1db42d9651eb"
< Date: Fri, 13 Jan 2012 09:54:39 GMT
< 

First request has been sent and the server replied with a 401 unauthorized and the nonce to use. Now curl continues:

* Connection #0 to host localhost left intact
* Issue another request to this URL: 'http://localhost:9990/management/'
* Re-using existing connection! (#0) with host localhost
* Connected to localhost (127.0.0.1) port 9990 (#0)
* Server auth using Digest with user 'heiko'
> POST /management/ HTTP/1.1
> Authorization: Digest username="heiko", realm="ManagementRealm", nonce="6089edca29aa27b064aa1db42d9651eb", uri="/management/", response="78b9546e7485b661121e34a72d2979f1"
> User-Agent: curl/7.21.7 (x86_64-apple-darwin10.8.0) libcurl/7.21.7 OpenSSL/1.0.0e zlib/1.2.5 libidn/1.22
> Host: localhost:9990
> Accept: */*
> Content-Type:application/json
> Content-Length: 25
> 
< HTTP/1.1 200 OK
< Transfer-encoding: chunked
< Content-type: application/json
< Date: Fri, 13 Jan 2012 09:54:39 GMT
< 
* Connection #0 to host localhost left intact
* Closing connection #0
{"outcome" : "success"}
$ 

So we've issued the equest and the server has shutdown. Using the same technique you can also e.g. query the port, the http-connector is listening on (which has the symbolic name of 'http'):

curl --digest -u heiko http://localhost:9990/management/ -HContent-Type:application/json  --data @-<< -EOF-
{
   "operation":"read-attribute",
   "address":[
       {"socket-binding-group":"standard-sockets"},
       {"socket-binding":"http"}
    ],
   "name":"port"
}
-EOF-

In this example you also see how to pass the address of the node to inspect and the name of the attribute to the server.

Beware that if you make a typo in the json-encoding (e.g. separating key and value by comma instead of colon), the server may just respond with a 401 without telling you what went wrong

Perl

It's a long time since I did serious perl coding, so that next example may not be the most elegant. The example shows again, how to retrieve the http port via a 'read-attribute' operation. As I don't want to obsfuscate the code even more, I did just provide the password in the script.

#!/usr/bin/perl

use JSON qw(objToJson jsonToObj from_json to_json decode_json);
use LWP;

$host = "localhost";
$port = "9990";

$realm = "ManagementRealm";
$user = "heiko";
$password = "okieh";

# Construct url of management api
$url = "http://$host:$port/management";


# the command to send to the server in JSON encoding
$json_data = '
{
   "operation":"read-attribute",
   "address":[
       {"socket-binding-group":"standard-sockets"},
       {"socket-binding":"http"}
    ],
   "name":"port"
}
';
# set up a User agent
my $browser = LWP::UserAgent->new();

# Create the request
my $req = HTTP::Request->new(POST => $url);
$browser->credentials("$host:$port",$realm,$user,$password);
$req->content_type( 'application/json');
$req->content($json_data);

# send the request to the server
$res = $browser->request($req);

# If we don't get a 200 back, we finish here
die "No success ", $res->status_line unless $res->is_success;

# Get the content from the response
my $seite_code = $res->content;
print "Received : $seite_code \n";

# decode the json retieved
my $json = JSON->new->utf8;
$obj = $json->decode($seite_code);
%pairs = %{$obj};  # json->decode returns a hash ref
# get the result
$httpPort = $pairs{"result"};
print "Http port is $httpPort \n";

The basic part to handle the digest authentication is $browser->credentials("$host:$port",$realm,$user,$password);, which makes LWP transparently handle the creation of the digest and re-sending of the request.

Ruby

Unlike perl, which I was using a lot in the past, I am not yet familiar with Ruby, so there may be a much better solution -- please provide some feedback. Especially I have not found a good way to automatically handle the digest authentication, so this is done explicitly

#!/opt/local/bin/ruby1.9

require 'json'
require 'net/http'
require 'net/http/digest_auth'

url = URI.parse('http://localhost:9990/management/')
url.user = 'heiko'
url.password = 'okieh'

# data to send to retrieve the server name
data = { "operation" => "read-attribute",
         "address" => [],
         "name" => "name"}

h = Net::HTTP.new url.host, url.port

# send first request to get nonce
req = Net::HTTP::Post.new url.request_uri
res = h.request req

So far we have sent a first request to obtain the 'nonce' from the server, so we can compute the digest in the 2n step.

# compute the digest
digest_auth = Net::HTTP::DigestAuth.new
auth = digest_auth.auth_header url, res['www-authenticate'], 'POST'

# Now send the real request with the nonce
body = JSON.generate(data)

puts "Sending "  + body
req = Net::HTTP::Post.new url.request_uri
req.add_field 'Content-Type', 'application/json'
req.add_field 'Authorization', auth
req.body = body

res = h.request req

print "Result " + res.body

# parse the JSON and obtain the 'result' object
data = JSON.parse(res.body)
server_name = data["result"]
print "Server name is " + server_name

Thanks

I want to thank Darran Lofthouse for helping me to get going with why apparently correct requests fail with a 403 (because of the wrong content type).

Some graphing fun with D3.js

Now that the RHQ REST api can expose raw numerical metrics for the last 7 days, it is possible to create additional graphs for numerics. As before I have used D3.js to create the following graphs. The source is now in RHQ git master and will also be in RHQ 4.3.

The first graph shows the last 7 days of metrics in one go:

raw_graph.html

This one shows the last 7 days where each span of 24h is represented by a colored line, which allows to compare the values directly - the darker the color, the older the data:

raw_graph7.html

This needs some more work, as the x-axis labeling does not yet take the exact start time into account.
Also time spans that have no values (e.g. because agent is down) should not show a straight line, but no line at all for that period. There should perhaps be a legend about the colors as well.

I am still very much on the learning side of JavaScript and D3.js and the above is far from the beautiful examples Mike Bostock creates, but I think one can already see the potential power here.

If anyone is interested to add tooltips for the values or fix the label for the second graph, please ping me.

Analyze your RHQ metrics with R

Plot of metric aggregates in R

You probably have seen that with RHQ 4.2 you can export the aggregate metrics for the last n (default 8) hours via the REST api by calling:

http://localhost:7080/rest/1/metric/data/<scheduleId>

where <scheduleId> must be a valid numerical schedule.

The statistical tool R allows via "RCurl" to load data from remote URLs e.g. like this:

json_file <- getURL("http://localhost:7080/rest/1/metric/data/10013", httpheader=c(Accept = "application/json"),userpwd="rhqadmin:rhqadmin")


RCurl allows you to specify username and password unlike the simple json_file <- "http://..." calls. The next step is then to transform the received data with the help of the "rjson" library into R data structures:

## convert json to list of vectors
json_data <- fromJSON(paste(json_file, collapse=""))


## convert the embedded data points into a data frame
df <- data.frame(do.call(rbind,json_data$dataPoints))

which you can then access and e.g. plot:


## plot the data
plot(df$timeStamp,df$value,xlab="time",ylab="Free memory (bytes)",xaxt='n',type='l')


You can find the full example as plot_metrics.r in the RHQ samples project on GitHub.

Some like it raw...

RHQ master (this will make it into RHQ 4.3) is now also able to export raw metrics via REST api


http://localhost:7080/rest/1/metric/data/10013/raw?duration=259200


Like above you provide the schedule id and optionally a startTime and endTime or an endTime and a duration (in seconds). If nothing is provided, the data for the last 8h is exported. The following screenshot shows an example plot:

The metrics are plotted in black, the average in blue, the 5% and 95% quantils in orange and green and with the help of the TTR library, the 50 samples moving average is plotted in red. The sample code (with slightly different parameters) is also available online as plot_raw.r.

If you find other cool usages like e.g. Holt-Winters prediction on the data, please consider submitting them as example to the samples project.

Announcing the RHQ samples project

I am proud to announce the inception of a the samples project for RHQ.
This is meant to collect samples and extensions around RHQ that could be used in RHQ and also JBoss ON. Users can go there and copy and paste the solutions to get a quicker start themselves.

While this project is started by the RHQ project team, everyone is encouraged to come and use the samples, fork the project and enhance it. To make this process easier, the project is hosted on GitHub at

https://github.com/rhq-project/samples

The initial content centers around the new REST api of RHQ, and shows how to use this API from Ruby or Python and how to build reports from JasperReports.

Wanja Pernath has also submitted his group control scripts using the CLI in RHQ and JBoss ON.

Join me and RHQ in London next monday

As written before, I will talk at the London JBoss User Group next monday about RHQ and JBoss ON.

You may register on the meetup page, which also lists the venue and the directions to get there.

RHQ 4.2 released

The RHQ team is proud to have released version 4.2 of the systems management and monitoring framework.

 

 

This release focuses on:

• UI enhancements / bug fixing
• Completion of drift management (this allows you to e.g. detect changes to files on your managed platform and to alert on such drift)
• Improvements in server config sync (you can basically take (parts of) the configuration of one RHQ server and apply it to a different RHQ server).

In addition to that, the release is the very first to have a (limited) REST interface. Another change is the added support for PostgreSQL 9.1.

We want to thank

• Ted Won for contributing Korean installer messages
• Rafael Torres for improvements to the Twitter plugins (OAuth support)
• All community members that provided input and bug reports

As always check the release notes for a more complete list of changes. You can get the downloads from sourceforge. And please provide us feedback; for the REST interface we are also looking for usage examples.

 

Heiko Rupp will showcase RHQ 4.2 at the November meeting of the London JBUG.

Evolving interface in REST with JAX-RS (updated)

[update] In the earlier version I had if (mediaType==MediaType.xxx), which is wrong. MediaType is no enum, so we need to compare via .equals()[/update]

 

So in RHQ we've introduced the REST api and one of the nice things is the ease of use like this:

  @GET
  @Path("/customer/{id}")
  public Customer getCustomer(@PathParam("id") int custId) ;

where the implementation code e.g. looks up a Customer object in  the database and then returns it. Clean expressive API.

Now things start getting more interesting when you consider returning XML, JSON and HTML versions of the document. The first naive way goes like this:

  @GET
  @Path("/customer/{id}")
  public Customer getCustomer(@PathParam("id") int custId) ;

Which fails big time for HTML as (at least RESTEasy) does not know how to serialize the Customer object into HTML. So the next version of the API could look like this:

  @GET
  @Path("/customer/{id}")
  @Produces({"application/json","application/xml"})
  public Customer getCustomer(@PathParam("id") int custId) ;

and

 @GET
 @Path("/customer/{id}")
 @Produces("text/html")
 public String getCustomerHtml(@PathParam("id") int custId) ;

Where both methods get the customer from the backend. The first one then returns the object directly (as above) whereas the second one renders the object with the help of Freemarker templates into a HTML string and returns this one. Just returning String in both cases is also not working as one may think. So this two methods that internally use almost the same code (actually the *Html one calls the other to retrieve the Customer object).

This works quite nicely, but now I want to start adding explicit return codes (and also caching information). JAX-RS offers the Response class for this, where you can e.g. say Response.ok() or Response.notFound(). So the interface changes to

  @GET
  @Path("/customer/{id}")
  @Produces({"application/json","application/xml"})
  public Response getCustomer(@PathParam("id") int custId) ;

and

 @GET
 @Path("/customer/{id}")
 @Produces("text/html")
 public Response getCustomerHtml(@PathParam("id") int custId);

Internally we have the same situation as before, but can now explicitly return the result codes we want. This is still not optimal. Luckily JAX-RS allows to inject the Request and the HttpHeaders into the called Java-method, so that we can now write

  @GET   
  @Path("/customer/{id}")   
  @Produces({"application/json","application/xml","text/html"})   
  Response getCustomer(@PathParam("id") int custId,
                    @Context HttpHeaders headers);

and then in the implementation do the following:

  MediaType mediaType = headers.getAcceptableMediaTypes().get(0);

  ResponseBuilder builder = ...

  if (mediaType.equals(MediaType.TEXT_HTML_TYPE)) {
     String html = renderTemplate("customer", customer);
     builder = Response.ok(html, mediaType);
  }
  else {
     builder = Response.ok(customer);
  }

So now we have one method doing all the work for us without code duplication and which can then use the same logic for caching and e.g. paging or linking.

Support for caching is now only one more step away:

  @GET   
  @Path("/customer/{id}")   
  @Produces({"application/json","application/xml","text/html"})   
  Response getCustomer(@PathParam("id") int custId,
            @Context Request request,
              @Context HttpHeaders headers);

and in the implementation

  // Check for conditional get

  String tagString = Integer.toString(customer.hashCode());
  EntityTag eTag = new EntityTag(tagString);       
  Date lastModifiedInDb = new Date(customer.getMtime();

  Response.ResponseBuilder builder = request.
           evaluatePreconditions(lastModifiedInDb,eTag);

  if (builder==null ) { 
      // we need to send the full resource
      if (mediaType.equals(MediaType.TEXT_HTML_TYPE)) {
            String html = renderTemplate("customer", customer);
            builder = Response.ok(html, mediaType);
      }  else {
            builder = Response.ok(customer);
     }
  } 

  builder.tag(eTag); // Set ETag on response

There is of course still a lot to do, but we have now achieved :

• uniform interface independent of the media type
• support for conditional sending and thus caching and re-validation by ETag and time stamp

The last step is to add some hints to the client how long it may cache data without the need to go out to the network to do any caching at all. Again JAX-RS has already support for that:

  // Create a cache control       
  CacheControl cc = new CacheControl();       
  cc.setMaxAge(300); // Customer objects are valid for 5 mins       
  cc.setPrivate(false); // Proxies may cache this
  builder.cacheControl(cc);
 

This gives the client a hint, that they can consider the customer object valid for 300s = 5min. Proxies on the way are also allowed to cache the returned object. In practice one may make the maxAge depending on e.g. some average update frequencies and also set the "you need to always verify" flag via cc.setMustRevalidate(true).

I am sure, this is not yet the last version of the interface, but you can see how it can evolve over time and add new features like support for conditional get. And the best part is that so far the clients don't even have to change a single line of code.

In the future we may want to introduce our own media types like appliation/vnd.rhq-customer+json that newer clients then can make use of. The server can dispatch as seen above the media type and return the appropriate representation. The existing clients, that do not know the new media type can still be serviced by the server just sending this "old" version of it.

 

RHQ and JBoss ON at JBug London (updated)

I will talk on monday, Nov 28th at the London (UK) JBoss User Group about RHQ and JBoss ON. The talk will give an overview over RHQ and JBoss ON, showcase the new features in RHQ 4.2 and also talk a little bit about the new REST api.

The talk venue is the Skills matter eXchange, where the meetup page has the link to the location and also a link to the registration.

The guys from C2B2 also created a cool poster:

 

Small recap of JBoss One Day Talk in Munich (updated)

Yesterday I was at JBoss One Day Talk, organized by the local JBoss Users Group. The schedule consisted of 18 sessions in three parallel tracks.

One of the main topics was of course JBoss AS 7, where Heiko Braun was giving an overview, Thomas Diesler talking about OSGi on AS7 and myself talking about management of the server doing a lot of live demoing.

Other JBoss speakers included Kris Verlaenen on JBPM5 and Sashin Shinde on Open Shift. And there was Gavin King announced to talk about Ceylon. While I was looking forward to meet Gavin again, I was also afraid that I would need to talk only to myself with the Ceylon talk being in parallel. (Un :-)fortunately Gavin was not able to make it, so I had a great audience.

My buddy Pavlo Baron was talking about Big Data (slides are available on Slideshare). While this presentation was not as technical as I hoped, it nevertheless got me thinking about some stuff we could integrate in a future version of RHQ.

This conference was organized for the second time this year and I like the intimate atmosphere a lot. And of course the speakers dinner at Löwenbräukeller :-)

Update:

The organizers of One Day Talk have posted some interviews (in German) -- and especially one with Heiko Braun

 

Some notes from the RHQ team meeting (and a team photo)

Last week, the whole RHQ development team met in the Red Hat office in Westford (near Boston). It was very nice for me to finally see some of the old colleagues again after two years and also to meet the new colleagues on the team.

And of course, we took a team photo

We had some very good discussions about possible future features (like e.g. supporting a REST style API into the system or on the future of alerts) - I will post more information soon. In the mornings we had show'n'tell style session where each developer was presenting some area of work to the team.

After work we did some more recreational things like:

• Seeing the Boston Red Sox win 18:9 in a 4h base ball game
• Playing minigolf at Kimball Farm (and ate huge amounts of ice cream)
• Had dinner at nice places

One of things we also did is to publish some sort or Roadmap for RHQ and discussed that we will move the source to GitHub at some time and also split up the build in an easier to build way. We will post about that when we are making more progress on this.

Ah and did I mention "Beer"?

REST coming to RHQ (updated)

In the past few weeks I was also working on implementing a sample REST interface for RHQ (see also here and here). At our team meeting I've showcased the work I've done in the heiko-rest branch of our git repository and we have decided to merge that into the RHQ master branch sooner or later.

UPDATE: this has been merged today (2011/10/09); if you build from source, you need to build the container again.

To stimulate your appetite I will show some samples what you could do with the REST api:

Returning resources in html representation

 

and as JSON

Render metric graphs in Javascript with the help of D3.js

Or an alternative resource tree in Javascript with the help of D3.js

Providing data to an Android client application (this is not very sophisticated yet, but should illustrate the idea):

RHQpocket as REST client to the RHQ server from Heiko W. Rupp on Vimeo.

List / add / delete favorite resources:

$ curl localhost:7080/rest/1/user/favorites/resource -u rhqadmin:rhqadmin
[{"resourceWithType":{"@resourceId":"10160","pluginName":"RHQServer"
       ,"resourceName":"Measurement Subsystem","typeId":10235
       ,"typeName":"RHQ Server Measurement Subsystem"}}
 ,{"resourceWithType":{"@resourceId":"10361","pluginName":"jboss-as-7"
       ,"resourceName":"\/devel\/jbas7\/jboss-as\/build\/target\/jboss-as-7.1.0.Alpha2-SNAPSHOT","typeId":10055
       ,"typeName":"JBossAS7-Standalone"}}]

Add resource with id 10013 to favorites:

$ curl localhost:7080/rest/1/user/favorites/resource/10013 -X PUT -u rhqadmin:rhqadmin

$ curl localhost:7080/rest/1/user/favorites/resource -u rhqadmin:rhqadmin
[{"resourceWithType":{"@resourceId":"10013","pluginName":"Platforms"
       ,"resourceName":"en0","typeId":10024
       ,"typeName":"Network Adapter"}}
 ,{"resourceWithType":{"@resourceId":"10160","pluginName":"RHQServer"
       ,"resourceName":"Measurement Subsystem","typeId":10235
       ,"typeName":"RHQ Server Measurement Subsystem"}}
 ,{"resourceWithType":{"@resourceId":"10361","pluginName":"jboss-as-7"
       ,"resourceName":"\/devel\/jbas7\/jboss-as\/build\/target\/jboss-as-7.1.0.Alpha2-SNAPSHOT","typeId":10055
       ,"typeName":"JBossAS7-Standalone"}}]

And remove it again from favorites:

$ curl localhost:7080/rest/1/user/favorites/resource/10013 -X DELETE -u rhqadmin:rhqadmin

 

Of course those are only some examples and should illustrate that:

• the api will be usable from different clients
• from different programming languages
• it will allow write / update access
• it will allow users to write extensions to RHQ like e.g. Über-consoles that we do not even think of

We were also thinking of the possibility for agents in other languages being able to push events and metrics over this interface.

The code is currently in the heiko-rest branch in RHQ-git (at fedorahosted), but we plan to merge it into master as soon as possible. If you want to try the code, check out the branch and either build RHQ from scratch. Or try running /etc/dev-utils/setup-rest/setup.sh to populate the server with the needed libraries (edit that file first), then build server/jar and gui/rest-war in dev-mode so that the artifacts are copied to the server. Then edit rhq.ear/META-INF/application.xml to include the rest war.  Of course all this will not be needed after we have merged the code in the master, as the normal build will then have all the artifacts.

The code for the Android app is available from my repository on GitHub

An important part of this whole effort is your input:

• Provide us with your requirements e.g. on the Design - REST page of the RHQ-Wiki
• Contribute to the code
• Show us your examples you have built on top of the API (and perhaps even contribute them)

 

Please help us making this great so that you can create great stuff with the interface.

RHQ 4.1 released

Actually this is not new, but already three weeks old - I wanted to add it here, for completeness sake:

RHQ 4.1.0 has been released on Sept 2nd. Make sure you read the release notes and then proceed to the download.

Major changes include translations of the UI into Chinese, Japanese and Portuguese. Big thanks go to Fusayuki Minamoto, Jijun Liu, Zhongqianglee and Rafael Torres Coelho Soares. Also in this release we have started to detect changes in configurations and files ("Drift"), made enhancements in the area of Bundles (provisioning) and added a first shot at a plugin to manage JBoss AS 7. Last but not least, Elias Ross has enhanced the SNMPTrapd plugin.

As always: please try the release and give us feedback.

 

RHQ 4.1.0 BETA released

The RHQ team has just released version 4.1.0.beta of the RHQ systems management and monitoring suite. This release serves as a preview of the upcoming version 4.1.0 and has a wealth of new features.

Some of the new features are work in progress and will only be finished after 4.1, which means that you can still influence the features with your ideas and feedback.

Release notes are on the RHQ wiki and also contain the link to the download at the bottom.

We want to especially thank Elias Ross, Fusayuki Minamoto, Jijun Liu, Zhongqianglee

and Rafael Torres Coelho Soares for their contributions (even if some are not in this build)

 

Some progress with REST for RHQ and new questions

As written some time ago on the rhq-devel mailing list, I've started implementing a RESTful interface for RHQ. A page on the RHQ wiki shows the requirement and some progress about the implemented API. Development currently happens in a personal feature branch in order to have some code to play around and gather some experience from.

One thing that I found out over the weekend is that at least with JAX-B setting up the interface and REST resource classes is far from trivial and I have been surprised a lot by interesting error messages.

Anyway, the current state of the API is already in a somewhat usable shape (mostly read-only at the moment). The implementation has been done via RESTeasy (RE). Security integration with the RHQ security model is also done and the session bean methods can just use a Subject called caller.

So, having solved some stuff rises even more questions:

1. Would stuff get easier when not trying to support XML, but only Json?
2. How to use much / more of the original RHQ domain model without running into LazyLoadExceptions all over the place (RE runs as a servlet to do the marshalling - at this point in time the connection to the entity manager is already closed).
3. How to implement linking between related classes / concepts. E.g. a Resource can have Alerts. Now instead of embedding Alert elements when returning a Resource object, I would like to at most have a list of Links to /alert/{id}. I feel that the answer may have to do with the next question
4. In RHQ we often pass just primitive ids around (e.g. int resourceId) How can easily turn them into a link to the target Resource (given that Resource lives below /resource/{id})
5. In the Atom-PUB linking I would add links to sub-resources like e.g. /resource/1234/availability or /resource/1234/schedules without the need to first obtain those values from the database.
6. Does it make sense to transfer date/times as long? Especially as Json seems to have some issues there. Complex date strings require a lot of processing on server and client.
7. Is there a preferred way to expose id elements ((XML) attribute, element, not at all? With Atom-PUB links present, I think the id would not need any special treatment at all, as it is available in the URI of the resource and in the link.

A part of the solution to questions 3+4 seem to be to only use Objects in REST domain objects, as the marshalling will skip elements where the corresponding object is null, so for e.g. the parent Resource it is enough to initialize its resource with the Resource id to only include this, but not all the other fields that should then be obtained from their real URI.

 

But then I am not sure how much my ideas are RESTful at all :-)

I would love to get some feedback here.

How to survive conf calls with elevator music

I guess you all know the situation: you have dialed into a conference call but the leader is not yet present. So the conference call bridge plays elevator music or soft jazz or ... When you hear that the first time it is nice but for people that spend a good part of the day in virtual meetings this gets boring. You end up in a state like this:

Actually there is a solution to this:

And meetings may even become more interesting if you do not pull out the head-phones when the meeting actually starts :-)

RHQ tip of the day: plugin version

I recently had the issue while working on the AS7 plugin that no matter what version I was using in the maven pom, the server always printed a message like the following on deployment:

22:09:51,082 INFO  [ProductPluginDeployer] Newer version of [jboss-as-7] plugin 
   found (version 4.0.0-SNAPSHOT) - older version (4.0.0-SNAPSHOT ....

And this happened with a pom version of 4.1.0-SNAPSHOT. The manifest file also showed 4.1* and the file name also had 4.1.0* in it and not 4.0.0.

It turned out that I had an explicit entry like this in the plugin descriptor, rhq-plugin.xml:

<plugin name="jboss-as-7"        
        displayName="JBoss-AS-7-Plugin"
        package="org.rhq.modules.plugins.jbossas7"
        version="4.0.0-SNAPSHOT" >

This explicit version overwrites all the other version qualifiers that maven creates; and a maven release cycle will not update it as well. After removing the version attribute from the plugin element, versioning was as expected.

So if you need a stable version, encode it in the plugin descriptor. Otherwise leave it to the build magic to provide the version.

 

Updated slides from my JFS talk about JBossAS7 available

I've put the updated slides of my presentation about "Neuerungen in JBossAS7" at this year's Java-Forum-Stuttgart online. You can download them as PDF.

 

Who's messing with my classes - IntelliJ! (UPDATED)

So I have the following code in one of my classes:

    public void setName(@NotNull String name) {
        this.name = name;
    }

In some situations when I use them, I get errors like this (I do not want to discuss right now that the annotation is wrong when I am complaining about it throwing errors):

Caused by: java.lang.IllegalArgumentException: Argument 0 for @NotNull
   parameter of org/rhq/core/domain/resource/Resource.setName must not
   be null
        at org.rhq.core.domain.resource.Resource.setName(Resource.java)

 

The @NotNull Annotation is coming from the Jetbrains org.jetbrains.annotations package.

Decompiling Resource.class with jad Resource.class shows this:

    public void setName(String name)
    {
        if(name == null)
        {
            throw new IllegalArgumentException("Argument 0 for @NotNull parameter of org/rhq/core/domain/resource/Resource.setName must not be null");
        } else
        {
            this.name = name;
            return;
        }
    }

So someone is inserting this check for if(name==null) on compile time. As we are building with maven and other team members do not see this, it looks like a local issue. But then, other team members have a very similar environment than I do.

[UPDATE]

I was suspecting IntelliJ already, as my colleague Ian was talking about the @NotNull compiler setting, but I turned that off and on and it had no effect.

After the comment by Scott Vachalek I went over this again -- especially as we are not using the IntelliJ compiler in our maven runs. I enabled the option, and compiled on command line. Nothing happened.

 

 

And then today I was running some unit tests from within IntelliJ and saw the "Looking for classes to compile ..." message in IntelliJ UI and then also messages about compiling. Then ran jad again and saw those checks again.

So it turns out that IntelliJ is compiling my classes with those @NotNull annotations again. And as this is in a maven build, it is putting the resulting classes in the places where a command line compile would also put them. My next command line build will then see that the class files are newer than the sources and not recompile them, but just include them in the resulting artifact.

 

Seemingly common RESTEasy error and a solution

So I was playing with RESTeasy a bit and using the standalone servlet method of exposing REST services was very easy and straightforward.

And then I thought, ok, let me now do the EJB integration on a JBoss AS 4.2.3 - and ran into the following error:

java.lang.RuntimeException: Class is not a root resource.  It, or one of its interfaces must be annotated with @Path: $Proxy862 implements: org.rhq.enterprise.server.rest.ResourceHandlerLocal org.jboss.ejb3.JBossProxy

Which was strange as I did have the @Path annotation present:

@Local
@Produces({"application/json","application/xml","text/plain"})
@Path("/resource")
public interface ResourceHandlerLocal {

Googling around showed, others have/had the same issue; but as so often, no solution was given. After some digging around I found the issue: the error message is a bit misleading, and I was actually missing a @ApplicationPath("/rest") annotation, which defines the "root context" and which is not needed in the simple standalone war case.

So I implemented a subclass of javax.ws.rs.core.Application which is marked with this ApplicationPath annotation and registered it in web.xml:

<!DOCTYPE web-app PUBLIC
 "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
 "http://java.sun.com/dtd/web-app_2_3.dtd" >

<web-app>
    <display-name>RHQ Rest interface</display-name>

    <context-param>
        <param-name>resteasy.jndi.resources</param-name>
        <param-value>rhq/ResourceHandlerBean/local</param-value>
        <description>List of jndi names of EJBs local interfaces that define REST stuff</description>
    </context-param>


    <servlet>
        <servlet-name>Resteasy</servlet-name>
        <servlet-class>
            org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher
        </servlet-class>

        <init-param>
            <param-name>resteasy.scan</param-name>
            <param-value>false</param-value>
        </init-param> <init-param>
            <param-name>javax.ws.rs.Application</param-name>
            <param-value>org.rhq.enterprise.server.rest.RHQApplication</param-value>
        </init-param>
    </servlet>

    <servlet-mapping>
        <servlet-name>Resteasy</servlet-name>
        <url-pattern>/*</url-pattern>
    </servlet-mapping>
</web-app>

And now the REST provider is starting nicely :)

Of course, if you are using JBoss AS 6 or the upcoming AS7, you do not need to go through all this, as they have RESTeasy already integrated and ready to go.

Two nice days at Android DevCamp Stuttgart 2

In Stuttgart, we had an Android Developer Camp organized by the local Google Technology User Group.

This was a two day camp with workshops and a hackathon on the first day and a bar camp style set of sessions on the second one.

Some of the almost 200 plush Androids

On the first day I went to the Hackathon and worked on Android 3 aka Honeycomb migration issues together with Fridger from Openintents. On the second day I was in a few sessions about app mashups, home screen widgets, git and also gave a session that was talking about what I have done in the Android 2->3 migration so far. My hope was to learn from some participants, but it looks like tablets are not yet that popular. Slides of my session are here.

Samsung, who were one of the main sponsors of the event, sent some developer advocates with new 8.9 and 10.1 tablets to play with, which was nice too.

Food was good and the Androids were plenty, so this was a good event. Thanks to all sponsors and the organization team.