Wednesday, September 30, 2009

RHQ plugins - platform services

There was recently a question on the Jopr devel mailinglist, that Ian Springer answered - I've taken his answers to write this post, as this is of general interest to plugin writers.

Bruno asks:
I have written a plugin which defines a single service like so:
<plugin ...>
<service
name="Interdomain Service"
...
supportsManualAdd="true">
</service>
</plugin>

But during auto-discovery, the resource discovery component does not get called. It only gets called, if I define the Interdomain Service to be a server. What am I missing here?

Lets first have a look at the resource hierarchy within RHQ:
PSS-hierarchy.png


So for <server>s that are standalone within a <plugin> tag, it is obvious that the parent is the <platform>, while <service> can have both other ResourceCategory as parent.

This means that you have to define the service's parent type(s), so RHQ knows where to stick it in the type hierarchy once it's discovered. There are a few ways to do this, depending on your needs.

If it's a platform service (i.e. a service whose parent is the platform
itself), you'd do something like:

<service name="Interdomain Service" ...>
<runs-inside>
<parent-resource-type name="Windows" plugin="Platforms"/>
<parent-resource-type name="Linux" plugin="Platforms"/>
<parent-resource-type name="Solaris" plugin="Platforms"/>
<parent-resource-type name="HP-UX" plugin="Platforms"/>
<parent-resource-type name="AIX" plugin="Platforms"/>
<parent-resource-type name="FreeBSD" plugin="Platforms"/>
<parent-resource-type name="Mac OS X" plugin="Platforms"/>
<parent-resource-type name="Java" plugin="Platforms"/> <!-- any OS that RHQ doesn't have native (i.e. SIGAR) support for -->
</runs-inside>
...
</service>


(If your service will only run on certain OS'es, you could remove the
unsupported platform types from the list of parent types).

Or you could do something similar to say it can be the child of multiple
server or service types, e.g.:

<service name="Interdomain Service" ...>       
<runs-inside>
<parent-resource-type name="JMX Server" plugin="JMX"/>
<parent-resource-type name="JBossAS Server" plugin="JBossAS"/>
<parent-resource-type name="JBossAS Server" plugin="JBossAS5"/>
</runs-inside>
...
</service>


Finally, if you want it to be a child of a single server or service type
defined in the same plugin, just nest it inside the parent server or
service element, e.g.:

<server name="CoolServer">
<service name="Interdomain Service" ...>
...
</service>
...
</server>



The reason it works if you change it to a server, rather
than a service, is because, when parsing the plugin descriptor, if RHQ
encounters a server element at the top level with no runs-inside types
defined, it assumes the server is a platform server (a server who has
all the platform types as its parents). So:

<server name="Interdomain Service" ...>
...
</server>


is essentially equivalent to:

<server name="Interdomain Service" ...>
<runs-inside>
<parent-resource-type name="Windows" plugin="Platforms"/>
<parent-resource-type name="Linux" plugin="Platforms"/>
...
<parent-resource-type name="Mac OS X" plugin="Platforms"/>
<parent-resource-type name="Java" plugin="Platforms"/> <!-- any OS that RHQ doesn't have native (i.e. SIGAR) support for -->
</runs-inside>
...
</server>


Most of the time when writing a plugin you will probably start with a server at the top level of your plugin, so it will just work.

1000 km :-)

Last weekend I crossed the 1000km mark since I started running last year (see here). I took me quite some time since the 800km mark at Stuttgart-Lauf. This had to a big extend to do with the fact that I was not really able to go out much while kids were on vacation from Kindergarden and also needed some rest after Üetliberg. But since start of september I am back on track :-)

So what are my next goals? I don't really know yet. Running Stuttgart-Lauf faster than this year probably. And just improving my overall fitness to be able to run at same pace with a lot lower heart rate.

Thursday, September 24, 2009

RHQ / Jopr agent waiting at startup

I have written in the past about RHQ and Jopr "agent waiting at startup. That actually was before high availability has been released, so it is a little outdated.

With high availability (HA) you will see one new type of issues: ip address forward and backward mappings do not match.

Basically you need to make sure that the IP address of your computer can be reverse mapped to the computer name and that this name mapps back to the same ip address.
And this needs to be true along all your hosts.

Suppose you have an IP address of 172.31.7.7, then the results of name resolution should look like the following:


$ dig -x 172.31.7.7
[...]
;; ANSWER SECTION:
7.7.31.172.in-addr.arpa. 86400 IN PTR snert
$
$ dig snert
[...]
;; ANSWER SECTION:
snert 74030 IN A 172.31.7.7


If you had agent-server communication working before and it stopped all of a sudden, go to the Jopr server UI, to the Administration -> High Availability -> Server section and check if the name shown there matches what you expect - same for the agents.


If this all works, and the agent is still hanging, have a look at the older post to rule out other possibilities for misconfiguration.

New api for manual discovery on RHQ and Jopr

The other day I was writing about manual discovery/addition of resources. Now that Jopr 2.3 (and its JBoss ON counterpart is out), we have updated the API for manual discovery to provide its own interface so that you don't need to fiddle around with finding the passed configuration.

The new API lives in the ManualAddFacet, which has one method
DiscoveredResourceDetails discoverResource(Configuration pluginConfiguration, ResourceDiscoveryContext<T> context)
throws InvalidPluginConfigurationException;
that you need to implement. This Facet is implemented in addition to the ResourceDiscoveryComponent within the discovery component. Ips has already converted all existing plugins to the new api, so you can have a look at many examples there, so I'll only show a short one here, which is an excerpt from the Twitter plugin:


public class TwitterDiscovery
implements ResourceDiscoveryComponent,ManualAddFacet {
 
// Auto-discovery not supported, so return an empty set
public Set discoverResources
(ResourceDiscoveryContext discoveryContext)
throws Exception {
 
// We don't support auto-discovery.
return Collections.emptySet();
}
 
// perform the manual add
public DiscoveredResourceDetails discoverResource(
Configuration pluginConfig,
ResourceDiscoveryContext discoveryContext)
throws InvalidPluginConfigurationException {
 
DiscoveredResourceDetails detail =
new DiscoveredResourceDetails(
discoveryContext.getResourceType(), //ResourceType
url + "_"+ user, // ResourceKey
url + " feed for " +user, // ResourceName
null, // Version
"One " + url + " user", // Description
pluginConfig, // Passed plugin config
null ); // Process scans
return detail;
}
}

I have also updated the plugin skeleton generator code in svn to support this new API and have uploaded the new version to jopr.org

Saturday, September 19, 2009

Jopr 2.3.1 released



The Jopr team has released version 2.3.1 of Jopr.

The major difference over Jopr 2.3 is a fix in the dbupgrade scripts.

If you are still on 2.2 and want to upgrade, use 2.3.1, as 2.3 will not correctly update your database schema.

You can download the release from SourceForge.

Friday, September 18, 2009

Jopr talking


I have added an experimental alert sender to Jopr that can call administrators on their phone, tell them the alert conditions and can then take input via DTMF tones. This input can be used to disable or re-enable the alert definition or to clean out this alert from the alert history.

This all has been made possible by via the Mobicents server and SIP servlet.

Mobicents logo

A special thanks goes to Jean Deruelle from Mobicents, who wrote the integration part on the Mobicents side.

Podcast episode

I have created an episode of the Jopr podcast that shows the setup and some UI demo (the call on the SIP phone has been faked a little bit, as my screen capture program is always crashing at this point, but the message read is real).

As usual, this podcast episode is available on iTunes.

Show me the code...

The code on the Jopr side lives in http://svn.rhq-project.org/repos/rhq/branches/HEIKO-EXP/. There is also code on the Mobicents side, but we still need to clean up things and write some documentation on how to integrate them.

Ping me if you are interested in this integration.

Monday, September 14, 2009

Snow Leopard - mixed feelings?

So far Snow Leopard is somewhat disappointing for me:

  • DNS - especially when VPN is on sucks, as SL is sometimes blacklisting very valid hosts all of a sudden. This results in e.g. "The computer is not connected to the internet" messages in Safari for sites like www.google.com (see e.g. here)

  • The upgrade broke Aperture - the upgrade process removed a framework needed by Aperture. For me re-installing /Library/Frameworks/Pluginmanager.framework from a TimeMachine backup worked. (see e.g. here)

  • Older Software like Parallels 2.5 no longer works

  • Nambu 1.2 no longer works (and Nambu 2 is still in closed beta)

  • X-Lite 3 has no sound in/out. Dialing works though. And the beta of X-Lite 4 is not even able to show the preferences.



The biggest item for me is currently the DNS one, as this renders the system more or less unusable.


But then there is at least some good thing about it:


  • The internal VPN now works against Cisco concentrators, which is very nice

  • JDK 1.6 is finally available on 32 bit macs like my 3 years old MacBook



I will update this post, when I see more pros and cons.

Friday, September 11, 2009

Jopr podcast iTunes URL has changed

The Jopr podcast is now available on iTunes at the following new place
http://itunes.apple.com/WebObjects/MZStore.woa/wa/viewPodcast?id=331189266.

Please update your feeds or iTunes subscriptions accordingly. Thanks.

Thursday, September 10, 2009

New episode of the Jopr podcast: Manik from Infinispan (updated)

At JBossWorld Chicago I had the pleasure to do a short interview with Manik Surtani, lead of the Infinispan project.

The interview is now available in the Jopr Podcast home page and is also available on iTunes (note, the iTunes url has changed).

Enjoy!

Again: if you like it, please rate it up on iTunes and/or provide Feedback via email to me.

Tuesday, September 08, 2009

What a weekend

I am currentl staying in New Jersey, as the opportunity to give a talk at JBossWorld in Chicago also gave me the opportunity to go to NJ to meet the biger part of theJopr developers.

I arrived friday evening and as the car rental company ran out of cars, I got the last one remaining: a Chrysler Sebring convertible. On Saturday I went running and also drove around a lot with the roof open and music turned on - very nice :-)

Sunday I spent with Jess and family. Jess and I went climbing - my first time, but this was fun. One one side I can't imagine to do it more often, but on the other side I can :)

Today was Labour Day, so no one in America is working (only joking) and I went to New York. Walked around a lot and visited important landmarks like the Brookly bridge and the Apple store on 5th avenue.

Saturday, September 05, 2009

Jopr 2.3 released



Jopr version 2.3 has been released. You can find the binaries http://sourceforge.net/projects/rhq/ on SourceForge.

IMPORTANT UPDATE: 2.3.1 is out with a fix for dbupgrade

Notable changes are:
  • Support for JBoss EAP 5

  • A new command line interface: you can now talk to Jopr from a Linux or Windows command line. This interface uses the Java 6 script support, so that you can create scripts for re-use.

  • Group alerts: it is now possible to fire an alert when any resource in a group runs into an error condition by defining alert templates on the group

  • Experimental support for embedded H2 database and MS SQL Server (same as in 2.2.1)

  • Plugins can now depend one of several others: if you have e.g. a plugin that uses JMX it is now possible to either depend on JBossAS when the resource types are used within the JBossAS or depend directly (and only) on JMX plugin to discover standalone JVM versions of those resources.

  • Support for taking snapshots of resource configurations: it is now possible to gather e.g. all config files from one of your JBossAS servers and have that zipped up and put in a place on the Jopr server

  • Dynagroups have been extended to support searching more levels of ancestry

  • Postgres 8.4 is now officially supported

  • Autodiscovery of JBossAS jnp credentials: Jopr is now looking at the config files to obtain the security credentials for secured JBossAS instances, so that you do not need to type them in by hand in the inventory



And many many more improvements. You can see lists of closed Jiras for
RHQ and Jopr projects.

Matching SVN tags are http://svn.rhq-project.org/repos/rhq/tags/RHQ_1_3_0/ and Jopr_2_3_0.

As so often :) join us on irc.freenode.net #jopr or in the forums or or or :)

Thursday, August 27, 2009

Twitter plugin + alert sender for Jopr updated



I've updated the twitter plugin for Jopr:

Manual add instead of auto discovery

The Twitter feed is no longer autodiscovered, but you have to go to the platform and explicitly add it there.

Bild 11.png


The reason behind it is that most users don't want to have a twitter feed discovered on each agent.

To learn more about auto discovery vs. manuall add, have a look at my post "Jopr plugins: Autodiscovery vs. manual add".

Support for other Twitter-like services

The plugin is now also able to post to other twitter-like services like identi.ca (or your own private one based on the laconi.ca code).
The serverBase is http://identi.ca/api/ for Identi.ca, as well as the search base.
For Twitter, serverBase is http://twitter.com and searchBase is http://search.twitter.com/

If you select the right template, this will automatically be filled in for you.
Bild 12.png


Alert sender updated too

I've also updated the Alert sending in the HEIKO-EXP branch to be able to send to other networks than Twitter.
This means that in the system configuration, there is a new field for the base url of the services api. See above for possible values.

related...

I've just seen that Twitter is offering a streaming api. This may be a nice addition to the plugin. Today every request to obtain values opens a new connection to Twitter. With the streaming api, this could probably go away.

Wednesday, August 26, 2009

Jopr plugins: Autodiscovery vs. manual add


One thing that I have neglected so far in the plugin development articles for Jopr that I wrote is the possibility to just manually add a resource instead of having it auto discovered.

Why would you want to manually add a resource?

Some resources are virtual like e.g. Twitter feeds in the twitter plugin. If you have 100 agents, it makes no sense to either take the twitter feeds into inventory on all 100 of them or to have to ignore them all in the auto discovery portlet.

When to use auto discovery then?

Auto discovery is best used for more concrete resources like e.g. processes that can exist on a system (a JBossAS for example) or file systems etc. Those are items you usually want to monitor on each system.

Ok, show me the code

Well, actually the code is not that different from what you already know. Within ResourceDiscoveryComponent.discoverResources() the change is to check for passed resource configurations and use those to create the concrete discovered resource:

for (Configuration config :
(Iterable)
discoveryContext.getPluginConfigurations()) {
DiscoveredResourceDetails detail =
new DiscoveredResourceDetails(
discoveryContext.getResourceType(), // ResourceType
url + "_"+ user, // ResourceKey
url + " feed for " +user,
null,
"One " + url + " user",
config,
null );

return Collections.singleton(detail);
}
return null;

As the plugin container will only pass us one configuration at a time (even if the call to getPluginConfigurations() is returning a List), we can safely return the constructed detail. And this is already all :-)


One thing you need to change though is in the plugin descriptor:

<server
name="Twitter"
discovery="TwitterDiscovery"
class="TwitterComponent"
description="Twitter monitoring subsystem"
supportsManualAdd="true"

This supportsManualAdd is what lets the resource type show up in its parent (usually platform for a standalone server) "Manually Add" dialog:

Bild 11.png


For more resources related to plugin development have a look at this post.

Thursday, August 13, 2009

Monday, August 03, 2009

Twitter, OAuth, Java and Twitter4j

For alert notifications in Jopr, I am using the Twitter4j library, as well as for the twitter plugin.

As Twitter now requires the usage of OAuth to register new applications, I was looking into this as well. Luckily, Twitter4j has support for OAuth.

Unfortunately this did not work for me ;-(

After some digging, I found the solution (code is the one from the samples page):


...

System.out.println("Open the following URL and grant access to your account:");

System.out.println(requestToken.getAuthorizationURL());

System.out.print("Hit enter when it's done.[Enter]:");

String pin = br.readLine();

 

try{

  accessToken = twitter.getOAuthAccessToken(requestToken, pin);

...


So basically Twitter is returning a PIN to me after I have granted access - and this PIN needs to be passed to the call to get the auth token. This also means to use a different method than in the original example.


Friday, July 31, 2009

Experimental alerting via Twitter in Jopr (and RHQ)


Something for the weekend :-)

I have just committed support for sending alert notifications via Twitter into Jopr and RHQ.

Bild 12.png


The code currently lives in an experimental branch in the RHQ code base.

The twitter notification can be enabled in the usual place in the alert UI

Bild 11.png


while the twitter account is set in the System preferences

Bild 13.png


This code is still experimental, but works for me :-)

And then there is the Twitter plugin (and here), that allows you to watch twitter feeds from within Jopr.

Please provide feedback.

Wednesday, July 29, 2009

Jopr Twitter plugin renamed



Two days ago I have commited a Jopr plugin to monitor Twitter feeds

Due to popular demand (and also because 'twit' has some negative connotation in some slang forms of English) I have renamed the Jopr TwitStats plugin to Twitter plugin.
This also means that the location of the source has moved to
modules/plugins/twitter within the RHQ svn repository.

Monday, July 27, 2009

Jopr meets Twitter -- new plugin to monitor Twitter



As you surely know, I am also active on Twitter as pilhuhn. So I have sat down and started to write a TwitStat plugin for Jopr and RHQ just to show the possibilities of the Jopr platform, that it is not exclusively about managing JBoss products (as some people say) and to show more example code.

Currently the TwitStats plugin lets you:


  • Post your own tweets

    Bild 4.png


  • Monitor the number of tweets


    • Your friends timeline

    • Public users timelines

    • Search for stuff

    • Bild 5.png


      Interesting detail is that the Event dots (green ones) show the post date of a tweet, while the blue monitor dots (number of tweets since last poll) show that number at the time of polling.


  • Show the monitored tweets as events in the Events Subsystem:

    Bild 3.png




I have built the plugin around the Twitter4J library, which helps a lot in getting things done.

There are still some open issues:

  • OAuth support is missing

  • Calls to get tweets are synchronous and block the plugin for too long - those need to converted into async calls



You can find the source code in RHQ svn in modules/plugins/twitstats.

Please provide feedback - either in the RHQ Jira, in the Forums or on Irc

Monday, July 13, 2009

Jopr Java1 sessions available

The mini theatre sessions about Jopr are now online at YouTube. This is a four part series of videos. Greg Hinkle, lead of the Jopr, RHQ and JBoss ON projects is presenting:

Part1: http://www.youtube.com/watch?v=5X0kXk16sEw (shown below)




Part 2: http://www.youtube.com/watch?v=6VGXc7__OCg


Part 3: http://www.youtube.com/watch?v=whc5UH197qY


Part 4: http://www.youtube.com/watch?v=FqpyLWtJyBw