Friday, September 13, 2013

More clever builds and tests ?

Hey,

when you have already tried to build RHQ and run all the tests you probably have seen that this may take a huge amount of time, which is fine for the first build, but later when you e.g. change a typo in the as4-plugin, you don’t want GWT being compiled again.
Of course as a human developer it is relatively easy to just go into the right folder and only run the build there.

Now on build automation like Jenkins this is less easy, which is why I am writing
this post.

What I want to have is similar to

  • if a class in GWT has changed, only re-build GWT

  • if a class in a plugin has changed, only rebuild and test that plugin
    (and perhaps dependent plugins like hibernate on top of jmx)

  • if a class in enterprise changes, only build and test enterprise

  • if a class in server/jar changes, only rebuild that and run server-itests

  • if a class in core changes, rebuild everything (there may be more fine grained rules as e.g. a change in core/plugin-container does not require compiling GWT again)

This is probably a bit abbreviated, but you get the idea.

What I can imagine is that we compile the whole project (actually we may even do incremental compiles to get build times further down and may also only go into a specific module (and deps) and just build those).
And then instead of running mvn install we run mvn install -DskipTests
and afterwards analyze what has changed, throw the above rules
at it and only run the tests in the respective module(s).

We could perhaps have a little DSL like (which would live in the root of the project tree and be in git)

    rules: {
rule1: {
if :modules/enterprise/,
then: {
compile: modules/enterprise,
skip: modules/enterprise/gui/coregui,
test: ["modules/enterprise"]
}
},
rule2: {
if: modules/plugins/as7-plugin,
then: {
compile:[ modules/plugins/as7-plugin,
modules/integration-tests/as7-plugin],
test: [ modules/plugins/as7-plugin,
modules/integration-tests/as7-plugin]
}
}

And have them evaluated in a clever way by some maven build helper
that parses that file and also the list of changes since the last build to
figure out what needs testing.

We can still run a full build with everything to make sure that we don’t loose coverage
by those abbreviations

There may be build systems like gradle that have this capability built in; I think for maven
this requires some additional tooling

QUESTIONS:

  • Are there any "canned" solutions available?

  • Has anyone already done something like "partial tests" (and how)?

  • Anyone knows of maven plugins that can help here?

  • Anyone interested in getting that going? This is for sure not only interesting for RHQ but for a lot of projects

Having such an infrastructure will also help us in the future to better integrate
external patches, as faster builds / tests can allow to automatically test those and
"pre-approve" them.




Wednesday, September 11, 2013

RHQ 4.9 released

It is a pleasure for me to announce on behalf of the RHQ development team the immediate availability of RHQ 4.9

Features

Some of the new features since RHQ 4.8 are:

Be sure to read the release notes and the installation documents.

Note

For security reasons we have made changes to the installation in the sense that there is no more default password for the rhqadmin super user. Also the default bind address of 0.0.0.0 has been removed. You need to set them before starting the installation.

If you are upgrading from RHQ 4.8 you need to run a script to remove the native components from Cassandrabefore the upgrade. Otherwise RHQ 4.9 will fail to start.

Thanks

Special thanks goes to

  • Elias Ross

  • Jérémie Lagarde

  • Michael Burman

for their code contributions for this release and to Stian Lund for his repeated testing of the new graphs implementation.

Downloads

As usual you can download the release from the RHQ-project downloads on SourceForge