This is the first part of a series of posts about a Java API for PBS servers, compliant with the DRMAA spec.
This post could also be entitled “Why we won’t use DRMAA for our Jenkins plug-in”, with spoilers included. After working on a prototype DRMAA PBS Java library for a few days, I finally understand why we can’t use DRMAA v1 for the plug-in.
When you use Jenkins for analytics it is important to deliver consistent analysis reports from each build. There are a few different ways that you can create graphical, tabular and/or textual analysis reports, but one thing that becomes clear immediately is that you also require a certain level of dynamic behavior. Dynamic behavior is required so that you can easily adapt the reports to the underlying data whose format and content will likely change even when the same type of analysis is performed.
In this this blog entry, I will describe how I have used the Summary Display and Scriptler plugins to create a simple but effective framework for displaying consistent but dynamic reports following numerical data analysis using the R plugin.
This is the first part of a series of posts about a Java API for PBS servers, compliant with the DRMAA spec. PBS is a type of batch server, or distributed resource manager (DRM). It manages resources like CPU and memory in a network of computers, and is able to schedule jobs to run utilizing the maximum of these resources.
We have a PBS Plug-in for Jenkins, that enables us to submit and monitor jobs from Jenkins to a PBS server. James Hetherington proposed enhancements for the pbs-java-api and PBS Plug-in, including supporting other batch servers like SGE.
On June 18th, I attended my first ever Jenkins User Conference in Boston, USA.
Not only I attended, but I also had the privilege to present at this meeting my work on:
Using Jenkins as a scientific data and image management platform
The meeting attracted more than 400 people, a quite impressive number of attendees. Most of these people were of course supporting the development operations of various companies, and they were clearly interested in what I call the standard Jenkins functionality and software build workflows. So, I was a bit pleasantly surprised when at least 150-200 people attended my talk which was clearly outside the primary interest of most attendees. Furthermore, at least 10 people remained after my talk to ask questions, discuss ideas, provide feedback and inquire how they could get a hold of the presentation slides.
If you review some of my final slides, you’ll see that I’m making some recommendations on the types of improvements required to make Jenkins a better platform for use by scientists. The BioUno project is making significant contributions in this area. Finally, note that my slides on the JUC 2014 CloudBees site are an earlier version of my presentation, and do not include some last minutes updates.
In the post entitled The Secret of Building a scientific community, Manuel Corpas describe his experiences coordinating the BiosJS project. It is a great writing and if you are interested in Open Source communities, related to research or not, you will find it very interesting.
I learned about BioJS months ago, but never really used it. After reading Manuel’s post I followed some tutorials and was able to produce some neat HTML pages with sequences and trees. A few things that I learned:
- There are several components implemented by contributors
- Components have different dependencies
- Some components can use Java applets
- Some components can be use Ajax to retrieve remote content
Using Jenkins to serve BioJS artifacts
The JQuery Plug-in simply adds JQuery into Jenkins web page, but the same approach won’t work with BioJS since it is framework agnostic (which is great) and each component may have different dependencies (YUI, JQuery UI, …).
The simplest way to produce artifacts using BioJS in Jenkins, and serve the content from Jenkins is by using CDN’s for retrieving the JS files, and the HTML Publisher plug-in to archive and serve the HTML’s. As in this sample build.
Some components use Ajax requests to dynamically update the UI. For these components a callback would have to be implemented in a plug-in for Jenkins. So the simplest approach is to deploy these artifacts to a Web server with the callback.