When you use Jenkins for analytics it is important to deliver consistent analysis reports from each build. There are a few different ways that you can create graphical, tabular and/or textual analysis reports, but one thing that becomes clear immediately is that you also require a certain level of dynamic behavior. Dynamic behavior is required so that you can easily adapt the reports to the underlying data whose format and content will likely change even when the same type of analysis is performed.
In this this blog entry, I will describe how I have used the Summary Display and Scriptler plugins to create a simple but effective framework for displaying consistent but dynamic reports following numerical data analysis using the R plugin.
When you perform some numerical, statistical or bioinformatic data analysis, the typical result can be assigned to one of two broad result categories.
In either case, it is desirable to be able to display this information to our users after the build for immediate feedback and increased understanding of the underlying data.
I will start with an illustration of the reporting requirements we are trying to implement. As an example, I will use a Jenkins project that assists users to upload and annotate a variety of file types (images, annotations, text, tabular data etc). Each build generates a multi-tab report where various properties and annotations of the uploaded file are displayed. In a follow-up entry, I will describe in more detail this FILE_UPLOAD Jenkins project.
The basic requireement is that build reports dynamically adapt to display different tabs and file/user-provided metadata depending on the type of file uploaded.
Binary file uploads (such as PDF, images, etc.) should generate reports with 4 tabs (DESCRIPTION, PROPERTIES, DOWNLOAD, ACTIONS).
CSV or TSV file uploads (typical formats for delimited numerical data) should generate reports that display an additional 3 tabs (CHARACTER_COLUMNS, NUMERIC_COLUMNS, SAMPLE) with information on the numeric and character columns of the table, as well as, a small sample of the data itself. In addition, the DESCRIPTION tab for CSV/TSV file reports should display extra fields to provide information about the number of columns, rows and rows of complete data (data rows with no ‘NA’ values, an R convention for missing data) in the data set.
Below are two report examples.
The report displays the 4 standard tabs
The report displays an additional 3 tabs and extra fields in the DESCRIPTION tab.
The Summary Display plugin is a powerful report generator. It uses an XML configuration file to generate a number of different HTML reports on the build page. The available report types and the required XML elements for generating them are described in detail on the plugin wiki page. The build process is responsible for generating the XML configuration file, while the Summary Display plugin is used in a post-build action for parsing the XML and generating the HTML report that is displayed on the build page. In this blog entry I will focus on the generation of multi-tab reports, as they are capable of reporting in a consistent, compact and familiar way a variety of numerical data and annotations.
Despite its promise, the Summary Display plugin is missing one crucial piece. And that is a prescriptive way for generating the required XML summary display configuration file. Instead, each XML configuration file has to be generated by custom code restricted to the artifacts and environment of each build. This discussion addresses this limitation and allows you to build the required XML diplay properties not from code but from a report configuration file.
What about if rather than writing custom code each time we wanted to create a new XML configuration file we could just rather write a set of simple rules of what type of report we want and what content we want to include. Basically, we want to tell the build what type of report to generate and let it figure out how to write the XML configuration file for us.
Let’s start by defining some basic requirements for such a tool and the types of content it can display.
Given the requirements above, I will now describe the syntax of the configuration file for encoding the report generation options.The supported options for each configuration property are shown in brackets.
Selectors for table content. These options allow you to select how much of a table to display. This is convenient for displaying samples of large data sets
The example file sumReport_test_data_upload.properties is used to configure a tabbed report for each FILE_UPLOAD build. The file is located in the userContent/properties folder on the Jenkins-CI server
# A properties file for configuring tabbed report of uploaded data # Thursday, June 26, 2014 summary.properties=JData.properties #summary.properties=none report.style=tab tab.header=DESCRIPTION,CHARACTER_COLUMNS,NUMERIC_COLUMNS,PROPERTIES,SAMPLE,DOWNLOAD,ACTIONS content.DESCRIPTION=field field.key.DESCRIPTION=data field.key.color=black field.value.color=blue content.SAMPLE=table table.data.SAMPLE=JData.csv table.length.SAMPLE=10 content.CHARACTER_COLUMNS=table table.data.CHARACTER_COLUMNS=meta.csv content.NUMERIC_COLUMNS=table table.data.NUMERIC_COLUMNS=measure.csv content.PROPERTIES=table table.data.PROPERTIES=JData.properties separator.PROPERTIES== content.DOWNLOAD=field field.key.DOWNLOAD=download content.ACTIONS=table table.data.ACTIONS=http://JenkinsURL/userContent/properties/after_test_data_upload.properties separator.ACTIONS==
Once a Summary Display report configuration is authored, it is used in a Scriptler-Script build step that executes the writeXMLProperties_scriptlet groovy script.
_Note: The writeXMLProperties_scriptlet takes 2 parameters
This build step generates the custom formatted XML report file that the Summary Display plugin then uses in rendering the build report.
NOTE: The groovy code for writeXMLProperties_scriptlet is available from my github repository
The Scriptler step generates the writeXMLSummary.xml file that the Summary Display Plugin will parse.
In the project’s post-build actions we now add the Summary Display Plugin step, referred to as ‘Publish XM Summary Reports’. The configuration of this action requires thta we provide one or more appropriately formatted XML files. We will provide the XML file (writeXMLSummary.xml) that the Scriptler step above generated.
In addition to the reports displayed under ‘Requirements’, the following figures display additional Multi-Tab Reportss that are dynamically generated from a single report configuration (the one shown above)
We see that the tabs dynamically adjust to the availability of properties and content data but importantly, maintain a consistent ‘look and feel’ that aids users in data review and improved comprehension of the results of a data analysis.
The PROPERTIES tab content is read from a standard Java properties file that is generated during the build.
The CHARACTER_COLUMNS tab content only appears when CSV files are parsed. The content is read from the meta.csv file that is generated during the build