Static code analysis in Python with Jenkins

I am checking the code quality of a python script called harvester.py which is in the root folder of my code .

To do so I installed the following packages (I am with Python 27 on Windows machine):

pip install pylint
easy_install -U clonedigger
pip install flake8

On Jenkins make sure you have setup Warning plugin and Violation plugin.

In Jenkins configuration, under Compiler Warning section we add a parser for flake8 with:

Name: flake8
Link name: Flake8 warnings
Trend report name: Flake8 warnings trend
Regular expression: ^(.*):([0-9]*):([0-9]*):(.[CE][0-9]*)(.*)$
Mapping script:
import hudson.plugins.warnings.parser.Warning
import hudson.plugins.analysis.util.model.Priority

String fileName = matcher.group(1)
String lineNumber = matcher.group(2)
String category = matcher.group(4)
String message = matcher.group(5)

return new Warning(fileName, Integer.parseInt(lineNumber), category, "PyFlakes Parser", message, Priority.NORMAL);;

Note from the regular expression the [CE] pattern in order to find Complexity and Error id problems. If you are not interested in the complexity just replace [CE] with E.

Now on the jenkins job configuration we add a build step “Execute Windows batch command” (you could do it similarly for Linux) with the following content:

rmdir /s /q output
mkdir output
pylint --msg-template="{path}:{line}: [{msg_id}({symbol}), {obj}] {msg}" --reports=y harvester.py >> output/pylint.log
clonedigger --cpd-output harvester.py -o output/clonedigger.xml
flake8 --max-complexity 12 harvester.py --output-file output/flake8-output.txt

The batch creates an output folder where pylint, clonedigger and flake8 generate their files (if you want you could put the script inside in a bat file having as parameters the output folder and the harvester.py file).

Add a post build action “Scan for compiler warnings”, select “scan workspace files” and with the following parameters:

File patterns: output/flake8-output.txt
Parser: flake8

Add a post build action “Report violations”  with the following parameters:

cpd: output/clonedigger.xml
pylint: output/pylint.log

Run the build and you should see the trend graph for Flake8 and the violations for cpd (clonedigger) and pylint!

P.S.:

If Flake8 exits with exit 1 failing your build, you might probably change the file:

C:\Python27\Lib\site-packages\flake8\main.py

At the end of the def main(): function you should have:

if exit_code > 0:
    raise SystemExit(exit_code > 0)

Which you can replace with:

raise SystemExit(0)

Now Flake8 should exit without problem.

 

 

 

 

 

JSLint4java, Jenkins and Violation plugin

In a previous article I told you about the existence of the violation plugin for Jenkins which can take in consideration the output generated by jslint4java.

jslint4java has an ant plugin which can be easily integrated in your environment and you need just to configure it for your javascript.

Let’s suppose you have a javascript file, in my case user-extensions.js coming from Selenium, in your repository.

At the same level create a folder named “jslint”. Under this folder place the jslint4java-2.0.1.jar file that you can find inside the zip file of jslint4java and create a build.xml file with the following content:

<project name="proj" default="main" basedir=".">
<target name="main" depends="jslint"/>

<taskdef name="jslint"
classname="com.googlecode.jslint4java.ant.JSLintTask"
classpath="jslint4java-2.0.1.jar" />

<target name="jslint">
<jslint haltOnFailure="false" options="sloppy">
<predef>selenium,Selenium,storedVars,LOG,Assert</predef>
<formatter type="xml" destfile="jslint.xml"/>
<fileset dir="../" includes="user-extensions.js"/>
</jslint>
</target>

</project>

In practice in the taskdef tag we say where the jar file is located (same folder), in the jslint tag I specified:

  1. haltOnFailure=”false”, to not  make stop and fail the plugin if a violation is found (in this way the entire job in Jenkins will not fail)
  2. options=”sloppy”, to disable the strict control in the file (used for ECMAScript 5)
  3. using predefined variables like “selenium” with the predef tag
  4. specifying the output file (in the same folder) in xml format (useful for the violation plugin)
  5. specifying just the input file that I have (which is in the top folder)

Once the folder is created with the 2 files (build.xml and jslint4java jar file) in the repository let’s go to the Jenkins job to install the violation plugin and configure the job.

First let’s configure the ant task in the Job (my directory is under trunk/test):

Ant configuration

Then we configure the Violation plugin in the Job:

Violation plugin configuration

Let’s save and run the build and we should see a graph the following in the Job page:

Violation graph in job page

If you click on the left menu on Violations (or in the graph) you will see the same graph with the number of violations per file:

Number of violations per file

If you click on the filename you will see all the violations with the respective row numbers:

Violations in a file

If you click on the row number you can see the line of the code (just below):

Photobucket

Enjoy fixing your javascript file :-)

Selenium, JSLint and editors

Testing with Selenium can require adding a user-extensions.js to your Selenium IDE and Server.

While modifying the user-extensions.js you could need a javascript editor with a JSLint plugin suggesting how to make your code better.

Among open source solutions with JSLint plugin I found: Notepad++, jEdit, Netbeans, Eclipse and Aptana (please suggest others if you know).

Notepad++ (5.9.6.2) has a good plugin updated to 2011 (I can see the Node.js option) and you can put the variable Selenium in the options (Predefined field) of the plugin. The errors are displayed in the JSLint panel which is good but you cannot order them by error description. The only problem is that Notepad++ is for Windows (I have seen people using it with wine for Linux).

Notepad++ doesn’t provide a function navigator but it is possible to setup a Function List plugin manually (put the dll in the program files directory and the rest in the %appdata%/Notepad++/plugins/config with the xml in readonly) and add a rule as explained in this blog.

jEdit (4.5.1 pre1) is a good multiplatform editor. It has a function navigator (you have to install Sidekick plugin with the XML plugin) and it has a JSLint plugin (0.8)

The JSLint plugin is old (in the page you see that the latest release is in 2009) and indeed you don’t see the Node.js option, further you need to add /*global Selenium */ inside your script to make disappear the undefined variable Selenium message. The errors are displayed in the Error List panel and underlined in the file andyou cannot order them by error description

Netbeans (7.0.1) has his own JSLint plugin, with a configuration under Tools->Options->Miscellaneous->JSLint but the Predefined field doesn’t seem to work so you need to add /*global Selenium */ in your script. The error are displayed next to the file and under the Task list (see the instructions) and you can order them by error description.

With Netbeans you can see the function navigator in the Navigator panel. Compared with others IDE (Eclipse/Aptana) you can do the check on a file or in a project while the others can work only in the project.

For Eclipse (Indigo) I setup the JSLint plugin using the update site:

http://update.rockstarapps.com/site.xml

When you have setup the plugin, you see a RockStarApps menu and, as you can notice, the plugin is not updated to the last version of JSLint (Node.js is not there) but the Predefined field works.

The errors are displayed next to the file and under the Problems panel and you can group them by error description.

For the function navigator Eclipse has a JavaScript Development Tools plugin that you can setup using the following site under WTP 3.3.1:

http://download.eclipse.org/webtools/repository/indigo

Then open the perspective Javascript and you find the function navigator in the Script Explorer panel.

Concerning Aptana Studio 3, there is a native JSLint plugin to be enabled under Window->Preferences->Aptana Studio->Validation->Javascript. I don’t see the configuration panel like in the other plugins so you need to add all the options in your javascript file but you can fiter some warnings.

The errors are displayed next to the file and under the Problems panel and you can group them by error description. It doesn’t look to me stable for the moment (it seems not all the warnings I get with the others plugin are there).

Aptana has a function navigator in the Outline panel.

At Continuous Integration level I found a Jenkins Violation Plugin or just using jslint4java which has an ant plugin on which is based the Sonar plugin.