JSLint4java, Jenkins and Violation plugin

In a previous article I told you about the existence of the violation plugin for Jenkins which can take in consideration the output generated by jslint4java.

jslint4java has an ant plugin which can be easily integrated in your environment and you need just to configure it for your javascript.

Let’s suppose you have a javascript file, in my case user-extensions.js coming from Selenium, in your repository.

At the same level create a folder named “jslint”. Under this folder place the jslint4java-2.0.1.jar file that you can find inside the zip file of jslint4java and create a build.xml file with the following content:

<project name="proj" default="main" basedir=".">
<target name="main" depends="jslint"/>

<taskdef name="jslint"
classname="com.googlecode.jslint4java.ant.JSLintTask"
classpath="jslint4java-2.0.1.jar" />

<target name="jslint">
<jslint haltOnFailure="false" options="sloppy">
<predef>selenium,Selenium,storedVars,LOG,Assert</predef>
<formatter type="xml" destfile="jslint.xml"/>
<fileset dir="../" includes="user-extensions.js"/>
</jslint>
</target>

</project>

In practice in the taskdef tag we say where the jar file is located (same folder), in the jslint tag I specified:

  1. haltOnFailure=”false”, to not  make stop and fail the plugin if a violation is found (in this way the entire job in Jenkins will not fail)
  2. options=”sloppy”, to disable the strict control in the file (used for ECMAScript 5)
  3. using predefined variables like “selenium” with the predef tag
  4. specifying the output file (in the same folder) in xml format (useful for the violation plugin)
  5. specifying just the input file that I have (which is in the top folder)

Once the folder is created with the 2 files (build.xml and jslint4java jar file) in the repository let’s go to the Jenkins job to install the violation plugin and configure the job.

First let’s configure the ant task in the Job (my directory is under trunk/test):

Ant configuration

Then we configure the Violation plugin in the Job:

Violation plugin configuration

Let’s save and run the build and we should see a graph the following in the Job page:

Violation graph in job page

If you click on the left menu on Violations (or in the graph) you will see the same graph with the number of violations per file:

Number of violations per file

If you click on the filename you will see all the violations with the respective row numbers:

Violations in a file

If you click on the row number you can see the line of the code (just below):

Photobucket

Enjoy fixing your javascript file :-)

Advertisements

Selenium, JSLint and editors

Testing with Selenium can require adding a user-extensions.js to your Selenium IDE and Server.

While modifying the user-extensions.js you could need a javascript editor with a JSLint plugin suggesting how to make your code better.

Among open source solutions with JSLint plugin I found: Notepad++, jEdit, Netbeans, Eclipse and Aptana (please suggest others if you know).

Notepad++ (5.9.6.2) has a good plugin updated to 2011 (I can see the Node.js option) and you can put the variable Selenium in the options (Predefined field) of the plugin. The errors are displayed in the JSLint panel which is good but you cannot order them by error description. The only problem is that Notepad++ is for Windows (I have seen people using it with wine for Linux).

Notepad++ doesn’t provide a function navigator but it is possible to setup a Function List plugin manually (put the dll in the program files directory and the rest in the %appdata%/Notepad++/plugins/config with the xml in readonly) and add a rule as explained in this blog.

jEdit (4.5.1 pre1) is a good multiplatform editor. It has a function navigator (you have to install Sidekick plugin with the XML plugin) and it has a JSLint plugin (0.8)

The JSLint plugin is old (in the page you see that the latest release is in 2009) and indeed you don’t see the Node.js option, further you need to add /*global Selenium */ inside your script to make disappear the undefined variable Selenium message. The errors are displayed in the Error List panel and underlined in the file andyou cannot order them by error description

Netbeans (7.0.1) has his own JSLint plugin, with a configuration under Tools->Options->Miscellaneous->JSLint but the Predefined field doesn’t seem to work so you need to add /*global Selenium */ in your script. The error are displayed next to the file and under the Task list (see the instructions) and you can order them by error description.

With Netbeans you can see the function navigator in the Navigator panel. Compared with others IDE (Eclipse/Aptana) you can do the check on a file or in a project while the others can work only in the project.

For Eclipse (Indigo) I setup the JSLint plugin using the update site:

http://update.rockstarapps.com/site.xml

When you have setup the plugin, you see a RockStarApps menu and, as you can notice, the plugin is not updated to the last version of JSLint (Node.js is not there) but the Predefined field works.

The errors are displayed next to the file and under the Problems panel and you can group them by error description.

For the function navigator Eclipse has a JavaScript Development Tools plugin that you can setup using the following site under WTP 3.3.1:

http://download.eclipse.org/webtools/repository/indigo

Then open the perspective Javascript and you find the function navigator in the Script Explorer panel.

Concerning Aptana Studio 3, there is a native JSLint plugin to be enabled under Window->Preferences->Aptana Studio->Validation->Javascript. I don’t see the configuration panel like in the other plugins so you need to add all the options in your javascript file but you can fiter some warnings.

The errors are displayed next to the file and under the Problems panel and you can group them by error description. It doesn’t look to me stable for the moment (it seems not all the warnings I get with the others plugin are there).

Aptana has a function navigator in the Outline panel.

At Continuous Integration level I found a Jenkins Violation Plugin or just using jslint4java which has an ant plugin on which is based the Sonar plugin.

Testing a multilanguage website with Selenium

I believe the technique to test a multilanguage website is still a quite open discussion and it depends what are the objectives. Normally you would say “I want to check all the labels that are on the website are on the right place” which means you have to extract them from somewhere, store them and compare them with those on the website.

When you extract them from a database you might probably already have a form that allows to create them and a page to list all of them. Somebody would say that it is convenient to retrieve them directly from the database doing in practice data driven testing/database validation.

If you are writing test cases directly with a programming language like Java (just for example) you have already the methods to contact the database and retrieve the labels but if you are writing your test cases in Javascript either you need a webservice to contact an application to retrieve the database data or you have to create a command to go to the page of your application that list all the labels. For sure you have to synchronize your test cases with the stored labels and not updating the test cases manually.

I am still not convinced how far a tester can go (and learn) when testing. At the beginning I was thinking that a tester could learn just selenium commands and a developer will extend some of them for her/him. In reality it depends on the skills of your tester. In my opinion javascript is a language easy to learn (together with xpath or css selector) it can parse easily DOM objects and Selenium can reach this level of granularity. So a tester needs to know the structure of a DOM object and from here s/he can learn xpath/css selector and javascript.

This long introduction will help to understand what I am going to explain.

Usually with Selenium you can store xpath or css selector in a variable with a simple command (like storeText) which stores this variable in a local javascript array (storedVars) invisible to a simpler tester. A contributed user extension can store a text in a global array that can be reused among all the test case. This global array (globalStoredVars) can be indeed used to store the labels of your website.

Now supposing that you have a website in 3 languages (en, fr, de), the approach that I would do is to store the labels with a language suffix (for example label_login_en, label_login_fr, label_login_de) and store in the original variable (label_login) one of 3 variables every time I change language, using a javascript function to update the original variable.

String.prototype.endsWith = function(suffix) {
    return this.indexOf(suffix, this.length - suffix.length) !== -1;
};

Selenium.prototype.doSetLanguage = function(language) {
  var languages=new Array("en","de","fr");
  if((languages.indexOf(language) == -1) || (language=="random") )
    language = languages[Math.round(Math.random() * new Date().getMilliseconds())%(languages.length)];
  var suffix="_"+language;
  for (var i in globalStoredVars){
    if(i.endsWith(suffix)){
      LOG.info("found variable "+i);
      var new_var=i.substring(0,(i.length-suffix.length));
      LOG.info("new_var: "+new_var);
      globalStoredVars[new_var]=globalStoredVars[i];
    }
  }
  globalStoredVars['xpath_cell']="//td[.='"+globalStoredVars['label_login']+"']";
};

The example above shows a Selenium command setLanguage which takes as input a language (or gives the possibility to choose a random language if not inserted or if “random” is used), it takes all the labels stored in globlalStoredVars with the suffix for the language chosen and it to replace them with the original variable without suffix.

The inconvenient is that if you have for example an xpath that it turns contain one of the original labels (see the last row in the code where xpath_cell depends on the label_login variable), you have to update it as well every time  (which means moving the storing of the xpath from the selenium command into a javascript command directly, which means the tester needs to know javascript).

Putting a part the usage of an xpath containing a label (that can be arguable), the question would be if it is correct changing a language (and what are the implications) at the beginning of the test suite or at the beginning of each test case.

It is obvious that in terms of performance it is better to choose it once at the beginning of the test suite but in some cases (a user chooses the own language in the profile or the website chooses a language for a particular user) it is not possible so you need to update each testcase.

Selenium: testing PDF with Calibre

During my test activities I have arrived at the point that I need to test generated PDFs containing the same text that I have in a html page.

So I searched different PDF utilities that can be used through a command line to extract text out of a PDF, among these I found: pdftotext (a command line tool coming with xpdf), Calibre (an ebook manager/converter), Tika (used by Solr to index pdf content), pdfbox (a java application library that extract pdf).

Giving a better look Calibre is using poppler as library and pdftotext is using poppler as well while Tika is based on pdfbox library to extract text.

At the end I chose Calibre for different reasons:

  1. Calibre allows to extract hyperlinks from the pdf (main reason) when using a wiki syntax like textile (I didn’t find this possibility in pdftotext, tika or pdfbox) on the other side the generated text is with wiki syntax
  2. The development life cycle of Calibre is quite fast (compared with xpdf)
  3. Calibre points to release the application for Windows, Mac and Linux (with xpdf you can have different versions but the one for Mac is released on 2007)

Note that I don’t care the possibility to extract text from images, I know that with pdfimages command it is possible to extract them or other formats that tika can support.

I have to say that in terms of performance on the pdf I tested pdftotext is the fastest (0.01 sec), followed by pdfbox (1.7 sec), Calibre (2.3 sec) and then Tika (2.8 sec) on page , so if you don’t care about hyperlinks but just PDF text I would suggest to go to pdftotext.

After choosing Calibre I created a simple php application that:

  1. receives as parameter the URL of a PDF
  2. calls Calibre with different options on the PDF downloaded
  3. removes  some text (like empty lines, etc.)
  4. gives back an xml file with all the text generated.

I created then a selenium command that calls the php file (through an xmlhttp request) and search for a particular text (specified as input together with the url of the pdf file).

Php application (convert.php):

<?php
$content = file_get_contents($_GET['url']);
$filefrom = 'extract.pdf';
$fileto = preg_replace("/\.pdf$/","",$filefrom).".txt";
file_put_contents($filefrom,$content);
//$t_start = microtime(true);
//system("java -jar tika.jar -t ".escapeshellcmd($filefrom)." > ".escapeshellcmd($fileto),$ret);
system("ebook-convert ".escapeshellcmd($filefrom)." ".escapeshellcmd($fileto)."  >nul 2>&1",$ret);
//system("pdftotext -nopgbrk -raw ".escapeshellcmd($filefrom)." ".escapeshellcmd($fileto),$ret);
//system("java -jar pdfbox.jar ExtractText  ".escapeshellcmd($filefrom)." ".escapeshellcmd($fileto),$ret);
//$t_end = microtime(true);
if($ret==0){
  $value=file_get_contents($fileto);
  $value_empty_line=preg_replace("/(^[\r\n]*|[\r\n]+)[\s\t]*[\r\n]+/", "\n", $value);
  $text=preg_replace("/(\[.*\]\n)/","",$value_empty_line);
  unlink($fileto);
  header('Content-type: text/xml');
  echo "<result>".htmlspecialchars($text)."</result>";
}
else{
  header('Content-type: text/xml');
  echo "<result>Convertion failed</result>";
}
?>

Selenium command (assertSearhInPDF):

Selenium.prototype.assertSearchInPDF = function(uri,names){
  var baseurl = globalStoredVars['host'];
  var params = [{"param" : "url","value" : storedVars[uri],}];
  var lista = "";
  for(var i=0; i<params.length; i++){
    lista +="&" + params[i].param + "=" + encodeURIComponent(params[i].value);
  }
  var indirizzo = baseurl+"/test/convert.php?"+lista;
  LOG.info( 'indirizzo = ' + indirizzo);
  var responseXML = chiamaWebService(indirizzo);
  LOG.info( 'response = ' + responseXML);
  var text = responseXML.getElementsByTagName('result')[0].firstChild.nodeValue;
  text = text.replace(/(\n)/g, " ");
  var array = names.split('|');
  var result=0;
  var length=array.length;
  LOG.info( 'text = ' + text);
  for (var i = 0; i < length; i++){
    if(text.indexOf(array[i]) !==-1){
      LOG.info( 'Found = ' + array[i]);
      result=result+1;
      text=text.substring(text.indexOf(array[i]));
    }
    else{
      LOG.info( 'Element ' + array[i]+' not found');
      break;
    }
  }
  if(result!=length)
    Assert.fail("Not all the elements have been found");
};

In particular the selenium command can accept a list of elements (separated by “|”) to searched in the respective order, if they are not found the command fails.

Selenium wiki formatter (textile e altri)

Di recente avevo necessità di copiare dei test case creati con Selenium IDE su Redmine, si potrebbe allegarli ma l’intenzione è quella di mostrali come una tabella.

Ho cosi’ consultato velocemente la pagina su come creare un custom format.

Da li’, a meno di eccezioni (quali l’escape della pipe | ) ho creato questo semplice script da aggiungere nel menu opzioni di Selenium IDE nel tab formats (inserite Wiki nel campo wiki format name e sovrascrivete con il seguente contenuto):

var wiki = {
confluence:{
  start_table: "",
  end_table: "",
  header: "||Command||Target||Value||\n",
  header_row:"||#",
  start_row: "| ",
  end_row: " |\n",
  separator: " | ",
},
dokuwiki:{
  start_table: "",
  end_table: "",
  header: "^ Command ^ Target ^ Value ^\n",
  header_row:"^ # ",
  start_row: "| ",
  end_row: " |\n",
  separator: " | ",
},
mediawiki:{
  start_table: "{|border=\"1\" \n",
  end_table: "|}",
  header: "!Command\n!Target\n!Value\n",
  header_row:"!#\n",
  start_row: "|-\n|",
  end_row: "\n",
  separator: "\n|",
},
moinmoin:{
  start_table: "",
  end_table: "",
  header: "||\'\'\'Command\'\'\'||\'\'\'Target\'\'\'||\'\'\'Value\'\'\'||\n",
  header_row:"||\'\'\'#\'\'\'",
  start_row: "||",
  end_row: "||\n",
  separator: "||",
},
textile:{
  start_table: "",
  end_table: "",
  header: "|_. Command|_. Target|_. Value|\n",
  header_row:"|_. #",
  start_row: "|",
  end_row: "|\n",
  separator: "|",
},
trac:{
  start_table: "",
  end_table: "",
  header: "||= Command =||=  Target =||=  Value =||\n",
  header_row:"||= # =",
  start_row: "||",
  end_row: "||\n",
  separator: "||",
},
twiki:{
  start_table: "",
  end_table: "",
  header: "|  *Command*  |  *Target*  |  *Value*  |\n",
  header_row:"| *#*",
  start_row: "|  ",
  end_row: "  |\n",
  separator: "  |  ",
},
xwiki:{
  start_table: "",
  end_table: "",
  header: "|=Command|=Target|=Value\n",
  header_row:"|=#",
  start_row: "|",
  end_row: "\n",
  separator: "|",
}
};

function formatCommands(commands) {
  var result = '';
  var start_table =  wiki[options['wiki']]["start_table"];
  var end_table =  wiki[options['wiki']]["end_table"];
  var header = wiki[options['wiki']]["header"];
  var header_row = wiki[options['wiki']]["header_row"];
  var start_row = wiki[options['wiki']]["start_row"];
  var end_row = wiki[options['wiki']]["end_row"];
  var sep = wiki[options['wiki']]["separator"];
  var row_num = '';
  for (var i = 0; i < commands.length; i++) {
    var command = commands[i];
    if('true' == options.row)
            row_num= i+sep;
    if (command.type == 'command') {
      result += start_row + row_num+ command.command + sep + command.target + sep + command.value + end_row;
    }
  }
    if('true' == options.row)
         header = header_row+header;
    result = start_table + header + result + end_table;
  return result;
}

function parse(testCase, source) {
  var doc = source;
  var commands = [];
  var start_row = wiki[options['wiki']]["start_row"];
  var sep = wiki[options['wiki']]["separator"];
  while (doc.length > 0) {
    var line = /(.*)(\r\n|[\r\n])?/.exec(doc);
    var line2 = line[1].slice(start_row.length);
    var array = line2.split(sep);
    if (array.length >= 3) {
      var command = new Command();
      command.command = array[0];
      command.target = array[1];
      command.value = array[2];
      commands.push(command);
    }
    doc = doc.substr(line[0].length);
  }
  testCase.setCommands(commands);
}

function format(testCase, name) {
  return formatCommands(testCase.commands);
}

options = {wiki: 'confluence', row:"false"};

configForm = '<description>Choose a wiki syntax:</description>' + '<menulist id="options_wiki">'
+ 	'<menuitem label="Confluence" value="confluence"/>' +
+ 	'<menuitem label="DokuWiki" value="dokuwiki"/>' +
	'<menuitem label="MediaWiki" value="mediawiki"/>' +
	'<menuitem label="MoinMoin" value="moinmoin"/>' + '<menuitem label="Textile" value="textile"/>' + '<menuitem label="Trac" value="trac"/>' + '<menuitem label="Twiki" value="twiki"/>' + '<menuitem label="Xwiki" value="xwiki"/>' + '</menupopup>' + '</menulist>'+ '<separator class="groove"/>' + '<checkbox id="options_row" label="Row number"/>';

Dal menu delle opzioni, vedete che confluence  è impostato di default (potete abilitare anche i numeri di riga tramite la checkbox) e potete scegliere tra Confluence, DokuWiki, MediaWiki, MoinMoin, Textile (Redmine), Trac, Twiki e Xwiki.

Una volta salvato, andate in Options -> Format e scegliete Wiki, cliccate poi sul tab Source e vedete il test case nel formato wiki da voi scelto, potete poi copiare e incollare nel vostro wiki preferito.

Selenium e la ricerca sulla tabella

Una delle operazioni più comuni che puo’ capitarvi in un test è la ricerca un testo in una tabella seguito dal click, nella stessa riga del testo trovato, su un pulsante o link per fare determinate azioni come vedere dettagli, modificare o cancellare un oggetto nella tabella.

Normalmente su un sito web vi trovate di fronte a tabelle divise in pagine per non avere una pagina troppo lunga e di conseguenza avrete anche un pulsante per proseguire alla prossima pagina, pulsante che non appare nell’ultima pagina ovviamente.

Con l’aiuto dell’estensione FlowControl, che permette di realizzare cicli o condizioni, ho quindi creato un test case che cerca il mio cognome in questa pagina (sezione s) e, se trovato, fa alcune operazioni, come memorizzare informazioni (nome, organizzazione, area) o cliccare sul link per vedere i contributi.

Il test case è il seguente:

looptest
store false ElementFound
while storedVars.ElementFound!=true && selenium.isElementPresent(“//div[@id=’content’]/table[2]/tbody/tr/td[1]/div[3]/a”);
searchSurname Stani seeContributions
gotoIf ${ElementFound}==true skipClickNext
clickAndWait //img[@alt=’Next’]
label skipClickNext
endWhile
gotoIf ${ElementFound}==true skipLastSearch
searchSurname Stani seeContributions
label skipLastSearch
verifyExpression ${ElementFound} true

Inizializziamo la variabile ElementFound a false, tale variabile ci permette di verificare alla fine se abbiamo trovato o meno il cognome (o elemento in generale).

Creiamo il ciclo (comando while) dove ad ogni iterazione verifichiamo se l’elemento è stato trovato e se è presente il link alla pagina successiva. Se l’elemento non è stato trovato allora facciamo una ricerca tramite il comando creato da me searchSurname.

Il comando searchSurname cerca nella prima colonna della tabella il cognome e, una volta trovato, clicca sul link dei contributi tramite l’azione seeContributions (per i dettagli vedere sotto).

Il comando searchSurname imposta a true la variabile ElementFound nel caso in cui il cognome è stato trovato. Se la variabile è  a true non dobbiamo cliccare sulla prossima pagina (salto alla label skipClickNext) e usciremo nel ciclo alla prossima iterazione. Se invece il cognome non è stato trovato (variabile a false) cliccheremo sul pulsante Next.

Dopo la fine del ciclo (endwhile) abbiamo bisogno di un ulteriore ricerca nel caso in cui raggiungiamo l’ultima delle pagine dove l’elemento non è stato ancora trovato e non esiste il link alla pagina successiva (salto alla label skipLastSearch).

Verifichiamo dunque alla fine se la variabile è a true (verifyExpression).

Andiamo a vedere ora il comando searchSurname:

function search(divId, textToFind, searchCol){
	obj_by_id = selenium.browserbot.getCurrentWindow().document.getElementById(divId);
	rowlist = obj_by_id.getElementsByTagName("TR");
	rowfound = -1;
	for (rowNo=1; rowNo<rowlist.length; rowNo++) {
		var cella = rowlist[rowNo].cells[searchCol];
		LOG.info("reading element "+ rowNo+": "+cella.innerHTML);
		if(cella.innerHTML == textToFind){
			LOG.info( "Found element " + textToFind + " at row "+rowNo);		
	        rowfound=rowNo;
			break;			
		}
	}
	if(rowfound==-1)
		LOG.info("Element "+textToFind+" not found!");
	return rowfound;
}

function action(divId, rowNo, actionType){
	rowPath = "//table[@id=\'"+divId+"\']/tbody/tr["+rowNo+"]";
	if(actionType == "storeName")
		selenium.doStoreText(rowPath+"/td[2]", "NameFound");
	else if(actionType == "storeOrganization")
		selenium.doStoreText(rowPath+"/td[3]", "OrganizationFound");
	else if(actionType == "storeArea")
		selenium.doStoreText(rowPath+"/td[4]", "AreaFound");
	else if(actionType == "seeContributions")
		selenium.doClick(rowPath+"/td[6]/a");
	else
		Assert.fail("Action "+actionType+" is wrong!");
}

Selenium.prototype.doSearchSurname = function(textToFind, actionType){
	var divId = "Countrytable";
	var searchCol=0;
	var found=false;

	var rowNo=search(divId, textToFind, searchCol);
	if(rowNo!=-1){
		action(divId, rowNo, actionType);
		found=true;
	}
	storedVars['ElementFound']=found;
};

Il comando searchSurname chiama la funzione search passando l’id in cui è contenuta la tabella (Countrytable) e la  colonna in cui cercare il cognome (colonna 0 ). La funzione search cerca l’esatto testo dentro ogni cella della colonna indicata e restituisce -1 se il cognome non è stato trovato oppure il numero di riga in cui il cognome si trova.

Il numero di riga insieme all’id della tabella e il tipo di azione vengono usati per chiamare la funzione action nel caso in cui qualcosa è stato trovato (rowNo!=-1).

La funzione action a seconda del tipo di azione invoca uno dei comandi di Selenium. Nel nostro caso abbiamo le azioni:

  • storeName, che memorizza il nome nella variabile NameFound
  • storeOrganization, che memorizza l’organizzazione nella variabile OrganizationFound
  • storeArea, che memorizza l’area nellla variabile AreaFound
  • seeContributions, che clicca sul link dei contributi (azione che usiamo nel nostro testcase).

Se l’utente inserisce una azione non riconosciuta allora il test fallisce e si interrompe tramite il comando Assert.fail .

Alla fine del comando searchSurname memorizziamo nella variabile ElementFound il valore true se il cognome, nella ricerca della tabella corrente, è stato trovato o false se non è stato trovato.

Per eseguire il test case  in Selenium dovete quindi includere il file goto_sel_ide.js e il comando searchSurname come estensioni di Selenium dal menu Options.

Il test case puo’ essere migliorato ma credo che comunque avrete bisogno di adattarlo alle vostre esigenze.

Buona ricerca :-)