Test-suite

Installation

The test suite is available via svn

%svn co svn+ssh://username@qeforge.qe-forge.org/svnroot/yambo-tests/test-suite ./test-suite

Remember to change username with your username.

After the cehckout the first thing to do is the configure

%cd test-suite
%./configure --with-yambo=/home/marini/Yambo/yambo/master/

If the configuration works you should see the following messages:

%./configure --with-yambo=/home/marini/Yambo/yambo/master/
%checking build system type... x86_64-unknown-linux-gnu
%checking host system type... x86_64-unknown-linux-gnu
%checking the anomaly.mlib.ism.cnr.it ROBOT... created
%checking for nccopy... no
%checking for ncftp... ncftp
%checking for ncftpls... ncftpls
%checking for ncftpput... ncftpput
%checking for awk... awk
%checking for txt2html... txt2html
%checking the Yambo source... /home/marini/Yambo/yambo/master/
%configure: creating ./config.status
%config.status: creating config/MODULES.pl
%config.status: creating config/TOOLS.pl

Databases download

Now it is time to download all databases. This can be easily done using the -b option

%./driver.pl -d all

Quick Testing

%./driver.pl -autotest

If everything works you should see as log the following message

%==================================================================================
%= Starting Yambo test-suite
%==================================================================================
% - Check requirements : ...ncftp...ncftpls...ncftpput...txt2html
% -     Test selection : from input
%               ncdump : /opt/gcc4/netcdf-4.0.1/bin/ncdump
% -  Executable checks : (yambo  OK ) (ypp  FAIL )(a2y  FAIL )(p2y  FAIL )
%        Running tests : hBN/GW-OPTICS 01_init 02_HF
%             Projects : nopj
%            Verbosity : low 
%            Precision : 0.01 
%             Hostname : anomaly.mlib.ism.cnr.it 
%             revision : 14505 
%                build : MPI+SLK 
%              bin dir : bin 
%           shortname  : 
%               source : master_-precompiled 
%        Parallel Loop : SERIAL
%=--------------------------------------------------------------------------------=
% >  [1  /1  ] hBN/GW-OPTICS[serial]   5 passes 
%=--------------------------------------------------------------------------------=

The 5 passes are important as they represent the number of passed tests.

Available tests

The list of tests can be done using the -l option:

%./driver.pl -l

Note that you can add a TEST to -l to list the specific inputs of that test. For example

%>./driver.pl -l GaAs001-c4x4/YAMBO
%
%Available input files for GaAs001-c4x4/YAMBO are:  01_init 02_RAS_full 03_RAS_full 04_RAS_front 05_RAS_dimer

Reports & Logs

The autotest and any other run of the test-suite produces three files. Each file is labelled with the date of the run.

In order to see the meaning and the in formations written in the files let's do a quick run of a Dummy test created in order to reproduce some of the possible error conditions.

Let's first clean the suite

%./driver.pl -c

and use the -force to run also tests tagged as BROKEN (like the Dummy one). First thing to do is to download the Dummy database adding the -force option.

%./driver.pl -d Dummy -force

Then we can run the Dummy test using

%./driver.pl -tests Dummy -force

The result will now be

%=--------------------------------------------------------------------------------=
% >  [1  /1  ] Dummy[serial]          11 passes  5 fails 
%=--------------------------------------------------------------------------------=

The 5 fails cover most of the potential errors detected by the test-suite. Details can be now found in the report/log files

REPORT file

This file contains very condensed information about the run. There is the duration time plus some detail of the reasons behind the 5 fails.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Yambo test suite global REPORT
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%==================================================================================
%Running tests           :Dummy
%Projects                :nopj
%Date - Time             :Fri-19-May - 11-46
%Build                   :master - MPI+SLK - rev.14505
%Parallel Conf           :[serial] - none
%=--------------------------------------------------------------------------------=
%FAIL: 5 & NO RUN: 0 & SKIPS: 0 % WHITELIST: 1 % OKs: 10
%FAIL details: 1 wrong output % 1 no reference % 1 no output % 2 no DB
%=--------------------------------------------------------------------------------=
%% Section Duration : 0:0:3 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% TOTAL   Duration : 0:0:3 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

In the dummy case indeed the suite should detect:

1. 1 wrong output
2. 1 output missing the REFERENCE
3. 1 missing output (not generated by the run)
4. 2 missing databases.

More details can be found in the LOG and and in the ERROR.

[[The REPORT file is the one used to send final logs to the mailing list. ]]

ERROR file

This is a useful file with detailed information about the errors only.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Yambo test suite ERRORs log
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%==================================================================================
%Running tests           :Dummy
%Projects                :nopj
%Date - Time             :Fri-19-May - 11-46
%Build                   :master - MPI+SLK - rev.14505
%Parallel Conf           :[serial] - none
%=--------------------------------------------------------------------------------=
%Dummy/02_hf-serial-master/o-02_hf.hf  : VALUE(#  17)@COL#  6 is  -5.4691     [REF] vs  -4.4691     [OUT] corresponding to  6.1013     [% RELATIVE TO MAX]
%Dummy/02_hf-serial-master  :  NO REFERENCE/o-02_hf.ndb.em1s_fragment_1 in OUTPUT
%Dummy/04_em1s-serial-master  :  NO o-04_em1s.ndb.em1s_fragment_1 in REFERENCE
%% Section Duration : 0:0:3 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% TOTAL   Duration : 0:0:3 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Running

The test-suite can be run in two different ways

1. Using a theme
2. Passing individual sets of tests
3. Using flows

Before entering in the details of the different running methods let's first describe the Parallel Options

Crontab

The test-suite can be run using the crontab daemon.

-------
35 23  * * * ~/bin/scripts/yambo_validation.tcsh >> /home/marini/Yambo/sources/svn/yambo-tests/test-suite 2>&1

with yambo_validation.tcsh a simple script:

#!/usr/bin/tcsh
cd /home/marini/Yambo/sources/svn/yambo-tests/test-suite
./driver.pl -flow validate -report

Adding New Tests

TODO

Advanced Use

General RUN descriptor & Automatic flows of calculations

{
 ACTIVE      => "yes", # can be yes or no
 MPI_CPU     => "NP",
 SLK_CPU     => "NM",
 PAR_MODE    => "TEXT", # (TEXT can be default, random, loop)
 THREADS     => "NT",
 BRANCH      => "/home/marini/Yambo/sources/git/yambo/branches/SLK", # (SAVED)
 THEME       => "G_parallelization", # (SAVED)
 CONFIG      => "gfortran_slk.sh", # (SAVED)
 TESTS       => "hBN/GW-OPTICS", #  (SAVED)
 KEYS        => "nopj elph hard bse rpa", # (SAVED)
}