Test-suite
Installation
The test suite is available via svn
%svn co svn+ssh://username@qeforge.qe-forge.org/svnroot/yambo-tests/test-suite ./test-suite
Remember to change username with your username.
After the cehckout the first thing to do is the configure
%cd test-suite %./configure --with-yambo=/home/marini/Yambo/yambo/master/
If the configuration works you should see the following messages:
%./configure --with-yambo=/home/marini/Yambo/yambo/master/ %checking build system type... x86_64-unknown-linux-gnu %checking host system type... x86_64-unknown-linux-gnu %checking the anomaly.mlib.ism.cnr.it ROBOT... created %checking for nccopy... no %checking for ncftp... ncftp %checking for ncftpls... ncftpls %checking for ncftpput... ncftpput %checking for awk... awk %checking for txt2html... txt2html %checking the Yambo source... /home/marini/Yambo/yambo/master/ %configure: creating ./config.status %config.status: creating config/MODULES.pl %config.status: creating config/TOOLS.pl
Databases download
Now it is time to download all databases. This can be easily done using the -b option
%./driver.pl -d all
Quick Testing
%./driver.pl -autotest
If everything works you should see as log the following message
%================================================================================== %= Starting Yambo test-suite %================================================================================== % - Check requirements : ...ncftp...ncftpls...ncftpput...txt2html % - Test selection : from input % ncdump : /opt/gcc4/netcdf-4.0.1/bin/ncdump % - Executable checks : (yambo OK ) (ypp FAIL )(a2y FAIL )(p2y FAIL ) % Running tests : hBN/GW-OPTICS 01_init 02_HF % Projects : nopj % Verbosity : low % Precision : 0.01 % Hostname : anomaly.mlib.ism.cnr.it % revision : 14505 % build : MPI+SLK % bin dir : bin % shortname : % source : master_-precompiled % Parallel Loop : SERIAL %=--------------------------------------------------------------------------------= % > [1 /1 ] hBN/GW-OPTICS[serial] 5 passes %=--------------------------------------------------------------------------------=
The 5 passes are important as they represent the number of passed tests.
For the impatiens: quick cron run and final reporting
If you do not want to bother with all details of the testing script and want just to install the suite read this section and stop.
By default the configure copies a basic compilation script in ROBOTS/<YOUR ROBOT>/CONFIGURATIONS/default.sh. This uses internally compiled libraries so it should work easily.
In order to properly setup your machine for night test-suite runs you have to:
- Choose the branch(s) to test
For this purpose use
%./driver.pl -edit branches
- create a crontab entry
See here for detailled informations on cron. My personal crontab file looks like
%# m h dom mon dow command % 30 18 * * Thu ~/bin/scripts/yambo_validation.tcsh validate >> /home/marini/Yambo/sources/svn/yambo-tests/test-suite/LOG 2>&1
with
%>cat bin/scripts/yambo_validation.tcsh %#!/usr/bin/tcsh %cd /home/marini/Yambo/sources/svn/yambo-tests/test-suite %./driver.pl -flow $argv[1] -report
That's all you need.
Available tests
The list of tests can be done using the -l option:
%./driver.pl -l
Note that you can add a TEST to -l to list the specific inputs of that test. For example
%>./driver.pl -l GaAs001-c4x4/YAMBO % %Available input files for GaAs001-c4x4/YAMBO are: 01_init 02_RAS_full 03_RAS_full 04_RAS_front 05_RAS_dimer
Reports & Logs
The autotest and any other run of the test-suite produces three files. Each file is labelled with the date of the run.
In order to see the meaning and the in formations written in the files let's do a quick run of a Dummy test created in order to reproduce some of the possible error conditions.
Let's first clean the suite
%./driver.pl -c
and use the -force to run also tests tagged as BROKEN (like the Dummy one). First thing to do is to download the Dummy database adding the -force option.
%./driver.pl -d Dummy -force
Then we can run the Dummy test using
%./driver.pl -tests Dummy -force
The result will now be
%=--------------------------------------------------------------------------------= % > [1 /1 ] Dummy[serial] 11 passes 5 fails %=--------------------------------------------------------------------------------=
The 5 fails cover most of the potential errors detected by the test-suite. Details can be now found in the report/log files
REPORT file
This file contains very condensed information about the run. There is the duration time plus some detail of the reasons behind the 5 fails.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% Yambo test suite global REPORT %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %================================================================================== %Running tests :Dummy %Projects :nopj %Date - Time :Fri-19-May - 11-46 %Build :master - MPI+SLK - rev.14505 %Parallel Conf :[serial] - none %=--------------------------------------------------------------------------------= %FAIL: 5 & NO RUN: 0 & SKIPS: 0 % WHITELIST: 1 % OKs: 10 %FAIL details: 1 wrong output % 1 no reference % 1 no output % 2 no DB %=--------------------------------------------------------------------------------= %% Section Duration : 0:0:3 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% TOTAL Duration : 0:0:3 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
In the dummy case indeed the suite should detect:
- 1 wrong output
- 1 output missing the REFERENCE
- 1 missing output (not generated by the run)
- 2 missing databases.
More details can be found in the LOG and and in the ERROR.
The REPORT file is the one used to send final logs to the mailing list.
ERROR file
This is a useful file with detailed information about the errors only.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% Yambo test suite ERRORs log %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %================================================================================== %Running tests :Dummy %Projects :nopj %Date - Time :Fri-19-May - 11-46 %Build :master - MPI+SLK - rev.14505 %Parallel Conf :[serial] - none %=--------------------------------------------------------------------------------= %Dummy/02_hf-serial-master/o-02_hf.hf : VALUE(# 17)@COL# 6 is -5.4691 [REF] vs -4.4691 [OUT] corresponding to 6.1013 [% RELATIVE TO MAX] %Dummy/02_hf-serial-master : NO REFERENCE/o-02_hf.ndb.em1s_fragment_1 in OUTPUT %Dummy/04_em1s-serial-master : NO o-04_em1s.ndb.em1s_fragment_1 in REFERENCE %% Section Duration : 0:0:3 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% TOTAL Duration : 0:0:3 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Running
Before entering in the details of the different running methods let's first describe the Parallel Options
Parallel Options
% (parallel options) % -np <N> Fixed number of CPU used % -np_min <N> Minimum number of CPU used % -np_max <N> Maximum number of CPU used % -nt <T> # of OMP threads % -nl <L> # of CPU used for linear algebra % -def_par Use the default parallelization scheme % -rand_par Use randomly generated parallel structures
Examples:
%./driver.pl -tests Dummy -force -np 2
produces all possible combinations of 2 cpu's for the Dummy tests. Each run is labelled in such a way to be unique. The Dummy folder will look like
%%l tests/Dummy/ %01_init-Nmpi2-1-master/ 02_hf-Nmpi2-3-master/ 03_optics-Nmpi2-3-master/ 04_em1s-Nmpi2-3-master/ INPUTS/ yambo.in %02_hf 03_optics 04_em1s 04_em1s-Nmpi2-4-master/ LOG/ %02_hf-Nmpi2-1-master/ 03_optics-Nmpi2-1-master/ 04_em1s-Nmpi2-1-master/ BROKEN REFERENCE/ %02_hf-Nmpi2-2-master/ 03_optics-Nmpi2-2-master/ 04_em1s-Nmpi2-2-master/ Dummy.tar.gz SAVE/
Note that each run is labelled by the number of MPI cpus (Nmpi2) and by the increasing counter of potential combinations of the two cpu's (-1-master,-2-master,...).
Other potential combinations are
- MPI, default parallelization (no cycle)
%./driver.pl -tests Dummy -force -np 2 -def_par
- MPI, random parallelization (no cycle, good for testing)
%./driver.pl -tests Dummy -force -np 2 -rand_par
- MPI, using 2 and 4 cpu's, no cycle on combinations, using random parallelization
%./driver.pl -tests Dummy -force -np_min 2 -np_max 4 -rand_par
- OpenMP (if compiled with OpenMP)
%./driver.pl -tests Dummy -force -nt 2
- MPI+SLK
%./driver.pl -tests Dummy -force -np 4 -nl 4 -def_par
Tests selection
The test-suite can be run in three different ways
- Using a theme
- Passing individual sets of tests
- Using flows
Moreover tests can be selected using projects.
Flows are discussed in the Flows.
Theme
A theme is a just a series of tests. Themes are stored in the THEMES/ folder and are used simply using
%./driver.pl -them test
test is a text file in the THEMES folder.
Individual Tests
This is the way we used to run the Dummy test. Just use
%./driver.pl -tests Dummy
to select a combination of specific inputs and tests you can use more complex tests options:
%./driver.pl -tests "Dummy 01_init 02_hf; Si_bulk/GW-OPTICS 00_init 02Cohsex"
Projects & Keys
Project are listed with the test lists and are accessed with the lowercase options: elph,sc,rt,nl...
So in order to run all tests belonging to the sc project just type
%./driver.pl -tests all -keys sc
Actually the -keys option is more powerful as, in the future, should also select all inputs with specific observables (like gw, optics and so on).
The list of project are accessed by simply the first column in the list obtained by using the -l option. In the following are in bold.
%%./driver.pl -l %Available test materials/systems: %[YAMBO] Al111 % C6H3Cl3 % PA_chain[H] % SiH4 % Al_bulk/GW-OPTICS % H2/OPTICS % H_chain/2.05 % H_chain/2.5 % hBN/GW-OPTICS % LiF/GW-OPTICS % Si_bulk/GW-OPTICS % Si_surface/OPTICS % Si_wire/OPTICS % MoS2/pwscf/OPTICS % MoS2/abinit/OPTICS/With-SOC % MoS2/abinit/OPTICS/Without-SOC-sp1 % MoS2/abinit/OPTICS/Without-SOC-sp2 %[QED] Si_bulk/QED %[NL] hBN/NL %[RT] AlAs[H] % H2/RT % hBN/RT % Si_bulk/RT % WSe2/RT[H] % MoS2/pwscf/RT %[SC] H2/SC % hBN/SC % Si_bulk/SC %[MAGNETIC] Si_bulk/MAGNETIC %[ELPH] Diamond/ELPH1[H] % Si_bulk/ELPH/BSE % Si_bulk/ELPH/ELPH_base % Si_bulk/ELPH/ELPH_for_BSE % Si_bulk/ELPH/QP_CTL %[KERR] Cobalt % Nickel % Iron/pwscf % Iron/abinit/With-SOC % Iron/abinit/Without-SOC %[BROKEN] Dummy % GaAs001-c4x4/YAMBO % hBN/PL[PL] % Si001-c4x2/YAMBO % WS2/PL[PL]
Note that to access any of this project you must use the lowercase option. For example
- SC → -keys sc
- RT → -keys rt
- SC+RT → -keys 'sc rt'
HARD tests
Some of the tests are long. In the tests list they are labelled with a [H]. In order to run them the key hard must be used. For example
%./driver.pl -tests all -keys 'sc rt hard'
runs all SC+RT tests including the heavy ones.
FLOWS
All properties of a single launch of the test-suite and also a more complex test-suite launch looping on several parallel, tests, branches, combinations can be controlled via the FLOW feature.
A FLOW is a perl file (like validate.pl) containing combinations of descriptors.
All FLOW files are in the ROBOTS/<YOUR ROBOT>/FLOWS.
A FLOW is run using
%./driver.pl -flow validate
A general flow descriptor is
% {
% ACTIVE => "yes", # can be yes or no
% MPI_CPU => "NP",
% SLK_CPU => "NM",
% PAR_MODE => "TEXT", # (TEXT can be default, random, loop)
% THREADS => "NT",
% BRANCH => "/home/marini/Yambo/sources/git/yambo/branches/SLK", # (SAVED)
% THEME => "G_parallelization", # (SAVED)
% CONFIG => "gfortran_slk.sh", # (SAVED)
% TESTS => "hBN/GW-OPTICS", # (SAVED)
% KEYS => "nopj elph hard bse rpa", # (SAVED)
%}
The fields labelled with a (SAVE) are not changed if this descriptor is included in a more complex FLOW. In order to explain this feature let's just have a look at the default validate FLOW provided by default:
%#!/usr/bin/perl
%#
%@flow = (
%{
% ACTIVE => "yes",
% CONFIG => "default.sh",
% KEYS => "nopj elph sc hard",
%},
%{
% MPI_CPU => $SYSTEM_NP,
% PAR_MODE => "default",
%},
%{
% MPI_CPU => $SYSTEM_NP,
% PAR_MODE => "random",
%},
%{
% MPI_CPU => $SYSTEM_NP_half,
% PAR_MODE => "loop",
%},
%{
% THREADS => $SYSTEM_NP_half,
%},
%{
% MPI_CPU => $SYSTEM_NP,
% SLK_CPU => $SYSTEM_NP_half,
% PAR_MODE => "default",
%},
%{
% MPI_CPU => $SYSTEM_NP_half,
% THREADS => $SYSTEM_NP_half,
% SLK_CPU => $SYSTEM_SLK,
% PAR_MODE => "default",
%},
%);
When this flow is launched the driver.pl will run all tests of the "nopj elph sc hard" projects using: a serial run, all system CPU's with default parallelization, with random parallelization, cycling among all possible combinations and so so...
Note that also the -autotest is just a simple calculation flow.
Advanced Use
TODO