Rome 2023

A general description of the goal(s) of the school can be found on the Yambo main website

Use CINECA computational resources

Yambo tutorials will be run on the MARCONI100 (M100) accelerated cluster. You can find info about M100 here. In order to access computational resources provided by CINECA you need your personal username and password that were sent you by the organizers.

Connect to the cluster using ssh

You can access M100 via ssh protocol in different ways.

- Connect using username and password

Use the following command replacing your username:

$ ssh username@login.m100.cineca.it

However, in this way you have to type your password each time you want to connect.

- Connect using ssh key

You can setup a ssh key pair to avoid typing the password each time you want to connect to M100. To do so, go to your .ssh directory (usually located in the home directory):

$ cd $HOME/.ssh

If you don't have this directory, you can create it with mkdir $HOME/.ssh.

Once you are in the .ssh directory, run the ssh-keygen command to generate a private/public key pair:

$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key: m100_id_rsa
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in <your_.ssh_dir>/m100_id_rsa
Your public key has been saved in <your_.ssh_dir>/m100_id_rsa.pub
The key fingerprint is:
<...>
The key's randomart image is:
<...>

Now you need to copy the public key to M100. You can do that with the following command (for this step you need to type your password):

$ ssh-copy-id -i <your_.ssh_dir>/m100_id_rsa.pub <username>@login.m100.cineca.it

Once the public key has been copied, you can connect to M100 without having to type the password using the -i option:

$ ssh -i <your_.ssh_dir>/m100_id_rsa username@login.m100.cineca.it

To simplify even more, you can paste the following lines in a file named config located inside the .ssh directory adjusting username and path:

Host m100 
 HostName login.m100.cineca.it
 User username
 IdentityFile <your_.ssh_dir>/m100_id_rsa

With the config file setup you can connect simply with

$ ssh m100

General instructions to run tutorials

Before proceeding, it is useful to know the different workspaces you have available on M100, which can be accessed using environment variables. The main ones are:

• $HOME: it's the home directory associated to your username;
• $WORK: it's the work directory associated to the account where the computational resources dedicated to this school are allocated;
• $CINECA_SCRATCH: it's the scratch directory associated to your username.

You can find more details about storage and FileSystems here.

Please don't forget to run all tutorials in your scratch directory:

$ echo $CINECA_SCRATCH
/m100_scratch/userexternal/username
$ cd $CINECA_SCRATCH

Computational resources on M100 are managed by the job scheduling system Slurm. Most part of Yambo tutorials during this school can be run in serial, except some that need to be executed on multiple processors. Generally, Slurm batch jobs are submitted using a script, but the tutorials here are better understood if run interactively. The two procedures that we will use to submit interactive and non interactive jobs are explained below.

- Run a job using a batch script

This procedure is suggested for the tutorials and examples that need to be run in parallel. In these cases you need to submit the job using a batch script job.sh, whose generic structure is the following:

$ more job.sh
#!/bin/bash
#SBATCH --account=tra23_Yambo           # Charge resources used by this job to specified account
#SBATCH --time=00:10:00                 # Set a limit on the total run time of the job allocation in hh:mm:ss
#SBATCH --job-name=JOB                  # Specify a name for the job allocation
#SBATCH --partition=m100_usr_prod       # Request a specific partition for the resource allocation
#          
#SBATCH --nodes=<N>                     # Number of nodes to be allocated for the job
#SBATCH --ntasks-per-node=<nt>          # Number of MPI tasks invoked per node
#SBATCH --ntasks-per-socket=<nt/2>      # Tasks invoked on each socket
#SBATCH --cpus-per-task=<nc>            # Number of OMP threads per task

module purge
module load hpc-sdk/2022--binary
module load spectrum_mpi/10.4.0--binary
export PATH=/m100_work/tra23_Yambo/softwares/YAMBO/5.2-mpi/bin:$PATH

export OMP_NUM_THREADS=<nc>

mpirun --rank-by core -np ${SLURM_NTASKS} \
       yambo -F <input> -J <output>

Please note that these instructions must be compatible with the specific M100 architecture and accounting systems. The complete list of Slurm options can be found here. However, you will find ready-to-use batch scripts in locations specified during the tutorials.

To submit the job, use the sbatch command:

$ sbatch job.sh
Submitted batch job <JOBID>

To check the job status, use the squeue command:

$ squeue -u <username>
             JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
             <...> m100_usr_      JOB username  R       0:01    <N> <...>

If you need to cancel your job, use:

$ scancel <JOBID> 

- Open an interactive session

This procedure is suggested for most of the tutorials, since the majority of these is meant to be run in serial from the command line. With the command below, you can open an interactive session of 1 hour to execute commands in serial (complete documentation here):

$ salloc --account=tra23_Yambo --nodes=1 --ntasks-per-node=1 --cpus-per-task=1 --partition=m100_usr_prod --qos=m100_qos_dbg --time=01:00:00
salloc: Granted job allocation 10164647
salloc: Waiting for resource configuration
salloc: Nodes r256n01 are ready for job

With squeue you can see that there is now a job running:

$ squeue -u username
             JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
          10164647 m100_usr_ interact username  R       0:02      1 r256n01

To run the tutorial, ssh into the node specified by the job allocation and cd to your scratch directory:

$ ssh r256n01
$ cd $CINECA_SCRATCH

Then, you need to manually load yambo as in the batch script above. Please note that the serial version of the code is in a different directory and does not need spectrum_mpi:

$ module purge
$ module load hpc-sdk/2022--binary
$ export PATH=/m100_work/tra23_Yambo/softwares/YAMBO/5.2-ser/bin:$PATH

Finally, set the OMP_NUM_THREADS environment variable to 1:

$ export OMP_NUM_THREADS=1

To close the interactive session when you have finished, log out of the compute node with the exit command, and then cancel the job:

$ exit
$ scancel <JOBID>

Tutorials

DAY 1 - Monday, 22 May

16:15 - 18:30 From the DFT ground state to the complete setup of a Many Body calculation using Yambo

To get the tutorial files needed for the following tutorials, follow these steps:

$ ssh m100
$ cd $CINECA_SCRATCH
$ mkdir YAMBO_TUTORIALS
$ cd YAMBO_TUTORIALS
$ wget https://media.yambo-code.eu/educational/tutorials/files/hBN.tar.gz
$ wget https://media.yambo-code.eu/educational/tutorials/files/hBN-2D.tar.gz
...
Saving to: ‘hBN-2D.tar.gz’

hBN-2D.tar.gz                      100%[================================================================>]   8.56M  46.7MB/s    in 0.2s    
...

$ ls
hBN-2D.tar.gz  hBN.tar.gz
$ tar -xvf hBN-2D.tar.gz
...
$ tar -xvf hBN.tar.gz
...

Now that you have all the files, you may open the interactive job session with salloc as explained above and proceed with the tutorials.

• First steps: Initialization and more
• Next steps: RPA calculations
• Band structures with yambopy: Tutorial 1 (BN)
• Band structures with yambopy: Tutorial 2 (Iron)

DAY 2 - Tuesday, 23 May

14:00 - 16:30 A tour through GW simulation in a complex material (from the blackboard to numerical computation: convergence, algorithms, parallel usage)

• GW computations on practice: how to obtain the quasi-particle band structure of a bulk material

DAY 3 - Wednesday, 24 May

17:00 - 18:30 Real-time Bethe-Salpeter equation Fulvio Paleari (), Davide Sangalli (CNR-ISM, Italy)

• Linear response from real time simulations (extract from this the part with yambo_rt)

DAY 4 - Thursday, May 25

14:00 - 16:30 Real-time approach with the time dependent berry phase Myrta Gruning (), Davide Sangalli (CNR-ISM, Italy)

• Linear response from real time simulations (extract from this the part with yambo_nl)
• Non-linear response TD-HSEX: hBN (to be created, inputs from Ignacio)

• Real time approach to non-linear response (to decide if we do also AlAS part 1)
• Correlation effects in the non-linear response (to decide if we do also AlAS part 2)