Modena 2025: Difference between revisions
No edit summary |
|||
Line 87: | Line 87: | ||
module load yambo/5.3.0--intel-oneapi-mpi--2021.12.1--oneapi--2024.1.0 | module load yambo/5.3.0--intel-oneapi-mpi--2021.12.1--oneapi--2024.1.0 | ||
export OMP_NUM_THREADS=${SLURM_CPUS_PER_TASK} | export OMP_NUM_THREADS=${SLURM_CPUS_PER_TASK} | ||
mpirun -np ${SLURM_NTASKS} \ | mpirun -np ${SLURM_NTASKS} \ | ||
yambo -F <input> -J <output> | yambo -F <input> -J <output> |
Revision as of 08:58, 17 May 2025
A general description of the goal(s) of the school can be found on the Yambo main website
Use CINECA computational resources
Yambo tutorials will be run on the Leonardo-DCGP partition. You can find info about Leonardo-DCGP here. In order to access computational resources provided by CINECA you need your personal username and password that were sent you by the organizers.
Connect to the cluster using ssh
You can access Leonardo via ssh
protocol in different ways.
- Connect using username and password
Use the following command replacing your username:
ssh username@login.leonardo.cineca.it
However, in this way you have to type your password each time you want to connect.
- Connect using ssh key
You can setup a ssh key pair to avoid typing the password each time you want to connect to Leonardo. To do so, run the ssh-keygen
command to generate a private/public key pair:
ssh-keygen -t rsa -b 4096 -f ~/.ssh/leonardo_rsa Generating public/private rsa key pair. Created directory '/home/username/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/username/.ssh/leonardo_rsa Your public key has been saved in /home/username/.ssh/leonardo_rsa.pub The key fingerprint is: [...] The key's randomart image is: [...]
Now you need to copy the public key to Leonardo. You can do that with the following command (for this step you need to type your password):
ssh-copy-id -i ~/.ssh/leonardo_rsa username@login.leonardo.cineca.it
Once the public key has been copied, you can connect to Leonardo without having to type the password using the -i
option:
ssh -i ~/.ssh/leonardo_rsa username@login.leonardo.cineca.it
To simplify even more, you can paste the following lines in a file named config
located inside the .ssh
directory adjusting the username:
Host leonardo HostName login.leonardo.cineca.it User username IdentityFile ~/.ssh/leonardo_rsa
With the config
file setup you can connect simply with
ssh leonardo
General instructions to run tutorials
Before proceeding, it is useful to know the different workspaces you have available on Leonardo, which can be accessed using environment variables. The main ones are:
$HOME
: it's thehome
directory associated to your username;$WORK
: it's thework
directory associated to the account where the computational resources dedicated to this school are allocated;$SCRATCH
: it's thescratch
directory associated to your username.
You can find more details about storage and FileSystems here.
Please don't forget to run all tutorials in your scratch directory:
echo $SCRATCH /leonardo_scratch/large/userexternal/username cd $SCRATCH
Computational resources on Leonardo are managed by the job scheduling system Slurm. Most part of Yambo tutorials during this school can be run in serial, except some that need to be executed on multiple processors. Generally, Slurm batch jobs are submitted using a script, but the tutorials here are better understood if run interactively. The two procedures that we will use to submit interactive and non interactive jobs are explained below.
- Run a job using a batch script
This procedure is suggested for the tutorials and examples that need to be run in parallel. In these cases you need to submit the job using a batch script job.sh
, whose generic structure is the following:
#!/bin/bash #SBATCH --account=tra25_yambo # Charge resources used by this job to specified account #SBATCH --time=00:10:00 # Set a limit on the total run time of the job allocation in hh:mm:ss #SBATCH --job-name=JOB # Specify a name for the job allocation #SBATCH --partition= dcgp_usr_prod # Request a specific partition for the resource allocation #SBATCH --gres=tmpfs:10g # List of generic consumable resources #SBATCH --qos=normal # Quality of service #SBATCH --reservation=s_tra_yambo # Reservation specific to this school # #SBATCH --nodes=<N> # Number of nodes to be allocated for the job #SBATCH --ntasks-per-node=<n> # Number of MPI tasks invoked per node #SBATCH --ntasks-per-socket=<n/2> # Tasks invoked on each socket #SBATCH --cpus-per-task=<c> # Number of OMP threads per task module purge module load profile/candidate module use /leonardo/pub/userexternal/nspallan/spack-0.22.2-06/modules module load yambo/5.3.0--intel-oneapi-mpi--2021.12.1--oneapi--2024.1.0 export OMP_NUM_THREADS=${SLURM_CPUS_PER_TASK} mpirun -np ${SLURM_NTASKS} \ yambo -F <input> -J <output>
Please note that the instructions in the batch script must be compatible with the specific Leonardo-DCGP resources. The complete list of Slurm options can be found here. However you will find ready-to-use batch scripts in locations specified during the tutorials.
To submit the job, use the sbatch
command:
sbatch job.sh Submitted batch job 15696508
To check the job status, use the squeue
command:
squeue --me JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON) 15696508 dcgp_usr_ job.sh username R 0:01 1 lrdn4135
If you need to cancel your job, do:
scancel <JOBID>
- Open an interactive session
This procedure is suggested for most of the tutorials, since the majority of these is meant to be run in serial (relatively to MPI parallelization) from the command line. Use the command below to open an interactive session of 4 hours:
srun -A tra25_yambo -p dcgp_usr_prod --reservation=s_tra_yambo -N 1 -n 1 -c 4 -t 04:00:00 --gres=tmpfs:10g --pty /bin/bash srun: job 15694182 queued and waiting for resources srun: job 15694182 has been allocated resources
We ask for 4 cpus-per-task (-c) because we can exploit OpenMP parallelization with the available resources.
Then, you need to manually load yambo
as in the batch script above:
module purge module load profile/candidate module use /leonardo/pub/userexternal/nspallan/spack-0.22.2-06/modules module load yambo/5.3.0--intel-oneapi-mpi--2021.12.1--oneapi--2024.1.0
Finally, set the OMP_NUM_THREADS
environment variable to 4 using the appropriate Slurm environment variable:
export OMP_NUM_THREADS=${SLURM_CPUS_PER_TASK}
To close the interactive session when you have finished, log out of the compute node with the exit
command:
exit
- Plot results with gnuplot
During the tutorials you will often need to plot the results of the calculations. In order to do so on Leonardo, open a new terminal window and connect to Leonardo enabling X11 forwarding with the -X
option:
ssh -X leonardo
Please note that gnuplot
can be used in this way only from the login nodes:
username@login01$ cd <directory_with_data> username@login01$ gnuplot ... Terminal type is now '...' gnuplot> plot <...>
- Set up yambopy
In order to run yambopy on Leonardo, you must first activate the python environment:
module load profile/candidate module load python/3.11.7 source /leonardo_work/tra25_yambo/env_yambopy/bin/activate
Tutorials
Quick recap. Before every tutorial, if you run on Leonardo, do the following steps
ssh leonardo cd $SCRATCH mkdir -p YAMBO_TUTORIALS #(Only if you didn't before) cd YAMBO_TUTORIALS
Since the compute nodes are not connected to the external network, the tarballs must be downloaded before starting the interactive session. Alternatively, once the interactive session has started, it is possible to access the tarballs by copying them from the following directories:
/leonardo_work/tra25_yambo/YAMBO_TUTORIALS /leonardo_work/tra25_yambo/YAMBOPY_TUTORIALS
After that you can start the interactive session
srun -A tra25_yambo -p dcgp_usr_prod --reservation=s_tra_yambo -N 1 -n 1 -c 4 -t 04:00:00 --gres=tmpfs:10g --pty /bin/bash [...]
set the environment variable for openMP
export OMP_NUM_THREADS=${SLURM_CPUS_PER_TASK}
and load yambo or yambopy as explained above in the general instructions.
DAY 1 - Monday, 19 May
16:15 - 18:30 From the DFT ground state to the complete setup of a Many Body calculation using Yambo
To get the tutorial files needed for the following tutorials, follow these steps:
wget https://media.yambo-code.eu/educational/tutorials/files/hBN.tar.gz wget https://media.yambo-code.eu/educational/tutorials/files/hBN-2D.tar.gz ls hBN-2D.tar.gz hBN.tar.gz tar -xvf hBN-2D.tar.gz tar -xvf hBN.tar.gz ls hBN-2D hBN hBN-2D.tar.gz hBN.tar.gz
Now that you have all the files, you may open the interactive job session with srun
as explained above and proceed with the tutorials.
At this point, you may learn about the python pre-postprocessing capabilities offered by yambopy, our python interface to yambo and QE. First of all, let's create a dedicated directory, download and extract the related files.
cd $SCRATCH mkdir -p YAMBOPY_TUTORIALS cd YAMBOPY_TUTORIALS wget https://media.yambo-code.eu/educational/tutorials/files/databases_yambopy.tar.gz tar -xvf databases_yambopy.tar.gz cd databases_yambopy
Then, follow the first three sections of this link, which are related to initialization and linear response.
DAY 2 - Tuesday, 20 May
14:00 - 16:30 A tour through GW simulation in a complex material (from the blackboard to numerical computation: convergence, algorithms, parallel usage)
To get all the tutorial files needed for the following tutorials, follow these steps:
wget https://media.yambo-code.eu/educational/tutorials/files/hBN.tar.gz wget https://media.yambo-code.eu/educational/tutorials/files/MoS2_2Dquasiparticle_tutorial.tar.gz tar -xvf hBN.tar.gz tar -xvf MoS2_2Dquasiparticle_tutorial.tar.gz cd hBN
Now you can start the first tutorial:
If you have gone through the first tutorial, pass now to the second one:
cd $SCRATCH cd YAMBO_TUTORIALS cd MoS2_HPC_tutorial
To conclude, you can learn an other method to plot the band structure in Yambo
DAY 3 - Wednesday, 21 May
14:00 - 16:30 Bethe-Salpeter equation (BSE) Fulvio Paleari (CNR-Nano, Italy), Davide Sangalli (CNR-ISM, Italy)
To get the tutorial files needed for the following tutorials, follow these steps:
wget https://media.yambo-code.eu/educational/tutorials/files/hBN.tar.gz # NOTE: YOU SHOULD ALREADY HAVE THIS FROM DAY 1 wget https://media.yambo-code.eu/educational/tutorials/files/hBN-convergence-kpoints.tar.gz tar -xvf hBN-convergence-kpoints.tar.gz tar -xvf hBN.tar.gz
Now, you may open the interactive job session with salloc
and proceed with the following tutorials.
- Perform a BSE calculation from beginning to end
- Analyse your results (exciton wavefunctions in real and reciprocal space, etc.)
- Solve the BSE eigenvalue problem with different numerical methods
- Choose the input parameters for a meaningful converged calculation
Now, go into the yambopy tutorial directory to learn about python analysis tools for the BSE:
cd $SCRATCH cd YAMBOPY_TUTORIALS/databases_yambopy
17:00 - 18:30 Bethe-Salpeter equation in real time (TD-HSEX) Fulvio Paleari (CNR-Nano, Italy), Davide Sangalli (CNR-ISM, Italy)
The files needed for the following tutorials can be downloaded following these steps:
wget https://media.yambo-code.eu/educational/tutorials/files/hBN-2D-RT.tar.gz tar -xvf hBN-2D-RT.tar.gz
- Read the introductive section to real-time propagation for the one-body density matrix (the part about time-dependent Schrödinger equation will be covered on DAY 4 and you can skip it for now)
- Perform the setup for a real-time calculation
- Calculate the linear response in real time
- Calculate the BSE in real time
DAY 4 - Thursday, May 22
14:00 - 16:30 Real-time approach with the time dependent berry phase Myrta Gruning (Queen's University Belfast), Davide Sangalli (CNR-ISM, Italy)
For the tutorials we will use first the hBN-2D-RT
folder (k-sampling 10x10x1) and then the hBN-2D
folder (k-sampling 6x6x1)
You may already have them in the YAMBO_TUTORIALS
folder
ls hBN-2D hBN-2D-RT hBN-2D.tar.gz hBN-2D-RT.tar.gz
If you need to downoload again the tutorial files, follow these steps:
wget https://media.yambo-code.eu/educational/tutorials/files/hBN-2D.tar.gz wget https://media.yambo-code.eu/educational/tutorials/files/hBN-2D-RT.tar.gz tar -xvf hBN-2D.tar.gz tar -xvf hBN-2D-RT.tar.gz
- Real time approach to non-linear response (additional tutorial)
- Correlation effects in the non-linear response (additional tutorial)
DAY 5 - Friday, 23 May
Lectures
DAY 1 - Monday, 19 May
- D. Varsano, [Description and goal of the school].
- C. Franchini, [First principles and data-driven correlated materials]
- F. Mohamed, [A tour on Density Functional Theory]
- E. Cannuccia, [Electronic screening and linear response theory]
DAY 2 - Tuesday, 20 May
- A. Marini, Introduction to Many-Body Perturbation Theory
- C. Cardoso, Quasiparticles and the GW Approximation
- A. Guandalini,G. Sesti, GW in practice: algorithms, approximations and W-averaged GW in metals
- M. Govoni, GW without empty states and investigation of neutral excitations by embedding full configuration interaction in DFT+GW
DAY 3 - Wednesday, 21 May
- M. Palummo, Optical absorption and excitons via the Bethe-Salpeter Equation
- D. Sangalli, Real-time simulations
- F. Paleari, Introduction to YamboPy (automation and post-processing)
DAY 4 - Thursday, 22 May
- E. Luppi, An introduction to Non-linear spectroscopy
- M. Grüning, Non-linear spectroscopy in Yambo
- F. Affinito, Frontiers in High-Performance Computing