Phonon calculation in parallel mode

[Tobias Kraemer]

Hi,


I am trying to run a phonon calculation on a system containing around 300 atoms in order to obtain its full Hessian matrix. Unfortunately wall time on our supercomputer architecture is limited to 48 hrs, hence the calculation does not finish in time when run on 16 nodes (384 cores). Since I am using finite differences, it should be possible to parallelise this job efficiently and run it on a big number of cores. Is this simply a matter of increasing the number of cores in this case? The system also contains some symmetry, so it might be useful to use IBRION=6 instead? Perhaps somebody can give me some advise, I am pretty certain that this should work somehow.

Below is my INCAR file.

Thanks

Tobias


General:
SYSTEM = [Rh-dcype-exo-nba][BArCl]
! if enough K-points are present -5 should be used
ISMEAR = 0 ! 0: Gaussian -> semicond/insulator ; 1-N: MP; -5: Tetra+Blochl -> metal/very accurate energy/forces
SIGMA = 0.05
EDIFF = 1.0E-7
PREC = Normal ! to make ROPT=2.0E-4 for LREAL=Auto
LREAL = Auto ! cheaper for large cell's we are using
ENCUT = 600 !
ISYM = 2 # use symmetry as done for PAW PP

NELMIN = 8
LWAVE = .FALSE.
LCHARG = .FALSE.
! LVTOT = .TRUE. ! Write LOCPOT file for Toon
! LVHAR = .TRUE. ! but LOCPOT without exchange and correlation contribution for forcefields Toon
VOSKOWN = 1 ! important for GGA (PW91) for interpolation of XC...since we only use PBE=switch on (LDA: VOSKOWN=0)
LASPH = .TRUE. ! For VASP.5.X the aspherical contributions are properly accounted for in the Kohn-Sham potential
! as well. This is essential for accurate total energies and band structure calculations for f-elements
! (e.g. ceria), all 3d-elements (transition metal oxides), and magnetic atoms in the 2nd row (B-F atom),
! in particular if LDA+U or hybrid functionals or meta-GGAs are used, since these functionals often result
! in aspherical charge densities.

Van der Waals Interaction (vasp 5.3.3 patched verion):
IVDW = 12 ! switches between 0:off, 1: DFT-D3 and 2: TS-VDW (default=1)

dynamic:
IBRION = 5 ! -1: Fix atoms; 0: MD; 2: ConjGrad relax; 44: improved dimer method
NSW = 1 ! Number electronic steps
ISIF = 0 # relax ions, no change cell shape, no change cell volume (volume changes require ENCUT*1.3)
POTIM = 0.015

parallel:
LPLANE = .TRUE.
# NPAR = 4

[alex]

Hi Tobias,

I once had a piece of code for VASP 4.x, which was not ported to VASP 5.x. Sorry for that.

Some general hint's:
- If you could exploit symmetry, do it. It's for free and you still get what you want.
- You may search the web for scripts (!), which do the job. The parallelization could, e.g., done by going through atom 1 to 50 in the first batch, than 51 to 100 and so on. Just make sure the script checks for existing OUTCARs first, calculate later. Afterwards you can cheat and put all OUTCARs together and calculate the Hessian with that script having no extra-fuzz.
- more cores? Well, That depends on the scaling, but yes, this is a way, too.

Cheers,

alex

[Tobias Kraemer]

Hi Alex,


thanks for your reply and the advice. I have tried to use symmetry by using IBRION=6 and ISYM=2 in my INCAR file for the phonon job. It seems that VASP picks up some symmetry, since it recognizes the initial symmetry as S_2, however, pretty soon after that the symmetry is reduced to C_1. So I don't know what is going on here. Is there anything I can do about this? If you are doing the calculation in batches, wouldn't you miss some off-diagonal elements in the Hessian matrix? I guess to first approximation this would be good enough to decide whether the structure was in a minimum.

Best

Tobias

Hi Tobias,

I once had a piece of code for VASP 4.x, which was not ported to VASP 5.x. Sorry for that.

Some general hint's:
- If you could exploit symmetry, do it. It's for free and you still get what you want.
- You may search the web for scripts (!), which do the job. The parallelization could, e.g., done by going through atom 1 to 50 in the first batch, than 51 to 100 and so on. Just make sure the script checks for existing OUTCARs first, calculate later. Afterwards you can cheat and put all OUTCARs together and calculate the Hessian with that script having no extra-fuzz.
- more cores? Well, That depends on the scaling, but yes, this is a way, too.

Cheers,

alex

[alex]

Hi Tobias,

in case you are using numerical differences (IBRION = 5 or 6) you'll always end up with C1, because you are changing the input structure by moving all atoms in +/- POTIM (according to symmetry you'll save some moves if IBRION = 6).
To avoid symmetry loss you might consider IBRION = 8, the analytical 2nd derivative calculation. But it's not fast either (compared to a single SCF run).

If you are after a first approximation you may cut down on precision of your numerics, e.g. ENCUT, k-points, POTCAR, ...
You might also consider - depending on the nature of your system - to fix certain atoms in POSCAR before you start the frequency calculation and do numerical differencies. This works fine if you study e.g. reactions on surfaces.

Hth,

alex

[Tobias Kraemer]

Alex,

yes, this seems to make sense. I had a closer look at the output file, and it appears that the symmetry is recognized correctly by VASP (and also kept throughout the calculation). I thought the the reduced symmetry was due to the fact that one atom at a time is displaced. Thanks for confirming this. The number of vibrations is correct, and corresponds to half of what would be expected for the full unit cell contents (in the present case there is a centre of inversion present).
Looks like I had to play around with the number of cores to make the calculation most efficient, and I seem to have found some settings which enable the job to finish within the wallclock time.

Thanks again for all your comments. Very helpful.

Tobias

[nigel_hew]

Hi Tobias,

I'm having the same issue with a 200-atom supercell not finishing in the 48 hour wall time. If it is not too much trouble, would you be able to share some of the changes you made?

Thanks,
Nigel
Postdoctoral Scholar
Penn State