...
The following containers are likely to be useful to MWASCI members as a replacement for the MWA_Tools module that used to work on Magnus/Zeus. MWA_Tools module doesn't work any more because of changes that pawsey have made to the available modules and versions (mostly python issues). The MWA_Tools module will not be resurrected.
| Container image | software within container |
|---|---|
| /pawsey/mwa/singularity/casa/casa.img | casa casaviewer |
| /pawsey/mwa/singularity/cotter/cotter_latest.sif | aoflagger aoqplot aoquality cotter fixmwams rfigui |
| /pawsey/mwa/singularity/wsclean/wsclean_2.9.2.img | wsclean aoflagger casacore chgcentre python2.7 python3.6 |
| /pawsey/mwa/singularity/mwa-reduce/mwa-reduce.img | addimg addimg aegean2model apparently applybeam applyion applysolutions autoprocess bbs2model beam calibrate cluster editmodel fitsmodel flagantennae flagbaselines flagionsolutions flagmwa flagsolutions flagsubbands ionpeel matchsources mrc2model mwafinfo pbaddimg pbcorrect peel phasecal regridimg render scaleimage sedcombine solutiontool storetime subtrmodel vo2model |
| /pawsey/mwa/singularity/python/python_latest.sif | stilts python3 numpy cython scipy matplotlib astropy aipy h5py healpy ipython pandas pyyaml tqdm colorama patsy statsmodels AegeanTools mwa-hyperbeam pyuvdata PyBDSF |
| /pawsey/mwa/singularity/rfi_seeker/rfi_seeker.img | RFISeeker python3 astropy numpy scipy matplotlib wcstools |
| /pawsey/mwa/singularity/robbie/robbie-next.sif | Robbie-next topcat stilts python3 AegeanTools astropy healpy numpy scipy lmfit pandas matplotlib |
Setup:
- Remove the
module loadandexportcommands from your .bashrc and .profile files and use the example files at/group/mwa/software/_RC_FILESas a guide for making new ones.Copy/group/mwa/software/_RC_FILES/mwasci.casarcover the top of your~/.casarcfile so that casacore will work - See README on the manta-ray client github for how to setup asvo access via a batch script.
- Contact your admin rep (Paul/Chris/Sammy) before contacting help@pawsey as there are some things that we can fix faster.
- Only members of the group
mwaadmincan build software and edit the module files in/group/mwa/software. We have automated the build process for future reference. If you require software that is not in there, and which is to be used by more than 1-2 people, then contact one your admin rep and we will work something out.
...
- Don't rely on .profile or .bashrc to load the modules that you need within a batch script
- Use containers as described above.
To ensure a 'clean' environment at the start of your batch script you should use:source /group/mwa/software/module-reset.sh# to reset the modules to some default/minimal list. NOTE: you must source this script, don't run it directly.
At the start of each job script load the modules that you need using:module use /group/mwa/software/modulefilesmodule load MWA_Tools/mwa-sci# will give you access to the MWA software stack.module load python# will give you python/astropy/numpy/scipy (included in MWA_Tools)module load stilts #will give you java and stilts (included in MWA_Tools)
- Data use:
- /group and /astro should be used for storing code and data. Intermediate/long term data products should be on /group and /astro should only be used for temp/short term data as it's not backed up.
- /home/ or ~ should be used only for config files like .profile .bash_aliases etc..
- DO NOT store scripts, logfiles, or code in your home directory.
- if you make symlinks in your home directory for convenience, don't use these links in your job scripts, it will put a strain on this file system for all users.
- downloading and moving of data should be done via the Zeus copyq (for data <1GB you may get away with the occasional wget/scp)
- copying data FROM pawsey should be done via hpc-data.pawsey.org.au
...