Skip to end of metadata
Go to start of metadata

Browser version

Link: https://asvo.mwatelescope.org/


Once you go to the link, on the left hand side are the search options. A vast range of search options are provided. Some of the frequently used ones are: Obs ID, Pointing, Data quality. For VCS observation, we will have to set the observing mode to  VOLTAGE_START. If set to any VCS, then legacy as well as MWAX data is downloaded. 

Once the required options are set, clicking the search button will show a list of observations of possible interest. On clicking on the Obs ID you will be taken to a page where you can enter the offset and the duration of observation you want to download and then press create. 

On pressing create, a job will be created to download the data on /astro on Garrawarla. This also gives you a Job ID which can be used to check on Garrawarla about the status of your job.

Here are some slides to explain the download process (with screenshots):



MWA_CLIENT 

GitHub link: https://github.com/MWATelescope/manta-ray-client


mwa_client is a helper script which provides the following functions:

  • Submit MWA ASVO jobs in bulk
  • Monitor the status of your jobs
  • Download your completed jobs

There are two types of MWA ASVO jobs:

  • Conversion: Average, convert and download a visibility data set (and optionally apply calibration solutions).
  • Download: Package and download a raw visibility data set. (This is recommended for advanced users, as the raw visibility files are in an MWA-specific format and require conversion and calibration).
  • Voltage: Raw voltage data from VCS observations. This option is restricted to members of the mwavcs team who have a Pawsey account.

Prerequisite: You must have an account on the MWA ASVO website

  • Get your API key from your profile. Set up your API key as an environment variable in linux (usually in your profile / .bashrc).

        ~$ export MWA_ASVO_API_KEY=<api key>

Then you may install natively on your computer OR install via Docker.



Installation ( Natively)


  • Clone the repository


~$ git clone https://github.com/ICRAR/manta-ray-client.git


  • Create a virtual environment


$ python3 -m venv env


  • Activate the virtual environment


~$ source env/bin/activate

(env)~$


  • Install mwa_client and all required packages


(env)~$ cd manta-ray-client

(env)~/manta-ray-client$ pip3 install -r requirements.txt

(env)~/manta-ray-client$ python3 setup.py install

You are now ready to use manta-ray client to submit download jobs directly onto Garrawarla from any machine. 


Using manta-ray client

Users can submit multiple jobs using a CSV file. An example.csv file can be downloaded from the github repository. 

Download job options

  • obs_id: <integer>
    • Observation ID
  • job_type: d
    • Always 'd' for download jobs.
  • download_type: <vis_meta || vis>
    • vis_meta: download visibility metadata only (metafits and RFI flags).
    • vis: download raw visibility data sets and metadata (raw visibility files, metafits and RFI flags).
  • delivery: <acacia || astro>
    • acacia (default): Data will be delivered to Pawsey's Acacia system and you will receive a link to download a zip file containing the data.
    • astro: Data will be left on the /astro file system at Pawsey in /astro//asvo/<job_id>. This option is only available for Pawsey users who are in one of the mwa science groups (mwasci, mwavcs, mwaeor, mwaops). Please contact support if you would like to use this option.

Example in CSV file:

obs_id=1110105120, job_type=d, download_type=vis_meta, delivery=astro

Voltage job options

  • obs_id: <integer>
    • Observation ID
  • job_type: v
    • Always 'v' for voltage jobs.
  • offset: <integer>
    • Number of seconds from the beginning of the observation for which you would like data
  • duration: <integer>
    • Number of seconds of voltage data to be included in the job.

Example in CSV file:

obs_id=1323776840, job_type=v, offset=0, duration=1200


Once the appropriate options are set in the CSV file, you can submit the job using mwa_client


Submitting job

mwa_client -c example.csv

  • No labels