Basic Reference

Basic Reference#

0. Setup#

Please make sure your environment is set up according to the instructions here.

1. Overview#

NASA services can be queried from Python in multiple ways.

  • Generic Virtual Observatory (VO) queries.

  • Astroquery interfaces:

    • Call sequences not quite as consistent, but follow similar patterns.

  • Ad hoc archive-specific interfaces

2. VO Services#

This workshop will introduce 4 types of VO queries:

  • VO Registry - Discover what services are available worldwide

  • Simple Cone Search - Search for catalog object within a specified cone region

  • Simple Image Access - Search for image products within a spatial region

  • Simple Spectral Access - Search for spectral products within a spatial region

  • Table Access - SQL-like queries to databases

2.0 Import Necessary Packages#

# Generic VO access routines
import pyvo as vo

# For specifying coordinates and angles
from astropy.coordinates import SkyCoord
from astropy.coordinates import Angle
from astropy import units as u

# For downloading files
from astropy.utils.data import download_file

# Ignore unimportant warnings
import warnings
warnings.filterwarnings('ignore', '.*Unknown element mirrorURL.*', vo.utils.xml.elements.UnknownElementWarning)

2.1 Look Up Services in VO Registry#

Simple example: Find Simple Cone Search (conesearch) services related to SWIFT.

services = vo.regsearch(servicetype='conesearch', keywords=['swift'])
services

<DALResultsTable length=224>
                 ivoid                  ...
                                        ...
                 object                 ...
--------------------------------------- ...
ivo://archive.stsci.edu/catalogs/mastcs ...
    ivo://bsdc.icranet.org/sds82/q/cone ...
               ivo://cds.vizier/b/swift ...
                ivo://cds.vizier/ii/339 ...
                 ivo://cds.vizier/ix/43 ...
                 ivo://cds.vizier/ix/51 ...
                 ivo://cds.vizier/ix/58 ...
          ivo://cds.vizier/j/a+a/427/87 ...
          ivo://cds.vizier/j/a+a/448/l9 ...
                                    ... ...
          ivo://nasa.heasarc/swiftuvlog ...
          ivo://nasa.heasarc/swiftxrlog ...
           ivo://nasa.heasarc/swsdssqso ...
           ivo://nasa.heasarc/swuvotssc ...
          ivo://nasa.heasarc/swuvotssob ...
            ivo://nasa.heasarc/swxcscat ...
           ivo://nasa.heasarc/swxrt1fgl ...
           ivo://nasa.heasarc/uvotbscat ...
           ivo://nasa.heasarc/xmmvaragn ...

2.1.1 Use different arguments/values to modify the simple example#

Argument

Description

Examples

servicetype

Type of service

conesearch or scs for Simple Cone Search
sia for Simple Image Access
, ssa for Simple Spectral Access
table or tap for Table Access Protocol

keyword

List of one or more keyword(s) to match service’s metadata. Both ORs and ANDs may be specified.

  • (OR) A list of keywords match a service if any of the keywords match the service.
  • (AND) If a keyword contains multiple space-delimited words, all the words must match the metadata.

['galex', 'swift'] matches ‘galex’ or ‘swift’
['hst survey'] matches services mentioning both ‘hst’ and ‘survey’

waveband

Resulting services have data in the specified waveband(s)

‘radio’, ‘millimeter’, ‘infrared’, ‘optical’, ‘uv’, ‘euv’, ‘x-ray’ ‘gamma-ray’

2.1.2 Inspect the results#

Using pyvo#

Although not lists, pyvo results can be iterated over to see each individual result. The results are specialized based on the type of query, providing access to the important properties of the results. Some useful accessors with registry results are:

  • short_name - A short name

  • res_title - A more descriptive title

  • res_description - A more verbose description

  • reference_url - A link for more information

  • ivoid - A unique identifier for the service. Gives some indication of what organization is serving the data.

# Print the number of results and the 1st 4 short names and titles.
print(f'Number of results: {len(services)}\n')
for s in list(services)[:4]:  # (Treat services as list to get the subset of rows)
    print(f'{s.short_name} - {s.res_title}')

Number of results: 224

MAST CS - MAST ConeSearch
sds82 cone - SDS82 catalog
B/swift - Swift Master Catalog
II/339 - Swift/UVOT Serendipitous Source Catalog
Filtering results#

Of the services we found, which one(s) have ‘stsci.edu’ in their unique identifier?

stsci_services = [s for s in services if 'stsci.edu' in s.ivoid]
for s in stsci_services:
    print (f'(STScI): {s.short_name} - {s.res_title}')

(STScI): MAST CS - MAST ConeSearch
Using astropy#

With the to_table() method, pyvo results can also be converted to Astropy Table objects which offer a variety of addional features. See astropy.table for more on working with Astropy Tables.

# Convert to an Astropy Table
services_table = services.to_table()

# Print the column names and display 1st 3 rows with a subset of columns
print(f'\nColumn Names:\n{services_table.colnames}\n')
services_table['short_name', 'res_title', 'res_description'][:3]

Column Names:
['ivoid', 'res_type', 'short_name', 'res_title', 'content_level', 'res_description', 'reference_url', 'creator_seq', 'created', 'updated', 'rights', 'content_type', 'source_format', 'source_value', 'region_of_regard', 'waveband', 'access_urls', 'standard_ids', 'intf_types', 'intf_roles', 'cap_descriptions']
Table length=3
short_nameres_titleres_description
objectobjectobject
MAST CSMAST ConeSearchAll MAST catalog holdings are available via a ConeSearch endpoint. \nThis service provides access to all, with an optional non-standard parameter for an individual catalog to query. \nThe available missions are listed at http://archive.stsci.edu/vo/mast_services.html, \nand include Hubble (HST) data, Kepler, K2, IUE, HUT, EUVE, FUSE, UIT, WUPPE, BEFS, TUES, IMAPS, High Level Science Products (HLSP), Copernicus, HPOL, VLA First, XMM-OM, and SWIFT.
sds82 coneSDS82 catalogThe SDS82 catalog is the result of the Swift DeepSky pipeline over\nall observations made by Swift-XRT within the Stripe82.
B/swiftSwift Master CatalogThis table records high-level information for each Swift observation and provides access to the data archive. Each record is associated with a single observation that contains data from all instruments on board Swift. The BAT is the large field of view instrument and operates in the 10-300 keV energy band. The narrow field instruments, XRT and UVOT, operate in the X-ray and UV/optical regime, respectively. An observation is defined as a collection of snapshots, where a snapshot is defined as the time spent observing the same position continuously. Because of observing constraints, the length of a snapshot can be shorter than a single orbit and it can be interrupted because the satellite will point in a different direction of the sky or because the time allocated to that observation ends. The typical Swift observing strategy for a Gamma Ray Burst (GRB) and/or afterglow, consists of a serious of observations aimed at following the GRB and its afterglow evolution. This strategy is achieved with two different type of observations named Automatic Targets and Pre-Planned Targets. The Automatic Target is initiated on board soon after an event is triggered by the BAT. The Figure of Merit (FOM) algorithm, part of the observatory's autonomy, decides if it is worth requesting a slew maneuver to point the narrow field instruments (NFI) on Swift, XRT and UVOT, in the direction of the trigger. If the conditions to slew to the new position are satisfied, the Automatic Target observation takes place; all the instruments have a pre-set standard configuration of operating modes and filters and about 20000 seconds on source will be collected. The Pre-Planned Target observations instead are initiated from the ground once the trigger is known. These observations are planned on ground and uploaded onto the spacecraft.

3. Astroquery#

Many archives have Astroquery or Astroquery-compliant modules for data access, including:

For more, see https://astroquery.readthedocs.io/en/latest/

3.1 NED#

Example: Get an Astropy Table containing the objects from paper 2018ApJ…858…62K. For more on the API, see astroquery

from astroquery.ipac.ned import Ned
objects_in_paper = Ned.query_refcode('2018ApJ...858...62K')
objects_in_paper
Table length=41
No.Object NameRADECTypeVelocityRedshiftRedshift FlagMagnitude and FilterSeparationReferencesNotesPhotometry PointsPositionsRedshift PointsDiameter PointsAssociations
degreesdegreeskm / sarcmin
int32str30float64float64objectfloat64float64objectobjectfloat64int32int32int32int32int32int32int32
1WISEA J115844.06+273505.3179.683627.58481G106.00.000354SLS17.2g--2902214700
2KUG 1207+367182.4849736.4337G341.00.001137SUN15.5--4002818740
3NGC 4244184.3727237.80735G244.00.000815SUN10.88--4721594533171
4SDSS J121731.98+323157.7184.3832932.53271G447.00.001492SLS17.95--60156240
5SDSS J121811.04+465501.2184.5461446.91685G402.00.001341SLS17.75--22037191140
6VCC 0530185.5317115.799G1299.00.004333SUN15.8--68120241433
7PGC1 0040692 NED033186.3714628.48244G483.00.001611SUN16.47--410715800
8UGC 07512186.42212.159G1505.00.00502SUN15.00--64121171240
9NGC 4395186.4535833.54686G319.00.001064SUN10.64--8873121011195170
...................................................
33LSBC D575-05193.92119.20908G419.00.001396S1L16.52--451217800
34MESSIER 064194.1819621.68299G409.00.001364S1L9.36--838242989239100
35UGC 08061194.1844111.93224G569.00.001898UUN15.5B--48215201335
36KK 176194.98527-19.4129G825.00.002753SUN17.5--3201913920
37UGCA 319195.56029-17.23779G755.00.002518UUN15.33--6924626930
38IC 4107195.6744621.99739G267.00.000892SLS17.4g--2102012540
39UGCA 320195.81975-17.42303G742.00.002475UUN13.78--120764271360
40KDG 218196.43278-7.75884G----20.35--12017000
41NGC 5068199.72819-21.03881G670.00.002235UUN10.52--31512724617180