MAST logo

The MAST Data Discovery Portal

Release Notes

Contents

Introduction

The Barbara A. Mikulski Archive for Space Telescopes (MAST) Data Discovery Portal (DDP, or Portal) is a web application for searching repositories of information about astronomical data, publications and images.

Startup
Fig. 1. Anatomy of the MAST Portal as it appears prior to the first search. Note: The AstroView panel may be expanded manually prior to a first search, but will automatically expand upon a first search.

The initial screen layout consists of the following components, numbered as in Fig. 1:

1. "Select Collection" Box and other basic tools
Use the dropdown menu to choose the collection to search (results vary depending on the information contained within each particular collection). Also has a link to the "about" screen, allows a user to import a CSV or VO table they have saved on their local machine, a button that brings up the user's download basket (if populated), and a button for the user's guide.
2. Search Box
Depending on the collection chosen, a hint appears describing the kind of query that should be performed. Also contains examples that populate the search box for you.
3. Data View Panel
Presents data and images returned from the query or queries, sometimes allowing multiple views of a dataset, such as a grid layout or an album view. If filtering is available for the data panel, applicable filters are grouped by semantic type, and will appear in this panel. Note that the Filters panel can be resized or (un)hidden using screen widgets.
4. Astroview Panel
This powerful all-sky viewer provides visualization context and imagery for either aribtrary positions specified within the viewer itself or data from Portal queries.

Initiating a Search

All searches begin by choosing a collection (All MAST Observations, All Virtual Observatory Collections or Hubble Source Catalog) and entering either the name of an astronomical object or the RA/Dec coordinates of a sky position (in decimal degrees, sexagesimal or galactic coordinates) in the search entry field. A radius can be appended to the search by adding "r=" and a radius, with units of "d" (degrees), "m" (minutes) or "s" (seconds). Some collections have search radius limitations, and the default radius depends on the size of the object, or is usually .2 degrees for other targets. The search can be initiated by clicking the Search button or by pressing the Enter key on your keyboard while the cursor is in the search entry box.

Users may also choose to load all the Hubble Source Catalog spectra in the archive by selecting that option from the collections drop-down menu.

Search Entry Box
Fig. 2. Search panel includes the Collection pulldown menu, Search box and button, examples of valid search text, and basic tool links.

Names of objects are passed to a name resolver (one of many Virtual Observatory services that support the Portal), which associates known object names with sky coordinates. A wide variety of catalog and common names can be resolved to known objects. Coordinates can be given in a variety of formats; right ascensions must be positive and southern declinations require a leading negative sign. Examples of valid names and coordinate formats are given below.

Object Names
M101, NGC45 Objects from standard catalogs such as Messier and NGC will be resolved
Antennae, Eta Carinae Common names often work
T Tau Variable star names often work
BD+19 706 Star catalogs with coordinate symbols
png 000.8-07.6 Other catalogs with coordinate and decimal symbols
2MASS J04215943+1932063 All-sky catalogs with coordinate symbols
TYC 1272-470-1 All-sky satellite catalogs with restricted symbols
Coordinates
14 03 12.6 54 20 56.7 Sexagesimal coordinates delimited with spaces
14:03.210 54:20.945 Sexagesimal coordinates delimited with colons; decimal minutes/arcminutes
14h03m12.6s +54d20m56.7s Sexagesimal coordinates with explicit hms/dms
g102.0373+59.7711 Galactic coordinates of the form g[lon][+/-lat] with no spaces
180.468 -18.866 Coordinates in decimal degrees

Further rules and caveats for valid searches may be found in Tips & Notes.

Exploring Search Results

A successful search, such as that shown for M57 below, populates a new Search Results panel with a table of resources, a corresponding Filters panel applicable to these resources, and focuses the AstroView component on the position. If the software detects image outlines (footprints), it will draw them, and will attempt to indicate non-images, such as catalog objects, by marking the position listed for that row. The results of subsequent searches will be stored in new tables, and can be accessed by clicking on the tabs at the top of the Results panel. To eliminate a results table (and discard the results for that search), click the "x" icon on the tab. Concurrent searches are possible. If a new search is initiated prior to the last one completing, a new tab will be created while the previous search continues. Just below the tabs is a status panel with the following items from left to right:

  • The number of rows retrieved (and update information for searches in progress),
  • A button with the canonical name and the search radius (expandable to show the RA/Dec coordinates),
  • A button that presents the user with a shareable link to the current search results,
  • The Cross-match button,
  • The Spectral viewer button,
  • The Plotter tool button,
  • The buttons for column manipulation,
  • The Download Basket button,
  • The Export Table button which can write the table of results to your local storage. This works in conjunction with the Import Table button at the top of the page, and
  • Controls for AstroView (more on these later).
Search Results
Fig. 3. Appearance of the Portal after a search for m57, showing the filters (left panel) that can be applied to the Search Results (center panel), and the AstroView component.

The records are listed in the table, one per row. Each record represents data about a particular image, catalog, or spectra, with the columns defining each datum about the record. In the example above, each visible record is about a SWIFT image that was found within the bounds of the search.

The search results are displayed in "List View" by default. This is a condensed view, with the option to display previews and cutouts via the "Show Preview" and "Show Cutout" checkboxes, as shown in Figure 4 below.

Search Results
Fig. 4. Appearance of the Portal after a search for m57, showing the previews and cutouts.

Alternatively, the results can be viewed in "Album View," which is accessible by clicking the tab at the top of the results grid. Album View allows the user to view search results in a tiled format. In this view, the previews are more prominent on the page.

Search Results
Fig. 5. Appearance of the Portal after a search for m57, showing Album View.

Search Details Pop-Up

Clicking the info button in the Search Results panel pops up a panel with summary information about the resource. Clicking the details tab at the top of this panel exposes all of the attributes for the resource. Different details panel formats exist for the various collections. For collections where the tool knows, for example, that there will be an image and a URL that links to a large-format version, it will pop up a details panel that links to it. Other image records may be displayed differently. Sometimes the format is unavailable and the tool can only provide a generic view of the data.

Summary PopUp Details PopUp
Fig. 6. Summary of an m57 image, showing the summary description of the resource (left). Clicking the details tab at the top brings up all available details (right).

Virtual Observatory (VO) Search Results

For search results from the VO, the initial list returned is only the first half of the search. It contains a list of resources from the VO that claim to have data within the searched cone. These data are catalogs, images, and spectra. From this list, the user must click the desired object to view a summary of its details. The user may then load the full data from that object by clicking on the Load button in the Summary Panel or in the Actions Column, which will display all of its associated data records into a new tab for further browsing. This action is referred to as a "drilldown", and is a key component of using the Portal to explore multi-tiered data hierarchies such as the VO. Additionally, the user may download the table, including all metadata, to a file on their local machine, or initiate another search using that resource. In order to search the VO, change the "Select Collection" dropdown to "All Virtual Observatory Collections."

Datascope Search
Fig. 7. Summary of a m57 VO search, with the underlying catalog having been partially loaded. The details panel for the search allows the user to load the records, which places it into a new tab (marked VO: m57 as shown).

Refining Results with Filters

Many searches, particularly on popular targets or of a sufficiently large area of sky, generate hundreds or even thousands of rows. The Filters panel allows the user to restrict the list of search results to those of most interest.

Filters

The list of potential filters is quite large, and includes as many attributes of the matched data collections as the data providers offer. The filters in the panel are grouped into semantically related filters depending on the data. You can view the list of allowed filters by clicking the Edit Filters button at the top of the panel; select the desired filters by clicking the checkboxes in the pop-up window and clicking the Apply button. Note that not all filters are displayed or even available by default.

There are three types of filters:

  • Discrete filters - Used for text and most integer-based filters (columns of data). The filter is applied by clicking the checkbox next to the desired filter values.
  • Ranged sliders - Used for continuous numeric data, some integer data, and dates. Each grouping presents the user with a histogram approximating the distribution of the data, a 2 handled slider for constraining endpoints, and a min/max box displaying the current endpoints, which also allows a user to manually type the desired endpoints. Some numeric filters are known to have certain values that represent a null or undefined entry, and the filter will alert the user that it is ignoring such a value.
  • A search box, which allows regular expression matching by searching all records and columns of the data (even ones not visible in the display grid).
Filtered Search
Fig. 8. The search has been filtered for rows where the Proposal ID is "PNMRD" with RA between the given values. This updates the counts of the other filter values based on how many other rows can be selected. Regular expressions may also be used as filters by entering a regular expression in the dialog box near the top of the panel. In the numeric filter above, the data were recognized as RA coordinates and is presented in these boxes in sexagesimal format, but a user may define his or her own bounds using sexagesimal or degrees.

Retrieving Results

A core capability of the MAST Portal is to enable users to download resources, or to direct them to other desktop applications for analysis. As used here, resources is a rather generic term used to refer to individual data files (images, tables, and the like), or collections of them, or to services that provide data upon request. Any of these resources may be of use scientifically, and the Portal has simple mechanisms for retrieving them.

Images

Continuing with the example shown in Figure 6 (summary of an M57 image), clicking the Zoom button will bring up a large-scale version of the image. In some cases, the larger image may be an interactive display using the Hubble Legacy Archive's FITS viewer.

Tables

Any table displayed in the results panel may be saved with the following steps:

  1. Click the Export button.
  2. Select the output format and other parameters from the pop-up panel.
  3. Click the Export button on the exporter dialog.

Using the Download Basket

For MAST data, the rows of the search result grid are only a summary of the data products available for download. Select the rows you wish to investigate by checking the box on the far left of the row, and add the associated products to the download basket by clicking the in the grid. Alternatively, each row has action icons to add the data to the basket or to immediately download the data . Adding data to the basket pops up a window containing the current download basket, which itself is filterable. Most summaries represent many actual data products, so each product gets its own leaf in the tree. The products are organized in a tree view from Mission > Observation > File. These can then be then be further selected and downloaded using the download button in the upper-right of the grid. Some of these data, such as proprietary data, can only be accessed via the batch download interface, and may require a login to authorize the batch retrieval. When this situation occurs, the user may click the "Batch Retrieval" button . This opens a dialog that provides options for sFTP, writing the data to DVD, or to stage the data for later download.

Adding to DownloadBasket
Fig. 9. Adding the products associated with a record to the download basket. Note that the 2 rows translated into 13 data products, of which 3 are considered Recommended Products.

The AstroView Tool

The AstroView Tool is an all-sky-viewing component. It was originally developed as a standalone application, then was later integrated into the Portal. It uses the same positional search and name-resolving capabilities as the Portal, but instead of displaying tabular results and image files it presents a view of the sky using imagery from various surveys. This view can be panned, zoomed, and searched, and the imagery displayed can be changed based on the survey selected using the tool.

One interaction between the Portal and AstroView is the notion of projecting image "footprints", i.e. the detector shape, size, and orientation for a given image, onto a background sky map from all-sky survey images. Selected rows in the grid are highlighted in this tool. Clicking footprints or markers in AstroView will not only highlight them, but will select them in the grid and bubble them to the top. Many footprints overlay each other exactly, so many rows may be highlighted in the grid with a single click. Figure 10 demonstrates footprints and highlighting.

AstroView Tool
Fig. 10. Footprints from WFPC2 were selected in the AstroView component, so they were highlighted in AstroView, then selected and bubbled to the top of the grid.

The AstroView controls listed for each grid contain 3 components:

  • Display - A toggle/drop-down menu indicating which footprints are to be displayed
    • None - show no footprints
    • All - show all footprints. Unselected footprints will be the color of the swatch, and selected footprints will appear slightly highlighted.
    • Selected - show only footprints that are selected in the grid. This acts as a filter for the tool, and can help viewing a crowded area.
    • ***It is important to note that the selection from this dropdown works on a grid-by-grid basis. To hide all footprints across several tabs, each tab must be set to "None".
  • Color - a swatch indicating the color of footprints associated with this search. Each search/grid may have its own color to allow a user to distinguish between different grid interactions.
  • Palette dropdown - allows user to pick the color for this grid's footprints and points in the plotting tool.

The Plotting Tool

The Plotting Tool, activated by using the charts button in the upper right of the grid, allows the user to scatter plot two numeric columns against each other. It provides a panel to allow the user to configure the X and Y axes. The axes may also be set to "Combine Columns" (the last value in each dropdown), which allows for arithmetic between two columns. Adjust the sliders or type the endpoints into the range boxes, then click "Apply Range" to zoom the chart to the new extrema. The user may also shift-click and drag a box on the plot to specify new bounds. The "Reset Range" button will restore the default bounds for the plot. Additionally, selection may be performed in two ways: 1) clicking and dragging (while holding shift) will remove any existing selection and select all points within the box bounds and 2) clicking data points will toggle selection of points at that position without removing the existing selection.

Plotting Tool
Fig. 11. Plotting Tool - The plot has been zoomed in from the original bounds, and points selected in AstroView/grid are also highlighted in the plot using the same color as in AstroView.

The Cross-Match Tool

The Cross-Match Tool allows the user to match the positions in their current grid against other catalogs. Click the cross-match button at the upper-right of the grid to bring up the cross-match dialog. Select the resource against which to match, specify the match radius in arcseconds and click the cross-match button to begin the search. The returned results will contain all the columns from the original search, then all the columns from the resource against which it was matched. It is important to note that in order to perform a cross-match, the software must have detected an RA and Dec column.

Cross-Match Tool
Fig. 12. Cross-Match Tool - There are multiple resources against which the user may match an arbitrary table.

The Spectral Viewer Tool

The Spectral Viewer Tool allows the user to visualize and do some preliminary analysis on spectral data from MAST results. Individual spectra can be loaded by clicking the spectral viewer button on a spectrum row in the grid, or alternatively, selecting up to 10 spectra and clicking the same icon in the upper right corner of the grid to plot spectra against each other. When plotting multiple spectra, the legend allows individual series to be toggled via the checkboxes.

A demo of this tool may be found later in this document here

Spectral Viewer Tool
Fig. 13. Spectral Viewer Tool - Two spectra plotted against each other, with an indicator at a user-specified wavelength.

The Timeseries Viewer Tool

The Timeseries Viewer Tool is similar to the Spectral Viewer, but allows users to view and manipulate timeseries data such as Kepler light curves. A user may select a single timeseries row to plot using the icon. All the series are downloaded and made available in the legend, but unlike the Spectral Viewer only one series may be viewed at a time.

Timeseries Viewer Tool
Fig. 14. Timeseries Viewer Tool - One quarter of Kepler data plotted as points.

Timeseries also have an option to phase fold the data to discover periodic events such as transiting exoplanets. Phase folding may be enabled by entering a period (and optionally an epoch to apply a shift) and checking the "Enable Phase Folding" checkbox in the Phase Folding Panel on the left.

Phase Folding
Fig. 15. Phase Folding - Folding a quarter of Kepler data at the Tres-2 object with a period of 2.470614 days shows a clear repetition of flux dips.

The Cutout Viewer Tool

In addition to viewing the detailed cutouts of the FITS images via the "Show Cutout" checkbox in the List View (as shown in Figure 4), the user may also view them, in several resolutions, via the "Cutouts" radio button in the Album View.

Cutout Viewer Tool
Fig. 16. Cutouts of some images around M57.

The Image Overlay Tool

In some views, the user may click the image overlay button to overlay the given image onto the AstroView tool. This can be used for comparing results against a catalog, for example.

Overlaying an Image
Fig. 17. Overlaying a WFPC2 image into AstroView.

Creating Columns

The Create Columns Tool allows the user to perform arithmetic operations on one or two columns at a time. Operations such as addition, subtraction, multiplication and division are supported for two columns, or one column with a constant, and logarithms and exponentiation are also available for single columns.

Combine Columns Tool
Fig. 18. Combining Columns - Several arithmetic operations are supported for combining two columns or altering a single column via a constant.

Choosing Position Columns

In some tables, such as when performing a crossmatch or when adding offsets to original RA/Dec, there will be some ambiguity as to which columns the user wishes to use as the primary position columns. By using the Choose Position button the user can switch the columns used for RA/Dec.

The Hubble Source Catalog (HSC)

You can now access the Hubble Source Catalog (HSC) through the MAST Discovery Portal. The HSC is designed to optimize science from the Hubble Space Telescope by combining the tens of thousands of visit-based source lists in the Hubble Legacy Archive (HLA) into a single master catalog.

Other HSC links:

Select the Hubble Source Catalog from the "Select Collection" dropdown menu and use the search box as you would for any positional query. Figure 19 shows a chart in front of a HSC matches search, with a match ID circled in orange, the action button to load the individual measurements comprising the match circled in yellow, with a particular match circled in blue in the charting tool and in green in the AstroView tool.

Hubble Source Catalog
Fig. 19. Hubble Source Catalog - Example usage of the HSC, with chart and AstroView interactions.

Advanced Search

The Advanced Search capability in the Portal allows users to "pre-filter" results by creating a search query that is more complex than a basic positional cone search. When available for a collection, it may be accessed by clicking the "Advanced Search" link below the search textbox at the top of the page to open the search window. At the top of this window is an informational panel that includes a count of how many rows would be returned by a query given the applied filters listed in the panel immediately below it, as well as a text box to also apply a cone search. When a query would return 50,000 or fewer rows, the user may load the results into a grid as if it were a normal MAST search using the Load Results Button button. Alternatively, if a query would return 500,000 or fewer rows, the user may download the results of the query as a CSV or VO Table using the Download Results Button button. The columns in the data are listed on the left and the expanded panels on the right allow users to apply filters.

Advanced Search
Fig. 20. Advanced Search Window - In this example, the user is searching for observations of stars from HST. The query results in too many rows to load in the browser, but the user would be able to download the results as a CSV or VO Table.

Available Data

What data can you find using the MAST Data Discovery Portal?

The MAST DDP uses, as its source for searching a database, a model called the Common Archive Observation Model (CAOM), which will eventually include all the MAST holdings. Currently, not all of MAST's holdings and products are in this database.

Currently available Data:

  • HST - Updated weekly
  • Hubble Legacy Archive - HLA enhanced products released for HLA 8.1
  • GALEX - GR6/7 images, spectra, and spectral images
  • FUSE - All FUSE observations
  • Kepler - All data
  • SwiftUVOT - Update monthly
  • IUE - All low and high dispersion data
  • EUVE - All data
  • HUT - All data
  • WUPPE - All data
  • TUES - All data
  • BEFS - All data
  • K2 - All data

Data that are not yet available:

  • XMM-OM
  • Copernicus
  • UIT
  • VLA-FIRST
  • HPOL
  • IMAPS
  • EPOCH
Listed below are more details about the data that are available and some caveats.

Hubble Space Telescope - There are three sources for entries displayed in the Portal for the Hubble Space Telescope: standard HST listings/data, HST Online data and related data from the Hubble Legacy Archive.

  • Standard HST listings/products - The initial load of HST observations into the new database includes all current data. The Portal will display the highest-level exposure or association that is available for a set of data, although the exposure data are also available for download. The download manager has several useful filters to let you choose the data products you need. At present, some of the HST data will be retrieved via batch download. This method of processing is currently in beta.
  • HST Online data - The FOC, FOS, GHRS and pre-servicing mission 4 STIS data are online and will be downloaded without having to submit a batch request. In spring of 2014, all HST holdings are planned to be available in an online cache. At that time, users will be able to download the data directly. These are flagged as project "HSTONLINE" in the Portal display.
  • Hubble Legacy Archive data - The Hubble Legacy Archive Project has created enhanced data products for ACS, WFPC2, & NICMOS images, ACS & WFPC2 source lists; ACS & NICMOS extracted grism spectra, prototype ACS mosaics. As of DR 8.1. See the HLA page for a complete list of the holdings. In the Portal these observations are labeled as being in the "HLA" project.
Kepler - The Kepler data are presented in the MAST DDP as targets, not individual observations although individual lightcurves and target-pixel files are available for download. In addition, tarfiles of the lightcurves and target-pixel files have been created for each Kepler ID and cadence. The tarfile name consists of the Kepler ID followed by a string that indicates how many lightcurves are available per quarter. For example, for kplr012644769 the filename for the short cadence tarfile is kplr012644769_sc_Q001000000033333320. This indicates that there were no short cadence data taken for this target in quarters 0, 1, 3, 4, 5, 6, 7, 8, 9, and 17. Three short cadence observations were taken in quarters 10, 11, 12, 13, 14, 15, and two in quarter 16.

Tips & Notes

When formulating a search, note the following:

  • Leading zeros are ignored in the name. For example, M5 returns results for object Messier 005.
  • Object names are not case-sensitive, and spaces between the characters (e.g. M51 and M 51) are ignored unless the space indicates a real character in the name.
  • If an object name is incomplete the name resolvers from NED and SIMBAD return the closest match. This can lead to surprising results however.
  • All coordinates are interpreted as J2000.
  • Maximum search radius is limited on a collection-by-collection basis.
  • VO resources may have arbitrary limits on the number of rows returned, on a resource-by-resource basis.
  • Queries are currently limited to a single object or position. However, multiple queries may be submitted sequentially, even if one or more queries are still executing.

When downloading an image, note the following:

  • Large images may take a while to download. Download progress can only be monitored using browser tools.
  • Downloading is disabled for data products that are embargoed by the provider. It may not be obvious when this happens, except that no file will be downloaded to your computer.

Tabs of search results are not saved between sessions; once the application has been abandoned, returning to it will not bring up previously searched objects. This means that clicking the browser back button will cause the session to be lost.

FAQ/Known Issues

  • In Firefox, the scrollbars may not be displayed by default in OSX. In OSX, the default setting for showing scrollbars is set to 'Automatically based on trackpad or mouse'. If you are using a mouse, the scrollbars should be visible. However, if you are using a trackpad or other touch-enabled device it will attempt to hide them entirely. This can lead to confusion about when a grid or other widget is scrollable or not.
  • The contents of the Download Basket are not persisted across sessions, nor is any customization of the grids.
  • The search box in the filter container accepts regular expressions in JavaScript format.
  • Circular footprints (e.g., GALEX) are currently defined as polygons with a finite number of vertices. When zoomed in close in Astroview, you can sometimes see the footprint line segments, rather than a curved trace of the true circular footprint. As such, targets that are very, very close to the edge of the footprint will appear to lie "outside" the footprint when they are actually within it.
  • For Galex, background images used in AstroView are from data release 4/5, but the data available from MAST are from release 6/7. This may cause footprints to be placed in the viewer over objects that are not visible in the viewer.
  • For Galex, the FUV and NUV observations are typically downloaded as a pair, rather than separately.

Video Demos

While these videos are embedded here, it is recommended that you watch them in fullscreen mode so they can be viewed in high resolution. To view in high resolution, click the gear icon on the embedded widget and select 1080p (preferred) or 720p, then click the 4-corner box in the lower-right corner of the widget to go fullscreen.

Positional Search

Crossmatch Search

Searching Using the Virtual Observatory

Visualization with AstroView

Visual Form Search (SID interface)

Charting and Selection Interactivity

Importing and Exporting Files

The Spectral Viewer Tool

Advanced Search

Download Basket 2.0

HST Reprocessing Notifications


To help improve the MAST Portal, please send feedback to The MAST Portal. Please note: the above mailto link may cause you to leave the current page if your browser does not have a default email client setup! In this case, please send email to archive[at]stsci.edu with the subject "Portal Feedback".