A DTM (or synonymously DEM for Digital Elevation Model) is a grid, or raster, file describing elevation values at regularly spaced points, or posts.

HiRISE DTMs are made from two images of the same area on the ground, taken from different look angles. All the stereo pairs acquired so far are available here. Not all of these have been made into DTMs due to the time-intensive process. Creating a DTM is complicated and involves sophisticated software and a lot of time, both computing time and man-hours.

As mentioned in a previous post, the great advantage of a HiRISE DTM is the high resolution of the source imagery. As a general guide, terrain can be derived at a post spacing about 4X the pixel scale of the input imagery. HiRISE images are usually 0.25 – 0.5 m/pixel, so the post spacing is 1-2 m with vertical precision in the tens of centimeters.

The three basic stages of creating a DTM are:

1. Prepare the images for ingestion into the stereo software
2. Triangulate the images
3. Extract terrain

In order to prepare the images, we must first correct the geometry by removing any optical distortions inherent to HiRISE. Then the spacecraft pointing information at the time of each observation is gathered.

Triangulation is also called bundle adjustment. This step requires the most operator skill and time. The result is a transformation of the original images to epipolar space. What this means is that all the stereo information is now captured in the horizontal direction, or x-parallax. During triangulation, we also align the stereo model to MOLA elevations, so the end result is tied to the global elevation map produced by the MOLA instrument team. This is the same map that you see in the context map pane of every HiRISE observation page.

Once the images are triangulated, then terrain can be extracted. This step is computationally intensive, but automated, so it just takes a lot of computer time. The output of terrain extraction is reviewed for any artifacts or errors. These are edited out if possible. Since editing is extremely time-consuming, it is only done on easily corrected errors and in the areas of most interest to the researcher. The less editing we have to do, the better, so a lot of effort goes into preparing the images so that the input is as high quality as possible. The excellent contrast and value range of HiRISE imagery usually result in high quality terrain extraction that requires minimal editing.

After we have terrain, we can make other products, such as orthoimages. An orthoimage is a picture that has been orthorectified. This means that the pixels have been projected so that at each pixel it is as if you are looking directly down at the terrain. In the original stereo images, we rely on the fact that there are topographical distortions (parallax) to derive the elevations in the terrain model. In the orthoimages, all topographic distortions have been removed.

The final products are map projected using the same mapping definitions as the regular HiRISE RDR products.

A really useful (and cool) thing to do with the orthoimages is to drape them over the terrain for 3D viewing. Below is a subimage from the Newton Gullies DTM showing the imagery draped over the terrain.

You can see animated fly-throughs made with HiRISE DTMs by going to the HiClips page and clicking on the JPL Flythrough Clips. This is a great way to see and understand the geological relationships from a ground perspective.

Researchers use DTMs to take measurements and model geological processes. DTMs are very powerful research tools. In fact, almost every HiRISE DTM produced results in publication. There is a long waiting list for these products because they are so valuable and so difficult to produce. Several institutions involved with HiRISE contribute to DTM production to maximize the number of projects produced and to avoid duplication of effort.

Standard PDS products linked to the DTM project page are usually quite large files. The links provided will download the files to your system. To get a quick view of what the project looks like, click on the Extras links to see a reduced version of the products, displayed as images, grayscale, shaded relief and colorized altimetry.

Standard PDS products:

• The DTM in standard PDS image object (.IMG) format with an embedded label
• The left orthoimage at the same resolution as the DTM, in JPEG2000 format with detached label
• The left orthoimage at the resolution of the original image, in JPEG2000 format with detached label
• The right orthoimage at the same resolution as the DTM, in JPEG2000 format with detached label
• The right orthoimage at the resolution of the original image, in JPEG2000 format with detached label

Extras available in the PDS Extras directory (letters in parentheses correspond to PDS file names such as <Product_ID>.br.jpg):

• Browse (br), annotated browse (ab), and thumbnail (th) jpegs of the DTM as a grayscale image
• Browse (sb), annotated browse (sa), and thumbnail (st) jpegs of the DTM as a shaded relief image
• Browse (cb), annotated browse (ca), and thumbnail (ct) jpegs of the DTM as colorized altimetry
• Browse (br), annotated browse (ab), and thumbnail (th) jpegs of the lower resolution orthoimages

PDS product naming convention for HiRISE DTMs:

PRODUCT_ID = aabcd_xxxxxx_xxxx_yyyyyy_yyyy_Vnn
where
aa = DT, indicating it’s a DTM product
b = type of data

• E = areoid elevations
• 1 = orthoimage pixels from first image
• 2 = orthoimage pixels from second image

c = projection (others are possible but these are the important ones)

• E = Equirectangular
• P = Polar Stereographic

d = grid spacing (think of this as pixel scale in meters)

• A = 0.25 m
• B = 0.5 m
• C = 1.0 m
• D=2.0 m

xxxxxx_xxxx = orbit number and latitude bin from SOURCE_PRODUCT_ID[1]
yyyyyy_yyyy = orbit number and latitude bin from SOURCE_PRODUCT_ID[2]
V = letter indicating producing institution

• U = USGS
• A = University of Arizona
• C = CalTech
• N = NASA Ames
• J = JPL
• O = Ohio State
• Z = other

nn= 2 digit version number

Below is an example of the set of annotated browse images for the Russell Crater Dunes DTM.

The grayscale image of the DTM looks weird, if you have not looked at lots of these before, but keep in mind that the color of the pixels represents elevation. The higher the elevation, the brighter the pixel. Lower elevations are darker. The shaded relief is another way of visualizing the topography. The pixels are illuminated from a certain direction, to show the relief of the topography, rather than the elevation. It is also emphasizes any artifacts in the DTM. In the example here, many artifacts (errors) can be seen such as the faceted areas and boxes in the lower left and top of the image. These artifacts are usually caused by areas of low contrast (such as in this project) or sharply differing shadows. Most HiRISE DTMs will not have a lot of these artifacts, fortunately! The area of most interest to the researcher who requested this DTM was the long slope with the gullies, which was well-illuminated and had good contrast. So in that area, there were few, if any, artifacts. Adding color-coded elevation to the shaded relief creates the colorized altimetry map, where the lowest elevations are purple, green is the median elevation value, and white is the highest elevation. In the Russell Crater Dunes project shown here, the difference in elevation from the highest to the lowest point is almost 590 meters (~1935 ft.). That is a tall dune!!

We are happy to be able to share HiRISE DTMs with the scientific community and with the public. We will continue to release more DTMs as they become available, so stay posted!

You might remember that we were planning on releasing HiRISE images to the public on a monthly basis. That plan was delayed by issues with our processing software, hardware and other events. A productive summer working on these issues culminated last week with one of our larger releases of Mars images!  Here are some statistics about our September 2009 release, which includes the images the HiRISE camera took of the Martian surface between Mars Reconnaissance Orbiter (MRO) orbits 12,600 to 14,199, or roughly April 4 through August 6, 2009:

• 2,996 RDRs, 1 TB
• 42,370 EDRs, 1 TB
• 34,481 RDR Extras, 1.6 TB
• 83,784 EDR Extras, 0.02 TB
• 636 Anaglyphs, 0.01 TB

Totals for this release: 163,631 image products, 3.6 TB

This brings our total released product numbers and data volume to:

• 22,676 RDRs, 12 TB
• 317,120 EDRs, 10.4 TB
• 192,270 RDR Extras, 15.3 TB
• 612,769 EDR Extras, 0.1 TB
• 2,892 Anaglyphs, 0.5 TB

Total: 1,148,363 images, 37.5 TB

In summary, we released nearly 1500 observations, most of those with both black & white and color RDR products. Several newer observations matched up with older observations from a slightly different angle of the same location on the surface, resulting in 636 awesome new anaglyphs. The RDRs are the fully processed, geometrically projected products best for scientific inquiry. If you really want to, though, anyone can download and process HiRISE data from scratch. You can do this using ISIS software, which is publicly available for free download. See the ISIS Web site for download information, processing instructions, and tutorials.

Starting this week, I will be looking over the observations taken August 6 through August 26 before MRO went into safe mode and make sure they are ready for release. We plan to release these images in early October. We are also in the process of reprocessing those Extended Science Phase mission images prior to all the latest processing pipeline fixes and updates.  Once we are satisfied with that data set, we will release them to the public and then start reprocessing the images from the Primary Science Phase…a major project that should keep me and the rest of Downlink busy for several months!

Planetary scientists used to keep new data from the spacecraft explorers of the solar system within the mission team for a lengthy period of time so they could make all the cool initial discoveries. Only later would the mission’s data sets be archived on the public Planetary Data System (PDS). Once archived, these data could then be used by the scientific community and public for further research and discovery.

Dr. Alfred McEwen, HiRISE principal investigator, decided early on that this incredibly powerful instrument should be “The People’s Camera”. This meant, among other things, that we would endeavor to make the data returned by HiRISE available to the scientific community and public as quickly as possible. We have PDS release requirements, but our goal has always been to beat those requirements. To do so, we needed to develop automated software pipelines to take the raw data and turn them into useful calibrated and geometrically mapped products. We also needed to develop the right PDS release tools, train a talented group of operations staff to validate the data and fix problems, and develop a website to effectively and beautifully showcase HiRISE images.

We now believe we have reached the point to be able to support a monthly release of recent HiRISE images to the public! This week we released the observations HiRISE took of Mars between orbits 11,600 and 12,599, or between January 16 and April 04, 2009. This makes us the first mission to release a data set to the PDS so quickly! Here are the statistics for this release, including the number of each product type released and their respective data volumes (EDRs are the individual uncalibrated image channels and RDRs are the calibrated, mosaicked, and geometrically-projected observations):

• 1,179 RDRs, 520 GB
• 16,861 EDRs, 459 GB
• 13,512 RDR Extras, 788 GB
• 33,152 EDR Extras, 7 GB
• 342 Anaglyphs, 51 GB

Totals for this release: 64,704 image products, 1.7 TB

This brings our total released product numbers and data volume to:

• 19,667 RDRs, 11 TB
• 278,807 EDRs, 9.5 TB
• 166,816 RDR Extras, 13.7 TB
• 529,095 EDR Extras, 0.1 TB
• 2,892 Anaglyphs, 0.5 TB

Total: 993,277 images, 34 TB

Those are various products for about 9998 Mars observations, and another reason why it makes no sense to hoard our data; there is too much of it and too few of us! The team scientists have plenty to do and there are plenty of discoveries to be made, old hypotheses to update, and new mysteries to solve.  The operations staff are now hard at work getting observations from orbits 12,600 through 12,999, or between April 04 and May 5, 2009, ready for the June PDS release. This involves making sure each observation has been processed by our software pipelines correctly, fixing any problems, and checking and double checking that the relevant image products are ready for release.  Sometimes we have to manually force an observation through the pipelines because some of its channels were lost during transmission to the Earth, or we might stumble across an observation we somehow forgot to send on to the color pipelines after it had been calibrated. There are spreadsheets to maintain, lists of problematic observations to keep (see the ERRATA.TXT file), and a variety of other tasks that need to be completed before the latest data set is ready for release.

Over the next few months we will see how this goes! It is a lot of work, but our desire for you to see these beautiful images of Mars as quickly as possible is strong. No promises, but we will also explore releasing completed observations even faster!

Here are a few excerpts from yesterday’s University of Arizona story about our PDS release:

The HiRISE team has so far released a total 26.9 terabytes of data…. That amounts to more data than has been released by all previous deep space missions combined.

“If I showed each HiRISE image for 10 seconds, it would take me about 4 years to show them all,” said UA’s Alfred McEwen, HiRISE principal investigator.

…

Spacecraft motion pushes this electronic array so that it records the view down to Mars’ surface at a ground speed of about 3.2 kilometers per second, or about 7,000 miles per hour.

Skeptics doubted that a technique called “time integration delay,” needed to compensate for extremely short exposure times – about one ten-thousandth of a second per pixel – could produce sharp, unsmeared images.

But the technique has worked “wonderfully well,” thanks to accurate spacecraft pointing and stability and precise exposure time calculations, McEwen said.

Click here for full story.

We’ve just released 1008 new HiRISE images to the PDS! (See main page, or click here for the catalog.) This release covers orbits 8200 – 9299 of the primary mission, or in other words, the end of April through the end of July. That means we’re releasing data that’s only about 6 weeks old! This is awesome – I’m so impressed with the downlink team! The amount of work required to process these images is astounding, let alone prepare and post everything for an official release.

Here are a few examples of cool images, which were previously unreleased:

• PSP_008248_2640, Polygons and spots on defrosting dunes (right)
• PSP_008269_1395, crazy weird stuff in Hellas Planitia (be sure to look at the whole browse image on this one!)
• PSP_008322_1865, Multiple generations of slope streaks on a crater in Arabia Terra (left)
• PSP_008343_1430, Gullies on mesas in Gorgonum Chaos

I’ve only looked through the first few pages in the release. I know there are a lot more amazing images in there, so if you’re browsing through the images, post some of your favorites below!

GeoTIFF is an industry standard for embedding geographic information in images. Beginning soon, HiRISE RDRs will include GeoTIFF info in the Jpeg-2000 files. All of the information about the image will continue to be in the RDR label (.LBL plain text file), but with this additional info in the JP2, image viewing software that supports GeoTIFF will be able to take advantage of it.

For example, such software could display the actual coordinates on Mars of the pixels you are looking at, allow you to measure features directly in physical units, or stitch together images based on their absolute location on the planet. A number of GIS (Geographical Information Systems) applications use GeoTIFF; many on our science team have been waiting patiently for this feature to be rolled out.

We have already begun to produce RDRs with GeoTIFF, and they will start appearing in our weekly releases. At some point, a major reprocessing effort will be underway to bring this feature (and others) to all of our pre-existing products.

This brings up the topic of versioning: namely, how to tell which version of a HiRISE product you are working with.

(more…)

The latest, and most massive, release of HiRISE image data to the Planetary Data System includes such gems as the previously mentioned “Caught in Action: Avalanches on North Polar Scarps (PSP_007338_2640)” and “The Earth & Moon as Seen from Mars (PSP_005558_9040 and PSP_005558_9045)“.

How much data was released? 2422 observations, making up 9.9 terabytes “in over 225,599 standard PDS and extras products” according to our database specialist. This was for data between orbit ranges 4400 and 6999, or between July 05, 2007 and January 23, 2008 (which is a lot of loops around the Red Planet!)

We have now released a total of 16.8 TB worth of data, or nearly 500,000 image products. Please check out the latest images on the HiRISE website on the “March 2008: New HiRISE Images Released to the Planetary Data System” page.

These data have been processed, and reprocessed when necessary, with the latest automated pipelines on our production processing cluster. We continue to make changes to the software, however, and will have to reprocess all of these data yet again in a few months. What you see today is gorgeous and as complete as currently possible, but we always want to tweak our calibration, color, and geometry pipelines to make these even better.

This release places us very far ahead of the MRO project’s expectations for the HiRISE team. We are now working on speeding up our releases even more, so that they occur more often. That means we will probably never have such a large release again, which, as far as us downlink folks are concerned is a very good thing. Making sure 9.9 terabytes of data is ready to release is hard work. The images and new findings make it worth it, though!

• PDS Imaging Node: http://pds-imaging.jpl.nasa.gov/
• MRO PDS page: http://pds-imaging.jpl.nasa.gov/Missions/MRO_mission.html
• HiRISE Volume: http://hirise-pds.lpl.arizona.edu/PDS/

Starting with the 10/10 release, color images are included for the first time. We’ll describe how we process these in the days and weeks to come. But what I’d like to do first is give a brief description of all our product types as they currently are available. You’ve no doubt noticed a mind-boggling array of new options on our product pages. They now include what we call our “NOMAP” products; NOMAP means that they are not map-projected. In other words, not rotated to the direction of north, not mapped to a coordinate system, and not scaled to any particular geometric resolution.

I’ve prepared this ugly table that outlines each of the products now available (excluding the raw EDRs). So reading the columns from left to right: there are three types of “NOMAP” products, two types of lossy “QLOOK” (Quicklook) RDRs, and two types of lossless RDRs.

With that as a reference, now I’ll try to define everything more precisely.

“NOMAP”

Non map-projected product. Always lossy compressed for smaller size and quicker viewing. These are not formal Planetary Data System products; they’re “special”, meaning there is no PDS label and no Software Interface Specification describing them. Available for IRB, RGB and RED.

RDR

Reduced Data Record: reduced in the sense of refined or processed, not raw data. Formal PDS products with accompanying labels and a detailed SIS document describing their format and processing steps. Available both in lossless and quicklook formats for both RED & COLOR.

“QLOOK”

Quicklook: a special product that is a lossy compressed version of the RDR. In a normal RDR, all of the original data is retained. But with a quicklook, some of the highest resolution detail is discarded to make for quicker viewing.

RED

The image obtained by the red-filtered CCDs. It will be over the full swath width, typically data from all ten red CCDs. Covers the visible wavelength band from 550 to 850 nanometers.

IR

Infrared. Covers the near-IR wavelengths from 800-1000 nanometers.

BG

Blue-Green, visible wavelengths from 400-600 nm.

COLOR

A color RDR. It contains data from the IR, BG and center RED ccds. Typically this will be a skinny strip (”center swath”) inside a skinny strip, or as I like to say, the bacon-strip effect.

IRB

An enhanced color NOMAP. It has the same color bands as the RDR: IR, RED and BG.

RGB

An enhanced color NOMAP. It contains only data from the RED and BG. The blue is derived from the difference between the RED and BG. The color bands are RED, BG and the synthetic blue.

EDR

Experiment Data Record, a formal PDS product that is raw uncompressed data with a label header.

Note: we will be working towards making all of these products available for all prior releases.

Spacecraft missions are complicated endeavors that result in a wealth of scientific and engineering data. Long after the mission has ended, these data can be extremely useful for later study and discovery. With so many missions over so many years, how can later generations find and make use of these data?

The solution for many NASA missions has been the development of the centralized Planetary Data System (PDS). The PDS is several things: a collection of websites, a search capability, an archive, a database, a learning tool, etc. The PDS Imaging Node is located at http://pds-imaging.jpl.nasa.gov/ and acts as “the curator of NASA’s primary digital image collections from past, present and future planetary missions.” These missions include Voyager, Galileo, Cassini, and many more. Now the Mars Reconnaissance Orbiter (MRO) has been added to the list, with the HiRISE team releasing our first several months of image data.

• MRO PDS page: http://pds-imaging.jpl.nasa.gov/Missions/MRO_mission.html
• MRO Product Search page: http://pds-imaging.jpl.nasa.gov/search/index.jsp
• HiRISE Volume: http://hirise-pds.lpl.arizona.edu/PDS/

What we have released is an archive of the HiRISE Experiment Data Records (EDRs) and Reduced Data Records (RDRs). EDRs are in the *.IMG file format and represent individual CCD channels (remember, there are 14 CCDs in the HiRISE camera and two channels per CCD, for a total of 28 channels). These EDRs are cleaned up, calibrated, stitched together, and mapped to Mars’ geometry, resulting in the RDR products. RDRs are in the *.JP2 and *.LBL formats. JPEG2000 is the technology that enables us to offer our gigantic images to the scientific community and the public in a timely and efficient manner. An observation’s image data are in the *.JP2 file and its meta data are in the detached *.LBL files. To view these products, JPEG2000 compatible software is required (see our site for a list of offerings).

While we have been trying to release up to five captioned images a week for the past few months, the PDS release represents several hundred images, most of them without captions. You can find them using the PDS search capabilities, and you can also find them on the new HiRISE site, unveiled today to coincide with this first PDS release. The redesigned site focuses on the images while providing, hopefully, a more user-friendly interface:

• HiRISE Site: http://hirise.lpl.arizona.edu/
• “About Our Redesign”: http://hirise.lpl.arizona.edu/profil.php
• Images released to the PDS: http://hirise.lpl.arizona.edu/pds_release.php

As word gets out about the new site and the PDS release, you may experience some site slowness. Please be patient, and thank you for your interest!

While most of our uplink and downlink procedures have been developed and tested fairly heavily over the last year and a half, we still have parts of our ground data system that are in heavy development. The distribution of our imaging products to the Planetary Data System’s Image Atlas, our scientific colleagues, and you, the public at large is still very much a work in progress. Through the month of October I have been developing the next part of our PDS data node culminating in a test where I practiced the release of 2 days worth of imaging products from our Post Mars Orbital Insertion imaging campaign back in March. For the purposes of this test I released 196 raw products and our first sample JPEG2000 product.

(more…)