NASA Takes Over Navy Instrument On ISS

A version of this article appears in the May 19 edition of Aviation Week & Space Technology, p. 59, Frank Morring, Jr.

HREP on JEMEFA hyperspectral imager on the International Space Station (ISS) that was developed by the U.S. Navy as an experiment in littoral-warfare support is finding new life as an academic tool under NASA management, and already has drawn some seed money as a pathfinder for commercial Earth observation.

Facing Earth in open space on the Japanese Experiment Module’s porchlike Exposed Facility, the Hyperspectral Imager for Coastal Oceans (HICO) continues to return at least one image a day of near-shore waters with unprecedented spectral and spatial resolution.

HICO was built to provide a low-cost means to study the utility of hyperspectral imaging from orbit in meeting the Navy’s operational needs close to shore. Growing out of its experiences in the Persian Gulf and other shallow-water operations, the Office of Naval Research wanted to evaluate the utility of space-based hyperspectral imagery to characterize littoral waters and conduct bathymetry to track changes over time that could impact operations.

The Naval Research Laboratory (NRL) developed HICO, which was based on airborne hyperspectral imagery technology and off-the-shelf hardware to hold down costs. HICO was launched Sept. 10, 2009, on a Japanese H-2 transfer vehicle as part of the HICO and RAIDS (Remote Atmospheric and Ionospheric Detection System) Experimental Payloads; it returned its first image two weeks later.

In three years of Navy-funded operations, HICO “exceeded all its goals,” says Mary Kappus, coastal and ocean remote sensing branch head at NRL.

“In the past it was blue ocean stuff, and things have moved more toward interest in the coastal ocean,” she says. “It is a much more difficult environment. In the open ocean, multi-spectral was at least adequate.”

NASA, the U.S. partner on the ISS, took over HICO in January 2013 after Navy funding expired. The Navy also released almost all of the HICO data collected during its three years running the instrument. It has been posted for open access on the HICO website managed by Oregon State University.

While the Navy program was open to most researchers, the principal-investigator approach and the service’s multistep approval process made it laborious to gain access on the HICO instrument.

“[NASA] wanted it opened up, and we had to get permission from the Navy to put the historical data on there,” says Kappus. “So anything we collect now goes on there, and then we ask the Navy for permission to put old data on there. They reviewed [this] and approved releasing most of it.”

Under the new regime NRL still operates the HICO sensor, but through the NASA ISS payload office at Marshall Space Flight Center. This more-direct approach has given users access to more data and, depending on the target’s position relative to the station orbit, a chance to collect two images per day instead of one. Kappus explains that the data buffer on HICO is relatively small, so coordination with the downlink via the Payload Operations Center at Marshall is essential to collecting data before the buffer fills up.

Task orders are worked through the same channels. Presenting an update to HICO users in Silver Spring, Md., on May 7, Kappus said 171 of 332 total “scenes” targeted between Nov. 11, 2013, and March 12 were requested by researchers backed by the NRL and NASA; international researchers comprised the balance.

Data from HICO is posted on NASA’s Ocean Color website, where usage also is tracked. After the U.S., “China is the biggest user” of the website data, Kappus says, followed by Germany, Japan and Russia. The types of data sought, such as seasonal bathymetry that shows changes in the bottom of shallow waters, has remained the same through the transition from Navy to NASA.

“The same kinds of things are relevant for everybody; what is changing in the water,” she says.

HICO offers unprecedented detail from its perch on the ISS, providing 90-meter (295-ft.) resolution across wavelengths of 380-960 nanometers sampled at 5.7 nanometers. Sorting that rich dataset requires sophisticated software, typically custom-made and out of the reach of many users.

To expand the user set for HICO and future Earth-observing sensors on the space station, the Center for the Advancement of Science in Space, the non-profit set up by NASA to promote the commercial use of U.S. National Laboratory facilities on the ISS, awarded a $150,000 grant to HySpeed Computing, a Miami-based startup, and [Exelis] to demonstrate an online imaging processing system that can rapidly integrate new algorithms.

James Goodman, president/CEO of HySpeed, says the idea is to build a commercial way for users to process HICO data for their own needs at the same place online that they get it.

“Ideally a copy of this will [be] on the Oregon State server where the data resides,” Goodman says. “As a HICO user you would come in and say ‘I want to use this data, and I want to run this process.’ So you don’t need your own customized remote-sensing software. It expands it well beyond the research crowd that has invested in high-end remote-sensing software. It can be any-level user who has a web browser.”

Advertisements

Application Tips for ENVI 5.x – An IDL application for opening HDF5 formatted HICO scenes

This is part of a series on tips for getting the most out of your geospatial applications. Check back regularly or follow HySpeed Computing to see the latest examples and demonstrations.

Objective: Open a HICO dataset stored in HDF5 format using an IDL application prepared by the U.S. Naval Research Laboratory.

This is a supplement to an earlier post that similarly describes how to open HDF5 formatted HICO files using either the H5_Browser or new HDF5 Reader in ENVI.

HICO Montgomery Reef, Australia

Scenario: This tip demonstrates how to implement IDL code for opening a HDF5 HICO scene from Montgomery Reef, Australia into ENVI format. Subsequent steps are included for preparing the resulting data for further analysis.

The HICO dataset used in this example (H2012095004112.L1B_ISS) was downloaded from the NASA GSFC archive, which can be reached either through the HICO website at Oregon State University or the NASA GSFC Ocean Color website. Note that you can also apply to become a registered HICO Data User through the OSU website, and thereby obtain access to datasets already in ENVI format.

The IDL code used in this example is available from the NASA GSFC Ocean Color website under Documents > Software/Tools > IDL Library > hico. The three IDL files you need are: byte_ordering.pro, nrl_hico_h5_to_flat.pro and write_nrl_header.pro.

The same IDL code is also included here for your convenience:  nrl_hico_h5_to_flat,  byte_ordering  and  write_nrl_header (re-distributed here with permission; disclaimers included in the code). However, to use these files (which were renamed so they could be attached to the post), you will first need to change the file extensions from *.txt to *.pro.

Running this code requires only minor familiarity working with IDL and the IDL Workbench.

The Tip: Below are steps to open the HICO L1B radiance and navigation datasets in ENVI using the IDL code prepared by the Naval Research Laboratory:

  • Start by unpacking the compressed folder (e.g., H2012095004112.L1B_ISS.bz2). If other software isn’t readily available, a good option is to download 7-zip for free from http://www.7-zip.org/.
  • Rename the resulting HDF5 file with a *.h5 extension (e.g., H2012095004112.L1B_ISS.h5). This allows the HDF5 tools in the IDL application to recognize the appropriate format.
  • If you downloaded the IDL files from this post, rename them from *.txt to *.pro (e.g., nrl_hico_h5_to_flat.txt to nrl_hico_h5_to_flat.pro); otherwise, if you downloaded them from the NASA website they already have the correct naming convention.
  • Open the IDL files in the IDL Workbench. To do so, simply double-click the files in your file manager and the files should automatically open in IDL if it is installed on your machine. Alternatively, you can launch either ENVI+IDL or just IDL and then select File > Open in the IDL Workbench.
  • Compile each of the files in the following order: (i) nrl_hico_h5_to_flat.pro, (ii) byte_ordering.pro, and (iii) write_nrl_header.pro. In the IDL Workbench this can be achieved by clicking on the tab associated with a given file and then selecting the Compile button in the menu bar.
  • You will ultimately only run the code for nrl_hico_h5_to_flat.pro, but this application is dependent on the other files; hence the reason they also need to be compiled.
  • Run the code for nrl_hico_h5_to_flat.pro, which this is done by clicking the tab for this file and then selecting the Run button in the menu bar.
  • You will then be prompted for an *.h5 input file (e.g., H2012095004112.L1B_ISS.h5), and a directory where you wish to write the output files.
  • There is no status bar associated with this operation; however, if you look closely at the IDL prompt in the IDL Console at the bottom of the Workbench you will note that it changes color while the process is running and returns to its normal color when the process is complete. In any event, the procedure is relatively quick and typically finishes in less than a minute.
  • Once complete, two sets of output files are created (data files + associated header files), one for the L1B radiance data and one for the navigation data.

Data Preparation: Below are the final steps needed to prepare the HICO data for further processing (repeated here in part from our previous post):

  • Open the L1B radiance and associated navigation data in ENVI. You will notice one side of the image exhibits a black stripe containing zero values.
  • As noted on the HICO website: “At some point during the transit and installation of HICO, the sensor physically shifted relative to the viewing slit. The edge of the viewing slit was visible in every scene.” This effect is removed by simply cropping out affected pixels in each of the data files. For scenes in standard forward orientation (+XVV), cropping includes 10 pixels on the left of the scene and 2 pixels on the right. Conversely, for scenes in reverse orientation (-XVV), cropping is 10 pixels on the right and 2 on the left.
  • If you’re not sure about the orientation of a particular scene, the orientation is specified in the newly created header file under hico_orientation_from_quaternion.
  • Spatial cropping can be performed by selecting Raster Management > Resize Data in the ENVI toolbox, choosing the relevant input file, selecting the option for Spatial Subset, subset the image for Samples 11-510 for forward orientation (3-502 for reverse orientation), and assigning a new output filename. Repeat as needed for each dataset.
  • The HDF5 formatted HICO scenes also require spectral cropping to reduce the total wavelengths from 128 to the 87 band subset from 0.4-0.9 um (400-900 nm). The bands outside this subset are considered less accurate and typically not included in analysis.
  • Spectral cropping can also be performed by selecting Raster Management > Resize Data in the ENVI toolbox, only in this case using the option to Spectral Subset and selecting bands 10-96 (corresponding to 0.40408-0.89669 um) while excluding bands 1-9 and 97-128. This step need only be applied to the hyperspectral L1B radiance data.
  • If desired, spectral and spatial cropping can both be applied in the same step.
  • The HICO scene is now ready for further processing and analysis in ENVI.

For more information on the sensor, detailed data characteristics, ongoing research projects, publications and presentations, and much, much more, HICO users are encouraged to visit the HICO website at Oregon State University. This is an excellent resource for all things HICO.

Hyperspectral Imaging from the ISS – Highlights from the 2014 HICO Data Users Meeting

The annual HICO Data Users Meeting was recently held in Washington, D.C. from 7-8 May 2014. This meeting was an opportunity for the HICO science community to exchange ideas, present research accomplishments, showcase applications, and discuss hyperspectral image processing techniques. With more than a dozen presentations and ample discussion throughout, it was an insightful and very informative meeting.

HREP-HICO

The HICO and RAIDS Experiment Payload installed on the Japanese Experiment Module (credit: NASA)

Highlights from 2104 HICO Data Users Meeting include:

  • Mary Kappus (Naval Research Laboratory) summarized the status of the HICO mission, including an overview of current instrument and data management operations. Notable upcoming milestones include the 5 year anniversary of HICO in September 2014 and the acquisition of HICO’s 10,000th scene – impressive achievements for a sensor that began as just a technology demonstration.
  • Jasmine Nahorniak (Oregon State University) presented an overview of the OSU HICO website, which provides a comprehensive database of HICO sensor information and data characteristics. The website also includes resources for searching and downloading data from the OSU HICO archives, visualizing orbit and target locations in Google Earth, and an online tool (currently in beta testing) for performing atmospheric correction using tafkaa_6s.
  • Sean Bailey (NASA Goddard Space Flight Center) outlined the HICO data distribution and image processing capabilities at NASA. HICO support was initially added to SeaDAS in April 2013, with data distribution beginning in July 2013. In less than a year, as of February 2014, NASA has distributed 4375 HICO scenes to users in 25 different countries. NASA is also planning to soon incorporate additional processing capabilities in SeaDAS to generate HICO ocean color products.
  • With respect to HICO applications: Lachlan McKinna (NASA GSFC) presented a project using time series analysis to detect bathymetry changes in Shark Bay, Western Australia; Marie Smith (University of Cape Town) described a chlorophyll study in Saldanha Bay, South Africa; Darryl Keith (US EPA) discussed the use of HICO for monitoring coastal water quality; Wes Moses (NRL) summarized HICO capabilities for retrieving estimates of bathymetry, bottom type, surface velocity and chlorophyll; and Curtiss Davis (OSU) presented HICO applications for assessing rivers, river plumes, lakes and estuaries.
  • In terms of image processing techniques, Marcos Montes (NRL) summarized the requirements and techniques for improved geolocation, ZhongPing Lee (UMass Boston) presented a methodology for atmospheric correction using cloud shadows, and Curtiss Davis (OSU) discussed various aspects of calibration and atmospheric correction.
  • James Goodman (HySpeed Computing) presented an overview of the functionality and capabilities of the HICO Online Processing Tool, a prototype web-enabled, scalable, geospatial data processing system based on the ENVI Services Engine. The tool is scheduled for release later this year, at which time it will be openly available to the science community for testing and evaluation.

Interested in more information? The meeting agenda and copies of presentations are provided on the OSU HICO website.

About HICO (http://hico.coas.oregonstate.edu/): “The Hyperspectral Imager for the Coastal Ocean (HICO™) is an imaging spectrometer based on the PHILLS airborne imaging spectrometers. HICO is the first spaceborne imaging spectrometer designed to sample the coastal ocean. HICO samples selected coastal regions at 90 m with full spectral coverage (380 to 960 nm sampled at 5.7 nm) and a very high signal-to-noise ratio to resolve the complexity of the coastal ocean. HICO demonstrates coastal products including water clarity, bottom types, bathymetry and on-shore vegetation maps. Each year HICO collects approximately 2000 scenes from around the world. The current focus is on providing HICO data for scientific research on coastal zones and other regions around the world. To that end we have developed this website and we will make data available to registered HICO Data Users who wish to work with us as a team to exploit these data.”

Application Tips for ENVI 5.x – Image to map registration using GCPs

This is part of a series on tips for getting the most out of your geospatial applications. Check back regularly or follow HySpeed Computing to see the latest examples and demonstrations.

Objective: Re-utilize ground control points (GCPs) originally obtained from the “Image Registration Workflow” to perform “Image to Map” registration of an image or its associated spatially equivalent images.

Scenario: This tip demonstrates the steps used to align a chlorophyll image derived from a hyperspectral HICO scene of the Turkish Straits with a multispectral Landsat 8 OLI image mosaic of the same area.

In some situations it is advantageous, or necessary, to re-use GCPs to geo-locate more than one spatially equivalent image. For example, instances where analysis is first performed on a non-registered source image and output products must be geo-located using the same GCPs as used for the source image.

Shown here is a depiction of the registration output for a HICO scene of the Turkish Straits, achieved following steps outlined in one of our previous tips: Improved geo-location using the Image Registration Workflow. What follows is an example of how to similarly geo-locate a derived chlorophyll image using the same GCPs as used in this registration.

HICO Turkish Straits

The Tip: In this example, the process isn’t as direct as using ENVI’s “Warp from GCPs” tools, or using the “Registration: Image to Map” tool, since these do not produce the desired output. Instead, as outlined below, we perform “Image to Map” registration via the “Registration: Image to Image” tool:

  • This example requires that you first use the “Image Registration Workflow” to generate a desired set of GCP points for geo-locating your selected source image.
  • Once the GCP points have been created, open and display the base image (here a Landsat 8 OLI mosaic) and non-registered warp image (here a HICO-derived chlorophyll image).
  • Start the “Registration: Image to Image” tool, found under “Geometric Correction > Registration” in the Toolbox.
  • In the opening dialog, select the desired band from the base image (in this example we use the OLI green 562.3 nm band), and then click Ok.
  • Note that the registration tool will attempt to automatically generate tie points between selected bands from the base and warp images; however, these points will be discarded and replaced using the existing file. This means that the specific bands selected for the base and warp images isn’t critical at this stage of the process.
  • In the next dialog, select the warp file to be registered, perform any desired spectral subsetting, and click Ok.
  • Now select the band from the warp image to be used for registration process (here the chlorophyll band).
  • When asked if you would like to “select an optional existing tie points file”, respond No.
  • Next, since you won’t be using the automatically generated tie points, accept the default “Automatic Registration Parameters”, being sure that “Examine tie points before warping” is set to Yes, and then click Ok.
  • The tool now opens windows for “Ground Control Points Selection” and the “Image to Image GCP List” as well as displays the base and warp images using the ENVI Classic 3-window interface.
  • In the “Ground Control Points Selection”, under “Options”, select “Clear All Points”. In the same window, under “File”, select “Restore GCPs from ASCII…”, and choose the appropriate GCP file that was previously generated using the “Image Registration Workflow”.
  • The “Image to Image GCP List” is now populated with your previously derived GCPs, which are also now shown in the two image displays.

Turkish Straits GCPs

  • In the “Ground Control Points Selection”, under “Options”, select “Warp File (as Image to Map)…”, choose the desired warp file to be registered (the chlorophyll image), perform any desired spectral subsetting, and click Ok.
  • The final window that appears is the “Registration Parameters” dialog. Here you will set the registration parameters equivalent to those of the previously registered source image. Note that the default parameters are likely not the same, and will need to be adjusted using the metadata from the original output image derived from the “Image Registration Workflow”. The metadata can be accessed directly from the image header file, or through the ENVI interface.
  • Enter the appropriate parameters for the “Output Project and Map Extent”, which includes the projection, coordinates of the upper left corner, output pixel size, and output image size.
  • Now enter the same “Warp Parameters” as used in the original registration process, select an output filename, and then click Ok.

Turkish Straits registration parameters

  • Once the registration process has completed, the output image now has the same geo-location as that obtained for the original geo-located source image.

HICO Turkish Straits chl

Visualizing HICO Ground Tracks Using Google Earth – A useful tool for project planning

Do you work with HICO imagery? Are you planning a project using HICO? Or perhaps you’re just interested in exploring where HICO will be acquiring imagery in the coming days?

If so, be sure to check out the ISS Orbit tool on the HICO website at Oregon State University. This tool allows you to interactively visualize the location of HICO ground track locations using Google Earth.

HICO ISS Orbit tool

The tool shows predicted HICO ground tracks in selected 1- or 3-day intervals up to six months in the future. However, even though orbital files are updated regularly, because of uncertainties in future ISS orbit specifics, the prediction is most accurate 2-3 days into the future and declines thereafter. So be cautious when planning fieldwork or image acquisitions for any extended time period.

For more information on ISS orbits characteristics, visit the NASA Space Station Orbit tutorial.

The ground tracks are displayed only for local daylight hours, and illustrate the nominal ground track (shown in teal above) as well as the full width available using HICO’s pointing capabilities (shown in grey above). Users have the option of also displaying the place names and locations of scheduled target areas for both ascending and descending orbits. Additionally, as the zoom level is increased, yellow dots appear in the visualization indicating the predicted time and date the ISS will pass over that location.

The HICO ISS Orbit tool requires the Google Earth plugin, which is available in Chrome, Firefox and IE (note that IE users may need to add the oregonstate.edu website to Compatibility View in the tool settings).

Let’s look at an example. Say you’re interested in exploring when HICO will be available to acquire imagery of Melbourne Harbor from April 5-11. Using the tool to step through the ISS orbits for those dates, it is revealed that Melbourne Harbor can be acquired on April 5 @ 22:26 and 5:45 GMT, April 6 @ 4:56 GMT and April 9 @ 4:05.

HICO Melbourne Harbor 040514

ISS Orbit tool: HICO – Melbourne Harbor 5-April-2014

HICO Melbourne Harbor 040614

ISS Orbit tool: HICO – Melbourne Harbor 6-April-2014

HICO Melbourne Harbor 040914

ISS Orbit tool: HICO – Melbourne Harbor 9-April-2014

Now let’s extend this example to see if Hyperion data is also available for Melbourne Harbor for the same dates. To do so, you will need to utilize COVE, a similar tool (best in Chrome or Firefox) with robust capabilities for visualizing ground tracks of numerous Earth observing satellites (but unfortunately not HICO or any other instruments on the ISS). Visit our earlier post for an overview of COVE’s capabilities.

Using COVE, it can be seen that Hyperion data is available for acquisition of Melbourne Harbor on April 9 @ 23:16 GMT. This closely coincident acquisition opportunity might provide some interesting data for comparing hyperspectral analysis techniques using HICO and Hyperion.

Hyperion Melbourne Harbor 040914

COVE tool: Hyperion – Melbourne Harbor 5-April-2014

So be sure to check out both the COVE and HICO ISS Orbit tools when planning your next mission.

HICO ISS Orbit tool: http://hico.coas.oregonstate.edu/orbit/orbit.php

COVE: http://www.ceos-cove.org/

About HICO (http://hico.coas.oregonstate.edu/): “The Hyperspectral Imager for the Coastal Ocean (HICO™) is an imaging spectrometer based on the PHILLS airborne imaging spectrometers. HICO is the first spaceborne imaging spectrometer designed to sample the coastal ocean. HICO samples selected coastal regions at 90 m with full spectral coverage (380 to 960 nm sampled at 5.7 nm) and a very high signal-to-noise ratio to resolve the complexity of the coastal ocean. HICO demonstrates coastal products including water clarity, bottom types, bathymetry and on-shore vegetation maps. Each year HICO collects approximately 2000 scenes from around the world. The current focus is on providing HICO data for scientific research on coastal zones and other regions around the world. To that end we have developed this website and we will make data available to registered HICO Data Users who wish to work with us as a team to exploit these data.”

About Hyperion (http://eo1.gsfc.nasa.gov/ and http://eo1.usgs.gov/): “The Hyperion instrument provides a new class of Earth observation data for improved Earth surface characterization. The Hyperion provides a science grade instrument with quality calibration based on heritage from the LEWIS Hyperspectral Imaging Instrument (HSI). The Hyperion capabilities provide resolution of surface properties into hundreds of spectral bands versus the ten multispectral bands flown on traditional Landsat imaging missions. Through these spectral bands, complex land eco-systems can be imaged and accurately classified.The Hyperion provides a high resolution hyperspectral imager capable of resolving 220 spectral bands [from 400 to 2500 nm] with a 30-meter resolution. The instrument can image a 7.5 km by 100 km land area per image, and provide detailed spectral mapping across all 220 channels with high radiometric accuracy.”

Application Tips for ENVI 5 – Using the Image Registration Workflow

This is part of a series on tips for getting the most out of your geospatial applications. Check back regularly or follow HySpeed Computing to see the latest examples and demonstrations.

Objective: Utilize ENVI’s Image Registration Workflow to improve geo-location of an image by aligning it with a higher accuracy base image.

Scenario: This tip demonstrates the steps used to align a hyperspectral HICO image of the Florida Keys with a multispectral Landsat 7 image mosaic of the same area.

HICO - LANDSAT alignment

Data: This example utilizes a HICO image of the Florida Keys (H2011314130732.L1B_ISS) downloaded in ENVI format from the HICO website at Oregon State University. The same image is also available in HDF5 format from the NASA GSFC Ocean Color website (read this tip for an overview on opening HICO HDF5 files in ENVI).

HICO – Hyperspectral Imager for the Coastal Ocean – is an imaging spectrometer located on the International Space Station. HICO images are currently distributed with only rough geo-location information, and can be off by as much as 10 km. The Florida Keys HICO image used in this example has been geo-located using this coarse information (the HICO website provides a good summary of the steps used for this process).

Additionally, as the base image for the registration process, this example utilizes a Landsat 7 mosaic generated from two L1T Landsat images (LE70150422000036EDC00 and LE70150432000036EDC01) downloaded from USGS EarthExplorer. The L1T processing level “provides systematic radiometric and geometric accuracy by incorporating ground control points while employing a Digital Elevation Model (DEM) for topographic accuracy.” The two Landsat 7 scenes were mosaicked using ENVI’s Seamless Mosaic Tool.

The Tip: Below are steps used to implement the Image Registration Workflow for this example:

  • Open both the HICO image and Landsat 7 mosaic in ENVI, and start the ‘Image Registration Workflow’ found in the Toolbox under Geometric Correction > Registration.
  • In the opening dialog window select the Landsat 7 mosaic as the ‘Base Image File’, select the HICO image as the ‘Warp Image File’, and then click ‘Next’.
  • The next dialog window that appears is for ‘Tie Points Generation’. In some cases it is possible to automatically generate acceptable tie points utilizing the default values and without selecting any seed points. You can explore this process by simply selecting ‘Next’ at the bottom of the dialog and reviewing the resulting points that are generated. If the output isn’t acceptable, then just select ‘Back’ to revert back to the ‘Tie Points Generation’ dialog. For our example, this automated process produced just 5 tie points with an RMSE of 3.15. Furthermore, the generated tie points did not properly correspond to equivalent features in both images. We can do better.
  • Returning to the ‘Tie Points Generation’ dialog, under the ‘Main’ tab, we adjusted the ‘Matching Method’ to ‘[Cross Modality] Mutual Information’, changed the ‘Transform’ to ‘RST’, and left the other parameters set to their default values.

Tie points generation - main

  • Under the ‘Advanced’ tab we set the ‘Matching Band in Base Image’ to ‘Band 2 (0.5600)’, the ‘Matching Band in Warp Image’ to ‘Band 28 (0.5587)’, and left the default values for all other parameters.

Tie points generation - advanced

  • To add user-selected tie-points, select ‘Start Editing’ under the ‘Seed Tie Points’ tab. This displays the base image in the active view, and sets the cursor to the ‘Vector Create’ tool. To add a point, left-click in the desired location on the base image, and then right-click to select ‘Accept as Individual Points’. This brings up the warp image in the active view, where you use the same steps (left-click on location and right-click to accept) to select the equivalent location in the warp image. The process is then iterated between the base and warp images until you have selected the desired number of tie-points. If needed, you can switch the cursor to the pan or zoom tools, or turn layer visibility on/off, to better navigate the images for point selection.
  • Select ‘Stop Editing’ once you have added at least 3 tie-points (we added just 4 in our example). Note that you can always go back and delete or add points as needed until you are satisfied. You can even return to the tie-point selection step after reviewing results from the automatic tie-point generation process.
  • Now select ‘Next’ at the bottom of the ‘Tie Points Generation’ dialog to automatically generate tie-points based on these seed points. Once the point generation process is complete, the next dialog window that appears is for ‘Review and Warp’.
  • The total number of tie-points is listed under the ‘Tie Points’ tab in this dialog. In our example the process produced a total of 14 tie-points. Select ‘Show Table’ to view a list of the tie-points, including information on the error associated with each point, as well as the total RMSE calculated for all current points.

Tie points table

  • Before proceeding with the warp process, it is recommended that you visually inspect each tie-point to confirm it correctly identifies corresponding locations in both images. Individual points can be selected and visualized by first selecting the point’s row in the attribute table (accomplished as shown above by selecting the box to the left of the point’s ID number). Once selected, the active point is highlighted and centered in the base image, and you can then use the ‘Switch To Warp’ and ‘Switch To Base’ buttons in the ‘Tie Points’ tab to alternate between the two images.

Tie point review

  • If you see a point that isn’t correct, simply select the small red X at the bottom of the attribute table to delete the active point. In the Key Largo example, two points were associated with non-permanent features on the water surface and were thus appropriately discarded. This reduced the total number of points in the example to 12, and improved the overall RMSE to 0.86, which was deemed acceptable for this application.
  • While the deletion process isn’t directly reversible, if you inadvertently delete a point you can always close the table, go back to the ‘Tie Points Generation’ dialog and regenerate all tie-points.
  • Once you are satisfied with the tie-points, close the attribute table and select the ‘Warping’ tab in the main ‘Review and Warp’ dialog. This displays the parameter options for performing the final alignment of the warp image. In our example we set the ‘Warping Method’ to ‘RST’, ‘Resampling’ to ‘Nearest Neighbor’, and ‘Output Pixel Size From’ to ‘Warp Image’.

Tie points warp

  • The final warp processing is then initiated by selecting the ‘Next’ button at the bottom of the dialog. Once complete, all that remains is to select an ‘Output Filename’ and format for the warped image and an ‘Output Tie Point File’ for saving the tie-points.

It is important to note that the above parameter values were selected to work best for this example and for other HICO images; however, different values may be more appropriate for your particular application. For detailed descriptions of parameters and general instructions on running the Image Registration Workflow, as well as a hands-on tutorial, be sure to look at the documentation included with ENVI.

Application Tips for ENVI 5.1 – Opening HICO scenes formatted in HDF5

This is part of a series on tips for getting the most out of your geospatial applications. Check back regularly or follow HySpeed Computing to see the latest examples and demonstrations.

Objective: Open a HICO dataset stored in HDF5 format and transform the data into standard ENVI formatted files ready for further processing and analysis.

(Update 14-June-2014): A companion post is now available describing how to open HDF5 HICO scenes using an IDL application prepared by the U.S. Naval Research Laboratory.

Scenario: This tip demonstrates two methods for opening a HDF5 formatted HICO scene in ENVI: (i) one method uses the new HDF5 tool released in ENVI 5.1 and (ii) the other method uses the H5_BROWSER routine in IDL. Subsequent steps are then used to save the data in standard ENVI format and prepare the resulting data for further analysis.

  • HICO – Hyperspectral Imager for the Coastal Ocean – is an imaging spectrometer located on the International Space Station. Visit the HICO website at Oregon State University for detailed information on data characteristics and tips for working with this data.
  • HDF5 – Hierarchical Data Format v5 – is a file format used to organize and store large multifaceted data collections. Visit the HDF Group for more information.

HICO_website

Why two methods? Not only is it informative to describe both methods, but since the new HDF5 tool in ENVI is generic, it does not always support access to all data files within a given dataset. Hence, for users who wish to remain within the ENVI/IDL system, it is often necessary to leverage the H5_BROWSER routine to open all available data in a particular HDF5 scene.

(Update 5-Mar-2014): Exelis VIS has released ENVI 5.1 Hotfix 1, which addresses the issue of being able to read all data files within a given HICO dataset. The new HDF5 tool is now fully functional for the entire dataset. See below for details.

Data: HICO data is currently available from two different online sources: (i) as ENVI files through the HICO website, which requires users to submit a brief proposal to become a registered HICO Data User, or (ii) as HDF5 files through the NASA GSFC Ocean Color website, which requires users to register with NASA’s EOSDIS User Registration Service.

This example utilizes a HICO scene of the Florida Keys (H2011314130732.L1B_ISS) downloaded from the NASA GSFC Ocean Color website.

H2011314130732_L1B_ISS_RGB

The HDF5 formatted HICO scenes contain six core datasets: (i) products (L1B top of atmosphere radiance), (ii) images (three-band true color image), (iii) navigation (latitudes, longitudes, sensor azimuth and zenith, and solar azimuth and zenith), (iv) quality (scan-line quality flags), (v) data (L0 uncorrected data), and (vi) metadata (FGDC and HICO metadata information). Note: In more recent files the L0 uncorrected data is not included and the HDF5 files therefore only contain five core datasets.

Tip 1 – HDF5 Tool in ENVI 5.1: Below are steps to open the navigation and quality datasets in ENVI using the new HDF5 tool:

  • Start by unpacking the compressed folder (e.g., H2011314130732.L1B_ISS.bz2). If other software isn’t readily available, a good option is to download 7-zip for free from http://www.7-zip.org/.
  • Rename the resulting HDF5 file with a *.h5 extension (e.g., H2011314130732.L1B_ISS.h5). This allows the HDF5 tools to more easily recognize the appropriate format.
  • Open the ENVI generic HDF5 reader by selecting File > Open As > Generic Formats > HDF5, and then select the desired *.h5 file in the file selection dialog.
  • The left column of the HDF5 reader lists the Available Datasets and the right column lists the Raster Builder. The right column is where you will create and populate the two HICO datasets to open in ENVI.
  • First, for the quality dataset, rename the default raster in the right column of the Raster Builder (e.g., rename Raster 1 to H2011314130732_L1B_ISS_quality). This can be easily accomplished by right clicking on the raster name and selecting Rename Raster. Next, add the quality flags to this raster by clicking on the <flags (512×2000)> dataset in the left column and then clicking the arrow in the center of the HDF5 tool to add this data to the recently created raster.
  • Now add a second raster to the right column using the New Raster button at the bottom of the right column (it’s the rightmost icon with the small green plus sign). Rename this raster (e.g., rename Raster 2 to H2011314130732_L1B_ISS_navigation), and use the center selection arrow to add all six data layers from the navigation dataset in the left column to the new raster in the right column.
  • Note that if you will be opening data from more than one HICO scene, then you can also build a Template from these settings and use the Template as the basis to open more datasets.
  • Your HDF5 dialog window should look similar to the following:

ENVI_HDF5_reader

  • Now select Open Rasters at the bottom of the dialog window to open both of the rasters as data layers in ENVI.
  • These data layers can then be saved in ENVI format by selecting File > Save As…, selecting a given data layer in the file selection dialog, and assigning an Output Filename in the Save File As Parameters dialog.

(Update 5-Mar-2014): Follow the same steps as above to open and save the L1B product using the HDF5 reader. However, for the three band color image, note that the HDF5 metadata itself misidentifies this data as band interleaved by pixel (BIP) when in fact it is band sequential (BSQ). To save this data in ENVI format, first open and save the data using the same steps as above, and then use Edit ENVI Header to adjust the band interleave of the resulting ENVI file from BIP to BSQ.

Tip 2 – H5_VIEWER in IDL: Below are steps to open the L1B product in ENVI using the IDL H5_BROWSER tool:

  • As evident in the current example, the left column of the generic HDF5 reader does not indicate any data is present within the products data layer. Hence, we use an alternative approach here to access the L1B product data. You can also use this same approach to open other datasets within the HDF5 file.
  • This tip requires both ENVI and IDL, so to get started you will need to make sure you have launched ENVI+IDL rather than just ENVI. By launching ENVI+IDL you will start both the standard ENVI interface and the IDL Workbench.
  • In the IDL Workbench, within the IDL command-line console, enter the following commands to start the HDF5 Browser (substituting your own filename as appropriate):

IDL_H5_BROWSER

  • The HDF5 Browser window should now appear, where the left column of the window lists the available datasets (now with data listed under the products dataset) and the right column shows details about any given selected dataset.

H5_BROWSER

  • Within the H5 Browser, select the Lt data in the products dataset (as shown above), and then select Open.
  • Returning now to the IDL command-line console, enter the following lines (again substituting your own filename as appropriate) to save the data to disk in ENVI format and write an associated ENVI header file.

IDL_HICO_write

  • Alternatively, you can also use the following code to create your own simple IDL program. To do so, simply start a new program by selecting File > New File in the IDL Workbench, copy and paste text from this file (hy_open_hico.pdf) into the new program, save the program with the same name as listed in the code (i.e., hy_open_hico), replace the hardcoded filenames with your own filenames, compile the code, and then run.

IDL_code

Data Preparation: Below are the final steps needed to prepare the HICO data for further processing:

  • As noted on the HICO website: “At some point during the transit and installation of HICO, the sensor physically shifted relative to the viewing slit. The edge of the viewing slit was visible in every scene.” This effect is removed by simply cropping out affected pixels in each of the above data files (i.e., quality, navigation and products). For scenes in standard forward orientation (+XVV), cropping includes 10 pixels on the left of the scene and 2 pixels on the right. Conversely, for scenes in reverse orientation (-XVV), cropping is 10 pixels on the right and 2 on the left.
  • If you’re not sure about the orientation of a particular scene, the orientation is specified in the HDF5 metadata under Metadata > HICO > Calibration > hico_iss_orientation.
  • Spatial cropping can be performed by selecting Raster Management > Resize Data in the ENVI toolbox, choosing the relevant input file, selecting the option for Spatial Subset, cropping the image for Samples 11-510 for forward orientation (3-502 for reverse orientation), and assigning a new output filename. Repeat as needed for each dataset.
  • The HDF5 formatted HICO scenes also require spectral cropping to reduce the total wavelengths from 128 bands to a 87 band subset from 400-900 nm. The bands outside this subset are considered less accurate and typically not included in analysis.
  • Spectral cropping can also be performed by selecting Raster Management > Resize Data in the ENVI toolbox, only in this case using the option to Spectral Subset and selecting bands 10-96 (corresponding to 404.08-896.69 nm) while excluding bands 1-9 and 97-128. This step need only be applied to the hyperspectral L1B product data.
  • The header file for the L1B product data should now be updated to include information for the wavelengths (87 bands from 404.08-896.69 nm), fwhm (10 nm for 400-745 nm; and 20 nm for 746-900 nm) and gain values (0.020 for all bands).
  • Editing the header can be accomplished using the option for Raster Management > Edit ENVI Header in the ENVI toolbox, or by directly editing the actual header file using a standard text editor. For simplicity, as illustrated below, you can cut and paste the relevant header info from the following file (example_header.pdf) into the header.

ENVI_header

  • The HICO scene is now ready for further processing and analysis in ENVI.
  • As an example, the latitude and longitude bands in the navigation file can be used to build a GLT to geolocate the scene using the rough coordinates provided with the data distribution. The HICO website provides a step-by-step overview of this geolocation process.

For more information on the sensor, detailed data characteristics, ongoing research projects, publications and presentations, and much, much more, HICO users are encouraged to visit the HICO website at Oregon State University. This is an excellent resource for all things HICO.