Table of Contents
Introduction
Between February 11-13 a series of discussion style round tables were held at the NASA Jet Propulsion Laboratory (JPL) in Pasadena, CA. The meetings were hosted by the Instrument Software and Science Data Systems Section 398, and largely represented by members of the Multimission Image Processing Lab (MIPL) and Planetary Data Systems (PDS) Imaging Node, although others were able to join the discussions in person or via webex. The Astrogeology Science Center (ASC) of the U.S. Geological Survey (USGS; Flagstaff, AZ) was represented by Trent Hare and ASC software developers during the camera model discussions. These discussions were initiated by Dr. Thomas Logan and Paul Ramirez and who also provided travel support for Mr. Hare.
These notes are a summary/overview of the discussions as prepared by Mr. Hare, with minor changes provided by JPL participants (R. Deen and T. Logan).
Planetary and Earth Data Format Interoperability, Mon Feb 11
We began the discussion by listing many of the formats commonly used in the planetary (and Earth) community. The formats included: VICAR, ISIS3, PDS4, FITS (GeoFITS), GeoTiff, Jpeg2000, NITF, etc.
· Jpeg2000 was discussed to help understand its limitations and its seemingly decline in popularity (although it still is used within the NITF format). Issues listed included: 1) while current open source implementations are available, they are not very robust, 2) the popular encoder/decoder Kakadu, while it used to be open access, continues to be locked down, and (3), while 32bit floating point is defined in the specification, it is not widely supported. Jpeg2000 is currently allowed in PDS3 archives (HiRISE and LOLA archives), but is now banned from PDS4 archives. This is due to PDS4 archives only allowing raw binary streams, thus no compression of any style is allowed.
· Both Video Image Communication And Retrieval (
VICAR) and the Integrated Software for Imagers and Spectrometers (
ISIS3) formats are critical for the processing environments and missions they support. Changing formats would be difficult for either to support. Because VICAR uses an in-line label followed a binary stream, it is recognized that a simple reader could be written. Also, VICAR meets the PDS4 criteria to allow a VICAR image to be archived (as-is) along-side a PDS4 label which describes how to skip the in-line label.
Because ISIS3, by default, supports tiled data streams, reading and supporting PDS4 archives is a little tougher (note that ISIS3 also support raw binary streams). Also, VICAR does have limited compression support throught it’s Automated Fusion of Image Data System (AFIDS) branch of VICAR. The main branch of VICAR at JPL is used for in-situ missions like InSight, Mars2020, and other planetary mission activities.
· The AFIDS team has implemented a robust reader/writer in the Geospatial Data Abstraction Library (GDAL) for the VICAR format (which is available in the current Open VICAR distribution). However, it has not been pushed back into GDAL’s baseline code release. This is a stated goal for the group. There is already a limited VICAR driver in GDAL (by Sebastian Walter, Freie University Berlin), which needs to be taken into consideration prior to merging the AFIDS version.
· GDAL does have read support for PDS3 and the older ASC ISIS2 format. ASC has funded support for a basic ISIS3 reader and writer. Lastly, GDAL does have
basic support for PDS4 but requires a well-defined template label to make it ready for archival.
· The FITS format now has a proposed and implemented extension, called GeoFITS, allowing for more standardized geospatial map projection definitions. GDAL contains the first implementation as written by Chiara Marmo (University of Paris-Sud). It is still new and not yet widely used. Reference: Marmo, C., Hare, T. M., Erard, S., Minin, M., Pineau, F.‐X., Zinzi, A., et al. (2018). FITS format for planetary surfaces: Definitions, applications, and best practices. Earth and Space Science, 5, 640–651,
https://doi.org/10.1029/2018ea000388.
· The JPL-created and now Esri-supported format, called the Meta Raster Format (MRF) by Lucian Plesea, was discussed for its ability to optimize streaming services like Web Mapping Services (WMS). It is analogous to a file-based database since it supports an index into packed image tiles. It supports tiles as compressed by JPEG, PNG, the new Limited Error Raster Compression (
LERC) to name a few. LERC, originally written by Thomas Maurer, is an open specification by Esri and an open source implementation is available within GDAL. Because it can be lossless or lossy (with user defined precision), it has the potential to take on compressions like LZW and even Jpeg2000.
· The group seemed to agree that the GeoTiff format is the current best choice for interoperability between applications. As a free and open source library, GDAL also maintains the reference implementation. GeoTiff is well supported within AFIDS (a VICAR branch) and the NASA AMES Stereo Pipeline (ASP), but for other planetary applications (like ISIS3), the GeoTiff format must be converted into their native format. Using a raw (uncompressed and untiled) GeoTIff allows one to use the format in a PDS4 archive. Actually, in GDAL, when converting to a PDS4 image, GDAL optionally allows one to store the output image within a properly labeled GeoTiff (with a PDS4 label pointing into the GeoTiff). Lastly, it was noted that several projects like the Lunar Mapping and Modeling Project (LMMP) and related Trek projects, the Phobos Working Group, and the ASC-supported PDS Annex portal all prefer GeoTiff as their primary format (optionally with a PDS label).
· In the process of discussing image file formats, a number of useful file attributes were identified. These include: 1) Support for multiple bit structures such as 8-64 bit integer, floating point, real and imaginary; 2) Open source (100% non-propriety); 3) Support for Compression; 4) Support for tiling; 5) Support for Georeference; 6) Straight-forward binary layout; 7) Straight-forward label design (possibly integrated and/or detached); and 8) Support for multiple bands.
Collaboration Topics
· The group agreed that we are not all going to move everyone over to one format. There is no “killer” format or “one format to rule them all”. Thus, it was agreed that supporting GDAL seemed to be the best method to help support interoperability across applications. One collaboration idea would be a test suite providing results in GDAL for these conversions. For example, converting VICAR to GeoTIff back to VICAR and document what is lost. This could be applied across many of the popular formats like ISIS3 to VICAR and back to ISIS3. If too much information is lost, potentially teams could update the drivers in GDAL to help. Also supporting a side-car metadata file during conversion might also help. This metadata side-car file is already supported in GDAL, but the drivers would need to take advantage of it.
· The AFIDS group agreed to find resources to push their current in-house GDAL-based VICAR reader/writer back into the public release of GDAL. This would require assessing the impact to the current VICAR driver already in GDAL.
Open Source Software Issues, Tue Feb 12
- Discussions in this topic were not about the benefits of open source as all persons and projects represented have goals for making their code open source. Mostly this discussion focused on perceptions on open source projects and ideas around supporting open source projects.
- In 2015, Bob Deen (JPL) helped make a majority of VICAR open source. However, there are parts of VICAR that still have not been released. These include: 1) code which contain ITAR constraints; 2) code that uses licensed software (notably Numerical Recipes); 3) parts of the in-situ code-base; and 4) the AFIDS branch of VICAR. Besides the ITAR constraints, there are goals to open source both the in-situ and AFIDS versions.
- For ASC, all products, including software, are mandated to be public domain. Only recently though has ASC pushed their code-base to Github and will strive to “behave” like an open source project (see below for more).
- The image processing package Opensource Multi-INstrument Analysis Software (OMINAS), created by Joseph Spitale, is now mainly maintained by PSI and JPL, has also stated plans to open source their code. OMINAS is built on the commercial application IDL, currently owned by Harris Geospatial Solutions. The OMINAS maintainers currently support a free run-time for IDL and their code which users can take advantage of without an IDL license.
- It was discussed that releasing one’s code can be difficult for all members of a team when it is perceived as a specialized capability that potentially maintains future funding opportunities. Within the group, most stated that the benefits and importance of releasing their code outweighed this potential risk.
Collaboration Topics
- The group ended with discussions on what it is to be an open source project and ideas on how to fund it. It was pointed that the key behind successful open source projects is your user community. While releasing the source code is a great first step, if there is no support to foster and work with your user community, the project probably won’t make it for the long-term. Thus, calling out resources within your funding opportunities to foster your community and fund other open source projects you rely on is viewed as an extremely important aspect for any open source project. This includes the ability to take community input, whether they are bug reports, new ideas, or even the ability to accept user supplied code into your project (granted it must meet your coding standards).
- Since we all support the same or closely tied communities within both the planetary and Earth-based domains for image processing, we should continue to discuss how to better support our user community across our different applications. For example, allowing them to interoperate (e.g. formats, camera models, shared capabilities), would be a great step forward.
- A good resource for this topic includes this free publication: National Academies of Sciences, Engineering, and Medicine. 2018. Open Source Software Policy Options for NASA Earth and Space Sciences. Washington, DC: The National Academies Press. https://doi.org/10.17226/25217.
Talk: Activities at the USGS Astrogeology Science Center, Mon Feb 11
- presentation: available from dropbox.
“Big Data” Streaming Technologies and GeoSpatial Data Portals, Tue Feb 12
This round table covered several ongoing projects at JPL which are tackling these big-data topics. Like JPL, ASC has also been supporting streaming methods since the late 90s. Several projects were discussed including:
- Science Data Analytics Platform (SDAP). “SDAP is a technology software solution currently geared to better enable scientists involved in advancing the study of the Earth's physical oceanography”. As an Apache incubator project, it has been an open source project from the start. While centered around oceanography data, many of the core capabilities could be used in other projects. Instead of the generation of derived products on-disk, they are working towards real-time calculations. These products can be recalculated based on an encoded URL. The goal is to try to limit the need to large mosaics to be downloaded. If results must be provided, try to summarize results as small tables. Again, the encoded URL can be used to recreate the derived maps.
- The Global Imagery Browse Services (GIBS) are a set of standard services to deliver global, full-resolution satellite imagery in a highly responsive manner. This project is riding on technologies created at JPL and use OGC web standards including Web Map Tile Service (WMTS) and the similar Tiled Web Map Service (TWMS). Most all of their code is made available from within GDAL or from their Github site.
- There was some discussion on the JPL Trek system. Like GIBS, Trek builds on many of the same standardized services and Open Geospatial Consortium (OGC) technologies. The interface they support, and have extended for planetary, is Cesium. CesiumJS is an open-source JavaScript library for 3D globes and maps as founded by Analytical Graphics, Inc.
- ASC has been recently working with Esri and Caltech to test Web serving a couple of extremely large mosaics including the Caltech Mars CTX mosaic (~1TB in size, lossy, at 5m/p) a USGS- and Esri-create uncontrolled Mars HiRISE mosaic (19TB in size, lossy, at 0.25 m/p). These should be released by March 2019.
Collaboration Topics
- The grouped talked about goals of providing cloud services (applications) next to these cloud-provided data sets. They also discussed how such systems can be maintained for the long-term. For example, how are researchers able to recreate their results if the service is no longer provided. This is an area for rapid growth and support as data sets continue to get larger.
End-to-End Processing of Planetary Data, Camera Models to GIS Products, Wed Feb 13
While this round table provided time for both camera model discussions and GIS products, largely we discussed the Camera Sensor Model (CSM) standardized API. ASC provided an
introduction to the CSM standard. In short, the CSM API was conceived and implemented by the U.S. Air Force to standardize the use of camera models across different photogrammetric applications. The CSM standard provides a standardized API
only, not the actual implementation of the camera model. It is now maintained by the CSM Working Group, which has several government, university, and industry members. The CSM is currently used by several photogrammetric applications including commercial applications like BAE’s Socet GXP, Harris Geospatial Solution’s ENVI, and Hexagon’s ERDAS Imagine. ASC is currently working on CSM support in ISIS3 and is helping to support the CSM within NASA’s Ames Stereo Pipeline (ASP).
· ASC is currently working both a CSM open source framing and a pushbroom (line scanner) implementation. The pushbroom was originally developed by BAE and updated by ASC. With support from BAE, it is now released on
Github using a permissive BSD license. ASC in collaboration with the CSM Working Group, the 3.0.3 API supports variable radii for planetary applications. Previously it was locked to WGS84 (Earth).
ASC has also created a Simplified Wrapper and Interface Generator (SWIG) for the CSM library which provides Python access to the API. It can also provide access to JAVA, Ruby, and many other languages. Note that ASC supported camera models in ISIS3 (more than 50 instruments) are currently developed using a unique in-house design, but they are working on supporting CSM implementations also.
· The AFIDS Team provided background for their software called “
Mars Nest”. The software is a georeferencing pipeline that automatically co-registers most two map-projected images to each other. For unmapped (EDR) images, they have developed procedures to use a Replacement Sensor Model (RSM) as a close approximation for a rigorous camera model. Current implementations use the NITF format to store the RSM metadata (called Tagged Record Extensions or TREs), which could potentially be stored inside a PDS4 file. Similar to a CSM, an RSM implementation is another method to provide interoperability across photogrammetric applications.
· The CAHVOR camera model is used by several at JPL for in-situ and orbital cases.
· The NASA Ames Stereo Pipeline (ASP) currently uses both ISIS3 available camera models and their own camera models mainly for Earth-based instruments. The ASP team is currently implementing capabilities to incorporate CSM.
· Lastly, for GIS data products, we did have a short discussion from Caltech providing details for their recently released
CTX near-global mosaic for Mars. Not only did they optimize the input data to remove hazy images, they also optimized their workflow to maintain the ability to map every pixel in the mosaic back to the input image. They have also been provided funding, via a PDART, to make an even better version (~2 years). Input from the community is welcome. The current version will soon be available as a streaming cloud service hosted by Esri.
Collaboration Topics
- After the introduction to CSM, there were many questions about whether the standardized CSM API would be capable enough for current applications. While the relatively simple API does lock the user into the standard (for the sake of interoperability), the implementation is left up to the user. Thus, it should be able to support most applications (but more research or pilot studies should be done). It was also brought up that the CSM API doesn’t make the implementation of a camera model any easier. But by providing several open source implementations, there is hope it might help with adoption.
- Specifically, for in-situ applications, there was discussion if the CSM would be beneficial. Within the planetary community, there has been a long-time goal for the better integration of orbital and in-situ mission data. By allowing both orbital and in-situ photogrammetric applications to share the same camera model should actually help to realize this goal. Of course, a collaboration across projects to pilot this would be needed.
