Sensors, Satellites, and Sidekicks: Chemical Fingerprinting From Space for Valuable Applications on Earth

Beside some of the greatest heroes, you’ll find a loyal sidekick ready to jump in and help whenever a challenge arises. Batman has Robin. Han Solo has Chewbacca. And Mario has Luigi. When Dan Katz and Tushar Prabhakar founded their global monitoring company in Katz’s San Francisco garage in 2016, they envisioned being a trusted partner for their customers.

From high above our planet, satellites equipped with hyperspectral sensors can capture images of the Earth that reveal more than what we can see with our eyes. These sensors detect electromagnetic radiation beyond the visible light that typical cameras capture. Hyperspectral data allows scientists to identify specific chemicals and materials that may be present in the image, which can serve as an indicator for a wide variety of applications.

Using hyperspectral satellites, Katz and Prabhakar sought to establish a company that would provide monitoring services to help solve its customers’ most pressing challenges. Whether it’s finding leaks in pipelines, identifying areas susceptible to wildfire, detecting valuable materials for mining, or assessing the health of crops, the company would serve as a sidekick to its customers—an Orbital Sidekick.

“Hyperspectral imaging is a powerful tool for global monitoring,” said Katz, CEO of Orbital Sidekick. “One of our core tenets is partnering with our customers to provide actionable insights so they can have better operational integrity and make smart decisions about their assets.”

But before they could get their company—and hyperspectral satellites—off the ground, they needed to test their sensor technology in space. They realized early on that launching their own satellite for testing would be a significant task because they would also need to figure out how to get the satellite to work and get the data from the sensor back to Earth.

“Having to worry about not just the new hyperspectral sensor technology but also everything else that goes along with the mission is a lot to handle, especially for a small team,” Katz said. “So, we started thinking, what other ways could we demonstrate the sensor in space and not have to worry about the infrastructure?”

It was while Katz and Prabhakar were at SpaceCom, a global commercial space conference, that they stumbled upon the answer: the International Space Station (ISSInternational Space Station) National Laboratory. At the conference, they met with representatives from the ISS National Lab and Nanoracks, part of Voyager Space’s Exploration Segment. They realized the space station was the perfect platform to test their sensor technology. In an investigation sponsored by the ISS National Lab, Orbital Sidekick launched its sensor— Hyperspectral Earth Imaging System Trial (HEIST)—to station, where it was installed on the Nanoracks External Platform(Abbreviation: NREP) A platform that on the exterior of the ISS that provides power and a data connection and enables payloads to operate in the harsh space environment. This ISS National Lab commercial facility is owned and operated by Nanoracks. (NREP), which is mounted to the exterior of the orbiting laboratory.

“It was very appealing to us because the space station is a stable platform and has all the infrastructure and downlink capability in place, so all we had to really worry about was making sure the hyperspectral sensor worked, which was a huge relief,” Katz said. “Our goal was to see if we could demonstrate that there’s a viable path forward to commercialization by leveraging space-based hyperspectral sensing. The mission was wildly successful and really set the table for everything we’re doing today with our commercial satellites.”

Seizing an Opportunity
Traveling in the form of waves, electromagnetic radiation is all around—from radio waves that transmit our favorite songs to microwaves that heat our food, ultraviolet (UV) rays that can cause sunburn, infrared waves that are in some heaters, x-rays used in hospitals, and gamma rays that deliver high-energy radiation to kill cancer cells. However, the type of electromagnetic waves you are most familiar with come in the form of visible light (red, orange, yellow, green, blue, indigo, and violet). This is the only type of electromagnetic waves we can see with our eyes.

Electromagnetic radiation is organized in a spectrum of wavelengths, with high-energy gamma rays at one end, low-energy radio waves at the other, and visible light in the middle. High-energy waves have shorter wavelengths, meaning the waves are closer together, and low-energy waves have longer wavelengths, meaning the waves are spaced farther apart.

Orbital Sidekick’s HEIST can detect a range of wavelengths from around 400 nanometers (which is violet visible light) to about 1,000 nanometers (which is near-infrared, just beyond the visible light part of the spectrum, which ends around 700 nanometers at deep-red visible light). The sensor captures light through the lens, and a diffraction gradient (a tool that acts like a prism) has a pattern of grooves that split the light into discrete spectral bands that can be used to characterize the area imaged.

Images of Earth taken by HEIST have a spatial resolution of 28 meters. This means that each pixel in an image represents 28 meters on the ground. That may seem like a low resolution compared with images from satellites like those used by Google Earth, which can capture areas on the ground as small as 15 centimeters. However, typical satellites cannot provide the detailed spectral data that HEIST can. Most imaging satellites can only capture three spectral bands (red, green, and blue visible light, similar to the camera in your cell phone), compared with HEIST’s 150 discrete spectral bands that go beyond the visible spectrum. By analyzing the spectral bands in an image, Orbital Sidekick can do what Katz calls “chemical fingerprinting.”

Everything absorbs light in a unique manner, and different materials reflect different intensities of light. HEIST can capture the absorption and reflectance features in each pixel of an image, and the team can use a database of chemicals, compounds, molecules, and materials to see what the features correspond to. For example, the features may correspond to chemicals that are health indicators for crops or may reveal the presence of lithium, nickel, cobalt, or other battery storage materials for mining.

“I call it chemical fingerprinting because everything has unique spectral fingerprints, and we can identify those fingerprints using our sensors and provide the chemical identification to our customers,” Katz said. “It’s a more detailed and in-depth way to classify what you’re looking at, especially for things you cannot see with the naked eye, such as energy-sector contamination events or emissions.”

With the expanding commercial space market and the push for sustainable energy transition, Katz and Prabhakar saw an opportunity to leverage hyperspectral satellites to monitor oil and gas pipelines. Using hyperspectral data, they could help pipeline operators identify threats like oil spills and methane leaks and address the problems quickly.

“There aren’t many experts who know how to extract insights from hyperspectral data, so it was clear to us early on that we had to build a pretty vertical company,” Katz said. “We knew we would need to launch hyperspectral satellites and process the collected data with an analytics engine so we could extract meaningful insights and deliver them to our customers in the form of a monitoring service.”

Translating From Ground to Space
Orbital Sidekick’s first steps toward providing a hyperspectral monitoring service were testing HEIST in orbit and building an analytics engine. As Katz and Prabhakar prepared their sensor for launch, they worked closely with Nanoracks to ensure it met all safety and compliance requirements. Nanoracks also did integration work to ensure the sensor would interface with the NREP and NASANational Aeronautics and Space Administration systems and operate smoothly and effectively in orbit.

“There were a lot of things they really helped us think through,” Katz said. “It was things we wouldn’t have known to do or didn’t have the expertise in, everything from high-level information to low-level details. They really held our hand through the whole process.”

In June of 2018, HEIST was finally ready to launch. As the Orbital Sidekick team watched from NASA’s Kennedy Space Center, they were filled with excitement and a little anxiety. But the nervousness stripped away as they watched the SpaceX Falcon 9 rocket launch their project into orbit.

“It was sometime in the middle of the night, and we had a great view of the launch,” Katz recalls. “It lit up the whole night sky, almost like the sun was rising. It was a pretty incredible experience to see that spacecraft go up and know that our system was on it.”

Once HEIST was on station, it needed to be installed on the NREP. Payloads for the NREP are installed on the station’s exterior via an airlock in the Japanese Experiment Module (JEM). Placed on a special plate, the payloads are transported with the help of the Japanese robotic arm and installed on the NREP. Once plugged in, the NREP provides the payload with power and data communications.

“We pride ourselves on enabling space-based research and development,” said Michael Lewis, Nanoracks chief innovation officer. “The philosophy of the company since the start has been to make the space environment more accessible. The NREP is designed to interface directly with the space station, and payloads can just attach to the platform, so there’s almost no barrier for our customers.”

Clearing Hurdles and Pushing Boundaries

However, when designing payloads for spaceflight, things don’t always go as planned, and you have to be ready to come up with creative solutions—and HEIST was no exception. While working with Orbital Sidekick on the HEIST payload, Nanoracks ran into a problem. Payloads can only extend from the NREP 10 centimeters in the Earth-facing direction, but HEIST needed a lens with a focal length that exceeded that limit. To address this, Nanoracks built a custom plate with holes that allowed the sensor’s lens barrel to extend upward into the NREP to achieve the proper focal length while meeting the size requirements.

To help the ISS crew members install HEIST on the custom plate, Nanoracks developed special brackets designed to fit together easily. But once the payload was on station, the team ran into another problem. The astronauts couldn’t get the brackets to fit correctly. In talking with the crew, the Nanoracks team realized that the brackets had been assembled in reverse order on the ground.

“At first, I couldn’t believe it because I had assembled it myself,” said Keith Tran, Nanoracks director of mission operations. “But in real time, we were able to quickly make a change in the plan and told the crew. They swapped the payload around and rotated the cable, and we were able to install it successfully and complete the mission.”

HEIST also drove the development of new and enhanced capabilities on the NREP, Lewis said. The NREP obtains data about position from the space station, but because HEIST was imaging very precise locations on Earth, the platform needed a GPS receiver, which was installed. Another challenge led to an increase in bandwidth for data transfer from the NREP.

Hyperspectral imaging produces a lot of data, and it took quite a bit of coordination to get all of Orbital Sidekick’s data down to Earth, Lewis said. The NREP was equipped with Wi-Fi, but the connection was too slow, causing a bottleneck. Normally, the NREP would send down data between 2 gigabytes and 10 gigabytes per week, but Orbital Sidekick needed more on the order of 100 gigabytes daily.

“We tackled the data issue on both ends,” Tran said. “On the NASA side, we worked to get more bandwidth so we could bring the data down, and Orbital Sidekick worked on their end to reduce the amount of data by only taking images where needed and compressing the data into smaller files.”

Originally, HEIST testing was slated to take place over 15 weeks, but the project was so successful, it stayed on station for more than a year. In the end, Orbital Sidekick captured more than 30 million square kilometers of hyperspectral imaging, which was a little more than 20 terabytes of data. The team successfully demonstrated the company could build a hyperspectral sensor, send it to space, retrieve and analyze data, and extract valuable insights for several use cases.

“The data can be used for a whole host of different things—from the energy sector to the mining sector, agriculture, food quality and safety, fire risk assessment, and everything in between,” Katz said.

Springing From Station to Commercial Satellites
Since Orbital Sidekick’s ISS National Lab-sponsored project, the company has launched five commercial satellites that make up its GHOST™ (Global Hyperspectral Observation Satellite) constellation, with two of these satellites launching last month. Each GHOST satellite is equipped with an upgraded hyperspectral sensor that can detect wavelengths up to 2,500 nanometers, which is farther into the infrared. The upgraded sensor’s spatial resolution is eight meters, which is a big improvement from HEIST’s 28 meters. The company plans to expand GHOST to include a total of 14 satellites.

“This will allow us to map the entire globe every week with high-resolution hyperspectral data,” Katz said. “Our goal is to monitor millions of kilometers of pipeline every week with our satellites, which is a big commercial opportunity for us.”

Following the completion of its ISS investigation, Orbital Sidekick has raised nearly $50 million in investment dollars, obtained several government grants and contracts, and signed more than a dozen large energy companies for pipeline monitoring services. According to Lewis, this project is a great example of what can be accomplished by harnessing the space station as a research platform.

“This was an opportunity to take a sensor in the early stages of development and enable the company to go on to produce a strong commercial product,” he said. “I’d say their ability to use our NREP platform put them ahead in the industry—they had identified a market and found a way to get into that market by increasing the technology readiness level(Abbreviation: TRL) A measurement system used to assess the maturity level of a particular technology. There are nine technology readiness levels, and technology progresses from TRL 1 to TRL 9. of their sensor in space.”

Orbital Sidekick has come a long way since its early days as a fledgling startup in a San Francisco garage. Katz said he can’t imagine where the company would be had they not leveraged the ISS National Lab. “It allowed us to focus our resources on what’s really driving value for our company, which is the intelligence platform, analytics engine, and product development for our end user—that’s what’s important for our commercialization effort,” he said. “Our ISS National Lab mission was an amazing launch point for our company—it was the enabling instrument that unlocked so much for us. It was incredible.”