The Mantis Shrimp Will Change How You See the World (Literally)

The mantis shrimp will change how you see the world

Human eyes have three photoreceptors: Red, green, and blue. Compared to dogs — who only have the latter two — our eyesight is exceptional. But compared to the mantis shrimp, we live in a pitiable state.

Mantis shrimp have the world’s most complex visual system — with up to 16 photoreceptors, they can see in ultraviolet, in addition to (human) visible and polarized light. In fact, mantis shrimp are the only known animals capable of detecting light that is polarized circularly — which is when a portion of light rotates in a circular motion.

They can also see depth with one eye, and move either eye independently. Simply trying to imagine how mantis shrimp see is mind-boggling. Much like flies, mantis shrimp have compound eyes, made of tens of thousands of ommatidia — which are elements containing a cluster of photoreceptor cells, in addition to pigment and support cells.

Defining polarimetry and hyperspectral imaging

“Lots of artificial intelligence (AI) programs can make use of data-rich hyperspectral and polarimetric images, but the equipment necessary for capturing those images is currently somewhat bulky,” said a co-corresponding author Michael Kudenov of the paper on the work, who’s also an associate professor of electrical and computer engineering at North Carolina State University. “Our work here makes smaller, more user-friendly devices possible. And that would allow us to better bring those AI capabilities to bear in fields from astronomy to biomedicine.”

Hyperspectral imaging refers to novel technologies capable of breaking down visible wavelengths of light into narrow bands that the human eye alone can’t notice. But computers can see them — which makes hyperspectral imaging ideal for tasks like identifying the chemical composition of objects in an image.

Polarimetry is a term used to refer to the measurement of the polarization of light — consisting of data scientists and engineers can use to identify the surface geometry of an object in an image. For example, it can distinguish between rough and smooth surfaces, in addition to the angle of a surface relative to a light source.

Smartphone camera designs create subtle alignment errors

Light is hard to talk about — since it exists as both a particle and a wave. When it moves between two discrete points, the path between them is the direction of light. As a particle, light moves in a straight line from A to B. But it also exists as an electromagnetic field that varies from place to place, like a wave.

Imagining a wave rising and falling side-to-side while it moves from A to B, polarization becomes the measurement of the distribution of the wave along that fluctuating path.

Of course, we already have larger devices capable of capturing both polarimetric and hyperspectral images — but developing a smartphone-sized imaging technology of these kinds has been an incredibly challenging task.

Smartphone camera designs create subtle errors in alignment of the varying light wavelengths in the final image. This doesn’t interfere with family photos or selfies, but it’s an issue when precision worthy of scientific analysis is required. And this problem is made worse when the camera captures even more colors — which is exactly what hyperspectral technologies do.

Mantis shrimp-based design could make sense up to 15 spectral channels

The scientists who created the new light sensors took their engineering cue from the eyes of mantis shrimp — which possess an incredible power for accuracy, capturing subtle gradations in color. 

To mimic this ability, the researchers created an organic electronic sensor called the Stomatopod Inspired Multispectral and POLarization sensitive (SIMPOL) sensor. In case you’re wondering, a stomatopod is the technical term for a mantis shrimp.

The SIMPOL sensor can register four spectral channels and three polarization channels at the same time. This contrasts sharply with charge-coupled devices used in smartphones — which only have three spectral imaging sensors that detect green, red, or blue. Additionally, the SIMPOL prototype can capture four color channels and three polarization channels in one singular point — unlike CCDs, which require imaging sensors spread throughout several locations.

“SIMPOL’s color channels can discern spectral features 10 times narrower than typical imaging sensors; in other words, it is 10 times more precise,” said Kudenov.

While this new imaging system is nothing more than a proof of concept, the researchers built modeling simulations suggesting SIMPOL’s mantis shrimp-based design could make detectors capable of sensing up to 15 spatially registered spectral channels. It seems there’s no limit to nature’s continual relevance in the most advanced fields of science and engineering.