Self-supervised AI can learn to recognize cloud types in satellite images without human guidance

Clouds play a pivotal role in the Earth’s climate and weather. They form in a diversity of distinct patterns and types that can have profoundly different impacts on things such as the water cycle and how radiant energy moves through the atmosphere. Earth-observing satellites have been capturing images for decades and these datasets offer a unique opportunity to study clouds and their impact on the Earth system.

The massive quantity of satellite images can be prohibitive to evaluate because there is far too much imagery for scientists to sort manually. Self-supervised AI can learn to interpret images without human input. But typical image recognition AIs require vast amounts of human-generated labels to train.

In a new article, published in Artificial Intelligence for the Earth Systems, researchers demonstrate an AI that can be taught to recognize cloud types by looking at millions of satellite images of clouds without requiring human input.

AI developed in this research learns to generate a simple numerical representation of cloud images by performing what is essentially an image-matching task. It is trained on pairs of cloud images taken from nearby locations that are likely to contain the same cloud types. It is rewarded for learning similar representations of images with the same cloud type and differing representations for images with different cloud types.

The model was evaluated on several datasets of cloud images that have been hand-labeled by past cloud classification studies and performed comparably to classification models that had been trained with human-generated labels.

Researchers also demonstrated the model’s ability to generalize between satellite instruments and explored the internal representations of clouds learned by the model by forcing the model to hallucinate images based on its own parameters.

Clouds play a pivotal role in the Earth’s climate and weather. They form in a diversity of distinct patterns and types that can have profoundly different impacts on things such as the water cycle and how radiant energy moves through the atmosphere. Earth-observing satellites have been capturing images for decades and these datasets offer a unique opportunity to study clouds and their impact on the Earth system.

The massive quantity of satellite images can be prohibitive to evaluate because there is far too much imagery for scientists to sort manually. Self-supervised AI can learn to interpret images without human input. But typical image recognition AIs require vast amounts of human-generated labels to train.

In a new article, published in Artificial Intelligence for the Earth Systems, researchers demonstrate an AI that can be taught to recognize cloud types by looking at millions of satellite images of clouds without requiring human input.

AI developed in this research learns to generate a simple numerical representation of cloud images by performing what is essentially an image-matching task. It is trained on pairs of cloud images taken from nearby locations that are likely to contain the same cloud types. It is rewarded for learning similar representations of images with the same cloud type and differing representations for images with different cloud types.

The model was evaluated on several datasets of cloud images that have been hand-labeled by past cloud classification studies and performed comparably to classification models that had been trained with human-generated labels.

Researchers also demonstrated the model’s ability to generalize between satellite instruments and explored the internal representations of clouds learned by the model by forcing the model to hallucinate images based on its own parameters.