New algorithms track ships in harbors

The security of port areas involves monitoring at various levels. What kind of ships are coming in, are they perhaps guilty of illegal fishing, and what cargo do they carry? Security officers and harbor masters often can’t carry out these control duties all by themselves, which is why ports around the world are increasingly making use of smart surveillance systems to monitor maritime territory. TU/e researcher Amir Ghahremani developed new algorithms as well as a learning system to improve vessel identification. He will obtain his Ph.D. degree at the department of Electrical Engineering on Friday June 24.

Identifying a ship on a computer image may seem simple, but Ph.D. researcher Amir Ghahremani knows by now that nothing is what it seems. “Ships come in various shapes and travel at different speeds. And when they’re far away, all you see on the screen is a couple of vague pixels. Try to recognize a boat in that! On top of that, the water and weather conditions often make it extra difficult to identify ships. Bright sunshine causes reflection, and fog or heavy rain also reduces the quality of images. In short, a very dynamic environment.”

Giethoorn punting boat

Within the international Applying Plug & Play Surveillance (APPS) project, Ghahremani collaborated with various universities, institutes, and industrial partners from around the world on innovative algorithms to improve automated maritime surveillance systems. These systems aren’t just used to ensure the security of port areas, but also of maritime territories and busy river and canal routes.

The researchers set out to develop a robust system tailored to strongly varying conditions. However, they didn’t have visual footage to test their algorithms on, which is why Ghahremani first spent a month shooting and collecting photos. This included photos of ships in the ports of Amsterdam and Rotterdam, of boats in Turkish ports, and of his personal favorite: the village of Giethoorn, famous for its canals and boats.

But you won’t find any container ships in Giethoorn though. Instead, the canals are crowded with pleasure boats for tourists, especially during high season. Ghahremani eventually ended up in a traditional punting boat. And not without results, he admits with a smile, since it taught the Iranian born researcher with a predilection for mountains to appreciate the flat, water-rich Dutch countryside. In fact, he even became a Dutch citizen recently.

Dynamic behavior

His photos clearly show the various challenges within the maritime domain, Ghahremani explains. “Ships come in various shapes and sizes, and they move in different directions. Maritime surveillance often covers vast areas, which is why ships are sometimes located far away from a camera. And then there’s the challenge of changing weather conditions and the dynamic behavior of water.”

Ghahremani shows a picture of a ship in the middle of a perfect ocean under a bright and shining sun. “If you need to squint your eyes because of the sun, you also have more difficulty seeing things.”

Ghahremani developed vessel recognition algorithms and noticed that recognition strongly improved when it became clear where in the image water was located. “By demarcating the water region in an image, you can then zoom in on this. A car driving along the coastline will no longer be identified as a ship.”

He then started to set up a so-called neural network based on his images. Using deep learning techniques, he trained the system to help the new algorithms improve their precision in recognizing ships. And it worked. With familiar images at first, but at the end of his project with new footage as well, in which vessel recognition became increasingly accurate.

Safe haven

His self-learning algorithms didn’t just improve vessel recognition, they also proved to be of use for re-identification (re-ID) of vessels across non-overlapping camera views. “Sometimes, a ship will remerge on camera after 10 kilometers, especially in large maritime areas. The smart neural network significantly increases the chance of re-ID. This approach is new and promising. And the continuous improvement and training of the network will enable maritime surveillance systems to analyze the behavior of vessels, which will increase the security of port areas and other maritime regions.”