Tactile sensing and logical reasoning strategies aid a robot’s ability to recognize and classify objects

Today’s intelligent robots can accurately recognize many objects through vision and touch. Tactile information, obtained through sensors, along with machine learning algorithms, enables robots to identify objects previously handled.

However, sensing is often confused when presented with objects similar in size and shape, or objects unknown to the robot. Other factors restrictive to robot perception include background noise and the same type of object with different shapes and sizes.

In Applied Physics Reviews, researchers from Tsinghua University worked to break through the difficulties of robotic recognition of various common, yet complex, items.

Humans possess many different types of touch sensing, one of which is thermal feeling. This allows us to sense the wind blowing, perceive hot and cold, and discriminate between matter types, such as wood and metal, because of the different cooling sensations produced.

The researchers aimed to mimic this ability by designing a robotic tactile sensing method that incorporated thermal sensations for more robust and accurate object detection.

“We propose utilizing spatiotemporal tactile sensing during hand grasping to extend the robotic function and ability to simultaneously perceive multi-attributes of the grasped object, including thermal conductivity, thermal diffusivity, surface roughness, contact pressure, and temperature,” said author Rong Zhu.

The team created a layered sensor with material detection at the surface and pressure sensitivity at the bottom, with a porous middle layer sensitive to thermal changes. They paired this sensor with an efficient cascade classification algorithm that rules out object types in order, from easy to hard, starting with simple categories like empty cartons before moving on to orange peels or scraps of cloth.

To test the capabilities of their method, the team created an intelligent robot tactile system to sort garbage. The robot picked up a range of common trash items, including empty cartons, bread scraps, plastic bags, plastic bottles, napkins, sponges, orange peels, and expired drugs. It sorted the trash into separate containers for recyclables, food scraps, hazardous waste, and other waste.

Their system achieved a classification accuracy of 98.85% in recognizing diverse garbage objects not encountered previously. This successful garbage sorting behavior could greatly reduce human labor in real-life scenarios and provide a broad applicability for smart life technologies.

Future research in this area will focus on enhancing robotic embodied intelligence and autonomous implementation.

“In addition, by combining this sensor with brain-computer interface technology, tactile information collected by the sensor could be converted into neural signals acceptable to the human brain, re-empowering tactile perception capabilities for people with hand disabilities,” said Zhu.

Today’s intelligent robots can accurately recognize many objects through vision and touch. Tactile information, obtained through sensors, along with machine learning algorithms, enables robots to identify objects previously handled.

However, sensing is often confused when presented with objects similar in size and shape, or objects unknown to the robot. Other factors restrictive to robot perception include background noise and the same type of object with different shapes and sizes.

In Applied Physics Reviews, researchers from Tsinghua University worked to break through the difficulties of robotic recognition of various common, yet complex, items.

Humans possess many different types of touch sensing, one of which is thermal feeling. This allows us to sense the wind blowing, perceive hot and cold, and discriminate between matter types, such as wood and metal, because of the different cooling sensations produced.

The researchers aimed to mimic this ability by designing a robotic tactile sensing method that incorporated thermal sensations for more robust and accurate object detection.

“We propose utilizing spatiotemporal tactile sensing during hand grasping to extend the robotic function and ability to simultaneously perceive multi-attributes of the grasped object, including thermal conductivity, thermal diffusivity, surface roughness, contact pressure, and temperature,” said author Rong Zhu.

The team created a layered sensor with material detection at the surface and pressure sensitivity at the bottom, with a porous middle layer sensitive to thermal changes. They paired this sensor with an efficient cascade classification algorithm that rules out object types in order, from easy to hard, starting with simple categories like empty cartons before moving on to orange peels or scraps of cloth.

To test the capabilities of their method, the team created an intelligent robot tactile system to sort garbage. The robot picked up a range of common trash items, including empty cartons, bread scraps, plastic bags, plastic bottles, napkins, sponges, orange peels, and expired drugs. It sorted the trash into separate containers for recyclables, food scraps, hazardous waste, and other waste.

Their system achieved a classification accuracy of 98.85% in recognizing diverse garbage objects not encountered previously. This successful garbage sorting behavior could greatly reduce human labor in real-life scenarios and provide a broad applicability for smart life technologies.

Future research in this area will focus on enhancing robotic embodied intelligence and autonomous implementation.

“In addition, by combining this sensor with brain-computer interface technology, tactile information collected by the sensor could be converted into neural signals acceptable to the human brain, re-empowering tactile perception capabilities for people with hand disabilities,” said Zhu.