Scientists create a robotic arm capable of finding even items lost in a bag

Do you know that difficult task of finding a key inside a bag full of objects or finding the remote control forgotten in the middle of the sofa? Scientists at MIT in the US have developed a system capable of fulfilling this mission. They create a robotic arm that combines a camera with a radio frequency antenna to retrieve lost objects.

  • “Extra” robotic arm promises to help with the tasks of day to day
  • Intel robotic arm can help people with physical disabilities to write
  • Controlled robotic arm by the mind restores the touch sensation

The device called Rfusion merges the radio signals emitted by the antenna with the visual input provided by the camera attached to the arm to reach the missing item, even if it’s hidden under a pile of other objects and completely out of sight of the robot.

“This idea of ​​being able to find items in a chaotic environment is an open issue we’ve been working on for a few years. Having robots that can search for things under a pile of objects is a growing need in today’s industry,” says electrical engineering professor Fadel Adib, co-author of the study.

Want to catch up on the best tech news of the day? Go and subscribe to our new channel on youtube, Canaltech News.

Every day a summary of the main news in the tech world for you!


The prototype created by the researchers uses RFID tags (acronym for Radio-Frequency Identification or, in Portuguese, Radio Frequency Identification) that are cheap, do not need a battery to work and can be attached to an item to reflect the signals sent by an antenna.

As these RF signals can travel across most surfaces, like a lot of dirty clothes, for example, RFusion is able to identify any tagged object. A machine learning system helps the robotic arm trace the exact location and move the items above it until it finds the desired object.

“While finding lost keys is useful, RFusion it may have broader applications in the future, such as sorting batteries to fill orders in a warehouse, identifying and installing components in an automobile factory, or helping an elderly individual with daily tasks at home,” predicts Adib.

Search System

To find a lost object, RFusion uses an antenna that reflects the RFID tag signals, identifying a spherical location area. The researchers used reinforcement learning to train the neural network and optimize the robot’s trajectory to the hidden item.

Robotic arm uses radio frequency to locate objects (Image: Reproduction/MIT)

The optimization algorithm learned through a trial and error system that rewarded the machine for every hit, limiting the number of moves that were needed for locate the object and the distance traveled to pick it up. Once the system identifies the exact spot, the neural network combines the RF signals and camera information to predict how the robotic arm should grab the object and whether it needs to remove other items first.

“This is also how our brain learns when we are rewarded by our teachers or our parents. The same thing happens in reinforcement learning. We let the agent make mistakes or do something right and then punish or reward the neural network,” explains electrical engineering student Tara Boroushaki, co-author of the study.


In order not to overload the system with all the data obtained by the camera and RFID tags, the researchers based themselves on a method used by GPS devices to restrict RF measurements and visual information only to the area in front of the robot.

GPS system helps robotic arm accuracy (Image: Playback/MIT)

With this limiting approach to the tracking system, RFusion had a success rate of 96% to retrieve a fully concealed keyring in a box full of messy items or find a remote control hidden among various scattered objects s on a sofa.

“In the future, we hope to increase the speed of the system so that it can move smoothly, rather than stopping periodically to take measurements. This would allow RFusion to be deployed in an industrial environment. In addition, it could be incorporated into smart homes, helping people with household chores,” concludes Tara Boroushaki.

Source: MIT

Did you like this article?

Subscribe your email on Canaltech to receive daily updates with the latest news from the world of technology.

Related Articles

Back to top button