Real-Time Hazard Detection
via Machine Vision for Retail Robots
In 2017, when a Fortune 200 company was faced with a challenge of finding a hazard detection solution for its new line of retail robots, they were referred to Volo. Given that there was no such solution available in the market at the time, the company needed to make sure that their new tech partner would have the needed capacity and experience to handle a project that was complex and research intensive. While their senior management was in Armenia with the purpose of getting acquainted with Volo to discuss a possible partnership, they shared their concerns regarding an issue they were experiencing with their line of retail robots. Having assessed Volo’s previous experience with image processing solutions, all their lingering doubts were put to rest. With deadlines pending, they turned to us for help.
The Challenge
Today’s fast-paced retail environment demands that companies invest in solutions that speed up their services and respond quickly to issues that impact their customers. One such issue is quickly detecting spills, trash, and hazards in retail stores and notifying staff to promptly take care of it. Our partner company was a provider of retail store maintenance robots. In the testing stage, they learned that their robots were experiencing issues in recognizing smaller objects (under 2 cm in height) and transparent fluids. While the robots were equipped with monocular vision to collect and sort hazard information, they weren’t able to identify and bypass these particularly challenging objects, which often resulted in their malfunction. I.e., robots would often get stuck in sticky fluids like honey, oil, etc.
Limitations
The project required us to overcome several obstacles:
- The robots needed to detect objects under 2 cm tall.
- The robots needed to recognize and detect objects in cases when there were changes in the surface/pattern of the floor.
- The robots had limited processing power and battery life, which our solution had to accommodate.
- The existing hardware of the robots was not subject to change.
The Solution
We had a three-pronged approach to finding a solution to the problem. Considering the pressing deadlines of our partner, we formed three separate teams that would work on three possible solutions simultaneously.
Machine Learning
We used machine learning algorithms to improve the trash recognition capacity of the robots. While those algorithms were successful in improving hazard detection, they required a higher processing power, which was draining battery life and decreasing the operational time of the robot.
Image Processing
Given that image processing does not achieve 100% accuracy, we came up with algorithms that would increase the robot’s performance for specific scenarios. E.g., if the pattern and texture of the floor in a particular retail store were constant, our solution provided 90-95% accuracy. The algorithm configurations could be easily adjusted and uploaded into the robot controller application and calibrated for different environments.
Our algorithm also worked without preconfiguring the pattern/texture of the floor, however with less accurate performance.
Infrared Data Processing
We developed and tested several image processing algorithms that would use the data received from the infrared camera of the robot. The plan was to combine and process data from both the infrared and the regular cameras of the robot for more accurate results. However, yet again we were hit with the robot’s hardware limitations, which didn’t allow us to make the necessary changes for complete infrared support.
Conclusion
Having tried and tested the abovementioned three approaches, we came to the conclusion that our image recognition algorithm was best suited for this particular robot hardware. We are in the constant process of tuning our algorithms, and in case of having a robot with more advanced hardware, we would be able to combine all three algorithms, which would result in very high accuracy.
Partnerships & Certificates
How Can We Meet Your Need?
How Can We Meet Your Needs?
Contact Us for a Free Consultation
Tell us a bit about your needs, and our sales team will review your information and contact you promptly to figure out the next steps together.
If you prefer email, you can reach us at business@volo.global