Image Recognition App For Shelf Monitoring

Uniсsoft provides an AI-powered product recognition and distribution management app to retail pharmacies in Latin America

About the client

With dozens of successful product launches but no digital apps, the Latin American healthcare company turned to Unicsoft to create an unprecedented app that combines AI and product management functionality.

The company wanted to develop an application that would radically change the way merchandisers work by delivering a higher quality of pharmaceutical retail data for better decision making.

Business challenges to solve

Panama has only recently begun to implement high-tech business solutions into the operations of its major industries. Most of the data that pharmaceutical managers and retail chain owners receive on the status of products in their brick and mortars is still collected manually by merchandisers.

Merchandisers visit dozens of retail outlets to track:

  • Specific product inventories and shelf quantities
  • Product placements
  • Brand exposure at particular retail chains

Merchandisers have to spend time manually counting goods, analyzing their demand, referencing their in-store presence with shelf placing model guidelines, and so on. These processes place a high demand on employee resources and cause a number of bottlenecks in the operational processes of an entire business:

  • Inventory gaps
  • Inability to monitor inventory rotation in each store
  • Inability to know about inventory decay in advance
  • Low productivity of sales representatives visiting the stores

The solution

To address these shortcomings, the client wanted to arm
pharmaceutical merchandisers with a mobile application that could:
Analyze and count the number of products and brands on shelves using cameras on mobile devices.
Group the collected information by brand and specific product, then display it to the manager on a separate dashboard for analysis.

The biggest challenge was to implement
a machine learning system. This system had to recognize products and then analyze the pictures taken by merchandisers in real-time. It would then identify brands, titles, and accurately count the number of products on store shelves.

A highly-skilled team of Unicsoft data scientists and engineers followed the sophisticated process to create an ML model for the product:

01
Created a dataset of images
Image Recognition App For Shelf Monitoring
02
Annotated training dataset
Image Recognition App For Shelf Monitoring
03
Trained computer vision model for object detection
Image Recognition App For Shelf Monitoring
04
Benchmarked model accuracy
Image Recognition App For Shelf Monitoring
05
Finetuned and optimized model after going live
Image Recognition App For Shelf Monitoring

Development process

Unicsoft started development from the discovery phase, which lasted about four months. The main goal of the discovery was to define a straightforward app workflow, features, and interdependencies. Within the MVP scope, Unicsoft trained the unique ML model to recognize the titles of drugs and medical products using a test dataset of five products.

Unicsoft’s UX designers were proposing design outlines that prompted the client to immediately adjust their expectations. It took multiple iterations to create a design with usability and features that satisfied the client.

The quality of the team’s communication, adherence to the predefined plan and estimates, and sticking to the project’s budget was crucial for our client. He appreciated that we provided consistent communication and a well-organized development process.

How system works

The application supports three main roles: a merchandiser, an administrator (responsible for setting up and adding users), and a manager (a person who analyzes data from merchandisers and directs logistics and distribution). The initial release of the MVP included only the most necessary functionality that implemented the following workflow:

01

The Admin sets up a company in the app. Each company can have multiple products in its portfolio. An Admin can add those products and descriptions manually or by importing data from a CSV or CRM.

02

The Admin invites users under the Manager role. These users can set product KPIs and access product analytics.

03

The app then generates an unlimited number of QR codes. Each is assigned to a specific shelf with a particular address in the store. Merchandisers can scan those codes from the shelves. That's how the app identifies the locations of products it analyzes.

04

The merchandiser is equipped with a mobile app that can scan a QR code, take a photo of a shelf, and then automatically analyze the number of products located on the shelf along with their brands.

05

All information is instantly available for the manager, who can track any product inventory on the shelf and confirm its location.

06

Using the app’s dashboard that shows inventory for dozens of stores, the manager can quickly ship any number of goods to the needed store locations based on the data displayed in the app.

The result

As a result of our close cooperation with the customer within 6 months, we are proud to outline the following project highlights:

  • Team of 9 people released the PoC on time, providing the client with all the basic functionality they had initially requested.
  • The system generates results with 90% model accuracy, perfectly acceptable for the client.
  • The project grew from an idea to a functional mobile and web application that allows the client to test their business hypotheses.
  • The ML model is ready to be tested and proven in the field, and together with the primary application, scale for any volume of goods, retail outlets, and large number of users.

On working with Unicsoft, the client noted:

The Unicsoft team's proactivity and self-motivation
The team’s ability to accurately describe their thoughts and ideas and turn them into specific technical visualizations
Exceptional professionalism and overall approach of the ML engineer
High level of appreciation for the team's soft skills.

More projects

Anti-fraud system for e-commerce users
French startup helps customers avoid scams
Image recognition for sales automation
Sales rep's work optimization through business card info recognition