Smart Shoppers Skipping Checkouts with AI-powered Smartphone App
Attention, Shoppers:
You can now skip the line with Nomitri’s Intelligent Shopping Assistant App.
The first of its kind, Nomitri’s app turns shopping carts into mobile check-out stands and smartphones into store clerks. A shopper slides their smartphone into a cart-mounted cradle, scans items via smartphone camera while placing them in the cart, pays virtually, and leaves.
Nomitri is a Berlin-based, high-tech startup developing embedded visual Artificial Intelligence (AI) solutions for retailers. Trinh Le-Fiedler founded the company in summer 2019 with Moritz August and Max Fiedler, two AI-specialized engineers.
Nomitri’s idea for building an autonomous self-checkout system manifested itself after Le-Fiedler visited an Amazon Go store in Los Angeles for the first time.
“I loved the fact that you could check out without having to wait in long lines, which is something that I really try to avoid when shopping with my two kids,” she recalls. “The whole system was fascinating from a technical standpoint, but it’s also very expensive and privacy-invasive. So we said, ’you know, maybe we can find another solution.’ Then suddenly, everything came together.”
Nomitri’s app uses artificial intelligence to recognize and scan items.
A Nano Neural Network?
Nomitri’s goal of creating a neural network that performs intelligent visual recognition, is small enough to fit into one smartphone, and can do everything itself without sending data into a cloud was a tall task. Recognizing the enormity of the job, Nomitri sourced tech’s best and brightest, including machine learning and computer vision experts, to explore this idea and develop a solution.
“First, we built a smartphone self-checkout solution where shoppers could turn a smartphone camera into a virtual cashier that can identify what they put in their basket. We trained the model to recognize items just by the look of them, and the results were really, really good,” Le-Fiedler states.
Although the model’s initial performance held tremendous promise, retailers pushed for more retail-friendly results.
“When we showed it to retailers, they wanted to know what exactly they were selling, whether it was a strawberry or raspberry yogurt and which brand. They needed 100% accuracy,” Le-Fiedler explains. “And this is why we integrated barcode scanning into the app because it enabled us to keep things simple and convenient for both the retailer and the shopper.”
In-app Barcode Reading & One Mighty Megabyte
Nomitri’s development team needed superior barcode scanning technology. They looked at various options and began testing speed and performance on different mobile phones and under various light conditions.
“We wanted this to be a standalone solution, whether the customer or the end-user [shopper] has a Wi-Fi connection or not,” Le-Fiedler confides. “Code was the one that offered this functionality of working offline, which we thought was great because that gives us much more flexibility; plus, the performance is always outstanding.”
Nomitri’s solution can be integrated into any retail operation as a white-label in just a few days. Taking the complexity out of the app, Nomitri’s team compressed the model into a scant 1MB—it’s lean, efficient, and doesn’t need much chip capacity and calculation power. Using Code’s CortexDecoder software development kit (SDK), which has proven to work very well, the team stopped worrying about scanning and focused their efforts on technical improvements.
A Retail Standout
From loss prevention to data privacy, several differences make Nomitri stand out compared to traditional Scan&Go apps.
“Since the app works like a virtual cashier, it sees what shoppers put into their cart or what they take out—if they forget to scan products, it notifies them and asks them to correct it,” Le-Fiedler states. “Should multiple alerts be issued, the information gets fed on the backend to a customer service representative so he or she can take appropriate action.”
Nomitri’s lightweight app works offline—no Wi-Fi needed.
Moreover, she adds that customers benefit twice with Nomitri’s Scan&Go tech. First, skipping the line with self-checkout saves at least 30 minutes of precious time on a busy Saturday morning. Secondly, and perhaps more importantly, images of the shopper are self-contained on smartphones, maintaining privacy.
“While the transactional data belongs to the retailer and helps them gain better customer insight, there is no video data ever sent to any cloud,” Le-Fiedler points out. “The solution is embedded AI, and data always stays in the shopper’s smartphone.”
Understandably, the COVID-19 pandemic has changed the demand for autonomous self-checkout systems that minimize (or eliminate) human interaction. Moreover, retailers have realized they need to provide more digitized experiences and smartphones bring lots of new possibilities for interaction with their customers.
Lately, retailers have also been struggling to create higher-value roles for frontend workers. Introducing self-scanning in stores helps free up personnel, allowing them to provide additional customer service—rather than sitting at the checkout-stand.
Download the PDF
Share This Post
Author
Paul Garcia
Many of the talented and qualified staff at Code Corporation contribute to our blog. Our team members add a unique perspective to our communications efforts, and we are fortunate to have such a great cadre of writers as part of our company.