USA | Deep inside a Seattle building, behind a plain set of doors, lies what looks like an everyday convenience store, except this one is different.
Engineers and researchers have been solving one of retail's most complex challenges at the Just Walk Out Lab, creating a seamless and highly accurate checkout-free shopping experience.
While the lab looks exactly like a store using Just Walk Out technology, it’s a mock store environment that serves as a testing ground for Amazon's upgrades to the system.
Amazon has remained steadfast in its commitment to Just Walk Out technology. It has continued to invest in research and drive innovation because it believes frictionless, checkout-free shopping will be the future of physical retail.
The Just Walk Out technology has been offered as a service to third-party retailers. It is currently available in over 200 third-party locations at airports, stadiums, universities, hospitals, and other locations in the U.S., UK, Australia, and Canada. We will launch more Just Walk Out stores in 2024 than in any year prior.
The concept of Just Walk Out is simple—customers enter a store by tapping their credit card or mobile wallet, grab what they need, and simply leave without standing in a checkout line to pay, while the purchase is automatically charged to their payment method—the system behind it is anything but.
To solve the problem of “who took what,” Just Walk Out uses cameras, weight sensors, and a combination of advanced AI technologies to accurately determine the variety and quantity of items a customer selects and ultimately leaves the store with.
Just Walk Out associates a shopper with their payment method at the entry gate to figure out the “who” part of the “who took what” equation. It does not use biometrics-based technologies like facial recognition to identify shoppers—the system only tracks how their hand interacts with the products and fixtures (such as shelves or fridges) and correctly identifies the products and quantities they leave the store with.
If a shopper comes to the store again with another credit card, the system would not know it was the same person.
“Figuring out the ‘what’ and how many items were taken by a shopper is a challenging AI problem to solve,” said Chris Broaddus, Applied Science Amazon Web Services (AWS) senior manager.
“For example, the system needs to accurately recognise how a shopper’s hands interact with the shelves—are they picking up a product, returning it, or just rummaging within the shelf?"
To address these complexities, Just Walk Out’s new AI system simultaneously examines multiple inputs and prioritises the most important to determine the variety and quantity of items selected accurately.
It also uses continuous self-learning and transformer technology, a type of neural network architecture that turns inputs (sensor data, in the case of Just Walk Out) into tokens to generate outputs (receipts for checkout-free shopping).
The lab has allowed researchers to quickly simulate various other complex scenarios in an actual store and adjust the system as needed.
Just Walk Out engineers and researchers first test these scenarios in the lab by mapping a store’s layout, usually with Light Detection and Ranging (LiDAR) technology, which uses laser light to create detailed 3D maps of a space. This has helped researchers optimise the number of cameras needed, thereby driving down hardware costs and improving their placement to ensure comprehensive coverage of the shopping area.
Along with cameras, weight sensors have been another crucial component of the Just Walk Out system. They provide data inputs that are particularly helpful for small products (like candy bars and small packs of chewing gum, typically weighing just a few ounces) and potentially problematic to “see”. Researchers use weights to validate the accuracy of weight sensor installations on these shelves.
The Just Walk Out research team also tested this lab's new and advanced multi-modal AI model. Previously, the AI system analysed shopper behaviour sequentially, processing their movement and location in the store, what they picked up, and the quantity of each item.
The new Just Walk Out AI model is a significant advancement in checkout-free shopping. It uses the same transformer-based machine learning models underlying many generative AI applications and applies them to physical stores.
It analyses all sensor data simultaneously and supports even complex shopping scenarios with variables such as camera obstructions, lighting conditions, and the behaviour of other shoppers while allowing the team to simplify the system.
“The new multi-modal foundation model further enhances the Just Walk Out system’s capabilities by generalising more effectively to new store formats, products, and customer behaviours, which is crucial for scaling up Just Walk Out technology,” said Broaddus.
For retailers, the new AI model makes Just Walk Out faster, easier to deploy, and more efficient. For shoppers, the new model means faster receipts and worry-free shopping at even more third-party checkout-free stores worldwide.
As Just Walk Out technology scales, Amazon will continue to invest in the future of checkout-free shopping by driving research in areas like cutting-edge AI and delivering innovations that meet the needs of retailers across verticals.
