Deep within a nondescript building in Seattle lies a seemingly ordinary convenience store. But this is no ordinary shop—it’s a glimpse into the future of retail.
This is the Just Walk Out test lab, where engineers and researchers are hard at work perfecting one of the most complex challenges in retail: creating a seamless, highly accurate checkout-free shopping experience. This mock store environment allows Amazon to rigorously test and refine upgrades to its Just Walk Out system.
Amazon remains committed to Just Walk Out technology, seeing it as the future of physical retail. They offer this technology as a service to third-party retailers, with over 200 locations currently in operation across various venues in the U.S., UK, Australia, and Canada, and even more planned for 2024.
The premise of Just Walk Out is simple: customers enter the store by tapping a credit card or mobile wallet, grab what they need, and leave. Their purchase is automatically charged to their payment method. However, the technology behind this simplicity is anything but.
To determine “who took what,” the system relies on a combination of cameras, weight sensors, and advanced AI technologies. At the entry gate, the system links a shopper to their payment method. It doesn’t use facial recognition; instead, it tracks how a shopper interacts with products and fixtures, accurately identifying items and quantities.
Chris Broaddus, Senior Manager of Applied Science at Amazon Web Services (AWS), notes the complexity: “Figuring out the ‘what’ and how many items were taken by a shopper is a challenging AI problem to solve. For example, the system needs to accurately recognize how a shopper’s hands are interacting with the shelves—are they picking up a product, returning it, or just rummaging within the shelf?” Amazon helps the US Department of Justice thwart international cybercriminal group Anonymous Sudan.
To overcome these complexities, the new AI system considers inputs from cameras, weight sensors, and other data simultaneously, prioritizing what’s most important for accurate item identification. It uses continuous self-learning and transformer technology to convert sensor data into tokens and generate outputs, such as receipts for checkout-free shopping.
The test lab allows researchers to simulate various real-world scenarios, such as multiple people grabbing products from the same shelf or a shopper changing their mind. Customer behavior can also vary depending on store location and type. For example, sports stadium stores can be crowded with fans wearing the same team jerseys. Airport stores can be entered with travelers carrying luggage potentially obstructing cameras.
To expand checkout-free technology, radio-frequency identification (RFID) is being used for clothing, fan gear, and other items. Engineers map a store’s layout using Light Detection and Ranging (LiDAR) technology, creating detailed 3D maps. This helps optimize camera placement and reduce hardware costs.
Weight sensors provide crucial data, especially for small items. Researchers use weights to validate the accuracy of the sensors. The Just Walk Out team also tested an advanced multi-modal AI model where all sensor data is analyzed simultaneously. This new AI model builds on the same transformer-based machine-learning models used by many generative AI applications and supports even complex shopping scenarios, simplifying the overall system.
“The new multi-modal foundation model further enhances the Just Walk Out system’s capabilities by generalizing more effectively to new store formats, products, and customer behaviors, which is crucial for scaling up Just Walk Out technology,” said Broaddus.
For retailers, the new AI model makes deployment faster, easier and more efficient. These advancements translate to faster receipts and a worry-free shopping experience for customers in more third-party stores worldwide. As the technology scales further, investment in cutting-edge AI will continue so that innovations meet the needs of retailers across verticals.
