Invented by Gary M. Zalewski, Albert S. Penilla, Individual
Machine learning methods and systems play a crucial role in making cashier-less stores a reality. These technologies utilize computer vision, sensor fusion, and artificial intelligence algorithms to track shoppers and their interactions with items in the store. By leveraging these advanced technologies, retailers can accurately monitor customer behavior, optimize inventory management, and provide personalized recommendations, all without the need for traditional checkout counters.
One of the key advantages of machine learning methods and systems in cashier-less stores is their ability to track shoppers in real-time. Computer vision algorithms analyze video feeds from cameras placed strategically throughout the store to identify and track individuals as they move around. This enables retailers to understand customer behavior patterns, such as popular areas of the store, frequently visited sections, and the duration of time spent in each area. This valuable data can be used to optimize store layouts, improve product placements, and enhance overall customer experience.
In addition to tracking shoppers, machine learning systems can also monitor interactions with items. By using computer vision and sensor fusion technologies, these systems can detect when a shopper picks up an item, puts it back, or adds it to their shopping cart. This allows retailers to maintain accurate inventory records, automatically update stock levels, and minimize the risk of theft or misplaced items. Moreover, these systems can provide real-time notifications to store associates, enabling them to restock shelves promptly and ensure a seamless shopping experience for customers.
Furthermore, machine learning methods and systems can offer personalized recommendations based on a shopper’s behavior and preferences. By analyzing past purchase history, browsing patterns, and demographic information, these systems can suggest relevant products or promotions to individual customers. This level of personalization not only enhances the shopping experience but also increases customer engagement and loyalty.
The market for machine learning methods and systems for tracking shoppers and interactions with items in cashier-less stores is expected to witness significant growth in the coming years. According to a report by MarketsandMarkets, the global market for autonomous stores is projected to reach $18.8 billion by 2026, with a compound annual growth rate of 28.5%. This growth is driven by factors such as the increasing adoption of artificial intelligence, rising demand for frictionless shopping experiences, and the need for efficient inventory management.
Several major players in the retail industry have already embraced cashier-less technology. Amazon Go, the pioneer in this space, has opened numerous cashier-less stores across the United States, utilizing machine learning methods and systems to track shoppers and their interactions with items. Other retailers, such as Walmart and Alibaba, have also started piloting similar technologies in their stores.
However, there are still challenges that need to be addressed for wider adoption of machine learning methods and systems in cashier-less stores. Privacy concerns, data security, and the need for robust algorithms that can handle complex scenarios are some of the key challenges that retailers and technology providers need to overcome.
In conclusion, the market for machine learning methods and systems for tracking shoppers and interactions with items in cashier-less stores is poised for significant growth. These technologies offer retailers the opportunity to create frictionless shopping experiences, optimize inventory management, and provide personalized recommendations. As the retail industry continues to evolve, machine learning will play an increasingly vital role in shaping the future of cashier-less stores.
The Individual invention works as follows
Systems and methods are provided to process actions in a shop. In one example, the method captures sensor outputs from two or three sensors in a retail environment. A sensor can include a first camera that captures scene data, where the scene comprises a shopper moving around the store and interacting with an object in the shop. A second camera captures at least a part of the scene in a different angle. A processing entity connected to the store processes at least one camera output in order to generate feature data. One or more machine-learning models are used to process the feature data in order to generate engineered feature information. The engineered data also includes information about the skeletal movements of shoppers. The processing entity of the store processes the engineered feature data and feature data to predict that the item was taken by the shopper. The prediction is based upon a description that said engineered feature or feature data has been labeled in a way that suggests that the shopper’s movement is an interaction with the item classified as a take.
Background for Machine Learning Methods and Systems for Tracking Shoppers and Interactions with Items in a Cashier-less Store
Over the years there have been many advances in the field of processing devices and devices which communicate over networks. Electronic devices, for example, are usually designed for specific purposes. Some devices, like the smartphone and general-purpose computers, are more versatile. Although these devices are versatile and powerful, they require network connections to send or receive data. Internet service providers (ISPs) provide network connections. These connections are available for private use (e.g. homes or businesses) and can also be obtained in public areas. Before access can be granted, users must establish connections via either their devices or interfaces.
Devices that have network access to exchange data need to be able to manage setups and provide adequate power to process the data. Unfortunately, these requirements can be a hindrance to simplifying devices that could gain from data exchanges over the internet. This is why embodiments in this disclosure are needed.
The embodiments described in this document relate to communication devices, methods of communication, and methods that utilize digital detection, tracking, and sensing systems to determine interactions between retail items and enable the processing of transactions without a cashier. In one embodiment, one or more sensors are used to track items in a shop and detect interactions. These interactions can be classified and identified to determine whether the user intends to buy the item, and to charge the account of the user. A smart retail outlet can classify shopping activities of one or multiple shoppers into categories. If a user falls under a particular category or switches categories, a dynamic response to their state is possible. In the embodiments described herein, the store can track the taking of items in the shopper’s possession as well as the detection and return of items to the shelves, and be able to identify the misplacement or placement of returned items on the wrong shelf. In one embodiment machine learning algorithms can be used to predict the shopping list of a shopper. In one configuration, the retail outlet uses input data and machine learning models to classify and track shopping behaviors and events such as returns and takes. This allows a group or family of shoppers to take and return items into a single account. The present invention also includes the capability to determine the identity of every shopper who takes or returns items from the shelves, even when several shoppers are near the shelf and interact with it at the same moment.
In one embodiment, the method for identifying actions within a retail shop is described. This method involves sampling a retail environment with one or more sensors. The method comprises receiving the output of the sample as feature inputs into a machine-learning classifier model in order to derive labels describing a state of a product. At least one entity connected to the retail store detects that the item’s state has changed from item taken to item returned. Sensor data from the one or more sensors identifies the item as being returned to the wrong location within the retail shop. In other embodiments, the items taken by a customer are added to their electronic shopping cart in order to process a transaction without a cashier.
In some embodiments, processing tracking data includes accessing deep learning models using feature input of at most user location and shopping histories. The feature input includes time-based actions characterizing past user shopping history and global positioning data that the user has enabled to enable identification of the user’s location based on the location data received from their portable device.
In some embodiments, a portable device’s current route is a predicted or actual path that the user is expected or is expected to take from one location to another location. The predicted path is calculated using either global positioning system (GPS), dead reckoning, or GPS and machine-learning to classify that the portable device’s current route is still heading to the store.
In some embodiments, instructions for creating the task to pre-gather the items include an identification of the items from the shopping lists and a time frame within which the user must enter the store or when the user will receive one of the items.
Some embodiments of sending instructions trigger based on (A), a calculation of expected time needed to gather one or several of the items and (B), a forecasted future date in which the user will either arrive in the store or need the one or multiple items ready for pickup.
In some embodiments, if the current route is based on processed global positioning system data (GPS) of the portable device and if at least one processing entity associated to the store has not received data that indicates that the user isn’t planning to visit, not likely to visit, or plans to visit the shop at a different time, the user will remain headed to it.
In some embodiments, an indicator is a payload or signal on the portable device of the user that is associated to the user account or output on an instore device or output from the store that indicates that one or more items have been ready for pickup.
In some embodiments, an indicator is a state that has been received by at least one said processing entity in connection to a status and a description one or more of the items that have been gathered using the shopping list.
In some embodiments, an indicator is an actual receipt for the pre-gathered items.
In some cases, items available for pickup can be identified when the user enters or leaves the store after arriving at the store.
In some embodiments, items ready for pickup can be assigned to the user immediately upon entry to the store or after arriving at the store or before exiting the shop.
In some embodiments, pre-collected items are items pre-designed on the user?s shopping lists or a subset thereof. They are items the user can purchase to analyze the user?s shopping history, user shopping history, or data from at minimum one processing entity that is capable of keeping track or consumption of items associated with him.
In some embodiments, an item which is already gathered or is scheduled to be gathered is removed from the shopping cart or marked on the shopping basket as being gathered.
In some embodiments, based upon receipt of data from the mobile device of the user or an associated processing entity, the user remains heading to the shop. This confirms or predicts that the user is still headed to or intends to go to the store.
In some embodiments at least one or several of the identified items or items on the shopping lists are automatically identified using a deep learning model that predicts whether one or more items will be purchased by the user.
In some embodiments, the store has tracking sensors. These tracking sensors are in communication with at minimum one of the processing entities to monitor interaction data between the user with one or more items in store. If the processing entity determines that the user has taken the item that wasn’t already pre-gathered by the user, it sends a guidance message either to the portable device or to a speaker in the store or to a display or other store assets proximate to the user.
In some embodiments, the store has tracking sensors. The tracking sensors are in communication with at most one of the processing entities to monitor a user’s location in the shop. At least one processing entity sends a wireless payload to a store clerk to indicate a user’s current location in the store. In this way, the package of one or more items ordered and gathered by the clerk is delivered to the user at the user’s current location in the stores.
In some embodiments, the store can be further configured with tracking sensor, said tracking sensors being in communication to at least one said processing entity for monitoring the location of a user in store, wherein said processing entity provides the location of the customer in store to a robot. The robot has coupled thereto a package containing said one or more items ordered from the store and collected, where the robot delivers the items at the current location of user in store.
Click here to view the patent on Google Patents.
Leave a Reply