Mystery shoppers have long played a crucial role for retailers by capturing shelf images to monitor product placement, shelf space, and brand compliance. However, manual analysis of these raw and abundant images can be a slow and erroneous task. Auditors are failing to keep up with the demand and scale of today’s dynamic retail scape. With the advancements in Generative AI (combined with Human-in-the-Loop), shelf audits can now be automated and simplified all while maintaining pixel precision with human oversight. By combining AI’s speed and human judgment, retailers can now transform thousands of raw images into actionable insights without losing time or accuracy.
Automated Product Identification
Generative AI scans the images collected by mystery shoppers to identify products, brands, sizes. Where available, the AI also scans the packaging details like ingredients or nutrition facts. In cases where the AI struggles to identify a product or any information due to atypical circumstances such as unique packaging, poor angle, or improper placement, HITL can prove to be a useful tool. Human reviewers cross verify the AI’s findings and fill in any gaps to ensure that data reflects the correct SKU. This step is imperative for inventory accuracy as the AI may not be precise for long tail SKUs, new products, or packaging changes. Generative AI with HITL helps scale the operations and avoid any false compliance alerts.
Real-Time Pricing and Promotion Verification
Generative AI extracts pricing data, promotional details, and barcode information from the shelf images collected by mystery shoppers. This collected data enables retailers to monitor compliance in real-time. While AI is sufficiently adept at handling standard situations such as formatted product tags and digital price displays, non-standard circumstances such as handwritten prices or sale signage often require human intervention. For instance, an image showing a handwritten discount tag on a product may be misread by the AI as “30% Off” instead of “50% Off.” Human analysts can spot and fix such errors to ensure promotional strategy and brand compliance. Gen AI with HITL helps prevent revenue loss from such errors and maintains operational flow during critical retail events like the Black Friday sale.
Facings, Shelf Share and Planograms: Precision at Scale
Generative AI evaluates shelf organisation by assessing product facings and verifies if the shelves comply with proposed planograms. The AI calculates the shelf-share for each brand using the images collected by mystery shoppers. However, situations can emerge where human insight is required. For instance, in a crowded cereal aisle, AI might make mistakes while counting Kellogg's facings due to improper product placement or overlapping/stacked boxes. This error in calculation may lead to inaccurate shelf-share data for the entire planogram. Human experts can take charge and adjust these figures to ensure accuracy.
Dynamic Analysis of Temporary Displays
Temporary displays like promotional setups or seasonal endcaps are made to capture the shopper's attention. Generative AI workflows analyze these displays through images collected by mystery shoppers in real-time to flag any concerns such as improper signage and outdated packaging. For instance, a seasonal launch of a skincare product may accidentally feature an old product. Generative AI detects such mismatches and human experts correct the data before it is too late. Similarly, if a “Buy One, Get One” promo is missing its tag, AI alerts the team, and humans verify whether it’s a setup error or a delayed marketing update.
Bridging AI’s Gaps with Human Expertise
Generative AI often struggles with contextual nuances. Human analysts have the capability to interpret seasonal or regional trends and ethical considerations in complex circumstances. By merging generative AI with human oversight, retailers turn mystery shopper images into a strategic asset. The result? Faster audits, fewer errors, and insights that drive smarter merchandising—all while keeping the human touch essential for trust and accuracy.
