Manual QA doesn’t scale for modern data pipelines—too slow, too expensive, too brittle. @Hivemapper initially built a huge human based Map QA data platform with > 20,000 human editors. We deleted 85% of it this platform over the last 6 months and replaced it with various AI capabilities. There are three key parts of the new AI platform 1. AI Validators Validators are responsible for ensuring data accuracy for speed limits, gas prices, vertical height restrictions, road widths, and more. They are basically grading the homework that the Bee does, and when the Bee made an error in say a speed limit captured it as 25 mph as opposed to 35 mph then it corrects it. There are multiple models that are run and then all need to reach consensus. Here most of the AI models perform well, and we have seen excellent performance even using older models from OpenAI and Phi. At the moment, we don't recommend Claude for these vision based AI validation tasks. 2. AI Positioning This is a bit trickier, but basically we want to ensure that the objects are well positioned and the signs are associated to the correct lanes with the correct azimuth. Most of the foundational AI models do not yet have a strong sense of three dimensional space, but we expect this to change soon. Kevin Weil, CPO at OpenAI said, "The AI models you're using today are the worst AI models you'll use for the rest of your life.” Our approach here is to catch obvious positioning errors by that I mean is there a speed limit in the middle of the road, is there a utility pole inside the boundaries of a building, is there a stop sign not aligned with an intersection, etc. There are other techniques here that I will save for another post. 3. AI Re-Training The Bee is running the inference based AI models that build the map on the edge. However, we first train these models in the cloud and the key here is finding interesting edge cases that reduce future errors. For example, a 55 mph speed limit sign attached to the back of a truck as shown below is not a static speed limit sign. The context surrounding these objects tell us a lot about the objects themselves and so we are teaching the Bee's AI models to understand this context. Fundamentally, you need hundreds of millions of road km coverage to see a meaningful number of edge cases. It's a good thing Hivemapper has that.