The Assumption Gap
AI is very good at recognising patterns, but recognising a familiar situation does not always mean understanding the person involved. A recent conversation about a developer, a complex software system and a retained copy demonstrated this perfectly. A developer within the organisation had been asked to modify a some modues within an internal system. He appeared to have the knowledge and experience required, and when specific issues were identified, he addressed them. After completing the work, the owner suspected that the developer had kept a copy of the entire system. When asked to provide a copy of his new resource base, the developer provided one, and surly enough there was the complete file structure.. The obvious assumption was that the owner would feel betrayed or angry. That was also the conclusion AI initially reached. A valuable system, a developer and an unexpected copy naturally suggested a story about betrayal, intellectual property and loss of trust. But that wasn't how the owner actually felt. He wasn't angry that the developer had seen or learned from the system. In fact, he recognised that the developer had spent time working with something that represented years of knowledge and experience. If the developer learned from that experience, there was value in that. The real concern was security, not preventing someone from learning. Once somebody has learned something, that knowledge cannot simply be taken back. A developer who has worked inside a sophisticated system will naturally take some of that experience with them. The problem is the unnecessary copy. Computers can be lost, compromised, forgotten or accessed by someone else years later. The sensible approach is therefore to allow the learning while managing the security risk. Learn from the experience. Keep the knowledge. Protect the system. Remove unnecessary copies when they are no longer required. The most interesting part of the conversation was that the AI itself had demonstrated the problem being discussed. It predicted the most likely emotional response rather than the response of this particular person. People are not statistical averages. Two people can experience exactly the same situation and feel completely differently about it. Additional context revealed something AI could not know from the original story: trust, generosity, appreciation of learning and security concerns could all exist at the same time. AI can recognise patterns. Humans provide context. AI can predict what is likely. Individuals can still surprise it. The most valuable part of this conversation wasn't that AI initially got the situation wrong. It was discovering why it got it wrong. Sometimes the most important thing we can say to AI is simply: "No. That's not how I see it."The Assumption Gap: When AI Gets the Human Story Wrong
The Assumption AI Made
Knowledge Is Different From Possession
What AI Discovered About the Person
The Real Lesson