Garbage In, Garbage Out: Why your AI Is only as good as what you've documented

Most MSPs have tried AI. Most of those attempts quietly got abandoned. The problem is rarely the tool – it’s what the tool is running on. Unstructured knowledge, inconsistent documentation, and information locked in a senior tech’s head doesn’t get fixed by AI. It gets amplified. This session covers how to build AI workflows that actually hold up: starting with the knowledge foundation, building reusable skills on top of it, writing outputs back to a documentation system so they stay accurate, and measuring whether any of it is working. The worked example is a real workflow built and running in production – and yes, we build documentation software, so we have a bias. We’ll name it upfront and show you the thinking, not the tool. You’ll leave with a framework for sequencing AI adoption so it compounds over time instead of quietly dying.

Learning Objectives:

  • Recognize the specific ways AI amplifies documentation gaps – and how to spot them in your own operation
  • Build AI skills that are scoped tight, maintainable, and grounded in a structured source of truth
  • Write AI outputs back to a documentation system so knowledge compounds instead of drifting
  • Design a simple measurement loop so “is this working?” becomes a verdict based on data, not a feeling

Speaker Outline:

  1. The real failure mode: confident AI running on bad data, and why MSPs are uniquely exposed
  2. Documentation first, automation second: the sequencing most people get backwards
  3. Building skills that hold up: what to demand from whoever builds them – scope, source of truth, and handoff
  4. Write it back: why AI outputs that don’t return to a source of truth disappear
  5. The measurement loop: dated baselines and decision logs so results are verdicts, not vibes
  6. Q&A