This is what I learned from building a network of AI agents to do competitive intelligence to my own standard. Two years and many failures in, it changed how I think about AI, what my working day looks like, and where our profession is heading. – Alina Sharon-Green
Like most people, my journey with AI started with a Q&A in the humble chat interface. Immediately, the problems started showing up: invented facts, sources used wrongly, citations that led nowhere. Instead of deterring me, this turned into the question that would occupy me for the next two years: how much of my work as a Competitive Intelligence (CI) practitioner could I hand to AI?
What I Built: The Collection Network
I built the research process (plan, collect, analyze, deliver) into a network of AI agents. Shaped by trial and error, the architecture relies on narrow-speciality agents that collect or run specific analysis, with an overseeing orchestrator agent and multiple checkpoints, both agentic and human.
The network can be activated for different modes of operation and different depths of research to support various kinds of queries and topics. I spent hundreds of hours “downloading” my know-how and experience into the network’s resources. These include agent skills (which dictate how an agent is to perform a certain task), playbooks (concepts and standards for how various processes of collection and analysis need to work), and the context of lessons learned from previous agent and network failures.
What I Learned About AI Strengths and Weaknesses
The most useful way I have found to think about AI is as a very impressive intern: a top graduate walking into your team, sharp, fast, well-spoken, and still an intern. It reads faster than you, writes more cleanly, and never tires, but it does not arrive knowing our craft. You need to show how the work is really done: the methods, the traps, the mistakes to avoid. That inexperience runs deeper than sources and standards.
AI does not innately understand real-world relationships and importance: which signal matters more, which combination tells the real story, and which implication is most worth acting on first. Ask it for recommendations ranked by importance, and the results vary widely, sometimes sharp, sometimes confidently wrong. Would you take an intern’s answer for two months in at face value? No, you would make them show their work. The same goes for AI: it is not accountable. You are. Any mistake in the output, any gap in the process, is yours to answer for.
Common Misconceptions
Two pieces of common wisdom about AI did not survive this build. The first is some versions of the following: “AI is great at collecting information but is weak when it comes to analysis and synthesis.” The second half is fair: judgment and synthesis still need humans. The first half is what failed me, again and again. AI does not know how to collect to a CI standard, at least not natively. It must be taught.
AI is not built to search exhaustively or seek the truth. There is no reliability assessment for sources or information, and no assessment of coverage. It is built to return a fluent answer quickly, not a complete or reliable one. Folding into this misconception is the quiet assumption that AI sees the whole of the internet. It does not. Even within ungated content (no login or paywall required), a growing share of publication and content websites block AI crawlers outright, and some social media platforms do too (Longpre et al., 2024; Fletcher, 2024; Cloudflare, 2025). If you are unaware of this and do not solve it, your AI may end up bringing in a fraction of the information a human would source manually.
So, for CI, collection is also the hard part. It is where AI misses and does so quietly. The second debunked misconception is that “AI is easy.” Both the positive version (“I’ll just have AI make my job easier”) and the negative version (“Using AI is cutting corners, instantly producing slop”) are wrong. Getting AI to work for you the way you want, without sacrificing quality or accuracy, is far from easy. It takes time and effort to build, and to continuously oversee it.
Building Has Made Me a Better Analyst
To teach the intern my judgment, I first had to write it down. Playbooks, evidence standards for what counts as a source and what counts as a claim, definitions such as how many data points make a trend, and the order of the steps themselves, all had to be made explicit. Writing it down showed me what I really believe. Some working rules held up on paper; others turned out to be habit. It also sharpened the reasoning behind the craft, because explaining a step to a machine exposes every gap. It sent me back to a full, structured research process with no skipped steps. After years of shorthands, that turned out to be the best way to keep both the machine and me disciplined. It keeps both of us focused and honest.
My Work Looks Different Now
Total project time stayed the same, but what fills it has changed.
Collection now runs on its own, apart from the human checks at the gates; in some projects the whole chain runs to the finished report. To be clear, that is open-source collection. Primary and human-source work, interviews and expert conversations, stays with me, and it is where judgment matters most. To be clear, that is open-source collection. Primary and human-source work, interviews and expert conversations, stays with me, and it is where judgment matters most. The free hours go to work I always enjoyed most: testing claims, finding the real reason behind something I noticed, and working out how to present a finding with several layers of evidence under it. The result is a wider product (more sources per project) and a deeper one (more evidence under each finding), at the same time as before.
How I See CI Changing
CI is at a turning point. In my experience, the profession has rarely been firmly established as a role in corporate structures. With its value often underrated to begin with, CI is especially exposed to anything that looks like an easy, instant research solution, and to a lot of people, AI looks exactly like that.
I do not think the answer is to resist it. AI is going to change what a CI job looks like, and that is good news for the people who do it well. The parts of our work that never needed a professional, the quick search and the tidy summary, are now easier than ever.
The parts that did are worth more because AI made information abundant without making it relevant or true. Framing the question, judging sources, and deciding what finding means still belong to us. So does the step that matters most to a CI professional: turning a finding into a decision, and a decision into action.
The practitioners who thrive will be the ones who run the machine rather than tidy up after it, and, increasingly, the ones who build, or at least customize, their own. That is where the field is heading.
End notes
- Longpre et al. (2024): a peer-reviewed study from MIT’s Data Provenance Initiative audited 14,000 web domains and found that in a single year, more than one in four of the internet’s most actively maintained, highest-quality sources had shut AI crawlers out entirely. It is exactly the well-run, frequently updated, carefully sourced sites, the kind CI research depends on, that are disappearing from AI’s reach fastest. Longpre, S., Mahari, R., Lee, A., et al. (2024). Consent in Crisis: The Rapid Decline of the AI Data Commons. Advances in Neural Information Processing Systems 37 (NeurIPS 2024), Datasets and Benchmarks Track. https://arxiv.org/abs/2407.14933
- Fletcher (2024): among news publishers specifically, 48% of the most-used sites across ten countries were blocking OpenAI’s crawler by the end of 2023, rising to 57% among legacy print publishers. Fletcher, R. (2024). How Many News Websites Block AI Crawlers? Reuters Institute for the Study of Journalism, University of Oxford. https://doi.org/10.60625/risj-xm9g-ws87
- Cloudflare (2025): on Cloudflare’s network alone, more than a million site owners have turned on AI-crawler blocking since mid-2024. Cloudflare. (2025). Control content use for AI training with Cloudflare’s managed robots.txt and blocking for monetized content. Cloudflare Blog. https://blog.cloudflare.com/control-content-use-for-ai-training/
Alina Sharon-Green is an independent competitive and market intelligence specialist with 10+ years in CI, building CI programs and running hands-on research for corporates and startups in B2B SaaS, AI and hardware, enterprise software, defense and aerospace, and semiconductors. Website LinkedIn
