AI Can’t Connect Dots That Don’t Exist

Recently I had the opportunity to present some of the projects I've been working and discuss where AI fits into the future of technology and research.

As expected, much of the conversation centred around AI itself. Which models are the most capable? How quickly are they improving? Who or what will they replace? Is AI self-aware? Will it ever be? Where does privacy and security fit into all of this? Everyone wanted to know where it’s all going and had their own vision of the destination.

It was a great discussion, but I came away thinking that we may need to focus on a core part of this future now, which is building the (data) infrastructure foundations that allow AI to be genuinely useful in research.

Better answers start with better data

To use Healthcare as an example: it's has never suffered from a lack of data - if anything, it's the opposite problem!

Clinical records, imaging, genomics, laboratory results, wearable devices, patient-reported outcomes and disease registries all exist, but they’re often collected independently, stored in different systems/mediums and governed by different organisations. Each dataset tells part of the (patient’s) story, but very rarely do they tell the whole story together.

If we want AI to identify patterns that humans cannot easily see, we first need to create richer representations of patients and diseases by bringing these different sources together in a secure, privacy preserving and governed way - AI can’t connect dots that don’t exist.

That’s why I think infrastructure matters so much - Building trusted, secure environments where data can be collected consistently, linked securely and analysed responsibly. This is isn’t the glamorous part of research, and is needed even if AI never existed. Most people won’t ever see the work that goes into it, but it’s the bit that makes everything else possible.

In Healthcare, the stakes are different.

If AI is helping clinicians make decisions about a patient’s care, then reliability matters enormously. In many cases, it may move from being a research or productivity tool into a medical device. At that point, the question is no longer just whether the answer is useful or impressive. The question is whether it is safe, explainable, auditable and clinically appropriate.

That is where the discussion about issues like hallucinations becomes very real.

In other settings, an AI hallucination might be annoying or inefficient. In healthcare, it could mislead a clinician, overstate the evidence, miss an important caveat, or give a level of confidence that is not justified by the underlying data.

But is the answer to simply wait for a model that never makes mistakes? That may not be realistic, and in some ways it may not even be the right framing. Humans are not infallible either. Clinicians, researchers and software engineers all work within systems that assume error is possible. We use second opinions, peer review, clinical governance, validation studies, testing, audit trails and regulation because we know that judgement needs support.

It might also mean being very clear about what the AI is doing.

• Is it summarising information?
• Is it generating a hypothesis?
• Is it identifying patients who may need review?
• Is it supporting diagnosis?
• Is it recommending treatment?

Those are different levels of risk, and they may require a different solution for each of them.

This is where I think AI could borrow more from software engineering, clinical trials and safety-critical systems. We write tests, we review code, we validate models and we monitor systems after deployment. We don’t assume that because something produced an answer, the answer must be correct.

AI should not just provide an answer; it should also show its working(s), and expose uncertainty when it's not sure. Essentially, it should be possible to trace an output back to the data that produced it. And where the consequences are significant, a human expert should remain firmly in the loop.

Relatedly, I don’t think AI replaces researchers (or most/all jobs as is sometimes hypothesised), but I think it will change where we spend our time day-to-day. So much of research/work is:

• Searching for information.
• Bringing different datasets together.
• Holding many disconnected pieces of evidence in your head.

whereas;

• asking whether an answer makes sense.
• challenging assumptions and providing new insights.
• Deciding what should happen next in a research project.

are much 'better' problems to have.

By building richer datasets, better infrastructure and better ways of checking what AI tells us, AI becomes much more than an impressive chatbot - it becomes another tool that helps us understand increasingly complex diseases, while still keeping responsibility, judgement and care where they belong.