But that’s not true of modern generative AI systems like ChatGPT, DALL-E, and MidJourney. They are incredibly useful tools that can imagine and reimagine by using free association and lateral thinking.

I asked ChatGPT to give me ways that a proton was like a sneeze, and in seconds, it came up with an answer that would have taken me an afternoon to resolve.

For the first time in history, we have computers that can think outside of their box. But here’s the kicker: it’s also a box that dreams, imagines, and even “hallucinates”.

AI hallucinates in technicolor.

The double-edged sword of generative AI

The concept of “unreliable computers” might initially strike as an oxymoron. These systems have incredible power to generate new content, ideas, and solutions in text, images, audio, and code – with video and 3D rapidly on the rise. But with that creative capability comes a tendency to hallucinate – to make up facts and information that have no basis in reality.

Hallucination and generation are two sides of the same coin. According to OpenAI CEO Sam Altman, hallucinations are a feature, not a bug. If gen AI bots were too accurate, they would just repeat the training data they’d seen before. Creativity requires a little crazy. But crazy means unreliability.

Did you know that ChatGPT can just barely count? Up until a few weeks ago if you gave it some text and asked it to count the words, it failed every time. If you asked it today to generate a 300-word bio of a famous person, it would still fail. (I just got 331 words). It’s a computer system that’s bad with numbers.

Its image-generating sister model, DALL-E, is quite hilariously bad, but still claims confidently that it isn’t. Check out its artistry below.

That’s a lot more than three monkeys, you crazy monkey of an AI!

I recently posted on LinkedIn about the Microsoft Future of Work Report, which said “knowledge workers are 37% faster, 40% higher quality but ~20% less accurate” using ChatGPT. I still can’t read that without laughing out loud because it’s so bizarre. How did they separate quality from accuracy?

Still, OpenAI’s leadership in the GPT market hasn’t come without hiccups. Every day, my email inbox is flooded with messages from the bellwether tech company about ChatGPT’s elevated error rates and unstable performance.

These were all the OpenAI error alerts I could fit on one screen.

Reframing how we see computers

Gen AI turns the whole notion of computing on its head. It was like waking up to find the world’s legion of dependable accountants had morphed overnight into a troupe of whimsical artists, splashing paint across our digital landscape.

The challenge today lies in integrating the “wacky artists” into our organisations and workflows in a way that understands the technology’s challenges and leverages its strengths. An artist can change the world, even if they’re not the best at spreadsheets.

To try to set the right context, I often say, “Imagine you were just given a team of 10,000 interns fresh out of school. You could get an incredible amount done with them, but can you trust them?” Not without checks and balances, you can’t.

It’s this graceful dance of leveraging the intern bots’ brute-force power to create and innovate, but to also build processes that account for the reality of how they work.

So what do we do?

Where are the business processes that could benefit from having an unreliable computer generate ideas and content? I’ve encouraged clients to find those niches in their business processes where a sprinkle of unpredictability isn’t just tolerated; it’s welcome. Some tasks are perfect matches for this technology. AI is great for work such as:

• creating and drafting
• designing and planning
• modelling metadata (at the exploration phase)
• simulating personas (to drive requirements of all kinds)
• brainstorming, and
• journey mapping and designing experiences.

Use AI to give yourself a promotion

With 10,000 virtual interns on your team, you’re effectively the leader of a workforce. Give yourself a promotion! All you have to do is establish the processes, templates, and guardrails that will allow your team to produce the best output.

Many in the industry are comparing this to Daniel Kahneman’s Systems 1 and 2 thinking from one of my all-time favourite books Thinking Fast and Slow. In it, he discusses how our minds use two types of processing.

• System 1 is our “fast system”. It’s very much like a generative AI: extremely fast, with access to huge amounts of our unconscious bank of aggregate memories. But it’s often irrational and instinctual. It’s the thinking that experienced motorists use to drive their car; that virtuoso violinists use to play their instrument; that elite athletes use to compete in their sport; and that, sadly, racists use to make their snap irrational judgements. It’s a workhorse system designed for speed. It decides which way to dodge when we’re being attacked by a lion or on the receiving end of a spear. Quality is absolutely job 2.
• System 2 is our “slow system”. It’s the brain system where we do reflection, analysis, and strategy. Here we mull over options and compare attributes. Ironically, our conscious reflective mind has far worse access to our accumulated memories. When you’re making a complex strategic decision, you can’t remember the tens of thousands of experiences, significant and trivial, that are shaping your thoughts. You can remember only the main key narrative points and use them to make an executive conclusion.

This executive/worker relationship is a nice metaphor for the human/AI relationship: You, or the humans on your team, are the executives who lead, instruct, and correct your team of fast – but flaky – interns.

Ensure quality, not chaos: the best of both worlds

The key is learning to combine the strengths of both old and new technology. As I’ve said before, “Traditional software can handle your accounts beautifully and perfectly. It cannot do what gen AI does, and vice versa.”

We need to bring together the structure and reliability of traditional computing with the unpredictable creativity of modern AI. When we learn how to build these complementary strengths into our teams and workflows, we’ll see the true potential of this technology revolution.

The first way to get good results from gen AI is to prompt it with good, consistent, structured commands. We’ve already given away our ucLoops guide with some examples of how to do this.

Beyond prompting AI with good instructions, you also need to pair and integrate gen AI with its older, more mature cousins. Although gen AI has co-opted the general term “AI”, the other types of AI we’ve been using for years are stable and complement the “youthful exuberance” of gen AI.

For example:

• Autocomplete. Love it or hate it, you probably use it every day. From text messaging to emails, computers are finishing our sentences and nudging us towards what we could do next. This was in fact the first mass-market example of generative AI at work.
• Knowledge representation and modelling. The often weird and wonderful worlds of taxonomy, ontology, knowledge graph databases, and structured content are more important than ever. They give gen AI the structure and rules it needs to understand specific domains like pharma, finance, and manufacturing. Hallucinations may still occur, but their numbers drop dramatically when the AI accesses the structured input of a knowledge graph, reducing the AI’s reliance on its nonexistent intuition.
• Autotaggers and categorisers. These tools make sense of our unstructured data by leveraging things like taxonomies and ontologies to tag the content that humans didn’t tag or mistagged. They work on text, video, and almost anything else that’s digital. And, again, they work best if you’ve properly structured their reference models and training data.
• Machine translation. Once a joke, now machines can do quite a decent job of translation between many languages, and these features are getting integrated in daily life across everything from your email inbox to Airbnb reviews and chats between hosts and guests.

All these are AI bots that have been proven and stabilised over the years and can be used in workflows to support generative AI to produce more reliable solutions for your enterprise.

Support AI quality with content

The process works the other way as well. AI bots that are trained on high-quality content outperform generic gen AI by a huge margin. “High-quality” means

• well written
• well structured, and
• very importantly, well tagged.

Sébastien Bubec, a former senior principal research manager at the Machine Learning Foundations group at Microsoft Research, said that structuring content to a point of what he called “textbook quality” gave order-of-magnitude improvements to gen AI accuracy:

“Before working on this kind of work I was focusing on optimisation and improving the architecture. I worked on this for a few years and we could get 2%, 3% improvement. Small, around the edges, is nice but it’s tiny.

Suddenly when we focused around the data and focused on really trying to craft data in a way that’s more digestible by the LLM at training time, suddenly we saw these incredible 1000x gains. I think it’s massive and really pointing to where the gold is. The gold is in the data.”

Almost a decade ago, before I had ever heard of OpenAI, I was working with Barclays, one of Europe’s largest banks. Their business infrastructure lead told me that

“[o]f all the efforts we made to optimise the success rates of the AI in our chatbots, the main thing that moved the needle was improving the taxonomy tagging of the source content.”

A study by data.world showed that a structured database significantly improved performance on natural-language business questions. Sometimes success was still less than 50%, but that was up from 0% with a nonstructured database.

When AI uses a knowledge graph to respond to a complex question, it’s 38.7% more accurate than when it uses a SQL database.

Moving forward with AI-aligned workflows

As we navigate this new landscape, it’s crucial to remain adaptive and open-minded. The integration of generative AI into our digital ecosystem represents a shift towards a more dynamic, creative partnership with technology. It challenges us to rethink our roles and processes, to imagine a future where humans and AI collaborate in unprecedented ways.

The future will be about finding harmony between the structured and the unpredictable, the accountants and the interior decorators of our digital worlds. Or, for meganerds like me, the Newtonian and the quantum.

The journey into the world of gen AI is about finding the sweet spot where creativity and unpredictability intersect with our needs and aspirations. The question isn’t just about what AI can do for us, but how we can evolve our mental models alongside it to unlock the vast new possibilities.

Stay tuned to stay informed

This is the fastest-moving tech that humans have ever seen. As fast as the computer revolution was, AI is moving about 5 to 7 times faster. This is an acceleration rate that the human mind (at least this human mind and most of the ones I’ve talked to about it) struggles to comprehend.

Apparently, Google has in the works systems that will reduce hallucination drastically. I have no doubt OpenAI has the same. The ideal world is one where we have a “creativity dial” that we can turn up when we want creativity and turn down when we want accuracy.

It’s not yet on the open market, and I still hold firm on the idea that any intelligence, artificial or not, will always do better with tidier inputs. But in any case, we’re in for a wild ride.

Get in touch if you want to discuss how Urbina Consulting can help you make the most of AI in your business to maximise creativity and reliability, or – ideally – both.