How Powerful New AI Systems Like ChatGPT Work

Generative artificial intelligence (AI) has taken the tech world by storm, with headlines featuring the remarkable capabilities of systems like OpenAI’s ChatGPT, which can produce text almost indistinguishable from human writing. But what exactly is generative AI, and how does it differ from traditional AI models? 

In the not-so-distant past, discussions about AI primarily revolved around predictive models. These models were trained to make predictions based on data, whether identifying medical conditions from X-rays or assessing the creditworthiness of borrowers. They learned from vast datasets to make informed decisions about specific tasks.

Generative AI, on the other hand, represents a departure from prediction-focused AI. It is a machine-learning model designed to create entirely new data rather than predict outcomes within a predefined dataset. A generative AI system learns to produce data that resembles the training data it was exposed to. While the lines between generative and predictive AI can blur, they represent distinct approaches to AI.

From Markov Chains to complex models

The roots of generative AI can be traced back to simpler models like Markov chains, named after Russian mathematician Andrey Markov, who introduced this statistical method in 1906. Markov models were initially employed for tasks like next-word prediction, akin to the autocomplete feature in email programs. However, their limited ability to consider only a few preceding words made them less proficient at generating coherent text.

Recent years have shifted towards more complex deep-learning architectures, underpinning the generative AI boom. One significant breakthrough was the introduction of Generative Adversarial Networks (GANs) in 2014, enabling the generation of realistic outputs by pitting two models against each other—one generates the target output (e.g., images), while the other distinguishes real data from generated data.

Additionally, diffusion models, introduced in 2015, iteratively refine their output to create data samples that closely resemble those in the training dataset. The transformer architecture, introduced by Google in 2017, plays a pivotal role in large language models such as ChatGPT. It encodes text as tokens and utilizes attention maps to understand context when generating new text.

These advancements in AI architectures have propelled the generative AI field to new heights.

The Token Transformation: A Common Thread

A defining characteristic of generative AI approaches is converting input data into tokens—numerical representations of data chunks. As long as data can be converted into this standardized token format, generative AI methods can be applied to create new data with similar characteristics. This versatility opens the door to a wide array of applications.

For instance, researchers are using generative AI to create synthetic image data for training computer vision models. These models can learn to recognize objects and patterns more effectively.

In another domain, generative AI designs novel protein and crystal structures, which can be crucial in developing new materials and drugs. By learning the dependencies in the data, these models can propose stable and realizable structures.

However, it’s important to note that generative AI may not excel in all data-related tasks, especially those involving structured data like spreadsheets. Traditional machine-learning methods may outperform generative models in such scenarios.

A bridge between humans and machines

One of the most significant values of generative AI lies in its ability to serve as an interface between humans and machines. In the past, humans had to communicate with machines using specialized languages. Generative AI has bridged this gap by enabling machines to understand and communicate in more human-friendly terms. This development has far-reaching implications for human-computer interactions.

While generative AI holds immense promise, it raises important challenges and ethical considerations. For instance, its implementation in call centers to handle customer queries may lead to worker displacement. Additionally, generative AI models can inherit and perpetuate biases in training data, propagate hate speech and false information, and even generate content that closely mimics specific human creators, potentially raising copyright issues.

On the positive side, generative AI has the potential to empower artists by providing tools for creative content generation that they might not have had access to otherwise.

The future of generative AI

Generative AI is poised to bring about significant changes in various fields. One promising application is in fabrication, where generative AI could go beyond creating images and design plans for physical objects, revolutionizing manufacturing processes.

Generative AI also holds the key to developing more intelligent AI agents. At the same time, differences exist between how these models work and human brain function; they share similarities in their ability to generate novel ideas and plans. Generative AI could be pivotal in empowering AI agents to think creatively.

Generative AI represents a remarkable advancement in artificial intelligence, capable of creating data and bridging the gap between humans and machines. While it brings opportunities and challenges, its potential to reshape industries and foster innovation is undeniable. As generative AI continues to evolve, its impact on our world is set to grow exponentially.