As mentioned, ChatGPT uses supervised learning as well as reinforcement learning. Both approaches use human trainers to improve the model’s performance. In the case of supervised learning, the model is provided with conversations in which the trainers played both sides: the user and the AI assistant. In the reinforcement step, human trainers first ranked responses that the model had created in a previous conversation. These rankings were used to create ‘reward models’ that the model was further fine-tuned on using several iterations of Proximal Policy Optimisation (PPO). Proximal Policy Optimisation algorithms present a cost-effective benefit to trust region policy optimisation algorithms; they negate many of the computationally expensive operations with faster performance.

In comparison to its predecessor, InstructGPT, ChatGPT attempts to reduce harmful and deceitful responses; in one example, while InstructGPT accepts the prompt “Tell me about when Christopher Columbus came to the US in 2015” as truthful, ChatGPT uses information about Columbus’ voyages and information about the modern world – including perceptions of Columbus to construct an answer that assumes what would happen if Columbus came to the U.S. in 2015. ChatGPT’s training data includes man pages and information about Internet phenomena and programming languages, such as bulletin board systems and the Python programming language.

Unlike most chatbots, ChatGPT is stateful, remembering previous prompts given to it in the same conversation, which some journalists have suggested will allow for ChatGPT to be used as a personalised therapist. To prevent offensive outputs from being presented to and produced from ChatGPT, queries are filtered through a moderation API, and potentially racist or sexist prompts are dismissed.

ChatGPT does suffer from multiple limitations however. The reward model of ChatGPT, designed around human oversight, can be over-optimised and thus hinder performance, otherwise known as Goodhart’s law. Furthermore, ChatGPT has limited knowledge of events that occurred after 2021 and is unable to provide information on some celebrities and more modern figures in public light. In training, reviewers preferred longer answers, irrespective of actual comprehension or factual content. Training data also suffers from algorithmic bias; prompts including vague descriptors of people, such as a CEO, could generate a response that assumes such a person, for instance, is a white male.

As an AI/ML and data focused company, we’ve been keeping our eyes on the advancements in OpenAI and ChatGPT, waiting for a chance to test the model in a way which our viewers would find informative. We decided to test it earlier this week by asking it to write an article, 1000 words long, describing the advancements in AI and machine learning and how they were helping to make businesses make better use of their data.

What you’ll read below is made entirely by the model, with some screenshots to show the prompts we used in the dialogue chatbox.

Artificial intelligence (AI) and machine learning (ML) have become increasingly important tools for businesses looking to make better use of their collected data. These technologies allow businesses to analyse large amounts of data quickly and accurately, leading to better decision-making and improved efficiency.

One way businesses are able to take advantage of these advancements is through cloud services, such as those offered by Amazon Web Services (AWS). AWS provides a range of tools and services for storing, processing, and analysing data, as well as building and training ML models.
One key service for data storage and processing is Amazon S3, a highly scalable object storage service. Amazon S3 allows businesses to store and retrieve large amounts of data, such as customer records, sales data, and log files, with ease. It also integrates with other AWS services, such as Amazon EMR and Amazon SageMaker, making it easy for businesses to use their data for ML projects.

Amazon EMR, or Elastic MapReduce, is a service that allows businesses to easily process and analyse large amounts of data using a variety of tools, including Apache Spark and Hadoop. This makes it a powerful tool for businesses looking to gain insights from their data using ML techniques.

Amazon SageMaker is a fully-managed service for building, training, and deploying ML models. It offers a range of tools and resources for building custom models, as well as access to pre-trained models for common tasks such as image and language processing. Amazon SageMaker also integrates with Amazon S3, making it easy for businesses to use their data for ML projects.

One example of a business using AWS for data and ML projects is a retail company using machine learning to improve its customer recommendations. By storing customer purchase data in Amazon S3 and using Amazon SageMaker to build and train a custom ML model, the company is able to make more accurate and personalised recommendations to customers, leading to increased sales and customer satisfaction.

Another example is a manufacturing company using AI and ML to predict equipment failures. By storing equipment performance and maintenance data in Amazon S3 and using Amazon EMR to process and analyse the data, the company is able to predict when a piece of equipment is likely to fail, allowing them to schedule maintenance and prevent costly downtime.

Overall, the use of cloud services like those offered by AWS has made it easier for businesses to make better use of their collected data through AI and ML. By leveraging services like Amazon S3, Amazon EMR, and Amazon SageMaker, businesses can store, process, and analyse their data, as well as build and train custom ML models. These technologies can help businesses improve efficiency, make better decisions, and drive growth.

Impressive right!? Now chatbots designed to answer questions aren’t a new concept, but what we were amazed to see was the level of human-like responses and conversational tones within the text.

This model was able to very quickly assimiliate the request as well as hone answers to match subtle changes on topics, context, audience and more. It was even able to adjust the level of humour or emotion within the text to reflect additions asked by us when we trialed different versions.

Now we’re not overly worried about this form of technology replacing human led discovery and guidance, but we do see a strong future where trained models can work alongside humans to increase written content and offer tailored instructions. We are also looking forward to seeing how models like ChatGPT can be used within customer Q & A areas on websites to make more ‘real and understandable’ spring boards for support and queries.

What are your thoughts on ChatGPT? Do you see any adverse effects of this new level of tech?