In just five days, ChatGPT has made history with one million users and capabilities that the world has never seen before. We sat down with academics and industry experts to uncover what makes this language model so revolutionary and wander forward to explore its future applications.
Some might have been waiting for a major turning point in technological advancement, a point when the Black Mirror-esque scenarios start feeling a bit too real. However, very few expected that this cutting edge chatbot would shake up the world to such an extent and potentially re-invent the way we interact with technology.
OpenAI, a frontrunner in the development of generative AI, has made waves in the global media with its latest creation – a new chatbot based on the latest version of their natural language generation model GPT-3.5, ChatGPT. This chatbot captivates audiences with its superhuman responses, it can answer almost any question with high accuracy and opens a vast number of possibilities for commercial and personal uses. According to Kate Darling, a research scientist at the MIT Media Lab, “This type of generative AI, demonstrates the current advancement of large language models that are being developed by tech giants like Google and Facebook; it is completely changing the rules of the game.”
What is so revolutionary about the ChatGPT and GPT-3.5?
We invited academic researchers of the NLP (natural language processing) and visionary founders of companies based on this technology to share their thoughts as we enter the new era of human-machine interactions.
GPT-3.5, known as a new, upgraded version of GPT-3, is the basis of ChatGPT. All GPT (Generative Pre-Trained) language models are based on Google’s 2017 Transformer model. In a paper published by Stanford University in 2021, researchers refer to the Transformer model as the “foundational model” that they believe drives the paradigm shift in AI. To put it simply, the transformer models can self-learn context and meaning by observing relationships within sequential data, such as the words of a phrase, and they are trained on large amounts of unlabeled text data.
GPT-3’s use of transformer technology takes it to new heights, partially thanks to its massive 175 billion parameters. To put this number into perspective, it’s worth noting that Google’s Transformer-based language model, BERT, has a mere 340 million parameters (we’re talking about BERTLARGE, the largest model in this series), while another well-known model, ELMo, has only 94 million.
ChatGPT demonstrates the progression of the large language models like GPT 3.5 in a dialogue format. The system operates on large text data input, including Wikipedia articles, academic papers, and books. Based on the given cues (prompts) it is able to predict the next word or symbol in the sentence and then compose the most suitable response.
ChatGPT has been designed to follow instructions prompted by users to perform a wide range of actions, from writing poems and emails to summarizing articles, with an unprecedented level of accuracy. This is largely due to the vast number of parameters and the intricate refinements made to the model. OpenAI’s introduction of ChatGPT marks a major milestone in AI. This new tool is a game-changing “product” that will shape the future of how we interact with technology.
According to Hung-yi Lee, professor of electrical engineering at National Taiwan University with a focus on machine learning and semantic understanding, these AI models also bring “conceptual innovations” to the field of natural language generation. Previously, the language models had to undergo specialized fine-tuning for every application such as translation, article writing, and letter writing, but OpenAI models challenge this notion and with GPT-3 and ChatGPT, they may have completely disrupted the world of natural language technologies.
ChatGPT has the ability to perform tasks such as translation, article writing, and even programming without the need for fine-tuning for specific purposes, which is making it a highly versatile tool. (This capability in a dialogue environment was already demonstrated by a language model, LaMDA, developed and released by Google in 2021.)
Hung-yi Lee compares the capabilities of ChatGPT to a talented martial artist who can perform a wide range of moves after seeing them just once. “There is no doubt that ChatGPT will bring about a major shift in the AI landscape,” he added.
Because of ChatGPT’s strengths and intuitive UI, fans can’t help but wonder how will this technology affect the future and what opportunities lie ahead of us.
Aiello is a Taiwanese AI company behind Aiello Voice Assistant (AVA), an in-room virtual concierge that answers guests’ questions with NLP as the core technology. To develop the next line of products, Aiello acquired the GPT-3 language model license, over a year ago. The in-house NLU team has also studied the sibling model to ChatGPT that was released before, InstructGPT. “We did expect the ChatGPT to be a very powerful tool, but we are still impressed with the actual outcome and believe that this marks the beginning of an AI revolution. When we started our business in 2019, we knew that natural language technologies would grow rapidly in the next 5 years. However, we did not see it coming this fast,” said founder and CEO Vic Shen. (This interview took place shortly after the launch of the ChatGPT.)
Sharif Ma, co-founder, and chief technology officer of Aiello said: “There are many companies that use the GPT-3 model for commercial purposes such as customer service, and FAQs, but it does not mean it is a universal solution for all problems. At present, we see that most of these businesses are using multiple language models.”
Every input of text to GPT-3 and output results has costs. OpenAI announced that for the strongest language model in this series, Davinci, it costs $0.002 to process 750 words or 1000 tokens (English vocabulary). When the number of users is unknown and the number of words for user input is unlimited, it becomes difficult to estimate the costs.
“For example, if we know that the number of inquiries processed by AVA exceeds one million per month, switching to the most advanced language model makes sense,” said Sharif Ma.
Many people think that ChatGPT will bring drastic changes to virtual customer service but if we are just talking about a simple Q&A, such as what time does the bank open and what is the maximum withdrawal limit, using a powerful but expensive GPT series is not necessary.
Although ChatGPT can already answer most questions, before it is suitable for customer service in a specific industry or enterprise, it requires additional adjustments (prompting) that guide the output. For example, when answering questions asked by customers of the XYZ bank, the model must learn the relevant information for this use case and generate responses that follow the specific rules of this bank.
“I see a bright future for GPT-3, the advancements in prompting techniques will create new business opportunities. Just by changing a few words in the instructions, the responses can differ greatly. The possibilities are endless, and we are just scratching the surface.” Hung-yi Lee said.
If you can design the right hints and rules, the sky is the limit. As mentioned earlier, GPT-3 at default is already powerful, and its performance depends on the user and how they guide the tool to the desired outcome.
“We are still a long way from reaching the limits of these large language models. To gain more clarity in this new technology we have to begin by deeply understanding today’s models and their responses. In the future, many researchers will dissect and analyze AI just like AI brain scientists,” Hung-yi Lee said.
The popularity of ChatGPT grew faster than expected, and the next turning point may not be far away, “What I am really looking forward to is GPT-4. It’s going to revolutionize this industry, there is absolutely no doubt about it,” added Vic Shen.
AI and super-large language models will indeed change our way of life. Regardless of the circumstances, whether we are predicting, discussing, or designing the future, we will always require more conversations to navigate the unknown, and this may involve conversing with one another or learning to interact with ChatGPT.
*Originally published in Chinese on bnext.com.tw on January 3, 2023
Author: 陳君毅
If you want to dive deeper into conversational AI solutions for hotels get in touch with our expert team and we’ll get back to you within 2 working days.
Do you want to learn more about what we’re building? Contact us to schedule a demo.