ChatGPT: What Is It? What Isn’t It?
Two-minute read
Unless you’ve been unplugged from the internet since Thanksgiving, chances are you’ve heard a lot of people talking about something called ChatGPT. Since Open AI launched the program on November 30, 2022, it has been the proverbial talk of the town in a host of different industries, from digital marketing to education.
Its proponents fervently argue it is the best thing since sliced bread, while detractors are sounding the alarm that it represents our first willful step towards a “Terminator”-esque future ruled by robots. The truth, as always, likely lies somewhere in the middle.
What Is ChatGPT?
Simply put, ChatGPT is an AI chatbot built using training and machine learning. It utilizes sophisticated natural language processing to execute an array of prompts, such as summarizing information, basic coding, and writing essays.
To break down this concept in greater detail we asked our CTO, Vin DiPippo, to explain what ChatGPT is from his perspective:
“Ecclesiastes chapter 1 verse 9 says it best, ‘What has been will be again, what has been done will be done again; there is nothing new under the sun.’
What ChatGPT embodies are the qualities of a student of history. Through a complex set of heuristics and statistics, it can parse language and apply what it has learned from a vast amount of ‘what has been’ to produce for you something that ‘will be again.’”
What Is ChatGPT Good At?
ChatGPT is pretty good at general input-output commands. For example, inputting one or more source texts and asking it to output summaries. In many cases, it is capable of producing summaries that are accurate and realistic in tone and style.
There have been multiple instances already of educators putting ChatGPT’s writing prowess to the test. A private school in the U.K. recently conducted a blind grading test, in which a ChatGPT-written essay scored 85%.1 Stateside, the chatbot similarly scored between a B- and a B on an MBA exam administered by a professor at the University of Pennsylvania.2
What Is ChatGPT Bad At?
Although ChatGPT can write in a way that appears natural, it cannot yet write in a way that could be considered human-like. It lacks the cognition of a human that allows us to combine new information with our prior knowledge in a novel way while working within a set of constraints.
In a case study published by bioRxiv, researchers tasked ChatGPT with generating abstracts for 10 scientific papers from five different publications.3 ChatGPT-generated abstracts and the papers’ original abstracts were then analyzed side by side using several modalities, including AI detection software and human reviewers.
Only 8% of the ChatGPT-generated abstracts followed the journal’s formatting requirements. AI detection determined that it was 99.98% likely the generated abstracts were written by AI, with the original abstracts scoring 0.02%. What’s more, the human reviewers correctly identified 68% of the abstracts written by ChatGPT, stating, “the generated abstracts were vaguer and had a formulaic feel to the writing.”
How Can Digital Marketers Use ChatGPT?
In short, digital marketers should use ChatGPT sparingly. ChatGPT in its current iteration is a great tool to help you develop content, but not to author it.
In April 2022, Google’s Search Advocate, John Mueller explicitly stated that AI-written content violates its guidelines and will be penalized accordingly.4 However, in February 2023, Google walked back that statement with an update to its Search guidelines.5 In this article, Google explained that AI-generated content will only be penalized if it is crafted “with the primary purpose of manipulating ranking in search results.” AI-generated content will otherwise be rated the same as human-generated content, based on the content’s perceived E-E-A-T (Experience, Expertise, Authoritativeness, and Trustworthiness). And given that AI-generated writing is based solely on second-hand knowledge, the output will not have the same level of expertise, authoritativeness, and trustworthiness as a human author who is writing from a perspective of lived experience.
To maximize a piece of content’s E-E-A-T, ChatGPT could most effectively be utilized by digital marketers in the ideation stage of content writing. The system’s ability to quickly and accurately summarize source texts will allow you to determine in a matter of minutes if, and how, you can incorporate an article into your work.
In that sense, ChatGPT is more C-3PO than T-800 – a benevolent, slightly human-sounding AI system that we can collaborate with toward an end goal.
Are you looking to improve your digital presence with leading edge technology? Brave River can help. Contact us to learn what our digital marketing team can do for you.
1 https://www.businessinsider.com/chatgpt-helps-move-top-uk-private-school-to-ditch-homework-2023-1
2 https://www.nbcnews.com/tech/tech-news/chatgpt-passes-mba-exam-wharton-professor-rcna67036
3 https://www.biorxiv.org/content/10.1101/2022.12.23.521610v1.abstract
4 https://www.searchenginejournal.com/google-says-ai-generated-content-is-against-guidelines/444916/
5 https://developers.google.com/search/blog/2023/02/google-search-and-ai-content#rewarding-high-quality-content,-however-it-is-produced