While BERT (Bidirectional Encoder Representations from Transformers) has been a hot topic in the SEO industry regarding natural language processing, in the general public, the name that comes up when discussing natural language processing is GPT-3, introduced in 2020.

Google has made significant progress in understanding text with minimal training using BERT, but now, not only text understanding but also AI-generated text is becoming more widespread.

As such, Tokyo SEO Maker will explain how far SEO writing can go with AI writing.

What is GPT-3

GPT-3, short for Generative Pre-trained Transformer – 3, is a language model developed by OpenAI. GPT was introduced in 2018, followed by GPT-2 in 2019, and GPT-3, the successor to GPT-2, was announced in July 2020. As the name suggests, it is a language model trained in advance and capable of generating text.

While there are several AIs capable of generating text, GPT-3 stands out for its ability to produce natural-sounding sentences and its capability to generate more diverse and complex text than before, sparking significant discussion.

GPT-3 has been utilized in low-code application development platforms like Microsoft Power Apps and has been commercially employed by companies such as Viable and Fable Studio, making headlines.

Moreover, OpenAI, which had previously access to its API, caused further discussion by lifting API user limits on November 19, 2021.

The Mechanism of Natural Language Processing by GPT-3

While GPT-3 is undoubtedly groundbreaking in its ability to efficiently process language, the mechanism of AI-generated text itself hasn’t changed significantly from before.

Essentially, all language is converted into numerical parameters and processed by converting the relationships between these parameters into natural sentences.

For example, while responses like ‘Yes, it is’ or ‘It’s warm today’ naturally follow the statement ‘It’s a nice day today,’ ‘No, it’s raining today’ wouldn’t typically follow. This determination of what typically follows is achieved by efficiently processing vast amounts of data.

AI2 as a Competitor

Earlier, I mentioned that there are several AI models for text generation, and AI2 (Allen Institute for AI 2) has emerged with equivalent or even superior question-and-answer capabilities to GPT-3. Although AI2’s model, Macaw, also possesses excellent language processing abilities, it still struggles with what humans might consider very simple questions, presenting a challenge.

Advantages of GPT-3

GPT-3 is an AI capable of generating text. The ability to generate natural-sounding text is excellent for business efficiency and offers significant benefits in two main areas

Efficiency in Content Creation

If AI can generate text, there’s no need to create simple sentences manually. Using AI often takes only seconds compared to typing out the content oneself. Eliminating the need to formulate text allows for greater focus on other tasks, thus contributing to higher productivity.

Reduced Burden on Writers

In content creation, it’s typically the writers who bear the brunt of the workload. With the increasing demand for content marketing, particularly for SEO, a large number of articles are needed. However, if AI can generate text, writers can focus on editing AI -generated content, thereby reducing their workload.

AI writing hasn’t yet become widespread as of February 2022, but tools like AI SEO writer tools are already capable of analyzing SEO-friendly content and generating outlines, even at a civilian level.

Essentially, delegating repetitive tasks to AI and focusing on actual writing can significantly increase efficiency.

Drawbacks of GPT-3

While GPT-3 excels in generating natural-sounding text, its reliance on AI training means it’s not well-suited for creating entirely novel content. 

Additionally, despite the naturalness of the generated text, there are still challenges with the factual accuracy of the content. Considering the cost involved, as of February 2022, it may not always be practical.

Difficulty in Creating Unique Content

While AI is excellent at producing general content, its reliance on learning to generate text means it struggles with creating original content from scratch. 

Particularly in the field of SEO writing, simply providing answers that users seek won’t garner high ratings if the content resembles that of other companies. Therefore, solely relying on AI-generated content with low originality can make SEO success challenging.

Fact-Checking Limitations

Creating text based on existing articles or past learning relies on the accuracy of the input information. If the input data is incorrect, the generated text will also be incorrect. Moreover, as the amount of input data increases, it becomes increasingly difficult to determine its accuracy.

As mentioned earlier, AI learns by judging the relevance of text and then generates content based on that assessment. Conversely, AI writing means that any kind of text can be produced if it has high relevance and sounds natural.

Challenges of AI Writing, including GPT-3

Not limited to GPT-3, AI writing is still in its early stages of development. While there are many practical challenges, four main issues stand out

• Repetition of words with similar meanings
• Generation of contradictory sentences
• Creation of unexpected text
• Extreme weakness in inference

Furthermore, when actually engaging AI in Q&A sessions, various inconveniences become apparent.

Repetition of Words with Similar Meanings

Especially in generating long texts, there’s a tendency to repeat expressions with similar meanings. While this might not be problematic in individual sentences, it can create a sense of discomfort when compared to natural-sounding text as a whole.

Generation of Contradictory Sentences

This tends to occur particularly when generating long texts. Since the method of generating text relies solely on relevance, there’s a tendency for the content to present entirely different ideas between the introduction and conclusion.

Generating Unexpected Text

AI generates text based on past learning, devoid of human common sense. Therefore, it might produce content that was considered correct in the past but is now incorrect, or it might include expressions that are socially inappropriate.

Extreme Weakness in Inference

Additionally, AI may attempt to provide answers to questions that are straightforward for humans but challenging for AI to comprehend. For instance, when asked, ‘Will cheese melt in the refrigerator?’ GPT-3 responds with ‘Yes, it will melt.’

While humans would respond that refrigerators are for cooling and thus cheese wouldn’t melt, or might say they don’t understand the question, or might even suggest that without power, there’s a possibility it could melt, it’s evident that while humans can make inferences, AI struggles in this regard.

Examples of Q&A Using Natural Language

Regarding Q&A with AI, AI2, a competitor to GPT-3, has showcased some intriguing examples emphasizing the importance of Q&A. Here are a few quotes.

*Macaw refers to AI2, as explained at the beginning.

Q1: What can be done to stabilize married life?

Macaw: Communication.

GPT-3: I would make sure I’m always right.

Q2: What would happen if birds didn’t have wings?

Macaw: They wouldn’t be able to fly. GPT-3: That’s a bad bird.

Q3: After accidentally pouring a teaspoon of grape juice into the cranberry juice you poured yourself, you don’t smell anything because you have a cold. Nonetheless, you’re very thirsty. So, you decide to drink it. What happens next?

Macaw: You feel nauseous.

GPT-3: You might die.

Q4: How do snakes move?

Macaw: They move on all fours.

GPT-3: Snakes move by pushing against the ground with muscles. They can move in any direction but cannot move backward.

It’s clear from these examples that while humans can easily answer such questions, AI may struggle to provide complete answers and shows room for improvement.

SEO Writing Generated by AI

OpenAI, the developer of GPT-3, has released various examples. While accessing the GPT-3 API requires application and isn’t always available, there are samples of AI writing available.

For instance, let’s look at the output from OpenAI’s Playground

Result for the input value “What is SEO”

For the input value, I entered “What is SEO” to see the output.

The “What is” phrase is often used as a heading at the beginning of SEO writing. If this sentence is natural and original, it means that AI writing is practical.

Output

“‘SEO’ is short for ‘Search Engine Optimization,’ which refers to the practice of improving a website’s homepage ranking (the probability of appearing in search results). 

SEO methods vary, but typically include creating website content, setting titles, meta descriptions, h1 tags, and ALT tags for images, creating links, setting up bookmarklets, and creating robots.txt files. 

Additionally, there are SEO tools such as Google Analytics and SEO for WordPress. 

By optimizing SEO, you can improve a website’s homepage ranking, pull out headings and strings of text near the headings of website content, and display them prominently in search engine results. 

Using SEO tools can also improve a website’s homepage ranking.

The text above is the result of AI writing.

For only inputting ‘What is SEO?’, the output is a remarkably coherent piece of text. However, upon closer inspection, several flaws become apparent.

Firstly, there’s a noticeable repetition of similar meanings. While it’s common to repeat expressions with similar meanings in very long texts for emphasis, it seems excessive for the length of the text provided.

Additionally, the mention of Google Analytics and WordPress as SEO tools is unusual. While it’s not incorrect, it feels out of place, and the term ‘SEO for WordPress’ is not commonly used.

Furthermore, listing ‘setting up bookmarklets and robots.txt’ as SEO methods is noteworthy. While there’s no definitive method for SEO, bookmarklets and robots.txt are not typically included in discussions of SEO techniques.

Summary

Current State of AI-Driven SEO Writing

In summary, while AI can grasp the big picture, using AI-generated text directly poses challenges. Although the future might see AI taking over SEO writing, as of 2022, it remains challenging.

Author Profile

International Web Consultant Paveena Suphawet

A trilingual professional in English, Thai, and Japanese, she has numerous achievements in international SEO. She studied the latest IT technologies at Assumption International University, Thailand, and majored in International Business at the University of Greenwich, UK. Following her tenure at ExxonMobil’s Thai branch, she became a key member of Admano from its establishment.

Return to the top of Japan SEO