The German Tech blog heise.de writes about GPT-3, with the title: "Automated, but mediocre Articles" ("Automatisch mittelmäßige Artikel"), the original article is from TechnologyReview.com. While GPT-2/3 is really a break through in terms of language technology deep learning showcases, this kind of articles is missing the fundamental point of language: bringing information across.
Lets look at three of the underlying problems:
All three of these problems are fundamentally anchored in the end-to-end approach of GPT, so these will not go away in the next iterations just by more training data or some more "magic AI" features.
Linking to data, and annotating benefits and context are the fundamentals of information transfer - and just because an article is perfectly worded, it neither links that information to textual output, it can also not transfer that information to the reader.
This being said, language technology like this is actually really useful. It helps authors during the various stages of writing, e.g. by helping the creativity process with suggestions or variations, taking care of grammatical features like flection, generating perfect syntax or similar.
So what we are looking at is a future about Co-creation between human and machine, buzzworded as Augment Human. Just don't get distracted by machines producing shiny articles which are actually only a Potemkin village of readability.
So the Heise author misses the point: the article looks mediocre, but in reality they are just unfounded arguments, random fiction and racist, combined into fluent phrases.