The AI That’s Too Dangerous to Release

Apart from another fancy acronym, GPT-2 brought along somewhat coherent (semantically, at least) language generation capabilities, some semblance of hope for zero-shot transfer learning, and a transformer network  trained with approximately 1.5 billion parameters on a text corpus with over 40 gigabytes of internet wisdom. You see, what made BERT so great was that it used what’s called a bidirectional language model, as opposed to GPT’s unidirectional language model. Both methods aim to create large and powerful pretrained language models that are useful in a transfer learning context, but GPT-2 had a hidden superpower that was guaranteed to drive media outlets wild — writing articles about talking, four-horned, half-breed unicorns in the Andes…

If you’ve read this article completely and still feel like you don’t have GPT-2 engrained in your bones, fear not.

Source: blog.floydhub.com