When, if ever, is synthetic text safe, appropriate, and desirable?
The most valuable hour you can invest to gain a comprehensive understanding of language models, explained in terms that everyone can understand. Watch Professor Emily Bender, a renowned linguist, in this insightful YouTube video: ChatGP-why: When, if ever, is synthetic text safe, appropriate, and desirable?.
Outline :
- Brief overview & history of language models
- Form vs. meaning: Why language models don’t “understand”
- The race for scale: On the dangers of stochastic parrots
- Use cases for synthetic text
- Directions forward (regulation, combating AI hype)
Professor Bender is one of the author of "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?". (Stochastic Parrots : Wikipedia: Stochastic parrot)
Some important takeaways in the video:
- A large dataset is not necessarily diverse
- Human-human interaction is co-constructed and leads to a shared model of the world
- Text generated by an LM is not grounded in any communicative intent, model of the world, or model of the reader’s state of mind
- An LM is a system for haphazardly stitching together linguistic forms from its vast training data, without any reference to meaning: a stochastic parrot.
- Synthetic text can enter conversations without anyone being accountable for it
Thank you for imparting your knowledge in such a clear and insightful manner Emily M. Bender!
The presentation in the video could be downloaded here: Professor Benders presentation!