Design Lunches - Generative AI
Presentation
The "Design Lunches" of the Studio Design are occasions to share experiences, practices, skills and questions.
During that second edition, we welcomed Tiphaine Viard and Samuel Huron for a common talk on the technologies and uses of generative AI. They presented the underlying technical functioning of such algorithms, its evolution through time, as well as the difference between the main models. We then discussed the uses of all the participants, in everyday life as well as in their research work (including generating ideas for research communication, getting better wordins for articles or proofreading dyslexia).
References
- Key concepts : reward system, labeler (human formulating targets or validating machine output), tokenization, transformer
- Github's awesome list about AI
- ELIZA (an early natural language processing program developed from 1964 to 1967)
- Datacamp webpage about discriminative (classification) VS generative (content making) models
- Datacamp webpage about the concept of tokenization and its challenges
- Brendan Bycroft webpage presenting some visualizations of GPT models (explaining the role of the "transformer", including both an encoder and a decoder, and trying to match a target sequence)
- Casili et al's article called "The trainer, the verifier, the imitator: Three ways in which human platform workers support artificial intelligence"
- Springer article called "ChatGPT is bullshit" (the goal is not to create verified content but believable content, and what we identify as "hallucination" are not a side effect)
- There are some free and open training datasets, such as CommonCrawl or Laion-5b
- Open WebUI is an open-source, self-hosted and offline client
- Remarks on power consumption: the references commonly quoted by Yann LeCun don't take the hardware into account (current estimations say that a request consumes 15 times more than a google query, and openAI only could consumes more than tens of thousands of homes). Other models tend to optimize power consumption, like Bloom or BigScience
- Privacy issues are widely discussed in the press and academia (NB : a former chief officer of the NSA juste joined openAI)
- Scientists start to warn about the possible "enshitification" of AI models and systems
Organizers
Studio Design
Participants and attendance
Members of all the departments of Telecom Paris are welcome