Machine learning

Models

• https://arxiv.org/abs/1907.07355 Probing Neural Network Comprehension of Natural Language Arguments

• https://arxiv.org/abs/1902.01007 Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference

GPU

What is GPU https://course.fast.ai/gpu_tutorial.html

Cost

• https://twitter.com/eturner303/status/1143174828804857856 Holy crap: It costs $245,000 to train the XLNet model (the one that's beating BERT on NLP tasks..512 TPU v3 chips 2.5 days $8 a TPU -see the above thread

• The cost of training MT models: https://medium.com/syncedreview/the-staggering-cost-of-training-sota-ai-models-e329e80fa82

• Mozilla's struggles with Baidu model training - hardware issues

  • https://blog.mozilla.org/blog/2017/11/29/announcing-the-initial-release-of-mozillas-open-source-speech-recognition-model-and-voice-dataset/

  • https://hacks.mozilla.org/2017/11/a-journey-to-10-word-error-rate/

Ethics

Open AI and their ethical(?) release of model that can create fake news https://openai.com/blog/better-language-models/#task2

• news https://techxplore.com/news/2019-02-openai-gpt-algorithm-good-fake.html

• criticism https://www.theregister.co.uk/2019/02/14/open_ai_language_bot/

• https://thegradient.pub/openai-please-open-source-your-language-model/

• GPT‑3: Its Nature, Scope, Limits, and Consequences Luciano Floridi1,2 · Massimo Chiriatti