# Machine learning

## Models

* <https://arxiv.org/abs/1907.07355> Probing Neural Network Comprehension of Natural Language Arguments
* <https://arxiv.org/abs/1902.01007> Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference

## GPU

**What is GPU** <https://course.fast.ai/gpu_tutorial.html>

### Cost

* <https://twitter.com/eturner303/status/1143174828804857856> Holy crap: It costs $245,000 to train the XLNet model (the one that's beating BERT on NLP tasks..512 TPU v3 chips *2.5 days* $8 a TPU -see the above thread
* The cost of training MT models: <https://medium.com/syncedreview/the-staggering-cost-of-training-sota-ai-models-e329e80fa82>
* Mozilla's struggles with Baidu model training - hardware issues
  * <https://blog.mozilla.org/blog/2017/11/29/announcing-the-initial-release-of-mozillas-open-source-speech-recognition-model-and-voice-dataset/>
  * <https://hacks.mozilla.org/2017/11/a-journey-to-10-word-error-rate/>

## Ethics

Open AI and their ethical(?) release of model that can create fake news <https://openai.com/blog/better-language-models/#task2>

* news <https://techxplore.com/news/2019-02-openai-gpt-algorithm-good-fake.html>
* criticism <https://www.theregister.co.uk/2019/02/14/open_ai_language_bot/>
* <https://thegradient.pub/openai-please-open-source-your-language-model/>
* [GPT‑3: Its Nature, Scope, Limits, and Consequences Luciano Floridi1,2 · Massimo Chiriatti](https://link.springer.com/epdf/10.1007/s11023-020-09548-1?sharing_token=y3U0nDiQ_Vs_czk-OjqKsfe4RwlQNchNByi7wbcMAY6dl9yKaSKy9pZ9jIb5-fNBNoGfcfNJqJ36XsZSeuznP5ZbVUrOkiBUJHJv5qxKhISNBh56enqR2qbuFaFXrV4qVpLFWP_5Ai23WhvSt6YVNLlOB92FvmjwHHp-s3VRPvU%3D)