S., Patil S., Dettmers T., Baruwa A., Singh A., Cheveleva A., Ligozat A., Subramonian A., Névéol A., Lovering C., Garrette D., Tunuguntla D., Reiter E., Taktasheva E., Voloshina E., Bogdanov E., Winata G. I., Schoelkopf H., Kalo J., Novikova J., Forde J. Z., Clive J., Kasai J., Kawamura K., Hazan L., Carpuat M., Clinciu M., Kim N., Cheng N., Serikov O., Antverg O., Wal O. v. d., Zhang R., Zhang R., Gehrmann S., Mirkin S., Pais S., Shavrina T., Scialom T., Yun T., Limisiewicz T., Rieser V., Protasov V., Mikhailov V., Pruksachatkun Y., Belinkov Y., Bamberger Z., Kasner Z., Rueda A., Pestana A., Feizpour A., Khan A., Faranak A., Santos A., Hevia A., Unldreaj A., Aghagol A., Abdollahi A., Tammour A., HajiHosseini A., Behroozi B., Ajibade B., Saxena B., Ferrandis C. M., Contractor D., Lansky D., David D., Kiela D., Nguyen D. A., Tan E., Baylor E., Ozoani E., Mirza F., Ononiwu F., Rezanejad H., Jones H., Bhattacharya I., Solaiman I., Sedenko I., Nejadgholi I., Passmore J., Seltzer J., Sanz J. B., Dutra L., Samagaio M., Elbadri M., Mieskes M., Gerchick M., Akinlolu M., McKenna M., Qiu M., Ghauri M., Burynok M., Abrar N., Rajani N., Elkott N., Fahmy N., Samuel O., An R., Kromann R., Hao R., Alizadeh S., Shubber S., Wang S., Roy S., Viguier S., Le T., Oyebade T., Le T., Yang Y., Nguyen Z., Kashyap A. R., Palasciano A., Callahan A., Shukla A., Miranda-Escalada A., Singh A., Beilharz B., Wang B., Brito C., Zhou C., Jain C., Xu C., Fourrier C., Periñán D. L., Molano D., Yu D., Manjavacas E., Barth F., Fuhrimann F., Altay G., Bayrak G., Burns G., Vrabec H. U., Bello I., Dash I., Kang J., Giorgi J., Golde J., Posada J. D., Sivaraman K. R., Bulchandani L., Liu L., Shinzato L., Bykhovetz M. H. d., Takeuchi M., Pàmies M., Castillo M. A., Nezhurina M., Sänger M., Samwald M., Cullan M., Weinberg M., Wolf M. D., Mihaljcic M., Liu M., Freidank M., Kang M., Seelam N., Dahlberg N., Broad N. M., Muellner N., Fung P., Haller P., Chandrasekhar R., Eisenberg R., Martin R., Canalli R., Su R., Su R., Cahyawijaya S., Garda S., Deshmukh S. S., Mishra S., Kiblawi S., Ott S., Sang-aroonsiri S., Kumar S., Schweter S., Bharati S., Laud T., Gigant T., Kainuma T., Kusa W., Labrak Y., Bajaj Y. S., Venkatraman Y., Xu Y., Xu Y., Xu Y., Tan Z., Xie Z., Ye Z., Bras M., Belkada Y., Wolf T. (2022). BLOOM: A 176B-Parameter Open-Access Multilingual Language Model // https://arxiv.org/abs/2211.05100

2625

Zhang S., Roller S., Goyal N., Artetxe M., Chen M., Chen S., Dewan C., Diab M., Li X., Lin X. V., Mihaylov T., Ott M., Shleifer S., Shuster K., Simig D., Koura P. S., Sridhar A., Wang T., Zettlemoyer L. (2022). OPT: Open Pre-trained Transformer Language Models // https://arxiv.org/abs/2205.01068

2626

Zhang S., Diab M., Zettlemoyer L. (2022). Democratizing access to large-scale language models with OPT-175B / Meta AI, May 3, 2022 // https://ai.facebook.com/blog/democratizing-access-to-large-scale-language-models-with-opt-175b/

2627

Taylor R., Kardas M., Cucurull G., Scialom T., Hartshorn A., Saravia E., Poulton A., Kerkez V., Stojnic R. (2022). Galactica: A Large Language Model for Science // https://arxiv.org/abs/2211.09085

2628

AI21 Labs Makes Language AI Applications Accessible to Broader Audience (2021) / businesswire: a Berkshire Hathaway Company, August 11, 2021 // https://www.businesswire.com/news/home/20210811005033/en/AI21-Labs-Makes-Language-AI-Applications-Accessible-to-Broader-Audience

2629

Rae J., Irving G., Weidinger L. (2021). Language modelling at scale: Gopher, ethical considerations, and retrieval / DeepMind blog, 08 Dec 2021 // https://deepmind.com/blog/article/language-modelling-at-scale

2630

Rae J. W., Borgeaud S., Cai T., Millican K., Hoffmann J., Song F., Aslanides J., Henderson S., Ring R., Young S., Rutherford E., Hennigan T., Menick J., Cassirer A., Powell R., Driessche G. v. d., Hendricks L. A., Rauh M., Huang P., Glaese A., Welbl J., Dathathri S., Huang S., Uesato J., Mellor J., Higgins I., Creswell A., McAleese N., Wu A., Elsen E., Jayakumar S., Buchatskaya E., Budden D., Sutherland E., Simonyan K., Paganini M., Sifre L., Martens L., Li X. L., Kuncoro A., Nematzadeh A., Gribovskaya E., Donato D., Lazaridou A., Mensch A., Lespiau J., Tsimpoukelli M., Grigorev N., Fritz D., Sottiaux T., Pajarskas M., Pohlen T., Gong Z., Toyama D., d'Autume C. d. M., Li Y., Terzi T., Mikulik V., Babuschkin I., Clark A., Casas D. d. L., Guy A., Jones C., Bradbury J., Johnson M., Hechtman B., Weidinger L., Gabriel I., Isaac W., Lockhart E., Osindero S., Rimell L., Dyer C., Vinyals O., Ayoub K., Stanway J., Bennett L., Hassabis D., Kavukcuoglu K., Irving G. (2021). Scaling Language Models: Methods, Analysis & Insights from Training Gopher // https://arxiv.org/abs/2112.11446

2631

Kaplan J., McCandlish S., Henighan T., Brown T. B., Chess B., Child R., Gray S., Radford A., Wu J., Amodei D. (2020). Scaling Laws for Neural Language Models // https://arxiv.org/abs/2001.08361

2632

Hoffmann J., Borgeaud S., Mensch A., Sifre L. (2022). An empirical analysis of compute-optimal large language model training / DeepMind blog, April 12, 2022 // https://www.deepmind.com/publications/an-empirical-analysis-of-compute-optimal-large-language-model-training

2633

Hoffmann J., Borgeaud S., Mensch A., Buchatskaya E., Cai T., Rutherford E., de Las Casas D., Hendricks L. A., Welbl J., Clark A., Hennigan T., Noland E., Millican K., van den Driessche G., Damoc B., Guy A., Osindero S., Simonyan K., Elsen E., Rae J. W., Vinyals O., Sifre L. (2022). Training Compute-Optimal Large Language Models // https://arxiv.org/abs/2203.15556

2634

Pichai S. (2023). Google DeepMind: Bringing together two world-class AI teams. / Google Blog, Apr 20, 2023 // https://blog.google/technology/ai/april-ai-update/

2635

Chowdhery A., Narang S., Devlin J., Bosma M., Mishra G., Roberts A., Barham P., Chung H. W., Sutton C., Gehrmann S., Schuh P., Shi K., Tsvyashchenko S., Maynez J., Rao A., Barnes P., Tay Y., Shazeer N., Prabhakaran V., Reif E., Du