PaLM

PaLM, yet another language model developed by Google is trained on 540 billion parameters. This has evolved into a dense decoder-only transformer model trained with the Pathways system. It is the first one that used the Pathways system to train large-scale models with 6144 chips, along with being the largest TPU-based configuration. What makes PaLM stand apart from the rest is the fact that the model outperformed 28 out of 29 NLP tasks in English when compared to other models.