New Delhi: Google recently announced the release of its latest AI language model, PaLM 2, which competes with rival systems like OpenAI’s GPT-4. At the company’s I/O conference, Google CEO Sundar Pichai revealed that PaLM 2 models have enhanced proficiency in various text-based tasks, including reasoning, coding, and translation, thanks to broad training in those areas. PaLM 2 was trained on multilingual text from over 100 languages, which has significantly improved its performance.
According to Slav Petrov, Google’s senior research director, PaLM 2 is significantly better than its predecessor, PaLM 1, which was announced in April 2022. Petrov demonstrated how PaLM 2 understands idioms in different languages, giving the example of the German phrase “Ich verstehe nur Bahnhof,” which is better understood as “I don’t understand what you’re saying” or “it’s all Greek to me” in English.
Google’s engineers detailed PaLM 2’s capabilities in a research paper, attributing its proficiency in part to the abundance of non-English texts included in the training data. The paper noted that the system’s proficiency in a language is so high that it can be used to teach the language. PaLM 2 is available in four sizes named Gecko, Otter, Bison, and Unicorn, with different versions that will be deployed in consumer and enterprise settings.
PaLM 2 has been adapted to perform specific tasks for enterprise customers. Med-PaLM 2, trained on health data, can answer questions similar to those found on the US Medical Licensing Examination at an “expert” level, while Sec-PaLM 2, trained on cybersecurity data, can explain the behavior of potential malicious scripts and help detect threats in code. Initially, access to these models will be limited to a select group of customers through Google Cloud.
PaLM 2 is already being used to power 25 features and products within Google’s domain, including Bard, the company’s experimental chatbot, which offers updated coding features and enhanced language support. PaLM 2 is also being leveraged to enhance the functionality of Google Workspace applications such as Docs, Slides, and Sheets.
Google claims that the most lightweight version of PaLM 2, called Gecko, is compact enough to function on mobile devices, processing approximately 20 tokens per second, or roughly equivalent to 16 or 17 words. Reducing the size of language models has notable implications, as running these models in the cloud can be costly, and having the ability to use them locally can enhance privacy and offer other advantages. However, smaller variants of language models are likely to be less proficient than their larger counterparts.
PaLM 2 is part of Google’s effort to close the “AI gap” between the company and competitors like Microsoft, which has been aggressively pushing AI language tools into its suite of Office software. While PaLM 2 is a notable advancement for Google’s AI language models, it still grapples with common issues and hurdles that pervade the field of natural language processing.
One issue that experts are beginning to question is the legality of training data used to create language models. The data is usually scraped from the internet and often includes copyright-protected text and pirated ebooks. Google has not provided specific details about the sources of PaLM 2’s training corpus, noting only that it is comprised of “a diverse set of sources: web documents, books, code, mathematics, and conversational data.”
Language models also have inherent issues with their output, including the phenomenon of “hallucinations,” where the systems generate fabricated information without a basis in reality. However, Google’s efforts to develop PaLM 2 represent a significant advancement in the field of natural language processing.