Meta is going all in on open-source AI. The company is today unveiling Meta’s latest AI model, LLaMA 2. Also, LLaMA 2 is Meta’s first large language model that’s available for anyone to use—for free.
Since OpenAI released its hugely popular AI chatbot ChatGPT last November, tech companies have been racing to release models in hopes of overthrowing its supremacy. Meta has been in the slow lane. In February when competitors Microsoft and Google announced their AI chatbots, Meta rolled out the first, smaller version of LLaMA, restricted to researchers. But it hopes that releasing LLaMA 2, and making it free for anyone to build commercial products on top of, will help it catch up.
A Suite of AI Models
Meta’s latest AI model comes with a suite of AI models, which include versions of LLaMA 2 in different sizes, as well as a version of the AI model that people can build into a chatbot, similar to ChatGPT. Unlike ChatGPT, which people can access through OpenAI’s website, the model must be downloaded from Meta’s launch partners Microsoft Azure, Amazon Web Services, and Hugging Face.
“This benefits the entire AI community and gives people options to go with closed-source approaches or open-source approaches for whatever suits their particular application,” says Ahmad Al-Dahle, a vice president at Meta who is leading the company’s generative AI work. “This is a really, really big moment for us.”
But many caveats still remain. Meta is not releasing information about the data set that it used to train LLaMA 2 and cannot guarantee that it didn’t include copyrighted works or personal data, according to a company research paper shared exclusively with MIT Technology Review. LLaMA 2 also has the same problems that plague all large language models: a propensity to produce falsehoods and offensive language.
The idea, Al-Dahle says, is that by releasing the model into the wild and letting developers and companies tinker with it, Meta will learn important lessons about how to make its models safer, less biased, and more efficient.
A powerful open-source model like LLaMA 2 poses a considerable threat to OpenAI, says Percy Liang, director of Stanford’s Center for Research on Foundation Models. Liang was part of the team of researchers who developed Alpaca, an open-source competitor to GPT-3, an earlier version of OpenAI’s language model.
“LLaMA 2 isn’t GPT-4,” says Liang. And in its research paper, Meta admits there is still a large gap in performance between LLaMA 2 and GPT-4, which is now OpenAI’s state-of-the-art AI language model. “But for many use cases, you don’t need GPT-4,” he adds.
A more customizable and transparent model, such as LLaMA 2, might help companies create products and services faster than a big, sophisticated proprietary model, he says.
“To have LLaMA 2 become the leading open-source alternative to OpenAI would be a huge win for Meta,” says Steve Weber, a professor at the University of California, Berkeley.
Under the hood
To make LLaMA 2 safer than its predecessor, Meta used a mix of machine learning techniques, including training the model on 40% more data and fine-tuning it with feedback from human annotators. The model was also trained on a data set that was excluded from sites with lots of personal information.
Despite these measures, LLaMA 2 still spews offensive, harmful, and problematic language. Meta says it did not remove toxic data from the data set because it might help the model detect hate speech better.
LlMA 2 is an open-source model, which means that external researchers and developers can probe it for security flaws. This makes it safer than proprietary models.
original Source: technologyreview.com
