Meta AI boss says big language models won’t reach human intelligence

Meta’s AI chief said the big language models that power generative AI products like ChatGPT will never achieve the ability to reason and plan like humans, as he instead focuses on a radical alternative approach to creating “superintelligence” in the machines.

Jan Lekun, chief artificial intelligence scientist at the social media giant, which owns Facebook and Instagram, said LLMs have “a very limited understanding of logic. . . they don’t understand the physical world, they don’t have permanent memory, they can’t reason in any reasonable definition of the term, and they can’t plan. . . hierarchical”.

In an interview with the Financial Times, he argued that advanced LLMs should not be relied upon in the pursuit of human-level intelligence, as these models can only respond accurately to prompts if they have been fed the correct training data and are therefore “inherently dangerous”.

Instead, he is working to develop an entirely new generation of AI systems that he hopes will power machines with human-level intelligence, though he said that vision could take 10 years to achieve.

Meta is pouring billions of dollars into developing its own LLMs as generative AI explodes, aiming to catch up to rival tech groups including Microsoft-backed OpenAI and Alphabet’s Google.

LeCun leads a team of about 500 employees at Meta’s Laboratory for Basic AI Research (Fair). They are working to create AI that can develop common sense and learn how the world works in human-like ways, in an approach known as “world modeling.”

The Meta AI chief’s experimental vision is a potentially risky and expensive gamble for the social media group at a time when investors are eager to see a quick return on AI investments.

Meta lost nearly $200 billion in value last month when CEO Mark Zuckerberg promised to ramp up spending and turn the social media group into “the world’s leading AI company,” spooking Wall Street investors concerned about rising costs with little potential for immediate income.

“We’re at the point where we think we’re on the cusp of maybe the next generation of AI systems,” LeCun said.

LeCun’s comments come as Meta and its rivals push forward with increasingly improved LLMs. Figures such as OpenAI chief Sam Altman believe they provide a vital step towards the creation of artificial general intelligence (AGI) – the moment when machines have greater cognitive abilities than humans.

Last week, OpenAI released its new faster model GPT-4o, and Google unveiled a new “multimodal” AI agent that can respond to real-time queries in video, audio and text, called Project Astra, powered by an upgraded on its Gemini model.

Meta also launched its new Llama 3 model last month. The company’s head of global affairs, Sir Nick Clegg, said the latest LLM has “significantly improved capabilities such as reasoning” – the ability to apply logic to queries. For example, the system would assume that a person suffering from a headache, sore throat and runny nose has a cold, but it could also recognize that allergies might be the cause of the symptoms.

However, LeCun said this evolution of LLM is superficial and limited, with models only learning when human engineers step in to train it with that information, rather than AI coming to a conclusion organically like humans.

“It certainly looks like reasoning to most people — but mostly it uses accumulated knowledge from a lot of training data,” LeCun said, but added, “[LLMs] are very useful despite their limitations.”

Google DeepMind has also spent several years researching alternative methods for building AGI, including methods such as reinforcement learning, where AI agents learn from their surroundings in a game-like virtual environment.

At an event in London on Tuesday, DeepMind chief Sir Demis Hassabis said that what language models lack is “they don’t understand the spatial context you’re in. . . so that limits their usefulness in the end”.

Meta established its Fair Lab in 2013 to pioneer AI research, employing leading academics in the space.

In early 2023, however, Meta created a new GenAI team led by Chief Product Officer Chris Cox. He brought in many AI researchers and engineers from Fair and led the work on Llama 3 and integrated it into products such as his new AI assistants and imaging tools.

The creation of the GenAI team came as some insiders argued that the academic culture within the Fair lab was partly to blame for Meta’s late arrival to the generative AI boom. Zuckerberg is pushing for more commercial applications of AI under pressure from investors.

However, LeCun remains one of Zuckerberg’s key advisers, according to people close to the company, because of his record and reputation as one of the founding fathers of AI, winning a Turing Award for his work on neural networks.

“We’ve refocused the Fair on the longer-term goal of human-level AI, mainly because GenAI is now focused on the things we have a clear path to,” LeCun said.

“[Achieving AGI] it’s not a product design problem, it’s not even a technological development problem, it’s very much a scientific problem,” he added.

LeCun first published a paper on his vision for modeling the world in 2022, and Meta has since released two research models based on the approach.

Today, he said, Fair is testing different ideas about achieving human-level intelligence because “there’s a lot of uncertainty and research involved, [so] we cannot say who will succeed or be taken in the end”.

Among them, LeCun’s team fed the systems hours of video and deliberately skipped frames, then had the AI ​​predict what would happen next. This is to mimic how children learn by passively observing the world around them.

He also said Fair is exploring building a “universal text encoding system” that would allow a system to process abstract representations of knowledge in text that could then be applied to video and audio.

Some experts question whether LeCun’s vision is viable.

Aaron Culotta, an associate professor of computer science at Tulane University, said that common sense has long been a “thorn in the side of AI” and that it’s challenging to teach models of causality, leaving them “susceptible to these unexpected failures.”

One former Meta AI employee described the world modeling push as “fuzzy fluff,” adding, “It feels like a lot of flag-planting.”

Another current employee said Fair has yet to prove itself as a true rival to research groups like DeepMind.

In the longer term, LeCun believes the technology will power AI agents that users can interact with through wearable technology, including augmented reality or “smart” glasses and electromyography (EMG) “bracelets.”

“[For AI agents] to be truly useful, they must possess something like human-level intelligence,” he said.

Additional reporting by Madhumita Murgia in London

Video: AI: Blessing or Curse for Humanity? | FT Tech

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top