We tested Chat GPT-4
A lot of expectations accompanied the launch of the new version of ChatGPT. The Californian company OpenAI unveiled, Tuesday evening, GPT-4, a new version of the generative artificial intelligence technology which operates the famous chatbot. One more step towards ever-increasing “intelligence”.
While the OpenAI’s first chatbot launched in November is creating a buzz, this new version has already aroused many enthusiastic reactions and some reservations.
One of the major advances concerns image analysis, but this option is currently not available to everyone.
We tested it, with a view to general public use, by calling on experts.
Test 1: Cat GPT doesn’t really know who he is…
We tested it with the paid version of ChatGPT, at 20 dollars per month, notably offering greater availability when demand is high. First disappointment: even if the page invites you to choose between the different versions (GPT-3.5, GPT-4), and you choose the last one, when you ask the robot “are you based on GPT- 4? he replies: “To date, OpenAI has not yet announced or released a new version of ChatGPT, including GPT-4. Therefore, I am currently based on ChatGPT-3 and not on a later version”.
Checked with the Twitter thread “ChatGPT Community”, it may be a bug. “It’s a generative model that doesn’t know itself. It also says as little as possible for security reasons and to avoid transmitting too sensitive information to the competition, ”adds Vincent Terrasi, co-founder and technical director of Draft & Goal, specializing in artificial intelligence.
While many highlight the advances, the tool also arouses some criticism of its limits. “It’s certainly a step, but it’s not the big leap that some were expecting. Not everything changes in GPT-4 and not all features are deployed, especially on images. Also, in a demo, the tool took several seconds to respond, which is quite long,” said Forrester AI analyst Rowan Curran.
He also points to the limitation of the number of requests (to 100 messages every 4 hours), “which can probably be explained by the fact that the model is more expensive than previous versions”, he says.
Test 2: Cat GPT continues to have “hallucinations”
As in a previous test, we had ChatGPT write several articles on various topics.
An article on Daniel Kretinsky, Czech billionaire in the spotlight in recent days while he is in exclusive negotiations to take over Editis, gives a decent general introduction to the character. But in his participations, in addition to the “World” or “Casino” (which is correct), he also mentions the acquisition of the AS Monaco football club… (which he did not buy…).
“GPT-4 continues to have hallucinations: the conversational robot actually imagines what could be written even if it is not exact, explains Claude de Loupy, director of Syllabs, specialized in artificial intelligence to generate automatic texts. . It does not search for information on the Internet in real time”. Moreover, the robot says it itself: its knowledge is based on data collected until September 2021.
However, GPT-4 makes fewer errors. “According to our internal evaluations, GPT-4 answers are 40% less likely to contain factual errors than those of GPT-3.5,” says the Californian company.
Test 3: calculation and humor
“GPT-4 has a finer, deeper analysis of the data,” explains Claude de Loupy. For example, he can write poems with more elaborate rhymes, which was more difficult in the old version.
Similarly, GPT-4 has significantly improved in understanding funny stories, tested Vincent Terrasi, and is better in understanding different languages or dialects, “even with ch’ti!” “, notes the expert after conducting several tests.
Likewise in the calculations, he gives more precise explanations. If we do the same calculation of interest on a sum invested over 10 years, the previous GPT version gives an inaccurate amount, unlike GPT-4, which is didactic and above all… fairer.
“His ability to explain what he is doing is impressive. He better understands the question or how it works, which means he has a superior ability to interpret,” adds Claude de Loupy. “By nature, he is a literary person, he is not necessarily capable of solving complex mathematical problems, on the other hand, multimodality will develop in the future: he will be able to call an algorithm specialized in science”, adds Nicolas Gaudemet, Media Partner, Creatives Industries & Data. AI at OnePoint.
Test 4: stunning summaries
Finally, we asked for a summary of a long article at ChatGPT . And, the result is quite stunning. He manages to condense a paper into a few sentences. “GPT-3 generally allowed entry of up to 4,000 tokens (words or parts of words). In GPT-4, it’s 8,000 and 32,000 tokens, or a hundred pages of a book. We can make him sum up almost a novel…”, notes Nicolas Gaudemet.
