GPT-4 Can Solve Math Problems — But Not in All Languages | by Yennie Jun | Oct, 2023

A couple of experiments making GPT-4 resolve math issues in 16 totally different languages

It’s stated that arithmetic is a common language — mathematical ideas, theorems, and definitions will be expressed as symbols which can be comprehensible no matter language.

On this article, I check the mathematical capabilities of GPT-4 in sixteen totally different languages.

Early experiments confirmed GPT-4 scoring extremely on the SAT Math and AP Calculus assessments and on undergraduate-level arithmetic. Nonetheless, the vast majority of these experiments check GPT-4’s mathematical capabilities solely in English. To raised perceive GPT-4’s mathematical capabilities past English, I immediate it on the identical math issues in fifteen different languages.

So, how good is GPT-4 at math in numerous languages? In idea, it must be equally good (or dangerous) throughout all languages, however sadly (as you may need guessed), this isn’t the case. GPT-4 is significantly better at fixing math issues in English. Relying on the language, GPT-4 may resolve a number of the issues. For historically under-resourced languages, nonetheless, akin to Burmese and Amharic, GPT-4 was unable to unravel the issues I gave it.

I exploit mathematical issues from the Mission Euler web site to check GPT-4. (That is additionally a throwback to one in every of my one in every of my earlier articles from this yr, the place I used immediate engineering utilizing ChatGPT to unravel a couple of Mission Euler issues). Mission Euler, named for the eponymous mathematician, is an internet site with lots of of mathematical and laptop programming issues ranging in problem. Began in 2001, they boast over 850 issues (as of October 2023) and launch a brand new query roughly each week.

The beauty of Mission Euler questions is that every downside has a numerically “right” reply — this makes it simple to examine if GPT-4’s reply is objectively right or not. Additionally they are usually much more difficult than high-school or college-level math issues. Presently, there is no such thing as a large-scale complete understanding of GPT-4’s (or different massive language fashions, for that matter) math…