This AI Paper Introduces Learning from Mistakes (LeMa): Enhancing Mathematical Reasoning in Large Language Models through Error-Driven Learning

Human beings, as inherently fallible creatures, navigate the intricate journey of life marked by successes and failures. Within the grand tapestry of our existence, the thread of errors weaves a novel sample that contributes considerably to our progress and improvement. Studying from errors is prime to the human expertise, shaping our character, fostering resilience, and propelling us towards a extra enlightened future.

Can LLM additionally be taught from errors? Is it attainable? Sure, they do. Giant language fashions, like GPT-3, be taught from huge knowledge, together with examples of appropriate and incorrect language utilization. These fashions are skilled on various datasets containing a variety of textual content from the web, books, articles, and extra. The mannequin learns to acknowledge the coaching knowledge’s patterns, relationships, and contextual info. It understands grammar, syntax, semantics, and even nuances of language use.

Mimicking this error-driven studying course of, researchers at Jiaotong College, Peking College, and Microsoft current LEMA, which fine-tunes LLMs on mistake correction knowledge pairs generated by GPT-4. They are saying their concept of motivation got here from the training technique of human college students from errors.  

Their technique includes producing mistake-correction knowledge pairs after which fine-tuning LLMs utilizing correction knowledge. They make use of a number of LLMs, comparable to LLaMA and GPT collection fashions, to gather inaccurate reasoning paths to generate correction knowledge. The generated corrections comprise three items of details about the inaccurate step within the unique answer,  a proof of why this step is wrong, and the way to appropriate the unique answer to reach on the appropriate remaining reply.

They filter out the corrections with incorrect remaining solutions, they usually say this course of displays sufficient high quality for the following fine-tuning stage. They generate extra reasoning paths for every query within the coaching set with GPT-4 and filter out paths with mistaken remaining solutions. They apply this CoT knowledge augmentation to arrange a powerful fine-tuning baseline that solely makes use of CoT knowledge. It additionally facilitates additional ablation examine on controlling knowledge measurement for fine-tuning. They fine-tune the mannequin on question-rational knowledge alone.

In comparison with fine-tuning on CoT knowledge alone, LEMA constantly improves efficiency throughout numerous LLMs and duties. LEMA with LLaMA-2-70B achieves 83.5% on GSM8K and 25.0% on MATH, whereas fine-tuning on CoT knowledge alone yields 81.4% and 23.6%, respectively. 

Current developments in LLMs have enabled them to carry out a step-by-step strategy to problem-solving. Nonetheless, this multi-step era course of doesn’t inherently indicate that LLMs possess sturdy reasoning capabilities, as they might merely emulate the superficial habits of human reasoning with out genuinely comprehending the underlying logic and guidelines needed for exact rationale. LEMA employs GPT-4 as a world mannequin to show smaller fashions to stick to logic and guidelines fairly than merely mimic the step-by-step habits.

Take a look at the Paper. All credit score for this analysis goes to the researchers of this mission. Additionally, don’t neglect to hitch our 33k+ ML SubReddit, 41k+ Fb Group, Discord Channel, and E-mail E-newsletter, the place we share the newest AI analysis information, cool AI tasks, and extra.

Should you like our work, you’ll love our e-newsletter..

We’re additionally on Telegram and WhatsApp.