November 12, 2024
Recently, the results of the 2024 Major Academic Advances hosted by the Academic Committee of Huazhong University of Science and Technology,have been revealed. The achievement "General Multimodal Large Language Model ‘Monkey’," submitted by Professor Bai Xiang's team from SSE, was successfully selected. An introduction to their achievementsispresented as follows:
Achievement Title:General Multimodal Large Language Model “Monkey”
Primary Contributors:Bai Xiang, Liu Yuliang
Introduction:
LargeLanguagemodels(LLMs)aredeeplytransformingour ways ofproduction. However, challenges such as difficulties in adapting to domestic chips, invisible fine-grained information, incomplete input of long text data, and insufficient information analysis capabilities severely restrict their application in vertical fields. To address these challenges, Professor Bai Xiang's team proposesan adaptive image pyramid segmentation technique, experimentally validating the effectiveness and flexibilitytosupport thousands of high-resolution images and generating multi-level fine-grained data. Additionally,the researchreducesaccumulatederrorsin processing long sequence tokens through a multi-page sparse sampling technique.Combined with end-to-end optimization,it is able tohandleQAtasks for long PDF documentsefficiently. Building on this, the team further introduced a correction method based on reverse chain reasoning.Theyutilizea differentiable reverse chain reasoning mechanism,andthenthe modelgainscapabilitiesformulti-step reasoning,thussignificantly enhancing the model's hierarchical thinking abilityinsolving complex problems. The above achievements are based on Huawei's Ascend 910B 64GB chip and the MindSpore.The team makes breakthroughsinaddressing"invisibility, incompleteness, and inability"in multimodal applications of LLMs, andprovidestheoretical and technical support for the applicationin the industry.
The event, launched in April 2024, follows the principles of openness, fairness, justice, and the pursuit of excellence, focusing on publicizing the University’s original and leading academic progress. The evaluated achievements were recommended through two channels: the academic committees of various departments or the Academic Committee of the University. The process involves successive steps, including screening by the disciplinary committees of the Academic Committee, comprehensive evaluation by all members of the Academic Committee, nomination by the Director’s meeting, defense and review by the full meeting of the Academic Committee, and final approval by the President’s office meeting. Ten selected achievements emerged from this process. The ten selected achievements in this round reflect the University’s commitment to guiding high-quality innovation through upholding principles and embracing innovation. This aligns with the University’s educational philosophy of “upholding integrity, striving for excellence,” embodying the spirit of scholars at the University who harbor patriotism, serve the people, transcend limitations, and aspire to be pioneers in the scientific community.
