[ad_1]
GENERATIVE AI
Learnings from superb tuning a big language mannequin on a single shopper GPU
After we take into consideration Giant Language Fashions or some other generative fashions, the primary {hardware} that involves thoughts is GPU. With out GPUs, many developments in Generative AI, machine studying, deep studying, and information science would’ve been not possible. If 15 years in the past, players had been enthusiastic in regards to the newest GPU applied sciences, at this time information scientists and machine studying engineers be a part of them and pursue the information on this area too. Though normally players and ML customers are two completely different sorts of GPUs and graphic playing cards.
Gaming customers normally use shopper graphic playing cards (equivalent to NVIDIA GeForce RTX Sequence GPUs), whereas ML and AI builders normally observe information about Information Heart and Cloud Computing GPUs (equivalent to V100, A100, or H100). Gaming graphic playing cards normally have a lot much less GPU reminiscence (at most 24GB as of January 2024) in comparison with Information Heart GPUs (within the vary of 40GB to 80GB normally). Additionally, their value is one other vital distinction. Whereas most shopper graphic playing cards might be as much as $3000, most Information Heart graphic playing cards begin from that value and might go tens of hundreds of {dollars} simply.
Since many individuals, together with myself, might need a shopper graphic card for his or her gaming or day by day use, they is likely to be to see if they will use the identical graphic playing cards for coaching, fine-tuning, or inference of LLM fashions. In 2020, I wrote a complete article about whether or not we are able to use shopper graphic playing cards for information science initiatives (hyperlink to the article). At the moment, the fashions had been largely small ML or Deep Studying fashions and even a graphic card with 6GB of reminiscence might deal with many coaching initiatives. However, on this article, I’m going to make use of such a graphic card for giant language fashions with billions of parameters.
For this text, I used my Geoforce 3090 RTX card which has 24GB of GPU reminiscence. To your reference, information heart graphic playing cards equivalent to A100 and H100 have 40GB and 80GB of reminiscence respectively. Additionally, a typical AWS EC2 p4d.24xlarge occasion has 8 GPUs (V100) with a complete of 320GB of GPU reminiscence. As you may see the distinction between a easy shopper…
[ad_2]
Source link
