Hidet: An Open-Source Python-based Deep Learning Compiler

The demand for optimized inference workloads has by no means been extra essential in deep studying. Meet Hidet, an open-source deep-learning compiler developed by a devoted workforce at CentML Inc. This Python-based compiler goals to streamline the compilation course of, providing end-to-end assist for DNN fashions from PyTorch and ONNX to environment friendly CUDA kernels, specializing in NVIDIA GPUs.

Hidet has emerged from analysis offered within the paper “Hidet: Job-Mapping Programming Paradigm for Deep Studying Tensor Applications,” The compiler addresses the problem of lowering the latency of deep studying mannequin inferences, an important facet of making certain environment friendly mannequin serving throughout a wide range of platforms, from cloud providers to edge gadgets.

The event of Hidet is pushed by the popularity that creating environment friendly tensor applications for deep studying operators is a posh process, given the intricacies of recent accelerators like NVIDIA GPUs and Google TPUs, coupled with the fast enlargement of operator varieties. Whereas present deep studying compilers, reminiscent of Apache TVM, leverage declarative scheduling primitives, Hidet takes a singular method.

The compiler embeds the scheduling course of into tensor applications, introducing devoted mappings generally known as process mappings. These process mappings allow builders to outline the computation project and ordering immediately throughout the tensor applications, enriching the expressible optimizations by permitting fine-grained manipulations at a program-statement stage. This progressive method is known as the task-mapping programming paradigm.

Moreover, Hidet introduces a post-scheduling fusion optimization, automating the fusion course of after scheduling. This not solely permits builders to give attention to scheduling particular person operators but in addition considerably reduces the engineering efforts required for operator fusion. The paradigm additionally constructs an environment friendly hardware-centric schedule area agnostic to program enter dimension, thereby considerably lowering tuning time.

In depth experiments on trendy convolution and transformer fashions showcase the facility of Hidet, outperforming state-of-the-art DNN inference frameworks reminiscent of ONNX Runtime and the compiler TVM outfitted with AutoTVM and Ansor schedulers. On common, Hidet achieves a 1.22x enchancment, with a most efficiency achieve of 1.48x.

Along with its superior efficiency, Hidet demonstrates its effectivity by lowering tuning instances considerably. In comparison with AutoTVM and Ansor, Hidet slashes tuning instances by 20x and 11x, respectively. 

As Hidet continues to evolve, it’s setting new requirements for effectivity and efficiency in deep studying compilation. With its method to process mapping and fusion optimization, Hidet has the potential to change into a cornerstone within the toolkit of builders in search of to push the boundaries of deep studying mannequin serving.