Implementation and optimization of MTP for DeepSeek R1 in TensorRT-LLM

Подробнее о видео

Learn from our experts about how we use MTP speculative decoding method to achieve better performance in TensorRT-LLM. You'll learn the MTP method in LLM inference, the MTP implementation in TensorRT-LLM, and the optimization of MTP to further boost the performance.