[MT8893 / NPU Gen7] Recommended toolchain for deploying Qwen2.5: GAI-Deployment-Toolkit vs. LiteRT NeuroPilot Accelerator?

NeuroPilot NeuroPilot - GenAI
1

MT8893 NPU Gen7 — what is the MDLA version? Is GAI-Deployment-Toolkit-v2.0.x compatible? What backend string should I use in compile_generative.sh? Is LiteRT NeuroPilot Accelerator the recommended path for Qwen2.5 on MT8893?

For the MT8893 NPU Gen7 chip, which GAI Tools should I use and what are the specific configuration files needed to convert a model to run on the MT8893 NPU Gen7?

2

Chào Bảo Long,

Cảm ơn bạn đã chia sẻ thông tin chi tiết. Tôi sẽ xem xét và phản hồi sớm.

Trân trọng,
Pangh [曜發-庞虎]

4

Hi B_O_Long,

Thanks for the detailed questions! Here are the answers:

1. MDLA version & BACKEND string for compile_generative.sh

MT8893 NPU Gen7 is equipped with MDLA 5.1. The corresponding parameters for compile_*.sh are:

BACKEND="mdla5.1"
L1_SIZE_KB="2048"
NUM_MDLA="4"

2. Recommended GAI Toolkit version for Qwen2.5

For Qwen2.5 deployment on MT8893, we recommend using:

GAI-Deployment-Toolkit-v1.0.8_qwen2.5-0.5b-1.5b-7b-v0.1

Quick note — v2.0.x is not the recommended path for this use case.

3. Regarding LiteRT NeuroPilot Accelerator

LiteRT is a framework maintained by Google. On mobile platforms, it does provide a path down to the NPU backend; however, the integration work (wiring LiteRT to the NPU backend) needs to be handled on your end.

As an alternative, if you prefer a more turnkey solution that works consistently across platforms, we recommend going through Neuron Runtime / Shim API, which supports NPU acceleration on all our platforms out of the box. For Qwen2.5 on MT8893, the GAI-Deployment-Toolkit flow (built on top of Neuron Runtime) is the officially recommended path.

Best,
Jun

5 
Hello Jun

So now, can you guide me through converting and configuring the files correctly from A to Z?

I’m using the Qwen2.5-0.5B-Instruct model to test on the MT8893.

I am using GAI-Deployment-Toolkit-v1.0.8_qwen2.5-0.5b-1.5b-7b-v0.1.

6

Hi B_O_Long,

Thanks for the follow-up!

For deploying Qwen2.5-0.5B-Instruct on MT8893 with GAI-Deployment-Toolkit v1.0.8, I’d recommend starting with our official step-by-step guide:

Tutorial for Large Language Models

It walks through the complete end-to-end flow — model conversion, quantization, compilation, and on-device deployment — which should cover most of what you need.

Best,
Jun