Genio 520 : LLM NPU inference: pre-built models vs. converting our own model

NeuroPilot NeuroPilot - GenAI
,
1

Hi,

I’m working on deploying LLM inference on the NPU of the Genio 520-EVK (MT8371) and would like to confirm a few things about the workflow.

1. Pre-built models (public):
My understanding is that the pre-built LLM packages listed in the IoT AI Hub Model Zoo (e.g. llama3.2-1b, containing the compiled .dla files, tokenizer, and config) are publicly downloadable and can be run directly on the Genio 520 via the built-in llm_cmdline_tool in the Yocto image, with no special SDK access required. Is that correct?

2. Converting our own model:
For deploying our own LLM with NPU acceleration on the Genio 520, I’d like to understand the conversion path to produce the .dla files:

  • Do I need NDA-gated access to the GAI Toolkit and/or the NeuroPilot SDK (Premium) to perform this conversion? If so, could you point me to how to request that access for our account?
  • Which specific NeuroPilot version is required/matched for the Genio 520 (MT8371) LLM conversion?
  • Is there any publicly available converter that can convert a Hugging Face / ONNX / TFLite LLM into the .dla format for Genio 520 NPU inference, or is the GAI Toolkit the only supported path?

Thanks in advance for the clarification.

Best regards,

3

Hi Chris,

Thanks for reaching out!

1. Pre-built Models

Yes! The pre-built LLM packages on the IoT AI Hub Model Zoo (e.g., llama3.2-1b, which includes a compiled .dla, tokenizer, and config) are publicly available. No NeuroPilot SDK access needed. You can run them directly on Genio 520 using llm_cmdline_tool in the Yocto image.

Quick note: llm_cmdline_tool is only available in Yocto image v26.0 or above.

2-1. NDA Requirement for GAI Toolkit / NeuroPilot SDK

Yes, both the GAI Toolkit and NeuroPilot SDK (Basic/Premium) are NDA-protected. You’ll need to apply for access:

2-2. NeuroPilot Version for Genio 520

For Genio 520 (MT8371), use NeuroPilot 8. For more details, refer to: SDKs and Tools by Platform

2-3. Public .dla Conversion Tool

The GAI Toolkit is the only supported LLM conversion path. Since it’s NDA-protected, access requires the same application process as 2-1.

Best,
Jun