-
I am trying to deploy an LLM model on G520 (MT8371). Gemma-2B does not seem to be in the supported model list for G520.
Is it required to select only models listed as officially supported for G520?( TFLite(LiteRT) - Generative AI — IoT AI Hub documentation )
-
In neuropilot-sdk-basic-8.0.11-build20260211, I only found host platforms and device platforms (mt6878 / mt6897 / mt6899 / mt6989 / mt6991)
There is no MT8371 (G520) platform folder.
Does G520 require an additional SDK/platform package to run DLA models? -
I currently have this demo:
GAI-Deployment-Toolkit-v2.0.7_gemma2-2b-v0.1If Gemma2-2B is not supported on G520, where can I find demos for LLM models supported by G520?
-
Following Tutorial for Large Language Models.pdf and
GAI-Deployment-Toolkit-v2.0.7_gemma2-2b-v0.1, I tried to run Gemma2-2B on G520. After completing all steps, inference failed:Current yaml config: config_gemma2_2b_instruct.yaml Begin model init... Segmentation fault (core dumped)Checked logcat output:
$ adb logcat -d | grep -i neuron 07-21 03:21:02.445 18480 18482 E neuron : APUSysEngine::createInstance() failed 07-21 03:21:02.445 18480 18482 E neuron : APUSysEngine initialization failed. 07-21 03:21:02.445 18480 18482 W neuron : Cannot create device for APUSYS_2_0 07-21 03:21:02.445 18480 18482 W neuron : Cannot create device for MDLA_5_3 07-21 03:21:02.445 18480 18482 W neuron : Found an unsupported target: MDLA_5_3 07-21 03:21:02.446 18480 18482 W neuron : Fail to revise dla::CompiledResult 07-21 03:21:02.446 18480 18482 W neuron : Fail to preprocess dla::CompiledGraph 07-21 03:21:02.446 18480 18482 W neuron : Fail to preprocess dla::CompiledNetwork 07-21 03:21:02.446 18480 18482 E neuron : Cannot prepare execution. 07-21 03:21:02.446 18480 18482 E neuron : Successfully open network but cannot start execution. 07-21 03:21:02.446 18480 18482 E neuron : NeuronRuntime_loadNetworkFromBuffer fails: Cannot load network 07-21 03:21:02.446 18480 18482 E CHECK_API: NEURON API error (4, jni/prebuilt/src/executor/neuron_executor.cpp, line 51)
Hi ,
I have successfully run Gemma2-2B now.
The reason for the previous error was that I incorrectly set:
MDLA_VER=“mdla5.3” → (correct should be) MDLA_VER=“mdla5.3,edma3.6”
However, I would still like to ask the previous questions (1, 2, and 3). If I want to deploy and run newer models such as Gemma3 or Gemma4, it seems that they are not listed in the supported model documentation. How should I get started with deploy these models?
Additionally, I would like to know how to integrate DLA inference into a C++ application. Is there any example code or documentation showing how to load and run a .dla model using C++ APIs?
Thanks.
Hi Xin_Qi,
Thanks for reaching out!
-
Right, Gemma2-2B is not supported on G520. You can check the official supported model list here: Model Support and Performance
-
No additional SDK or platform package needed. Even though there’s no dedicated MT8371 platform folder in
neuropilot-sdk-basic-8.0.11-build20260211, you can still use this SDK to compile models for G520. Steps here: Compiling the Model -
Check the supported model list from point 1, then grab the corresponding toolkit from: GAI Tutorial Packages
-
The
Segmentation faultandNeuronRuntimeerrors suggest the model was compiled with incorrect backend settings for G520. When runningcompile.sh, make sure these variables match G520’s config:
BACKEND="mdla5.3,edma3.6"
L1_SIZE_KB="256"
NUM_MDLA="1"
You might also find this related thread useful: [Qwen2.5-0.5B] [DLA Compilation] Floating point exception (core dumped) when running compile_generative.sh
Best,
Jun
“Check the supported model list from point 1, then grab the corresponding toolkit from: GAI Tutorial Packages”
I logged into the website using my account, but I can only see the following resources:
NeuroPilot SDK & Document
- Documents, Examples & Downloads
- Knowledgebase
- NeuroPilot 4
- NeuroPilot 5
Platform
- MT9652
- MT9653
- MT9655
It seems that I cannot find the files I need.
I would like to obtain the NeuroPilot 8 SDK package for MT8189 and the GAI TOOLKITs for LLM models related to G520.
Could you please advise how I can get access to these SDK packages and toolkits? Are there any additional permissions, accounts, or download channels required?
Thank you.
Hi Xin_Qi,
The NeuroPilot 8 SDK and GAI Toolkit are NDA-protected, so they’re not publicly available for direct download. To get access, please reach out to your MediaTek contact. They can help get you the right download permissions.
For publicly available tools, documentation, and more details, you can check: SDKs and Tools by Platform
Best,
Jun
This is a question regarding DLA deployment. I previously posted it in the General Discussion section, but I have not received any feedback yet. Therefore, I would like to kindly ask for your help and advice on this issue.
I have a question regarding CV model deployment on G520 (MT8371).
I have a super-resolution (SR) model and would like to deploy it on DLA and integrate the inference process into a C++ application.
My questions are:
-
For CV dla model deployment on MT8371, do I need an MT8371-specific platform folder in the NeuroPilot SDK for DLA compilation and runtime integration? In neuropilot-sdk-basic-8.0.11-build20260211, I only found host platform and device platforms (mt6878 / mt6897 / mt6899 / mt6989 / mt6991). How to get MT8371 files if its necessary?
-
Is there any related demo, sample code, or reference implementation showing how to integrate DLA inference into a C++ application?
Currently, the examples I can find only demonstrate running the converted DLA model directly using NeuronRT. I could not find examples showing how to call DLA inference APIs and integrate them into a C++ project.
Any guidance or example code would be highly appreciated.
Thank you.
Hi Xin_Qi,
I’ve already replied to the same question in this issue: [MT8371/G520] DLA Deployment on NeuroPilot SDK – Platform Folder Availability & C++ Integration Sample