Unable to Load Model on Genio 510 NPU with Python

NeuroPilot NeuroPilot - Analytical AI
,
1

  1. I am trying to run a TensorFlow Lite model on the Genio 510 EVK using the NPU for hardware acceleration. I am using tflite_runtime and attempting to load the Neuron delegate as follows:
    from tflite_runtime.interpreter import Interpreter, load_delegate
    delegate = load_delegate(“/usr/lib/libneuron_stable_delegate.so”)

    However, I am getting the following error:
    Attempting to load NPU delegate: /usr/lib/libneuron_stable_delegate.so
    load_delegate failed: /usr/lib/libneuron_stable_delegate.so: undefined symbol: tflite_plugin_create_delegate
    Exception ignored in: <function Delegate.del at 0xffff907667a0>
    Traceback (most recent call last):
    File “/usr/lib/python3.12/site-packages/tflite_runtime/interpreter.py”, line 110, in del
    self._library.tflite_plugin_destroy_delegate.argtypes = [ctypes.c_void_p]
    AttributeError: /usr/lib/libneuron_stable_delegate.so: undefined symbol: tflite_plugin_destroy_delegate

    Additional Info:

    Board: Genio 510 EVK

    Python version: 3.12

    tflite-runtime version: 2.16.1 (vanilla Google runtime)

    Delegate path: /usr/lib/libneuron_stable_delegate.so

Attempted with both CPU and delegate, delegate always fails

Questions:
How do I properly load a TensorFlow Lite model on the Genio 510 NPU?
Do I need a MediaTek Neuron-patched tflite_runtime? If yes, how can I obtain it?
Are there any example recipes or steps for integrating Neuron SDK with Yocto for Python inference?

Any guidance, example code, or documentation references would be greatly appreciated.

Thank you!

2

Hello @Kunal_Kalkotwar ,

I believe you might find these links helpful:

  1. Official MediaTek Tao Tutorials
  2. MediaTek Tao Demos Source Code

On second link, you will be able to find instructions to enable the required libraries on your custom Yocto image for the custom tflite packages to be installed. Also, you can base on the Action Recognition Demo for a sample Python implementation.

Hope that helps.
Please do not hesitate to reach out if you need further support, we would love to help.

best regards,
Andres
Embedded Software Engineer at ProventusNova

1 Like
3

I’m not using a custom Yocto build; I’m working with the pre-built image provided for the MediaTek Genio 510 EVK. Please share the steps applicable for this setup.

4

Hello @Kunal_Kalkotwar,

Thanks for sharing that detail!

May I ask why are you not using a custom Yocto image ?
I am just curious since that is very common across our most successful clients.

And I don’t think I have an example with the pre-built image, however I can try to put something together for you during the day.

best regards,
Andrew
Embedded Software Engineer at ProventusNova

6

Hi @Kunal_Kalkotwar

Python API is currently not supported with NPU delegate. This is a TF restriction since our delegate is based on TF stable delegate API which only has C++ API.

Python API development work is currently under progress.
You should be able to use this C++ API snippet to integrate your C++ application with NPU delegate.

7

Thanks for replying
I have compiled whisper models encoder in .dla format using mtk_converter how can i deploy it on board.
I am trying to give continuous audio to the model (dla) how can i achieve it.