Issues with Instance Normalization in NeuroPilotSDK

NeuroPilot NeuroPilot - Analytical AI
, ,
1

Hi everyone,

I’m working on a model where I want to integrate an Instance Normalization layer. I’m using the MTK converter tool to convert the model into a quantized TFLite model. By default, the converter uses the MTKEXT_INSTANCE_NORMALIZATION operator.

However, in the quantized model, this operator significantly slows down execution because it adds dequantize before and quantize after the operator. This causes the network to run very slowly when using the neuronrt tool.

Using ncc-tflite tool, I noticed that the model is partitioned into many subgraphs, which leads to inefficient execution. I also tried enabling several optimization options in ncc-tflite, but this did not improve the execution speed in neuronrt.

The MTK converter provides an option converter.decompose_instance_normalization_ops. When I enable it, the converter replaces the instance normalization with basic operations (mean, multiply, etc.), which allows ncc-tflite to compile the model into fewer subgraphs.

The problem is that even though the model successfully compiles to .dla, running it with neuronrt fails and reports an error related to parallel execution.

I would appreciate your help with the following questions:

  • Is there a recommended way to build the graph efficiently when using an instance normalization layer with the ncc-tflite tool?
  • Why does the neuronrt tool report a parallel execution failure when I replace the instance normalization layer with simpler operators, even though ncc-tflite compiles the model successfully?

Any insights or guidance on handling instance normalization efficiently would be greatly appreciated.

Thank you in advance!

2

Hi @Sharon,
Thank you for the inquiry.

Here needs more information about the failure because for different model properties, we might have different ncc-tflite parameter settings.

Could you provide the platform information, also the ncc-tflite you used for compiling the model?
If available, attaching the full log might be helpful for us to troubleshoot.

MTK,
Joying

3

Hi @joying.kuo ,

The device I am using is based on the D9500 platform.
I ran ncc-tflite with the following parameters: --platform-config=mt6993 --verbose --show-memory-summary --show-exec-plan.
The ncc-tflite tool completed successfully and printed “success” at the end of the run.
However, when I ran the neuronrt tool, I received the following logs:

image
image1311×508 38.3 KB

Thanks,

Sharon

5

Hi @Sharon,
Thank you for the inquiry.

Sorry in advance, but the D9500 is not an IoT-supporting platform, so I can’t do the test for your case.

To collect more logs, here are some common queries that might work on your system

adb shell setprop debug.mtk_tflite.vlog true
adb shell setprop debug.neuron.adapter.AdapterSetDebugLevel 31
adb shell setprop debug.nn.vlog 1
adb shell setprop debug.neuron.runtime.ShowExecPlan 1
adb shell setprop debug.neuron.runtime.Verbose true

adb shell stop neuralnetworks_hal_service_mtk_neuron
adb shell stop neuralnetworks_hal_service_shim_mtk
timeout /t 1
adb shell start neuralnetworks_hal_service_mtk_neuron
adb shell start neuralnetworks_hal_service_shim_mtk

You can file the technical question about the platforms not supported by IoT, via e-Service(MTK customer inquiry channel) and attach the logs. The system will delegate your question to the support team.

Here is the table of IoT’s AI supporting scope: IoT AI Hub Overview — IoT AI Hub documentation

MTK,
Joying