MTK Converter Cheat Sheet

NeuroPilot NeuroPilot - Analytical AI
, ,
1

Here post some tips for the parameter setting of MTK Converter

1 Like
3

How to Ensure Full Integer Quantization (uint8) for All Tensors During Model Conversion?

Question

When converting a model to uint8, users may find that the inputs, outputs, and intermediate tensors remain in float32 despite quantization settings. How can one properly apply post-training quantization (PTQ) so that the entire model—including inputs, outputs, and all tensors—are quantized to uint8 as expected? Which converter parameters control this behavior?

Answer

If the use_dynamic_quantization=True parameter is set, only constant weights are quantized; all activation tensors, including inputs, outputs, and intermediates, remain as float32.
To achieve full integer quantization using PTQ:

  • Set use_dynamic_quantization=False when converting the model.
  • This ensures that activations, inputs, and outputs are quantized to uint8 as desired.

Refer to the converter’s official documentation for parameter support and compatibility.

4

How to Reduce or Avoid Generating MTK_EXT_OP When Converting Models?

Question

During model conversion for MediaTek platforms, excessive MTK_EXT_OP operators are sometimes generated. What are the recommended workflows or parameter adjustments to minimize or avoid MTK_EXT_OP, and are there alternative tools to achieve fully standard operator sets?

Answer

  • First, try setting the parameter use_per_output_channel_quantization=False.
    This reduces the likelihood of MTK_EXT_OP being generated by disabling certain optimizations.
  • However, disabling this option does not guarantee the elimination of MTK_EXT_OP, since the converter is specifically optimized for MediaTek NPUs, and compiler decisions are complex.
  • For conversion results without any MTK_EXT_OP, it is recommended to use the official open-source TensorFlow converter.
5

How to Convert a PyTorch Model with Multiple Inputs Using mtk_pytorch_converter

Question

When converting a PyTorch model that accepts multiple input tensors using mtk_pytorch_converter, users may encounter errors if input shapes are not correctly specified. Incorrect or missing shape information can cause conversion failures or wrongly-shaped outputs.

Answer

Use the --input_shapes argument to specify each input’s shape. Ensure that:

  • The number and order of input shapes match those expected by the model’s forward() method.
  • Each input shape is separated by a comma, and dimensions within each shape are separated by a colon (:).

Example Command

Convert a model with three input tensors, each of shape [1, 128]:

mtk_pytorch_converter \
    --input_script_module_file=xxx.pt \
    --output_file=xxx.tflite \
    --input_shapes=1:128,1:128,1:128

Notes

  • The order of shapes in --input_shapes must exactly match the order of input tensors expected by the model.
  • Always review the model’s forward() method or refer to TorchScript documentation to confirm the correct input order and shapes.
  • Providing incorrect shape or count may lead to conversion errors.