How to resolve DLA permission issues when packaging Qwen2.5 LLM into a standalone APK

NeuroPilot NeuroPilot - GenAI
8

Hi Jiahao_Zhao,

Below is a step-by-step guide on how to load a DLA and perform inference using the Neuron Adapter API. Each step is accompanied by the corresponding code snippet for your reference.

Note: The code snippets below are simplified examples for clarity, focusing on the core flow of loading a DLA and running inference. Please adapt the code to your project’s needs.

Parameter Definitions (recommended to place at the top of your file)

#define DLA_INPUT_BIN_SIZE                   376832
#define DLA_OUTPUT_BIN_SIZE                  1536000
#define RESTORE_DLA_EXTENSION_OPERAND_TYPE   0x0100
#define RESTORE_DLA_EXTENSION_OPERATION_TYPE 0x0000
#define RESTORE_DLA_EXTENSION_NAME           "com.mediatek.compiled_network"
Macro Description
DLA_INPUT_BIN_SIZE Byte size of the model input tensor; please adjust according to your DLA spec
DLA_OUTPUT_BIN_SIZE Byte size of the model output tensor; please adjust according to your DLA spec
RESTORE_DLA_EXTENSION_OPERAND_TYPE Operand type ID of the MTK extension “DLA Raw Data” (fixed value)
RESTORE_DLA_EXTENSION_OPERATION_TYPE Operation type ID of the MTK extension “Load DLA” (fixed value)
RESTORE_DLA_EXTENSION_NAME Extension name string (fixed value)

Step 1. Read the DLA file into memory

std::ifstream input_dla(dla_path, std::ios::binary);
input_dla.seekg(0, input_dla.end);
int length = input_dla.tellg();
input_dla.seekg(0, input_dla.beg);
char* buffer = static_cast<char*>(malloc(length * sizeof(char)));
input_dla.read(buffer, length);
input_dla.close();

For APK deployment, please change dla_path to the app’s private directory (corresponding to Context.getFilesDir() in Java/Kotlin), so that the file can be accessed without root permission.

Step 2. Create a NeuronModel

NeuronModel* model = nullptr;
NeuronModel_create(&model);

Step 3. Define the Input / Output tensor types

// Input
NeuronOperandType tensorInputType;
tensorInputType.type = NEURON_TENSOR_QUANT8_ASYMM;
tensorInputType.scale = 1.0f;
tensorInputType.zeroPoint = 0;
tensorInputType.dimensionCount = 1;
uint32_t dims_input[1] = {DLA_INPUT_BIN_SIZE};
tensorInputType.dimensions = dims_input;

// Output
NeuronOperandType tensorOutputType;
tensorOutputType.type = NEURON_TENSOR_QUANT8_ASYMM;
tensorOutputType.scale = 1.0f;
tensorOutputType.zeroPoint = 0;
tensorOutputType.dimensionCount = 1;
uint32_t dims_output[1] = {DLA_OUTPUT_BIN_SIZE};
tensorOutputType.dimensions = dims_output;

Step 4. Get the DLA extension operand type

int32_t operandType = 0;
NeuronModel_getExtensionOperandType(model,
    RESTORE_DLA_EXTENSION_NAME,
    RESTORE_DLA_EXTENSION_OPERAND_TYPE,
    &operandType);

NeuronOperandType extenOperandType;
extenOperandType.type = operandType;
extenOperandType.scale = 0.0f;
extenOperandType.zeroPoint = 0;
extenOperandType.dimensionCount = 0;

Step 5. Add operands to the model

NeuronModel_addOperand(model, &tensorInputType);   // 0: model input 1
NeuronModel_addOperand(model, &tensorInputType);   // 1: model input 2
NeuronModel_addOperand(model, &extenOperandType);  // 2: DLA Raw Data
NeuronModel_addOperand(model, &tensorOutputType);  // 3: model output
Index Role
0 Input tensor 1
1 Input tensor 2
2 DLA raw data (extension operand)
3 Output tensor

Step 6. Feed the DLA buffer into the model

NeuronModel_setOperandValue(model, 2, buffer, length);

This is the key step for “loading the precompiled DLA”.

Step 7. Get the DLA extension operation type and add the operation

int32_t operationType = 0;
NeuronModel_getExtensionOperationType(model,
    RESTORE_DLA_EXTENSION_NAME,
    RESTORE_DLA_EXTENSION_OPERATION_TYPE,
    &operationType);

uint32_t addInputIndexes[3] = {0, 1, 2};
uint32_t addOutputIndexes[1] = {3};
NeuronModel_addOperation(model,
    static_cast<NeuronOperationType>(operationType),
    3, addInputIndexes,
    1, addOutputIndexes);

Step 8. Identify the model’s inputs/outputs and finish the model

uint32_t modelInputIndexes[2] = {0, 1};
uint32_t modelOutputIndexes[1] = {3};
NeuronModel_identifyInputsAndOutputs(model, 2, modelInputIndexes, 1, modelOutputIndexes);
NeuronModel_finish(model);

Step 9. Create and finish the Compilation

NeuronCompilation* compilation = nullptr;
NeuronCompilation_create(model, &compilation);
NeuronCompilation_finish(compilation);
free(buffer);   // The DLA buffer can be released after compilation is finished

Step 10. Create the Execution

NeuronExecution* run = nullptr;
NeuronExecution_create(compilation, &run);

Step 11. Set the input via AHardwareBuffer

const uint32_t input_size = DLA_INPUT_BIN_SIZE;
const auto usage = AHARDWAREBUFFER_USAGE_CPU_READ_OFTEN
                 | AHARDWAREBUFFER_USAGE_CPU_WRITE_OFTEN;

AHardwareBuffer_Desc iDesc{
    .width = input_size,
    .height = 1,
    .layers = 1,
    .format = AHARDWAREBUFFER_FORMAT_BLOB,
    .usage = usage,
    .stride = input_size,
};

AHardwareBuffer* inputAhwb = nullptr;
AHardwareBuffer_allocate(&iDesc, &inputAhwb);

void* inputBuffer = nullptr;
AHardwareBuffer_lock(inputAhwb, usage, -1, nullptr, &inputBuffer);
memcpy(inputBuffer, input_buffer, input_size);
AHardwareBuffer_unlock(inputAhwb, nullptr);

NeuronMemory* iMemory = nullptr;
NeuronMemory_createFromAHardwareBuffer(inputAhwb, &iMemory);
NeuronExecution_setInputFromMemory(run, 0, nullptr, iMemory, 0, input_size);
NeuronExecution_setInputFromMemory(run, 1, nullptr, iMemory, 0, input_size);

Step 12. Set the output via AHardwareBuffer

const uint32_t output_size = DLA_OUTPUT_BIN_SIZE;

AHardwareBuffer_Desc oDesc{
    .width = output_size,
    .height = 1,
    .layers = 1,
    .format = AHARDWAREBUFFER_FORMAT_BLOB,
    .usage = usage,
    .stride = output_size,
};

AHardwareBuffer* outputAhwb = nullptr;
AHardwareBuffer_allocate(&oDesc, &outputAhwb);

NeuronMemory* oMemory = nullptr;
NeuronMemory_createFromAHardwareBuffer(outputAhwb, &oMemory);
NeuronExecution_setOutputFromMemory(run, 0, nullptr, oMemory, 0, output_size);

The output buffer does not need to be locked or written in advance; simply read it after inference completes.

Step 13. Run inference

NeuronExecution_compute(run);

Step 14. Read the output result

void* outputBuffer = nullptr;
AHardwareBuffer_lock(outputAhwb, usage, -1, nullptr, &outputBuffer);
// TODO: Read the inference result from outputBuffer and pass it to the downstream pipeline
AHardwareBuffer_unlock(outputAhwb, nullptr);

Step 15. Release resources

AHardwareBuffer_release(inputAhwb);
AHardwareBuffer_release(outputAhwb);
NeuronExecution_free(run);
NeuronMemory_free(iMemory);
NeuronMemory_free(oMemory);
free(input_buffer);
NeuronCompilation_free(compilation);
NeuronModel_free(model);

Best,
Jun