[LLaMA3-8B] [PTQ] Common Issues in Calibration Dataset Generation

[jing_liao]

Environment Information

• Platform: Genio-720, D9300
• Toolkit Version: GAI-Deployment-Toolkit-v2.0.2_llama3-8b-v0.1.tar.gz

Issue Description:

1. Tokenizer Class Mismatch:

The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. 
The tokenizer class you load from this checkpoint is 'PreTrainedTokenizerFast'. 
The class this function is called from is 'LlamaTokenizer'.

2. Unsupported Argument padding_side:

TypeError: _batch_encode_plus() got an unexpected keyword argument 'padding_side'

Solution:

1. For tokenizer class mismatch:

• Add "tokenizer": "pretrained_fast" to config.json.

2. For argument error:

• Install the compatible version of transformers:

pip install transformers==4.41.2

[joying.kuo]

The solution was tested and verified on July 7th, 2025