[LLaMA3-8B] [PTQ] Common Issues in Calibration Dataset Generation

[jing_liao]

Environment Information

• Platform: Genio-720, D9300
• Toolkit Version: GAI-Deployment-Toolkit-v2.0.2_llama3-8b-v0.1.tar.gz

Issue Description:

1. Tokenizer Class Mismatch:

The tokenizer class you load from this checkpoint is not the same type as the class this function is called from.

2. Unsupported Argument padding_side:

TypeError: _batch_encode_plus() got an unexpected keyword argument 'padding_side'

Solution:

1. For tokenizer class mismatch:

• Add "tokenizer": "pretrained_fast" to config.json.

2. For argument error:

• Install the compatible version of transformers:

pip install transformers==4.41.2

[joying.kuo]

The solution was tested and verified on July 7th, 2025