pytorch - Why using GPT2Tokenizer convert Arabic characters to symbols?

Question

I am trying to use GPT2 for Arabic text classification task as follows:

    tokenizer = GPT2Tokenizer.from_pretrained(model_path)
    model = GPT2ForSequenceClassification.from_pretrained(model_path, 
                                                          num_labels=len(lab2ind)) 

However, when I use the tokenizer it converts the Arabic characters to symbols like this '?ù??aù??±'

question from:https://stackoverflow.com/questions/65897844/why-using-gpt2tokenizer-convert-arabic-characters-to-symbols

Answer 1

Waitting for answers