RTL support and other languages ?

[PrestigeDevop]

Hello , it seems that the model library doesn't support the Arabic language .. not sure if this due to the model-qa script or because the inference - runtime doesn't support it ?

it does actually output Arabic prediction but it doesn't respond if there's question mark "؟" at the end for a Q&A test.

[natke]

Thank you @PrestigeDevop! We will look into this for you

[natke]

Hi @PrestigeDevop

We have a streaming detokenization issue for which a fix is currently in progress.

Can you please try running your input with the model-generate.py script also in the examples/python folder? This will output the generated text in one operation rather than streaming one token at a time.

python model-generate..py -m <path to your model> -pr <your prompt>

[natke]

Hi again @PrestigeDevop,

We have made a number of fixes in the decoding functionality. Can you please try again with the latest release onnxruntime-genai 0.2.0?

[PrestigeDevop]

Hi again @PrestigeDevop,

We have made a number of fixes in the decoding functionality. Can you please try again with the latest release onnxruntime-genai 0.2.0?

HI there, thanks for your support ..

running phi3-qa.py seems work just fine (using the same model too) so it seems the issue with model-qa.py (still occur )