Unicode Problems With Japanese Text

All announcements, questions and issues related to the TsiLang Components Suite.
Post Reply
delphian
Posts: 6
Joined: Thu Jan 20, 2005 6:13 am

Unicode Problems With Japanese Text

Post by delphian »

When i try to use Suggest Translation or Auto Translate i get a Question mark appear in front of my unicode japanese text

ie i am trying to translate the word / phrase " Booking Date " , which i have 予約作成日 in the dictionary and it returns ?約作成日


i believe the point where the problem occurs is in the following line of code.

S := FIntf.GetTranslation(BaseLang, ActiveLang, Cells[BaseCol, J]);

in siTransEditor , Function sbDictAutoClick
isiticov
Site Admin
Posts: 2383
Joined: Thu Nov 21, 2002 3:17 pm

Post by isiticov »

Does the same apply when you use Find Translation button or only for Suggest and Auto-Translate?
delphian
Posts: 6
Joined: Thu Jan 20, 2005 6:13 am

Japanese Text ( Unicode )

Post by delphian »

Yes it gives the same error when i use find translation. it only happens with japanese text , we have korean as well and its perfect.


Kind Regards



Dephian
isiticov
Site Admin
Posts: 2383
Joined: Thu Nov 21, 2002 3:17 pm

Post by isiticov »

Strange, could you please send us the DIC file use or at least problematic part of it? Then we will try to debug and test here in order to find the problem. Thanks.
delphian
Posts: 6
Joined: Thu Jan 20, 2005 6:13 am

Unicode Problems With Japanese Text

Post by delphian »

I've found the reason why the question marks appear , it seems our importing program was placing the unicode header of $FFFE on each unicode word in the XML import.


We have removed it and it now works perfectly.


maybe on your XML import you should scan the text and if the first wide char = $FEFF then remove it.


Kind Regards

Delphian
isiticov
Site Admin
Posts: 2383
Joined: Thu Nov 21, 2002 3:17 pm

Post by isiticov »

Actually, only first two bytes of the file must be #FEFF for Unicovde files.
delphian
Posts: 6
Joined: Thu Jan 20, 2005 6:13 am

Post by delphian »

Yes the its included in the first two bytes of the XML file ... not on each string



Delphian


we tested it at the office and it now imports properly
Post Reply