DIR command in XP does not display Chinese correctly in English Windows

This is a very annoy issue I have just come across mistakenly believing XP being a Unicode OS and that its DIR command will display any characters you can create mirroring that in the Windows Explorer.

Such is not the case. While this post discusses issues with the DIR command in XP in handling Chinese characters, the problem applies equally to other languages.

I have a test file like this shown in Windows Explorer:

In CMD the DIR command returns this:

Which is not what I want or expect. So what is the reason behind this? I thought PowerShell being a .Net implementation of a shell can surely handling this. Despite following these advices, it produces the same result as above.

It turns out that the DIR/CMD uses non-Unicode API to display the file name as a result one has to change the "Language for non-unicode programs" in "Regional and Language Options" via the Control Panel to the Chinese code page as follows:

With this setting, the DIR command displays the file name correctly:

Just to prove the point, I then use the chcp command to change the code page to 1252 and this produces the following result:

Which is what we have got before.

While the DIR displays the file name incorrectly, if you starts CMD with the /U switch and you redirect the DIR output to a text file, the content of the text file is in full Unicode and the Chinese file name is displayed in say Notepad correctly.

So the moral of the story is that you must set the "Language for non-Unicode Programs" to the correct code page in order to produce visually correct file name in the DIR. This has no effect on batch file or redirection if you launches the CMD with the /U.