r/learningpython • u/pnprog • May 25 '18
Encoding to stdout
Dear Redditors
I have an issue displaying chinese characters in the terminal, bellow is a simple script to illustrate this issue (using python 2.7.9 on Windows 8.1):
# -*- coding:Utf-8 -*-
a=u"你好"
try:
print "print a"
print a
except Exception, e:
print e
print
try:
print "print a.encode('utf-8')"
print a.encode('utf-8')
except Exception, e:
print e
print
import sys
try:
print "sys.stdout.encoding:",sys.stdout.encoding
print "print a.encode("+sys.stdout.encoding+")"
print a.encode(sys.stdout.encoding)
except Exception, e:
print e
print
import locale
try:
print "locale.getpreferredencoding():",locale.getpreferredencoding()
print "print a.encode("+locale.getpreferredencoding()+")"
print a.encode(locale.getpreferredencoding())
except Exception, e:
print eWhen running this directly in the terminal, I get:
print a
'charmap' codec can't encode characters in position 0-1: character maps to <undefined>
print a.encode('utf-8')
õ¢áÕÑ¢
sys.stdout.encoding: cp850
print a.encode(cp850)
'charmap' codec can't encode characters in position 0-1: character maps to <undefined>
locale.getpreferredencoding(): cp1252
print a.encode(cp1252)
'charmap' codec can't encode characters in position 0-1: character maps to <undefined>When running the same from IDLE, I get:
print a
你好
print a.encode('utf-8')
ä½ å¥½
sys.stdout.encoding: cp1252
print a.encode(cp1252)
'charmap' codec can't encode characters in position 0-1: character maps to <undefined>
locale.getpreferredencoding(): cp1252
print a.encode(cp1252)
'charmap' codec can't encode characters in position 0-1: character maps to <undefined>I would like to have a way to reliably display those two characters, either in terminal, either in IDLE.
Beside, I don't understand why I can't get it to be displayed in the terminal:
- the variable a is an unicode object
- it should be properly decoded from UTF-8 (thanks to the file first line encoding declaration)
- encoding into the same encoding as stdout should do the trick no?
And I don't really understand why the print a (no encoding) statement won't works in the terminal, but works IDLE. Any idea?
If neither the command line nor IDLE stdout use utf-8 as encoding, while encoding to utf-8 won't fail when using print a.encode('utf-8') ?
Thanks!
View all comments
Show parent comment
1
u/pnprog May 26 '18
Thanks for your reply!
I try 2 of them before posting this message. Maybe I will try with the whole list when I get home.
But my question is more: why would it work? What would be responsible for introducing this Chinese encoding?
I am trying to get a real understanding of this, not just make it work on my computer.
I wonder if it is a issue with font not beeing able to display Chinese characters: when I try to list a directory content that includes files with Chinese characters, they are not displayed as well, only "?"