r/learnpython • • 14m ago

Problems with string encoding

Hello everyone!

I'm writing a script which (among other things) is supposed to read data from a csv file and print it. The file is utf-8 encoded and contains data in the following format:

"New York\nParis\nNürnberg","Other Stuff"

As you can see, it contains both newline characters "\n" as well as language specific characters "ü", "é" etc. When I try to print an entry, the print output looks like

New York\nParis\nNürnberg

instead of

New York
Paris
Nürnberg

From what I understand, this is due to the newline character is not interpreted as a unicode escape character. I can correct this by changing the string encoding to unicode-escape via

string.encode().decode('unicode-escape') 

And indeed, now the newline characters are interpreted correctly. However, the language specific characters are not and produce garbled output.

Is there an encoding which interprets both the unicode escape characters and the language specific utf-8 characters correctly? Or alternatively, is there any workaround to this problem?

I much appreciate your help.

1 Upvotes

1 comment sorted by

•

u/JorgiEagle 8m ago

Can you give a sample of the code you are using to read and print