Create An Utf-8 Csv File In Python

I can't create an utf-8 csv file in Python. I'm trying to read it's docs, and in the examples section, it says: For all other encodings the following UnicodeReader and UnicodeWr

Solution 1:

You don't have to use codecs.open; UnicodeWriter takes Unicode input and takes care of encoding everything into UTF-8. When UnicodeWriter writes into the file handle you passed to it, everything is already in UTF-8 encoding (therefore it works with a normal file you opened with open).

By using codecs.open, you essentially convert your Unicode objects to UTF-8 strings in UnicodeWriter, then try to re-encode these strings into UTF-8 again as if these strings contained Unicode strings, which obviously fails.

Solution 2:

As you have figured out it works if you use plain open.

The reason for this is that you tried to encode UTF-8 twice. Once in

f = codecs.open('eggs.csv', 'w', encoding="utf-8")

and then later in UnicodeWriter.writeRow

# ... and reencode it into the target encoding
data = self.encoder.encode(data)

To check that this works use your original code and outcomment that line.

Greetz

Solution 3:

I ran into the csv / unicode challenge a while back and tossed this up on bitbucket: http://bitbucket.org/famousactress/dude_csv .. might work for you, if your needs are simple :)

Solution 4:

You don't need to "double-encode" everything.

Your application should work entirely in Unicode.

Do your encoding only in the codecs.open to write UTF-8 bytes to an external file. Do no other encoding within your application.