Aw man. I was using "WTF-8" to mean "Double UTF-8", as I described most recently at [1]. Double UTF-8 is that unintentionally popular encoding where someone takes UTF-8, accidentally decodes it as their favorite single-byte encoding such as Windows-1252, then encodes those characters as UTF-8.
It was such a perfect abbreviation, but now I probably shouldn't use it, as it would be confused with Simon Sapin's WTF-8, which people would actually use on purpose.
>  the future of publishing at W3C
That is an amazing example.
It's not even "double UTF-8", it's UTF-8 six times (including the one to get it on the Web), it's been decoded as Latin-1 twice and Windows-1252 three times, and at the end there's a non-breaking space that's been converted to a space. All to represent what originated as a single non-breaking space anyway.
Which makes me happy that my module solves it.
>>> from ftfy.fixes import fix_encoding_and_explain
>>> fix_encoding_and_explain(" the future of publishing at W3C")
('\xa0the future of publishing at W3C',
[('encode', 'sloppy-windows-1252', 0),
('transcode', 'restore_byte_a0', 2),
('decode', 'utf-8-variants', 0),
('encode', 'sloppy-windows-1252', 0),
('decode', 'utf-8', 0),
('encode', 'latin-1', 0),
('decode', 'utf-8', 0),
('encode', 'sloppy-windows-1252', 0),
('decode', 'utf-8', 0),
('encode', 'latin-1', 0),
('decode', 'utf-8', 0)])
Neato! I wrote a shitty version of 50% of that two years ago, when I was tasked with uncooking a bunch of data in a MySQL database as part of a larger migration to UTF-8. I hadn't done that much pencil-and-paper bit manipulation since I was 13.
The key words "WHAT", "DAMNIT", "GOOD GRIEF", "FOR HEAVEN'S SAKE",
"RIDICULOUS", "BLOODY HELL", and "DIE IN A GREAT BIG CHEMICAL FIRE"
in this memo are to be interpreted as described in [RFC2119].
[1] http://blog.luminoso.com/2015/05/21/ftfy-fixes-text-for-you-...
It was such a perfect abbreviation, but now I probably shouldn't use it, as it would be confused with Simon Sapin's WTF-8, which people would actually use on purpose.