Hacker Timesnew | past | comments | ask | show | jobs | submitlogin

Well dang: https://docs.python.org/3.12/library/locale.html

> There is no way to perform case conversions and character classifications according to the locale. For (Unicode) text strings these are done according to the character value only, while for byte strings, the conversions and classifications are done according to the ASCII value of the byte, and bytes whose high bit is set (i.e., non-ASCII bytes) are never converted or considered part of a character class such as letter or whitespace.



I actually like that python doesn't do locale aware case conversions. You can use ICU* for that, though it shows more warts (like the i in republic and that you have to handle Chinese banknotes differently as well):

  % env - LC_ALL=en_US.UTF-8 PATH="$PATH" python3
  Python 3.8.12 (default, Nov 13 2021, 10:49:08) 
  [Clang 11.0.3 (clang-1103.0.32.62)] on darwin
  Type "help", "copyright", "credits" or "license" for more information.
  >>> from icu import UnicodeString, Locale
  >>> s = b'the Republic of T\xc3\xbcrkiye'
  >>> s = s.decode()
  >>> s
  'the Republic of Türkiye'
  >>> lc = Locale("TR")
  >>> s = UnicodeString(s)
  >>> s
  <UnicodeString: 'the Republic of Türkiye'>
  >>> s = s.toUpper(lc)
  >>> s
  <UnicodeString: 'THE REPUBLİC OF TÜRKİYE'>
  >>> s = str(s)
  >>> s
  'THE REPUBLİC OF TÜRKİYE'
  >>> s.encode()
  b'THE REPUBL\xc4\xb0C OF T\xc3\x9cRK\xc4\xb0YE'
  >>> s = UnicodeString(s)
  >>> lc = Locale("CN")
  >>> s = s.toLower(lc)
  >>> s
  <UnicodeString: 'the republi̇c of türki̇ye'>
  >>> s = '壹,貳,參,肆,伍,陸,柒,捌,玖,拾,佰,仟,萬'
  >>> s.encode()
  b'\xe5\xa3\xb9,\xe8\xb2\xb3,\xe5\x8f\x83,\xe8\x82\x86,\xe4\xbc\x8d,\xe9\x99\xb8,\xe6\x9f\x92,\xe6\x8d\x8c,\xe7\x8e\x96,\xe6\x8b\xbe,\xe4\xbd\xb0,\xe4\xbb\x9f,\xe8\x90\xac'
  >>> s = UnicodeString(s)
  >>> s
  <UnicodeString: '壹,貳,參,肆,伍,陸,柒,捌,玖,拾,佰,仟,萬'>
  >>> s = s.toLower(lc)
  >>> s
  <UnicodeString: '壹,貳,參,肆,伍,陸,柒,捌,玖,拾,佰,仟,萬'>

* https://pypi.org/project/PyICU/

edit: Oh neat there's a hansfin keyword that I had not known about.

https://unicode-org.github.io/icu/userguide/locale/#keywords

https://stackoverflow.com/questions/6224177/how-to-convert-e...




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: