Switzerland has four official languages, with the English word "Switzerland" having a particular translation to each one. But if you choose a single official name in one of those languages, it is not fair to the others, so in addition an official name in each language, they have an official in a 'neutral' language: Latin.
> Due to its linguistic diversity, Switzerland is known by a variety of native names: Schweiz [ˈʃvaɪts] (German);[note 5] Suisse [sɥis(ə)] (French); Svizzera [ˈzvittsera] (Italian); and Svizra [ˈʒviːtsrɐ, ˈʒviːtsʁɐ] (Romansh).[note 6] On coins and stamps, the Latin name, Confoederatio Helvetica – frequently shortened to "Helvetia" – is used instead of the four national languages.
It's a thread of jokes. devoutsalsa started with a joke mixing the two meanings of "capital" using a fake example. I joked that "UniquenessError" isn't the right error, as some countries have more than one capital and two countries have no official capital.
One of those two is Switzerland, which has the official Latin name "Confoederatio Helvetica", leading zinekeller joke that the capital form was "HELVETICA". See https://en.wikipedia.org/wiki/Name_of_Switzerland .
I pretended my implementation of that non-extent programming language generated "Eidgenossenschaft", see https://en.wikipedia.org/wiki/Eidgenossenschaft . As that's a German word, I decided to use a blackletter typeface, specifically, the Fraktur in Unicode which is meant to encode mathematical alphanumeric symbols, then imply that my locale the reason I got a German word.
> That shouldn't be a uniqueness constraint as a country may have multiple capitals
And, in theory, two countries could share the same capital (as a condominium which is equal joint territory of both, or some kind of third territory belonging to neither), although I'm not aware that's ever actually happened. But, at the subnational level, it has happened in India – Chandigarh is the joint capital of two Indian states (Punjab and Chandigarh), each of which has it as its capital despite it being in neither of them, as well as being a union territory (and hence also capital of itself).
Reminds me of the bug in PHP where they used a call to toUpper without specifying locale to enable their case insensitivity and if your locale was Turkey, you couldn’t call any library calls with an i in them if you typed them in lowercase because, e.g., call to phpinfo() would get case-folded into PHPİNFO().
.ToUpper() is locale-dependent, so can only be used if the locale of the text in question is known. E.g. German ß capitalizes to SS, and .ToUpper().ToLower() should give you either 'ss' or 'ß' depending on what it was before. Always outputting 'ss' is okish and readable, but actually wrong.
Blindly calling .ToUpper() on anything is a typical anglo-centric mistake. Just don't use .ToUpper(), shoutcase is ugly anyways ;)
See also: one of the many "100 fallacies programmers assume about natural written language" documents or such.
How often do you see the new letter in German everyday life? Despite being German myself I don't visit Germany that often these days, I still read a couple of German publications regularly. I have never seen the new letter outside of discussions by software people about character handling.
I do sometimes, but I'm rather sensitive for the ẞ issue: My last name contains an ß and uppercasing would either mean keeping the ß lowercase – the Personalausweis does that (†) and it looks ugly — or doing the ß → SS transformation which is somewhat forbidden in identity documents; a name must be exact. Hence, someday in the future, hopefully, the ẞ. While personal names were a major motivation for the inclusion of the ẞ into Unicode, I’m always happy to see it in the wild in press or book titles or such.
† Although it’s Germany and of course there exists an obscure Verwaltungsvorschrift according to which you can write the non-machine readable field of the Personalausweis/Pass in lowercase, exactly for this use case. I didn’t know that last time but I fully intend to make some poor civil servants life a slight hell the next time I have to renew.
I was actually wondering if the driving factor is legal documents. ID cards show names in all-caps letters, which creates the dilemma that your ID might not show your actual name (notwithstanding international standards for travel documents that prescribe transliteration of non-latin characters; see ICAO Doc 9303 Part 3, section 6 [0] for examples)
That’s a good theory, especially as section 3.1 of that ICAO document explicitly permits the use of ß.
Bringing the thread back to the topic of this comment section: the ICAO document also calls the digits 0123456789 “Arabic” even though their shapes are closer to the original Hindi (Devanagari) forms than to actual Arabic digits — another “Hindi/Turkey” situation
That is correct and solves the roundtrip-problem (in this case and language). But uppercase 'ẞ' is just an additional option at the discretion of the writer, the recommended variant continues to be 'SS'.
> German ß capitalizes to SS, and .ToUpper().ToLower() should give you either 'ss' or 'ß' depending on what it was before
As long as there is no unicode SS character, we are into the "what color are your bits" problem or tolower needs to be language and word aware.
In .NET the uppercase and lowercase functions are culture aware (with defaults to system settings, which breaks more software than you might think) but not word aware AFAIK.
> As long as there is no unicode SS character, we are into the "what color are your bits" problem or tolower needs to be language and word aware.
It turns out there is such a unicode character -- ẞ/ß -- although based on other comments here it looks like it was added fairly recently.
Upper/Lower case stuff just seems to be at an annoying intersection where it has cultural and also programming significance. Or at least, people will use toUpper when they really want some case-insensitive sortable version of the string.
(based on some googling, probably localeCompare is the way to go in javascript at least).
>Blindly calling .ToUpper() on anything is a typical anglo-centric mistake.
Yes, one that you might make if you were for example, trying to make English text uppercase. Which is why it would be daft for anyone to suggest that their country has two different English spellings depending on the character case.
.toUpper() is a quick and mostly effective way to normalise strings for comparison if you're not sure what case the two strings to compare are in (eg: one has been input by a user). Yes, it's a shortcut, and occasionally you'll end up with a miss, but it's good enough to work 99% of the time, and the alternative is a LOT of code and data changes to handle a very small proportion of cases.
Hmm I think you miss the point. In some programming environments (like C# and Java) .toUpper() is always incorrect in code unless you are displaying the resulting string in a UI, as it uses the "current locale", which is whatever the user has selected for the machine. When e.g. comparing strings case-insensitively, you should always explicitly specify the locale where the conversion should happen instead of relying on an external configuration variable.
JavaScript actually seems to be the smart one here - its default .toUpperCase() uses the "locale-insensitive case mappings in the Unicode Character Database".
I don't think most Java and C# software is desktop apps? Surely in most cases it's the locale selected for the server or VM, which should be consistent?
(I'm not saying it's good coding practice, mind you, but it probably ends up accidentally working in a lot of cases.)
You write like you can know how and where the code will get executed in the future. :) Do you think that the authors of Windows 95 ever imagined the system would one day get ported to an obscure subset of a functional scripting language (Asm.js variety of JavaScript), and get booted in a hyper-text browser running on a PDA device with internet connection (web browser on a smartphone)? Yet - here we are: https://win95.ajf.me/
> I'm not saying it's good coding practice, mind you, but it probably ends up accidentally working in most cases
Fully agree. It's still bad practice and I high-five every linter that automatically flags it.
You make a good case (ha!). What if toUpper() and toLower() were omitted from standard libraries? Usually they are used, incorrectly, to do something like string comparison, which could be better served by a more specific method.
Only sz should use ß. Ss stays ss even in German-german. Switzerland got rid of the sz/ss distinction a long time ago.
So you need to be culture and word aware to do it „right“.
'sz' for 'ß' is sometimes used to make things roundtrip-proof in capslock, e.g. on military stencils. HTML calls it 'szlig'. Also, some use "Esszet" as the name of the character. But all are wrong in that ß isn't a ligature of s and z, it is a ligature of s and s. The shape of the character stems from the fact that in fractur writing and even some grotesk fonts, 's' at the end of a word was written 's', while 's' within a word was written 'ſ'. Thus the end of a word like Fuss was written Fuſs, giving a ligature of Fuß. No 'z' anywhere.
Originally ß arose as a ligature of s and z, or rather ſ and ʒ. In many older texts, or even current fonts, the second part of the ligature is indisputably a long-tailed ʒ
Only “wrong” in light of current usage, but not historically.
By this measure, the English name of “W” would be wrong because it’s not actually a “double-U” but a “double-V”. But at the time of the letter’s formation, U and V were not yet separate letters.
I always thought that German z used to look something between Ꙁ & з. ʒ looks pretty close so ſз became ß but Latin transliteration rules were ss instead. At least that's what I was taught in German class.
I often see this answer as a reason it's not particularly important to have on in a standard library.
I strongly disagree with that sentiment as often these simple things have subtle gotchas, and/or subtle differences in possible net effects creating much bigger problems than their diminutive implementations would suggest.
I generally feel the same way as you. My comment about getting it merged was not meant to imply that it shouldn't get merged, but that it might be difficult to. I find it very hard to predict which of these small QoL features will be accepted by the core devs and which features will be rejected for some reason or other.
> There is no way to perform case conversions and character classifications according to the locale. For (Unicode) text strings these are done according to the character value only, while for byte strings, the conversions and classifications are done according to the ASCII value of the byte, and bytes whose high bit is set (i.e., non-ASCII bytes) are never converted or considered part of a character class such as letter or whitespace.
I actually like that python doesn't do locale aware case conversions. You can use ICU* for that, though it shows more warts (like the i in republic and that you have to handle Chinese banknotes differently as well):
% env - LC_ALL=en_US.UTF-8 PATH="$PATH" python3
Python 3.8.12 (default, Nov 13 2021, 10:49:08)
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from icu import UnicodeString, Locale
>>> s = b'the Republic of T\xc3\xbcrkiye'
>>> s = s.decode()
>>> s
'the Republic of Türkiye'
>>> lc = Locale("TR")
>>> s = UnicodeString(s)
>>> s
<UnicodeString: 'the Republic of Türkiye'>
>>> s = s.toUpper(lc)
>>> s
<UnicodeString: 'THE REPUBLİC OF TÜRKİYE'>
>>> s = str(s)
>>> s
'THE REPUBLİC OF TÜRKİYE'
>>> s.encode()
b'THE REPUBL\xc4\xb0C OF T\xc3\x9cRK\xc4\xb0YE'
>>> s = UnicodeString(s)
>>> lc = Locale("CN")
>>> s = s.toLower(lc)
>>> s
<UnicodeString: 'the republi̇c of türki̇ye'>
>>> s = '壹,貳,參,肆,伍,陸,柒,捌,玖,拾,佰,仟,萬'
>>> s.encode()
b'\xe5\xa3\xb9,\xe8\xb2\xb3,\xe5\x8f\x83,\xe8\x82\x86,\xe4\xbc\x8d,\xe9\x99\xb8,\xe6\x9f\x92,\xe6\x8d\x8c,\xe7\x8e\x96,\xe6\x8b\xbe,\xe4\xbd\xb0,\xe4\xbb\x9f,\xe8\x90\xac'
>>> s = UnicodeString(s)
>>> s
<UnicodeString: '壹,貳,參,肆,伍,陸,柒,捌,玖,拾,佰,仟,萬'>
>>> s = s.toLower(lc)
>>> s
<UnicodeString: '壹,貳,參,肆,伍,陸,柒,捌,玖,拾,佰,仟,萬'>
But `"türki̇ye".upper()` works correctly, at least in Python 3.9.
Interestingly, `"türki̇ye".title()` does _not_ work correctly, returning `"Türki̇Ye"`, presumably because the "title-case" algorithm incorrectly detects \xcc\x87 as punctuation. Not sure if this has been fixed in 3.10 or 3.11.
Python 3.9.13 (main, May 24 2022, 21:13:51)
[Clang 13.1.6 (clang-1316.0.21.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> s = 'TÜRKİYE'
>>> s.lower()
'türki̇ye'
>>> s.lower().upper()
'TÜRKİYE'
>>> s.lower().title()
'Türki̇Ye'
>>> s.lower().encode()
b't\xc3\xbcrki\xcc\x87ye
Edit: It turns out that this behavior is documented [0], and the more-correct routine is `string.capwords` [1]:
> The algorithm uses a simple language-independent definition of a word as groups of consecutive letters. The definition works in many contexts but it means that apostrophes in contractions and possessives form word boundaries, which may not be the desired result ... The string.capwords() function does not have this problem, as it splits words on spaces only.
Compared this vs another comment and its interesting. Pulled the strings directly from the ISO website. I wonder if the 'i' in the lower case one is supposed to be special and not ASCII?
Python 3.8.10 (default, Mar 15 2022, 12:22:08)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> short_name_lower = "Türkiye"
>>> short_name = "TÜRKİYE"
>>> short_name.encode() ## The Ü and İ are UTF-8 chars
b'T\xc3\x9cRK\xc4\xb0YE'
>>> short_name_lower.encode() ## Note only the ü is special, i is just ASCII
b'T\xc3\xbcrkiye'
>>> short_name.lower()
'türki̇ye'
>>> short_name_lower.lower()
'türkiye'
>>> short_name.lower() == short_name_lower.lower() ## Looks the same, but it isn't
False
>>> short_name.lower().encode() ## The i has extra \xcc\x87 here
b't\xc3\xbcrki\xcc\x87ye'
>>> short_name_lower.lower().encode() ## The i doesn't have the extra here
b't\xc3\xbcrkiye'
>>> short_name_lower.upper().encode() ## So this is wrong too, since its just an ASCII i to start
b'T\xc3\x9cRKIYE'
> In Turkish, the character “i” becomes “İ” when capitalized, while the “ı” (a Turkish-specific character) becomes “I” (which looks just like the Latin upper case “I”).
> The out-of-the-box capitalization method implemented by developers or by localization tools by default is often the standard ‘toUpper()’, which doesn’t follow language-specific rules and will convert the “i” into an “I”. As for the lower case “ı”, it will simply fail to capitalize it at all. This will result in a very strange looking text in the game with uncapitalized characters and wrongly capitalized ones.
On my browsers (both Vivaldi and Safari, on MacOS), your string "türki̇ye" is rendered with two dots over the i (stacked vertically). I don't know if this is what you intended, but it doesn't seem to me as the correct lowercase form. But I'll defer to someone more versed in the local customs.
This might actually be a font thing, depending on whether any given font provides a precomposed version of "i" + "combining dot above" (and rendering it as just "i") or not.
Very interesting. On Windows in Chrome its just an i for me, but in Firefox its showing an extra dot between i̇y. But in my comment I can't get that to show up even in Firefox.
It's not i̇, it's i. Also capwords only capitalizes the first letter in each word. Here it's a bandaid. For locale-aware case conversions in python use ICU:
> It's going to be fun to watch developers wrap their heads around that one...
No, it will not. Nobody gives a damn. Nobody will implement it.
I am still trying to get people to correctly denote beginning and end of a day (much more useful, practical). Everybody I work with seems to be bent on using 23:59:59 as the end of the day rather than start of next day. Explaining that there is no 1s delay between end of one day and start of the next isn't helping either.
Sit on my lap youngling, and I will tell you tales of i18n horror... and I'm from an English-speaking country!
A realistic scenario is an Australian writing French poetry while on holidays in Turkey. Now you have an OS GUI with en-GB as the language, licensing and date/number formatting as per the AU region, French spelling dictionary, a US-101 keyboard layout, Turkey as the location, and GMT+3 as the time zone.
It's a rare piece of software that can handle this. Few vendors have staff that have even heard of such exotic places.
There are still people... many people... that deny the existence of places outside of the United States of America. Such filthy, heathen locations are surely a thing of myth, or legend!
Places where dates are formatted with the days before the month, followed by the year in some sort of weird, unnatural order.
Nations that have fallen into the trap of some sort of mass hallucination, or shared dream of common measurement units. Some sort of... metric, for space, time, and matter. Maybe they've been watching too much Star Trek!
Multicultural countries where strange unions of races are commonplace, and couples may want to watch Netflix in one language, but have subtitles in a different language. Neither of which are English for the hearing-impaired! Surely, nobody but the deaf are unable to comprehend the universal English language! Not to mention that such taboo couplings are, thankfully, still banned on this stream of holy virtue and shall not be permitted by the data scientists that have declared: "Your union is a statistically negligible!"
> There are still people... many people... that deny the existence of places outside of the United States of America.
Two fun anecdotes my mother has recounted when we were all in Denver for ~5 months in Denver 1996:
1. Asked where we were from on one occasion (after hearing an Australian accent): “Australia.” Response: “Oh, did you come by bus?” Now it is possible that there was a misunderstanding there, but mum doesn’t think so.
2. Of those that didn’t already know, only one person successfully identified where we were from, and that person was deaf. (It’s fascinating to try something like this on video without audio where there’s nothing obvious static to identify nationality: I find I can identify both Australian and American correctly at a rate considerably better than random, without ever having made a study of it.)
Can't we use our USA-centrism (centricity?) to make the behavior of the country drop-down just a little bit better? Surely I'm not the only one burdened with having to scroll past a hundred of these .. ahem.. OTHER... countries.. to find "United States of America".
Yes, I could begin typing "Unite.." but we all know that ends up on United Arab Emirates, and then when I continue on to "...d Sta..." and it doesn't work, only to find out this country picker uses "USA" or something similar so it breaks the autocomplete muscle memory.
Clearly, if we're the center of the known universe, we could use our power to make it a little easier to enter my billing and shipping information.
That just pisses off everyone else on the planet - plus the other countries in the Americas that consist of united states, and who equally consider themselves "American" much as French people are "European"
I know you're joking, but ideally forms like that should put the current country based on GeoIP at the top; these long selects really are kind of annoying: sometimes I accidentally use the keyboard to do weird shit because I accidentally didn't select the browser, some JS dropdowns don't allow typing at all, naming is somewhat inconsistent for some countries ("Netherlands" or "The Netherlands"?)
I'm only half joking! Yeah, I've found some put your GeoIP'd country at the top, which should be the default behavior. That throws me off occasionally, too, but I appreciate it!
Huh, but there is no 1 second gap between 23:59:59 and 00:00:00. The time is 23:59:59 for a whole second, and then once that second is done, the time is 00:00:00 the next day.
Thats why the day doesn't end at the instant of 23:59:59, nor does it end at 23:59:60 (if there is a leap second). It actually ends at 24:00:00. Which is the same instant in time as 00:00:00 of the following day. Every conforming implementation of ISO 8601 should know about that. And all software should conform to ISO 8601.
People seriously overestimate how many websites support Latin Extended-A to even be able to display ü or İ.
As someone whose last name contains a character from the same Unicode subset (ć), it's often a white square or just flat-out removed from my last name completely.
I think that's the point, people insist on using 23:59:59 as end of day instead of the correct 24:00:00 (which happens to be the same instant as 00:00:00 of the next day, since there's no gap between days)
My working theory on why this happens is that there are two different models and people sometimes have problem choosing the correct one.
When you say "Thursday" you mean entire 24h period. It is not a point in time, it is a label for a span of time.
But when you say "1 pm" you don't mean an entire hour, you mean a point in time that is more or less precisely 1pm.
It seems people extend the first model to a lot of cases where it is not correct. And so for many people their mental model of 23:59:59 is a label for a span of time that is one second in length.
If your model is that a day consists of 86400 "seconds", each second being a span of time of length of one second with a label like 12:37:28, then 23:59:59 is the last second of the day and 00:00:00 is the first second of the next day.
Working with spans is fraught with peril, though. It invites off-by-one errors.
If there's one person in the queue to the checkout, then span-wise, that person is the start of the queue and the end of the queue. But if the start and end are the same, what is the length of the queue?
(If you still insist it's one, you'll not be able to answer the question where the start and end of an empty queue is. There is no first or last person!)
```TSQL
select cast('2022-07-12 24:00:00' as datetime)
```
>>> The conversion of a varchar data type to a datetime data type resulted in an out-of-range value.
```python
from datetime import datetime
d: datetime = datetime(2022, 7, 12, 24, 0, 0)
```
>>> ValueError: hour must be in 0..23
I find it funny, Turkey was aldready a good test case for handling international users data :), I spend some hours on a bug in json with integer parsing of some users integer inputs in rares cases.
It was because of the way they write/parse integer using dots as separators. (Yes, the real problem was me having forgot to force server and client to use the same locale settings :) )
As a Mac user from the beginning I was about to say "that's an iPad, not a Mac", but lo and behold!
On a Mac now the alphabetic and digit keys do not autorepeat when held but instead pop up little menus of variants. Shifted alphabetic keys get different variants appropriate to their letter. Sadly for TÜRKİYE the single dot capital "i" is not one of them.
At least for my settings, none of the digits get options, nor do they autorepeat.
A completely useful overloading of "long press" of a keyboard key, but completely undiscoverable unless you make long strings of "vvvvvvvvv" as a pointer or some such, then you get a disappointment instead of what you wanted, but you will be enlightened.
That's horrible. The Turkish "i" is a never ending source of locale nightmares. I really wish they would pick a new letter for one of the sounds and use the standard "i" for the other. Like maybe use a diaresis.
Funny, there was a time when I was still learning English (well, it's a never-ending process, but still), and for quite some time I was using "red" as a past tense of "read"!
No spell-checker ever corrected me :-) In fact no one did, I just noticed one day that it's not how it's spelled in books...
Sometımes you see people wrıte lıke thıs or THİS when they are unused to the Turkish keyboard and it creates a lot of problems for some software, even crashes (to upper and back to lower is not the same word!).
It's going to be fun to watch developers wrap their heads around that one...