Wednesday, 5 April 2017

The many failings of language support

At the hardware level, in the operating system, and in the software we use every day, language support is sorely lacking.  If you're a developer of some kind, you might take offence at this because you reckon you built decent language support into whatever you built.  In which case, please read on, and if you've taken care of everything I mention, award yourself a gold star.  (And show me your product, because I'll be impressed if you've made something properly good.)

Lack of decent language support stems from a failure to grasp one fundamental truth: people do not operate exclusively in one language.  Whether you work in a multinational company, study a foreign language, have family from another country, live in one of the many countries where there is more than one native tongue, or just want to write foreign people's names correctly, the hardware, the OS and the software will likely get in your way more than they will help you out.

Devices with keyboards already start on the back foot.  If you live in the US you'll get a US keyboard; if you live in the UK you'll get a subtly-but-crucially-different UK keyboard; each of the Scandinavian countries gets its own special variant; and the Germans and French have different arrangements of the letter keys.  These keyboards are each suited pretty well to typing in their own language, but most don't support typing characters from other languages - the Scandinavian keyboards excepted, which have keys that allow entry of many accented characters.  That said, any proficient typist will have built up muscle memory of the keys, so changing the keyboard layout is not a way to make things better.

Windows users have by far the worst luck.  Windows's language model supports switching between different languages, each of which is tied to a keyboard layout.  The problem with this approach is that people only have one keyboard, so when I switch between typing in English and Danish, I want to keep the keyboard layout that works with my keyboard.  Except that, if I have an English keyboard, I can't then type the Danish characters.  What's the alternative?  Entering character codes like Alt+0230 for æ.  Except I can't do that, because you have to enter the numbers on the number pad, and my laptop doesn't have one.  So then you have to either switch to a keyboard layout that doesn't resemble your keyboard, or open Character Map, or make your own keyboard layout.

Microsoft did, at least, make a "US international" keyboard layout, and they have support, as Linux does, for "dead keys" - where, for example, you press Alt Gr+apostrophe followed by e to get é.  This is great... except that it's wrong.  You don't think "I want to write an acute accent with an E underneath", you think "I want to write an E with an acute accent".  The shortcut should not be AltGr+apostrophe, E.  It should be AltGr+E, apostrophe.  (The Danish keyboard is blessed with several dedicated accent keys, and they all work the wrong way around for our brains.)  But in any case, such dead-key support for accented characters does not cover the whole gamut.  Eastern European languages have characters like ą and ł (Polish) and č (Czech and others), which are not supported by "US international" keyboards.  So much for globalisation.

Smartphones are an entirely different story.  Use the on-screen keyboard on an iPhone or Android phone, hold down the letter you need an accent on, and you get a choice of accented characters.  This is great... except that, again, it's not complete.  I have three languages on my iPhone: English, Danish and Polish.  Each comes with its own keyboard.  From the English and Danish keyboards I can't access the Polish characters ą and ę - even though my phone knows that I use the language with those characters in.  And when I switch from the English keyboard to the Danish one, it squeezes another three keys on to the screen, which messes up the layout and my muscle memory.  And iOS, despite its efforts, still has a major flaw: hardware keyboards do not have the same behaviour as the screen keyboard.  My Brydge keyboard for my iPad Mini is an awesome piece of hardware, but when I hold the A key, I don't get the choice of whether I want an a, an ą or an æ.  I have to bring up the screen keyboard.

Mac users at this point can be a little smug, since their method of entering accented characters with a physical keyboard works much like that of the iPhone: hold a key, get the choice of accented characters.  However, it also suffers from the same problem that not all accented characters are shown.

OK, so some systems have taken into account that people don't operate exclusively in one language.  But here's the next thing that they don't take into account: people who use multiple languages do not use them one at a time.  They mix them.  They drop foreign words into their sentences, refer to place names from other countries... you get the idea.  Switching keyboard layout to type a foreign word then switching back again is just not a good user experience.

And we haven't even got on to the software yet.  The spell checker, and its evil descendant autocorrect, are prone to wreak havoc on any polyglot's work.  Start writing a sentence in any Microsoft Office program and it will red-wavy-underline words until you get to the end of a sentence, at which point it will try to guess the language you're using.  (This is not connected to the language you have Windows set to, oh no.)  A quotation, enclosed in quotation marks, will not be considered separately.  But there's a worse failing, that comes from another neglected fact: people using multiple languages likely do not have native proficiency in them.  In Danish, the word i means "in".  It is a single lower-case I.  Because of this, when many Danes write in English, they tend to write i when they mean in.  It's an easy and understandable mistake, but good old Autocorrect thinks "ah, you mean I", and capitalises it.  Its work done, there is no wavy-underlining to do, and the error goes unnoticed.

For those who write software that doesn't depend on text input, you do not escape my wrath.  Changing keyboard shortcuts between languages causes all manner of merry hell.  From Windows itself (Minimise in English Windows is Alt-Space, M; in Danish Windows it is Alt-Space, N) to the software that runs on it, if you change the keyboard shortcuts across languages you break usability.  Most of the Internet is in English, so most how-to guides are in English.  When a Danish user of some software is looking for help, chances are they'll find help in English.  If that help details keyboard shortcuts, chances are the instructions won't work for them.  (Here, iOS gets credit for keeping the keyboard shortcuts consistent across languages.)

All of these failings, and we haven't even got on to different alphabets yet.  The Cyrillic alphabet has more letters than the Roman alphabet; as such, the Russian keyboard layout is very different to those layouts using the Roman alphabet.  But what about those of us foreigners who occasionally need to write in Russian?  Google Translate has a great way of handling this: you can write in Romanised Russian, and it will convert.  Exactly the same approach could be done using a keyboard layout: type ya and you would get the letter я.  This approach would support everything except the letter shch (you could type sc for that, I guess), the hard sign and the soft sign.  But that's a tiny problem compared to the options we have today: even Russian keyboard layouts that put the letters as close as possible to their usual layout on a QWERTY keyboard still require a lot of learning.

I'm lucky enough that I am both sufficiently technically savvy and sufficiently bloody-minded to do things like use Microsoft's keyboard layout creator to work around these issues as far as possible.  Most people - even those clever-clogs who know several languages - are not.  The industry could, and should, do much better.