Nostr Web Client

I saw a post this morning (which I will never find again, due to lack of effective Mastodon search) about the New York Times stripping diacritics (accent marks) from text that included Vietnamese names and words whose meaning changes without those marks.

So I thought a fhort thread might be useful to describe how we got to this place.

It's technology, not anti-Asian bigotry, in case you're tempted to think that. Print content management/typesetting is the problem.

Start way back in the early 20th century, when fonts were made of metal and wire service dispatches were sent via telegraph. Hand telegraphy evolved into machine telegraphy, with typewriter-like automation (later called Teletype) pounding out text.

But what if the wire service could set stories in type directly? Without somebody rekeyboarding every story?

Mergenthaler Linotype, the leading maker of machines for setting type for printing, experimented with automated typesetting around 1902. It took nearly two decades for a newspaper publisher -- Frank Gannett (yes, THAT Gannett) to see the economic value and push that idea into operation.

#journalism #tech

Steve Yelvington 2y ago

Gannett used tape-punching tech to separate keyboarding from Linotype machine operation, speed up the system, cut costs, and allow centralization of what we would now call data entry. Wire services eventually followed, setting up Teletypesetting services that sent stories as data neatly formatted for standard column widths, including hyphenation and justification.

There were no computers then, of course. The data was transmitted over phone lines using signals broken into 6-bit codes in a standard called TTS. At the receiving end, the signals generated both typewritten copy readable by humans, and punched tape that could be fed to the Linotypes.

You may know that 6 bits can only represent 64 possibilities (2 to the 6th power). That's not enough characters. Shift, Unshift, Upper Rail and Lower Rail were used to enable majuscule (capital) and minuscule (little bitty) letters, and font changes such as bold or italic.

Accents? Diacritics? Nope. Not in the codeset. Not in the font magazine either. A Linotype could cast them, if properly equipped for the task, but they weren't in the least-common-denominator world where newspaper production lived.

So stylebooks of wire services and major newspapers simply said: don't use them.

Reply to this note

Please Login to reply.

Discussion

Steve Yelvington 2y ago

When computers came along, they did not immediately solve the problem. The 7-bit ASCII computer code set offered more possibilities, but it assumed you were writing in English. (The "A" stood for American.)

If you're older than 30 you may remember what a mess the Web was before the evolution of Unicode and widespread support for decent fonts. In theory, Unicode solves everything, but i still occasionally see text show up with junk characters where my browser can't figure out what was intended.

So what does this have to do with Vietnamese? The language is written in chữ Quốc ngữ (let's hope the diacritics stay put). It is a script based on the Latin alphabet with diacritics used to indicate pronunciation tone.

Vietnam's Lao, Thai and Khmer neighbors stuck with their own Sanskrit-based scripts, which you're not likely to see dropped into an English-language text. Instead, you'll see romanized transliterations (poorly standardized, which is the subject of an entirely different rant).

Most wire services have moved on from the 6/7-bit era, and in theory can easily represent text with rich diacritics. But print toolchains still may contain obsolescent pieces that don't understand the modern world, and font can be quite limited in what they can display.