I run a small YouTube channel.
And now and again, I record short videos documenting how to “do” certain things using Linux.
I make them as much for myself as for my 300-odd subscribers.
Because Linux, or rather doing things with it, tends to … you know … be quite complicated. And I can’t always remember how I got X to work three months later.
It’s nice to create documentation I can refer back to and it’s even better if others find it interesting as they occasionally tell me they do. But, for now at least, that’s about all there is to it.
Yesterday evening, I recorded a short video describing how to look up the time zone database (tzdb) to find the right way to denote the time zone on a certain world clock program (gworldclock). I wasn’t expecting that the video would give Netflix a run for its money. It hasn’t. But it has brought me into contact with a world so wonderfully weird that it could well the stuff of fiction. Thankfully it isn’t.
As most techies know, time zone setting is a fairly elementary feature of computing which most operating systems bake into their graphical user interfaces (GUI). Time zones are attached to locales. Setting a locale is often done on the basis of rough geolocation which users can manually override. Once set, users typically never need to change it unless they move countries. “Ah, I see Paul has been up to his usual tinkering” might be a clever excuse for being late to a Zoom meeting. But it would very seldom be a truthful one (Eggert, the world time zone coordinator, will be introduced later).
Linux, of course, tends to excel in letting users jump under the bonnet and denote changes themselves. That’s largely why I, and many, love it. So we get to see from a closer distance exactly how all this arcane technology works.
Typically, when I’m making my videos, I’ll do a bit of research before setting up shop and hitting the record button. I don’t script most of these, but I like to keep a few notes at the ready.
And this particular production process led me to a series of completely unexpected discoveries sometime in the early hours of yesterday morning.
Perhaps I had simply drunk too much coffee (at most hours of most days I usually have). But I found them all oddly engrossing as they came to me in slow waves of weirdness, one crescendoing after the other:
- All Linux and Mac-based computers pull their time zones from a massively important database — the time zone database. The process of defining time zones is centralized. This is actually quite a big deal in its own right because people tend to grossly underestimate how pivotal Linux is to … the entire internet and technology as we know it. It may constitute a small percentage of desktop users and be an OS largely favored by nerds and computer developers. But in server-land it’s actually the dominant operating system, especially on the public cloud infrastructure that is rapidly usurping the diminishing role that on-premises infrastructure has to play in getting packets of data from hosts to users (in normal language: making the internet work). AWS instances, for instance, default to Amazon’s Linux spin-off. Virtually all the world’s supercomputers used for everything from weather forecasting to simulating physics experiments run on Linux. Android is a fork of Linux. (I don’t play the “will this be Linux’s year on the desktop?” game. People have been wrong too many times and I don’t particularly care either way). If you’ve ever used an Android device, received a weather forecast, or accessed a website (you’ve probably done all those things), then you’ve benefited from the existence of Linux.
- The time zone database — which is sometimes called the Olson data or zoneinfo database — has a fascinating history. Unlike most databases, it’s been deemed interesting enough to have a wikipedia entry maintained about it. Just as curiously, the database has also been the subject of litigation. In fact, it’s been deemed so essential to the operation of computers worldwide that ICANN — which approves top level domains (TLDs) among many vital internet-relation functions — has brought it under its auspices. This generally only happens when something is really fricking essential to the internet. Or more accurately, the upkeep of the database is now the responsibility of the Internet Assigned Numbers Authority (IANA) — which sounds vaguely like the sort of organization that would feature either in an Orwell novel, The Martix, or be the go-to name for an intelligence service looking to create complete stultification about what a front company did (“So … what do you for a living.” “Oh, I’m a time zone specialist at the IANA”. *For the sake of clarification: I’m not alleging that either IANA or ICANN is a front company of any intelligence service although that would probably make good fodder for a bad conspiracy theory).
But wait, there’s more. Lots more, in fact.
You thought time zones were straightforward or even boring or at least eminently controversial?
So I did I. Until last night.
Not only are time zones apparently a longstanding menace for computer developers, but the time zone maintenance community is currently, it seems, mired among some procedural dispute regarding how this essential database should best be maintained. Of course that’s an interesting fact in its own right: there is a world time zone data community.
In fact, The Register recently described them as being no less than “up in arms” about the direction the project was proceeding down. When this saying is being rolled out in reference to a group of middle age computer scientists who discuss time zones in their spare time, you know that something serious must be going down.
The community’s water cooler, apparently is a listserv that looks pretty much like what you’d expect an actual listserv to look like.
It sports the characteristic bald look favored by computer scientists around the world. Its markup unsullied by a single line of CSS. Hyperlinks its most high-tech feature.
But to understand why the time zone data community can prove to be so oddly mercurial, its decision-making process volatile, you have to watch Tom Scott’s video first — no cheating, all the way through.
As Tom points out, the best way to deal with time zones is simply not to deal with time zones by integrating a library into your code. Like hangovers, they’re best avoided at any cost. And like bad hangovers, once you already have one, time zone problems are all sorts of hassle to fix.
The maddening series of problems and exceptions and inconsistencies he speaks of with the air of a man teetering ever closer to the brink of insanity? The ones that thankfully any developer can now avoid by simply integrating the time zone database?
That’s the thankless work of the people maintaining the time zone database. Tom Scott’s video may have driven him mad for five minutes. For these guys, tackling these issues is a career-long endeavor.
As befits the significance of their mission, the minds at the top of the time zone management community are serious guys. Tech titans and heavyweights.
Such as noted Java expert Stephen Colebourne who recently took fire at a recent series of updates to tzdn.
At least, that’s how I understand it. (I asked Colebourne to explain things more simply and he kindly indulged me with an answer; a difference of vision among time zone enthusiasts might be the neatest summary anybody can advance. Colebourne and the rest of the time zone community, I believe, remain on good terms. But you could say that professionally speaking they’re no longer on the same time zone. Sorry, I had to.)
In a blog bearing the utterly fantastic title “Big problems at the timezone database” Colebourne alleges that:
Paul Eggert is the project lead of the timezone database hosted at IANA, a position referred to as the TZ Coordinator. He is an expert in the field, having been involved in documenting timezone data for decades. Unfortunately, he is currently ignoring all objections to an action only he seems intent on making to solve an invented problem that only he sees as important.
Colebourne’s biting critique has already drawn the ire of other time zone mavericks on the Coordinator’s side. They charge that Colebourne is being overly hasty in his pronouncements. He’s jumped the clock, so to speak.
Not only can’t the time zone titans currently agree on the best way to carry the timezone database forward, it seems. But the entire process of codifying and standardizing time zones is also decidedly contentious political business with a long and tumultuous history to go with it.
Those who enter the fray need to be therefore not only technical heavyweights but also prepared to have the occasional audacity to stand up to countries like the Hashemite Republic of Jordan and tell them that their attempt to prematurely end DST is unacceptable and will not be promulgated in the database.
Practically speaking, the time zone maintenance process is tightly controlled and shared with major industry stakeholders (apparently). But ultimately — or at least on paper — the time zone database represents a sort of final arbitration of its decisions. At least as to how time zones are interpreted technically.
Like many other nations — the Germans probably aren’t to be reckoned among them — the Jordanians have had enough with summer time this year and are throwing caution to the wind and moving the clock over one month ahead of schedule. They’re going all YOLO on DST. Again.
But it could be worse.
Weary time zone mavericks are bursting to the seams with horror stories of African states who made rash time zone decisions on only four days’ notice. Warlords and dictators, in the midst of nervous breakdowns, have been known to take out their anger on time zone conventions. Perhaps they’re secretly just trying to troll people like Eggert.
And so time zone politics are an occupational hazard in this little corner of the tech universe. Political leanings — or rather adjudications — are sometimes called for. Those who rise to the heights of this part of the open source community rarely do so without having taken at least one stand on one contentious zone.
The ramifications of these decisions don’t go unnoticed, either. A thread from Hacker News by Y Combinator provides a rare look inside the obscure listservers where much of the coordination for the project happens and shows that those who are most closely involved are acutely aware of the weightiness of the project..
As one user commented in response to another poster raising doubts over whether time zones could really be that big of a deal:
“We have a client in Azerbaijan who was very angry their timezone was shown as Asia/Yerevan. As you may know, Azerbaijan is in a state of cold of war with Armenia.”
And here’s something else I learned: Pre-independence Ireland once operated its own time zone — Dublin Time — which was stamped out by the British as punishment for the Easter Rising.
All this is why the official source code for the world time zone community reads less like a text database and more like .. something you may never have seen before even if you’re the type to pore over with curiosity the commented lines that some developers leave in their code.
I would call it an entirely sui generis form of literature. Part historic intrigue (those lines are commented out). Part actual code that can be read by computers. And part trivia reference.
It is suffused, from file to file, with the sense of heaviness that could only come from men and women who have spent large parts of their life tracking the evolution of time zones and who (one sometimes feels) are lamenting the complete minefield they have waded into. Eggertian I would call it.
These are the people for whom adjudicating upon such matters as France’s attempted move to the decimal system and the International Atomic Agency’s periodic decision to add nuclear seconds to the world time system are the bread and butter that they grapple with at work every day. While we’re planning content marketing or just developing code — on computers with time zones set by their labor.
Time zone data insiders say that every single one of these high stakes deliberations represents a near Y2K disaster that must be averted. If Hollywood thought more imaginatively, it could even be the stuff of blockbusters. Another rogue country has pulled a time zone trick. Better think about a response before it’s too late.
Although most computer users have never heard of the project or its maintainers, the decisions of the time zone maintainer and his cohorts affect the daily computing lives of hundreds of millions of users worldwide.
After all, a keystroke on the Americas file could change EST to CST (or PST).
Zoom meetings could be missed in their millions, vast numbers of transactions incorrectly recorded in ledgers. The consequences of their actions are, frankly, rather enormous.
And the man heading all of this up: