In June 1989 I was the manager of Tandem's European Software Support Group OS and hardware department in Frankfurt am Main, Germany, and we were looking at a strange problem that appeared to affect only the architecture of a single customer, the Belgian Kredietbank, now part of the KBC Bank, in Brüssel, as the Germans spell it. Normally I don't mention customers, but in this case the story hangs around them, so instead I won't go into too much detail about the problem they were having.

In a country with a strong linguistic divide between French and Flemish, Kredietbank was known as a particularly Flemish bank. They also had a unusual network topology. From my recollection, there were two major nodes and a few others connected in two rings (thus giving fourfold protection against line outages), and a couple more out of town. From our point of view it was a kernel issue, but the transactions in question came across the network.

When I discovered this connection with EXPAND, Tandem's proprietary network, I discovered that the lead EXPAND developer, Nelson Bolyard, was in Frankfurt. I went to talk to him, and we decided that in view of the customer perception of the problem, we should both go to Belgium and discuss the matter with them.

We were going to go by car, but my car was in the workshop on that day, so I borrowed a staff car, a BMW 520i. We hadn't got far out of Frankfurt when I realized that the BMW was in very poor shape, and we couldn't rely on it getting us to Belgium. So I stopped in at Frankfurt airport and hired an Audi 100, a slightly more powerful version of my own car, and set off.

First problem: Nelson lived in Cupertino, and he wasn't used to German traffic. It was fairly heavy, so we didn't go much over 180 km/h, but that terrified him. It didn't last long, though—the journey was only about 400 km. We got to the Tandem office outside Brussels and met with the local systems engineer, Félix van Laethem, who took us to a hotel, dinner and then an extended tour of Belgian beers. I can't recall all the details, but they were pretty powerful. The last one I recall was called “Mort subite”—sudden death.

The next morning I woke up with the mother of all hangovers, and I don't think Nelson was feeling much better. We headed off to the customer site, in the Flemish-speaking north of Brussel (as the Flemish spell it). It was almost impossible to find a parking space, but finally I found one directly outside somebody's lounge room window.

The meeting didn't go well. It was clear that the customer was dissatisfied, and with good reason, though none of the Tandem representatives present were to blame; indeed, Nelson and I hardly knew the issue. In addition, we weren't feeling our best. But we had some ideas for fixing the problem, and we described them.

In the middle of the meeting, somebody came interrupted the meeting to tell us that one of the two main nodes had failed completely, and wouldn't come up—our worst case scenario, which we had long described as “System down, won't come up, customer screaming for blood”. The machine was quite close by in the same building, so we went and took a look.

One of the eight or so CPUs had failed with a halt %5113, if I recall correctly—effectively a page fault in kernel mode. The description I was given made it immediately clear that it was a fault I had recently dealt with, and we had supplied a fix—but they hadn't installed it.

But why should that bring down all eight CPUs? It didn't. The other seven went down because the “system freeze” switch was turned on. That's purely a debugging tool, used to capture a consistent state across all CPUs when something goes wrong. It was completely wrong to have it on in a production system. But we had a dissatisfied customer who didn't want to hear that—what he did want was an explanation of why I said “Ah, I know that bug”.

The meeting fizzled out without too much being achieved, and we went to the other main node, on the other side of Bruxelles, as the French called it, where we also didn't get much done. Finally we headed back to the car to take Nelson to the airport.

By this time Nelson must have felt as if he had had the worst day of his life. Strange country, strange beer, frustrating meeting—thank God he was heading back to California and civilization! We got to the car and—it wasn't there!

Félix looked at where it had been and said “Why did you park in front of a garage?”. I said “what do you mean, garage? It had windows and curtains in the windows”. It was also barely wide enough to get a car in. But it seems that's the way people do things in Brussels, as the English spell it. We headed off to the police station at the end of the road, and Félix, who—despite his surname—is a Walloon, asked in French if the car had been towed away. The policeman, evidently also a Walloon, asked where we had parked it. Félix: “Just around the corner from the Opzichterstraat”.

The policeman got angry and said “So that's the way it is! Why don't we continue speaking in Flemish?”. “Opzichterstraat” is the name by which we knew it, since Kredietbank was Flemish, but in French it's called rue de l'Intendant.

After Félix used the correct name and the policeman cooled down, we discovered that yes, indeed, the car had been towed away, conveniently to the other side of town. But it was rush hour, and Nelson had to get to the airport. That wouldn't have been an issue, except that his baggage was in the car. Somehow we found a way across, got the luggage out of the car, and took him to the airport in Félix's car before returning to handle the details of getting the car out of the pound.

It wasn't a minute too early for Nelson, I think. When he was safe and sound in the plane and off Belgian soil, I'm sure he swore never to return.

I returned to Germany the following day. It was a German national holiday (17 June 1989), and the traffic was much lighter—I managed 342 km in two hours, still my highest average distance over that time span. It was fun living in Europe.