Greg
Greg's diary
July 2006
Translate this page
Select day in July 2006:
Su Mo Tu We Th Fr Sa
1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30 31
Select month:
2006 Jan Feb Mar Apr
2006 May Jun Jul Aug
2006 Sep Oct Nov Dec
Today's diary entry
Diary index
About this diary
Previous month
Next month
Greg's home page
Greg's photos
Network link stats
Greg's other links
Copyright information
    
Groogle

Saturday, 1 July 2006 Echunga Images for 1 July 2006
Top of page
next day
last day

The rains have come with a vengeance, and with it garden flooding:


https://lemis.nyc3.digitaloceanspaces.com/grog/Photos/20060701/big/garden-1.jpeg

Image title: garden 1          Dimensions:          2816 x 2112, 1088 kB
Make a single page with this image Hide this image
Make this image a thumbnail Make thumbnails of all images on this page
Make this image small again Display small version of all images on this page
All images taken on Saturday, 1 July 2006, thumbnails          All images taken on Saturday, 1 July 2006, small
Diary entry for Saturday, 1 July 2006 Complete exposure details

 

That wasn't helped much by the way Dion laid a drainage pipe: he turned the end up, so that no water can flow in. Spent a little time outside rearranging it and digging trenches for water to run off; Dion can do the rest on Tuesday.


https://lemis.nyc3.digitaloceanspaces.com/grog/Photos/20060701/big/garden-4.jpeg

Image title: garden 4          Dimensions:          2816 x 2112, 1264 kB
Make a single page with this image Hide this image
Make this image a thumbnail Make thumbnails of all images on this page
Make this image small again Display small version of all images on this page
All images taken on Saturday, 1 July 2006, thumbnails          All images taken on Saturday, 1 July 2006, small
Diary entry for Saturday, 1 July 2006 Complete exposure details

 

This week has been full of interruptions; I was invited to no less than five functions in Adelaide during the week, though ultimately I only made one: two were cancelled, one I forgot my shoes, and the last one, last night, was just too inconvenient. In any case, the result was that I was behind on my work and spent half the day updating the internal version of the specs for online backup. The version on MySQL forge will need separate updating; yet another adventure in cooperation (or not) between firefox and Emacs lies behind that one.

The new fridge has a slight disadvantage: it's the same width externally, but 6 cm narrower inside, and I can't get two fermenters in side by side. On the other hand, it has a functional freezer compartment, so maybe I'll do some swapping around. Or build a new, larger fermenter.

Yesterday Yvonne brought back some 20 litre canisters for storing beer for kegs. Filled the first of them today, after a minor problem: I couldn't open them. On investigation, discovered that they were fitted with a ratchet lock that had to be removed before opening:


https://lemis.nyc3.digitaloceanspaces.com/grog/Photos/20060701/big/canister-3.jpeg

Image title: canister 3          Dimensions:          2816 x 2112, 240 kB
Make a single page with this image Hide this image
Make this image a thumbnail Make thumbnails of all images on this page
Make this image small again Display small version of all images on this page
All images taken on Saturday, 1 July 2006, thumbnails          All images taken on Saturday, 1 July 2006, small
Diary entry for Saturday, 1 July 2006 Complete exposure details

 

There's nothing unusual about that, of course, but what's it doing on an empty canister?


Sunday, 2 July 2006 Echunga
Top of page
previous day
next day
last day

Brew day again today, this time a Weißbier and an ale fermented from the same wort. As usual, I do this with a step infusion (from the Weißbier-specific 42° rest to 63°) and then double decoction, ostensibly to 72° and 78°. The decoction calculations have never worked well, and I came to suspect the mash tun, made out of aluminium. Looking at my ancient copy of Perrys “Chemical Engineers Handbook”, fourth edition, which I got at university 35 years ago, the specific heat of aluminium is 4.80 calories/gram (I said this book is old); that's 4.8 times as much as water. No wonder the calculations are off, but it makes me wonder whether decoction does what it should: the decoction quantity doesn't just affect the temperature, it also affects the consistency of the wort. So maybe for these small quantities (by commercial brewing standards) I need a combination of decoction (for the wort consistency) and infusion (to get the right temperature).


Monday, 3 July 2006 Echunga
Top of page
previous day
next day
last day

More work on Emacs indentation macros today. The trouble is that there is so much background to read, and I was looking for a quick fix. Managed to get things more or less working, but for some reason comment indentation is “broken”: I suspect it's really not configured the way I want, but configuration is very complicated.

New month, new objectives: I spent most of last month looking at backup design, to the detriment of bug fixes. It looks like I'll swing the other way this month, especially as most of the team are taking leave.


Tuesday, 4 July 2006 Echunga
Top of page
previous day
next day
last day

Back to looking at bugs today. It's amazing what details you forget in a couple of weeks. Spent most of the day looking at BUG#18980, having a lot of trouble getting it to run at all. Finally, with Lars' help, got it running, but the test case didn't fail as advertised.

Lessons learnt:

Another long conference call in the evening, and again VoIP setup problems. It's time to get things running more smoothly.


Wednesday, 5 July 2006 Echunga
Top of page
previous day
next day
last day

More bug work. I now have a couple of “test case fails at random” bugs that work perfectly for me. There seem to be too many of that nature.

Then there's a mysqldump bug that looks so simple, but every time I look at it I see more issues. I suppose this is the difference between “close bugs at all costs” and “do the correct fix”.

Is MySQL getting buggier?

We've spent a lot of time looking at bugs in the last few months; some people claim that MySQL is getting buggier. Doubtless we have more bugs than we should; that goes for just about any serious piece of software. But how buggy is the product really? How does it stack up with other software, both open source and proprietary? Is it worse than it was, say, a year ago?

I'll leave the discussion of the relative quality of open source and proprietary software for another day; there are too many variables. But let's look at the MySQL bugs database. As I write this, there are a total of 20,879 bugs in the database. Of those, nearly 10,000 have been submitted in the past 12 months. But looking back, the first real bug report (BUG#2) wasn't submitted until 12 September 2002, not really that long ago. It took until 11 August 2004 to reach 5,000 bug reports, until 19 April 2005 to reach 10,000 bug reports, until 16 November 2005 to reach 15,000 bug reports, and until 22 May 2006 to reach 20,000 bugs. Those are intervals of 23 months, 8 months, 7 months and (a little more than) 6 months.

Is that a real increase? The big jump was between 2 and 5,000; after that the increase is relatively small. Also, our own attitude is important here: if people have a problem which we can't immediately identify, one of the first things we do is to ask them “Please enter a bug report”. So we should be happy when they do, rather than to be concerned by the number of reports.

Of course, if the reason for this many bug reports is that the software is getting buggier, it's still a problem. But so many of the bug reports I look at fall into the categories “duplicate”, “not a bug” or “no feedback”. That's indicative more of active bug reporting than bugs themselves, though it would be nice if more submitters could identify duplicates before entering a bug report.

So how do the others do it? I need to look at the FreeBSD project, which currently has over 92,000 bug reports, collected over a much longer period of time; one has been open (i.e. not resolved) for nine years. Nobody has looked at it for over five. Clearly MySQL isn't that bad (nor, in general, is FreeBSD). But the real question to ponder is: how does the number of reported bugs reflect on the quality of the software? I'll report on that one too. Feedback is welcome.


Thursday, 6 July 2006 Echunga
Top of page
previous day
next day
last day

Somehow I'm not making any progress. Every bug I look at hides from me; spent most of the day chasing a bug that must almost certainly be a race condition in replication, but was not able to reproduce it. Not until evening did I get a chance to talk to Kristian and at least get him to reproduce it; I still don't have a good way of reproducing it myself, which makes testing frustrating.

Yvonne back with two new CRT monitors today; they're only 19", but one of them replaces my ancient iiyama 21" monitor very well. The prices of CRTs have also dropped dramatically; these cost less than the cost of repair for the iiyama. Monitors also seem to be getting brighter (or all of mine have become very dim); I had to turn it way down to match the brightness of the others. Next step is to put a couple of additional PCI display cards (also purchased today) into wantadilla.

Finally connected a second telephone to the analogue phone for my company-internal VoIP line; it got blown up by what must have been a lightning strike last summer . The only problem with the phone (two line) was that the ringer on this line didn't work, and replacing it begs the question what I should replace it with (I'm currently tending to a DECT phone, but they don't seem to be made in dual line configurations), so this stop-gap means that I at least hear the phone ringing.


Friday, 7 July 2006 Echunga Images for 7 July 2006
Top of page
previous day
next day
last day

Long day: in the morning, Will Backman called me for a podcast interview about my activity with the FreeBSD project, which proved to be an interesting exercise in fighting VoIP: for some reason connections set up from his end kept dropping out for up to 15 seconds, so called him from my NodePhone account, which also failed once. Here's the interview.

After that, shut wantadilla down and installed the two new video cards I bought yesterday. Used one of the new BenQ monitors and an old 17" ADI monitor for the new cards, and got things up and running pretty quickly.

In the process, set the pixel depths to 24 for all displays; that brought home the progress in display card technology: the new el-cheapo nVidia MX 4000 cards each have 64 MB of onboard memory and can thus easily handle the 2048x1536 resolution that's still, after all these years, the maximum most monitors can handle. When are we going to get resolutions comparable only to low-end digital cameras?

In any case, 24 bit pixel depth didn't work for display wantadilla:0.1 (the right-most): it's an old Matrox MGA64 with only 4 MB of memory, not enough to run that depth at 1600x1200. Display wantadilla:0.3 (the left-most), however, was running on one of the new nVidia boards at 1280x1024, so decided to reconfigure. Until that point, things had worked well, but the simple operation of connecting monitors to different cards caused startx to consistently lock the system. Reset to the old configuration and continued with only 16 bits for wantadilla:0.1; I don't have time to mess around crashing systems.

Why the display numbering? wantadilla:0.0 and wantadilla:0.1 have been there for ever, and renumbering them would require multiple changes. wantadilla:0.2 is left of wantadilla:0.0, and probably it will stay, while wantadilla:0.3 is more “because I can”; it's a little over 90° off from my field of view, and even getting the cursor over there is a problem. So it makes sense to order the displays in the sequence 3, 2, 0 and 1.

So now I have 9 displays on my desktop:


https://lemis.nyc3.digitaloceanspaces.com/grog/Photos/20060707/big/desktop-left.jpeg
Image title: desktop left          Dimensions:          2816 x 2112, 863 kB
Make a single page with this image Hide this image
Make this image a thumbnail Make thumbnails of all images on this page
Make this image small again Display small version of all images on this page
All images taken on Friday, 7 July 2006, thumbnails          All images taken on Friday, 7 July 2006, small
Diary entry for Friday, 7 July 2006 Complete exposure details

 
https://lemis.nyc3.digitaloceanspaces.com/grog/Photos/20060707/big/desktop-middle.jpeg
Image title: desktop middle          Dimensions:          2816 x 2112, 895 kB
Make a single page with this image Hide this image
Make this image a thumbnail Make thumbnails of all images on this page
Make this image small again Display small version of all images on this page
All images taken on Friday, 7 July 2006, thumbnails          All images taken on Friday, 7 July 2006, small
Diary entry for Friday, 7 July 2006 Complete exposure details

 
https://lemis.nyc3.digitaloceanspaces.com/grog/Photos/20060707/big/desktop-right.jpeg
Image title: desktop right          Dimensions:          2816 x 2112, 819 kB
Make a single page with this image Hide this image
Make this image a thumbnail Make thumbnails of all images on this page
Make this image small again Display small version of all images on this page
All images taken on Friday, 7 July 2006, thumbnails          All images taken on Friday, 7 July 2006, small
Diary entry for Friday, 7 July 2006 Complete exposure details

 

The overall layout takes some getting used to. From left to right:

In almost every case, I'm running the displays at the maximum resolution and frequency that the hardware can handle. wantadilla and echunga are connected together via x2x (in the Ports Collection as /usr/ports/x11/x2x). I could do the same with eucla, but since it's offset 135° from the keyboard, it doesn't see worthwhile.

This gives me the following total resolutions:

Display Resolution Total pixels
wantadilla:0.3  1280x1024  1310720
wantadilla:0.2  1600x1200  1920000
wantadilla:0.0  2048x1536  3145728
wantadilla:0.1  1600x1200  1920000
Total wantadilla 8296448
echunga:0.0  1920x1440  2764800
echunga:0.1  1600x1200  1920000
echunga:0.2  1280x1024  1310720
Total echunga 5995520
Total wantadilla and echunga 14291968
eucla.lemis.com  1920x1200  2304000
Total X displays 16595968
pain.lemis.com  1024x768  768432
Total, all displays 17382400

That's quite a bit more than the standard 768432 pixels of a “normal” display, of course—in fact, about 22 times as much. More interesting is the average display resolution of the X displays, 1843996 pixels. On a 4:3 display, this would correspond to 1568 x 1176 pixels.

In the afternoon, phone call with Arjen Lentz, who was having trouble with his ADSL connection. While talking, the (VoIP) line went dead: my ADSL connection went down too. So, it seems, did most Internode connections, and with it my login to freefall.freebsd.org, where I had been logged in since 7 February. How I hate things going down!

After that fun, back to work on the showstopper bug that I've been having so much trouble reproducing; Kristian Nielsen obligingly held up one of the threads in the correct place, making it reproducible, proving that it was really a race condition and also pointing to why: one thread announced that it was finished too early. Just moving a couple of statements solved the problem, much more elegant than taking Yet Another mutex. Got the patch tested, approved and committed before dinner.


Saturday, 8 July 2006 Echunga Images for 8 July 2006
Top of page
previous day
next day
last day

Spent far too much time this morning writing my diary entry for yesterday and also updating my hardware page; it's amazing how much things change, and much of what was in the page no longer applies; on the other side, it doesn't (yet) include references to the brewing temperature control or my CVR attempts, and the Mike Smith Memorial Room pages is years out of date.

Daniel O'Connor along in the afternoon to pick up a tuner, some Weihenstephan W68 yeast and the old iiyama monitor that I retired on Thursday. In turn he brought an AMD Duron 800 processor, still mounted on a motherboard of dubious quality:


https://lemis.nyc3.digitaloceanspaces.com/grog/Photos/20060708/big/capacitor.jpeg

Image title: capacitor          Dimensions:          2112 x 2816, 480 kB
Make a single page with this image Hide this image
Make this image a thumbnail Make thumbnails of all images on this page
Make this image small again Display small version of all images on this page
All images taken on Saturday, 8 July 2006, thumbnails          All images taken on Saturday, 8 July 2006, small
Diary entry for Saturday, 8 July 2006 Complete exposure details

 

After that, finally got the motivation I needed to continue with the work on the video recorder project. The last time I tried this was only barely less frustrating than the previous attempts, though there were signs that it might be more promising. On IRC and the LinuxSA mailing list I had got some suggestions to try tzap, which was typical of the documentation standards for multimedia programs:

=== root@ceeveear (/dev/ttyp2) /myth/Images 11 -> tzap
usage:
       tzap [options] <channel_name>
         zap to channel channel_name (case insensitive)
=== root@ceeveear (/dev/ttyp2) /myth/Images 12 -> man tzap
No manual entry for tzap
=== root@ceeveear (/dev/ttyp2) /myth/Images 13 ->

So I went googling for tzap and finally found a wiki page describing it; it seems that I couldn't use it until I had run dvbscan, which of course also had no documentation, and even the wiki page was pretty cursory. It did tell me that I needed a list of local transmitters, though.

Based on the information, extrapolated a file name au-Adelaide, went googling again, and to my surprise discovered indications that I should already have the file on the machine; and I did:

=== root@ceeveear (/dev/ttyp1) ~ 26 -> locate au-Adelaide | xargs ls -lrt
-rw-r--r--  1 root root  363 Apr 29  2005 /usr/local/share/dvb/dvb-t/au-Adelaide
-rw-r--r--  1 root root 4000 Sep 24  2005 /usr/local/share/dvb/zap/dvb-t/au-Adelaide
-rw-r--r--  1 root root  363 Sep 24  2005 /usr/local/share/dvb/scan/dvb-t/au-Adelaide

With a bit of playing round, got the tuner to work and copy MPEG data to disk. To get that far, I needed:

After this impressive success story, went back to the MythTV wiki and updated the documentation for dvbscan and tzap, which are now hopefully more useful (and indirectly show their origins). Tried to display the MPEG on teevee.lemis.com, my projector machine, but that failed, apparently because one of the myriad parameters didn't match. Left that investigation for another day; but what a relief! Finally, after not quite two years, I'm able to save broadcast TV as MPEG. Even if MythTV doesn't do what I want, I can live with this alone.

People don't complain as much about the sheer reliability of a Microsoft machine as they used to do; certainly I've never seen a Blue Screen Of Death on one of my machines. But today, pain.lemis.com, my Microsoft “Windows” box, developed some problems, and I had to reboot it. When I restarted it, it wasn't able to mount file systems from wantadilla, and indeed it couldn't even ping it. Investigation of the myriad network setup screens presented me with a message “Append these DNS suffixes (in order)”. This seems to imply a domain name search path; but why can't they adhere to established conventions?

Finally got the machine talking to the network again; it seems that since the X rebuild on wantadilla I can't start rdesktop on wantadilla:0.1; it works fine on the other displays. I wonder what the problem is there; it doesn't seem to be related to the configuration of the display itself, which hasn't changed.


Sunday, 9 July 2006 Echunga
Top of page
previous day
next day
last day

Today we had meant to go riding, but it was too cold and windy. Instead messed around trying to get MythTV working. As expected, it wasn't as simple as it should have been. Although they're mentioned on the web site, tzap and friends have nothing to do with MythTV, and the files they use are of no use in setting up MythTV. In addition, the documentation is very weak; it doesn't really tell you what you need to know to configure the tuners. From mythtv-setup, fortunately started from an xterm, I discovered that the cause of the hang was that it started a command on the xterm. I wonder where that would have gone if it had been started from the window manager.

Grab listings for which region?
0: ACT - Canberra
1: NSW - Regional
2: NSW - Sydney
3: NT - Darwin
4: NT - Regional
5: Queensland - Brisbane
6: Queensland - Regional
7: SA - Adelaide
8: SA - Regional
9: Tasmania - Hobart
10: Tasmania - Regional
11: Victoria - Melbourne
12: Victoria - Regional
13: WA - Perth
14: WA - Regional
Select one: [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14 (default=3)] 7
Grab listings for which service?
0: Austar Analogue
1: Austar Digital
2: Foxtel Analogue
3: Foxtel Digital
4: Free to Air
5: Free to Air Digital
6: Free to Air Digital (High Definition)
7: Optus
Select one: [0,1,2,3,4,5,6,7 (default=4)] 6

ERROR: can't find DataSet for Adelaide - freehd
2006-07-09 16:45:54.198 tv_grab_au --config-file '/root/.mythtv/Tuner-1.xmltv' --configure
2006-07-09 16:45:54.198 exited with status 65280

All I managed to do was 4 (“Free to Air”), which gave me a file ~/.mythtv/Tuner-1.xmltv with the following contents:

  # tv_grab_au 0.6.2 config file
  region Adelaide
  service free
  # "channel name" and "frequency" can be changed to suit
  # [+yes/-no] [channel ID] [channel name] [frequency]
  +channel free.Adelaide.10 "Network TEN" 10
  +channel free.Adelaide.2 "ABC SA" 2
  +channel free.Adelaide.7 "Channel Seven" 7
  +channel free.Adelaide.9 "Channel Nine" 9
  +channel free.Adelaide.SBS "SBS" 28

That's clearly analogue. I couldn't find any way to get the digital signals. The same happened with “Free to Air Digital (High Definition)”. Gave up and sent a message to the local mailing lists asking for help.

Trying to configure mplayer to display the data I copied yesterday was no better. mplayer did completely different things depending on where it was started from and where it displayed. In all cases, I got the error message:

VDec: vo config request - 720 x 576 (preferred csp: Mpeg PES)
Could not find matching colorspace - retrying with -vf scale...
Opening video filter: [scale]
The selected video_out device is incompatible with this codec.
VDecoder init failed :(

This, though, is just “ha ha, only joking”; it doesn't stop mplayer from working. What I got depended on where I ran the program and where I directed the output:

What a pain!

Will Backman sent me a link to the MP3 of the interview I did on Friday, so I loaded the sound driver on wantadilla. The system froze, and I had to reboot. When I tried to restart X, it froze again. No matter what I did, I couldn't start X until I removed the Matrox M2064 card from the configuration. What caused that? Yes, I had some issues involved with that card on Friday, but I fixed them. At least now the decision has been made for me about the extreme left monitor: it goes.


Monday, 10 July 2006 Echunga Images for 10 July 2006
Top of page
previous day
next day
last day

As usual when writing up my diary entry for the previous day, I tried reproducing some of the problems I had, in this case including the inability to set the channels. To do so, I started mythtv-setup on wantadilla:0.0, the 2048x1536 display. This takes forever! So did key presses, with the result that I accidentally selected the wrong entry, didn't notice it, and found myself in the channel scan menu. Despite all the error messages from yesterday, it seems that it had found the information it needed, and it was able to perform a (surprisingly slow) channel scan and locate the channels:


https://lemis.nyc3.digitaloceanspaces.com/grog/Photos/20060710/big/MythTV.jpeg

Image title: MythTV          Dimensions:          2048 x 1536, 976 kB
Make a single page with this image Hide this image
Make this image a thumbnail Make thumbnails of all images on this page
Make this image small again Display small version of all images on this page
All images taken on Monday, 10 July 2006, thumbnails          All images taken on Monday, 10 July 2006, small
Diary entry for Monday, 10 July 2006

 

Went looking for the ~/.mythtv/Tuner-1.xmltv file, but there was none. After a bit of searching, discovered that Chris Yeoh was right: the channel information is stored in a MySQL database, specifically the table channel in database mythconverg:

=== root@ceeveear (/dev/ttyp6) /myth/Images 31 -> mysql -D mythconverg
mysql> select callsign, freqid, serviceid from channel;
+-----------------+--------+-----------+
| callsign        | freqid | serviceid |
+-----------------+--------+-----------+
| 7 Digital       | NULL   |      1360 |
| 7 Digital 1     | NULL   |      1361 |
...
| ABC HDTV        | 7      |       592 |
| ABC TV Adelaide | 7      |       593 |
...
| SBS DIGITAL 2   | 5      |       834 |
| SBS EPG         | 5      |       835 |
+-----------------+--------+-----------+
24 rows in set (0.00 sec)

Getting closer to work, made a dump of the table with mysqldump; presumably it's the same for all installations in Adelaide, though notably the frequency information is missing.

Then on to looking at a mysqldump bug—well, that's how it was reported, but it turned out to be a character set problem with SHOW CREATE PROCEDURE. Spent a while looking at that, but didn't finish.

Looking for ways to listen to MP3s on firefox, downloaded and installed RealPlayer. That's all that I could do, though:

 

https://lemis.nyc3.digitaloceanspaces.com/grog/Photos/20060710/big/instructions.jpeg
Image title: instructions
Dimensions: 580 x 485, 142 kB
Dimensions of original: 580 x 485, 142 kB
Display this image:
thumbnail    hidden   alone on page
Display all images on this page as:
thumbnails    this size
Show for Monday, 10 July 2006:
thumbnails    small images    diary entry

And yes, that's the size of the popup, not high enough by about 20%. Why can't people get their dimensions right? Anyway, the installation worked fine, but how do I run it? What's the binary called? There's a directory called ~/RealPlayer/Bin (yes, Bin, not bin), but there's nothing in it. I still don't know what to do with it.

In the evening, confirmed that the sound problems yesterday were caused by the audio language settings of mplayer, so finally was able to play the MPEG stream with sound. Even then, though, the problems aren't over. Trying to play the recorded HDTV, the recorded audio failed again:

DEMUX OPEN, AUDIO_ID: -1, VIDEO_ID: -1, SUBTITLE_ID: -1,
PROBING UP TO 2000000, PROG: 0
VIDEO MPEG2(pid=2314)...NO AUDIO!  NO SUBS (yet)!  PROGRAM N.  0
Opened TS demuxer, audio: ffffffff(pid -1), video: 10000002(pid 2314)...POS=99452
...
Audio: no sound

So what's going on there? It really looks as if there is no audio in the feed. I'm not going to worry about it too much until I get MythTV working and it shows up there too.

As if to add to the general instability I've been seeing lately, tvremote, the laptop I use to control teevee, died on me. Investigation showed a loose mains plug adaptor. sigh I think that machine had been up for 6 months.


Tuesday, 11 July 2006 Echunga Images for 11 July 2006
Top of page
previous day
next day
last day

A day of interruptions, spent mainly walking through the server trying to understand BUG#16291. It had been entered as a mysqldump bug, but proved to be a character set issue in the SHOW CREATE PROCEDURE command. Learnt a lot about the server code structure, but by the end of the day I was still not even sure whether this was a bug or a feature.


Wednesday, 12 July 2006 Echunga Images for 12 July 2006
Top of page
previous day
next day
last day

More bug work today, the last day of the week I'm planning to look at bugs. Another SHOW CREATE problem reported against mysqldump, this time with default values. I always wonder whether I should do them myself or hand them over to somebody who knows that part of the server code.


Thursday, 13 July 2006 Echunga
Top of page
previous day
next day
last day

Back to work on the backup code today, but interruptions ensured that I didn't get nearly as far as I had wanted to. In particular, discovered that eucla's nightly backup has not been running, and it sounded like a good idea to do an immediate backup. That was hampered by the NFS problems that I reported a couple of months ago and which are still annoying me. The dump disk is NFS mounted, and it becomes inaccessible. When it does, Linux provides no way to dismount it; umount -f doesn't seem to work under Linux, or at least no better than without the -f flag. Installed ethereal and discovered that eucla (Linux) was sending messages that were being ignored by wantadilla (FreeBSD). Further investigation revealed:

=== root@wantadilla (/dev/ttyp3) ~ 193 -> netstat -finet | grep eucla
tcp4       0      0  wantadilla.x11         eucla.43945            ESTABLISHED
tcp4       0      0  wantadilla.52340       eucla.ssh              ESTABLISHED
tcp4       0      0  wantadilla.nfsd        eucla.1014             CLOSE_WAIT
tcp4       0      0  wantadilla.nfsd        eucla.1017             ESTABLISHED

Where did that CLOSE_WAIT come from? Even rebooting eucla didn't make it go away, and it must have been there for many hours. Spent some time trying to find out which process owned the socket, without success; I had some recollection that lsof would display that sort of information, but I couldn't find a way to do it. Also discovered sockstat(1) in the process, but still didn't find the culprit.

Perrini wines, just down the road from us, are cleaning out their warehouse and selling cleanskin wines for $39 a dozen. Down today and picked up another 9 crates, not all at that price. Now we have a storage problem.


Friday, 14 July 2006 Echunga
Top of page
previous day
next day
last day

I really, really wanted to get some coding done today, but first I had some important maintenance work to do: I've installed an approximation to the new Emacs C code indentation macros, and modulo a few things I need to think about, they work acceptably. With one exception: comment indentation. I have a macro (really little more than a key binding) that indents the next comment. I tried it on this code:

enum enum_backup_block
{
  BUB_INVALID,                                  /* 0: not a valid block type */
  BUB_HEADER,                                   /* global header */
  BUB_VTIME,                                    /* validity time block */
  BUB_TRAILER,                                  /* global trailer */
  BUB_ENGINEBLOCK,                              /* block from engine */
  BUB_DEFAULTBUB,                               /* default backup block */
}

The result didn't look encouraging:

enum enum_backup_block
{
  BUB_INVALID,                         /* 0: not a valid block type */
  BUB_HEADER,                                   /* global header */
  BUB_VTIME,                                 /* validity time block */
  BUB_TRAILER,                                  /* global trailer */
  BUB_ENGINEBLOCK,                             /* block from engine */
  BUB_DEFAULTBUB,                           /* default backup block */
}

After a while it dawned on me that the comments were being right-justified, at least in part, to column 70. Spent a lot of time RTFMing, to no result; there's no mention of such a feature anywhere. Finally bit the bullet and took a look at the LISP code. The function in question is comment-indent (in lisp/newcomment.el), and it contains this code:

    ;; Avoid moving comments past the fill-column.
    (unless (save-excursion (skip-chars-backward " ") (bolp))
      (setq indent
        (min indent
             (+ (current-column)
            (- fill-column
               (save-excursion (end-of-line) (current-column)))))))

So the real issue is fill-column, which is set by the macro set-fill-column, normally bound to c-x f. By default it's 70. I considered removing this limitation, but in fact it's useful, since we're constrained to making source lines no longer than 80 characters, so I set it to 80 in my CC mode setup:

(defconst my-c-style
  '((c-tab-always-indent        .  t)
...
    (fill-column               .  80)
    )
  "MySQL")

As usual, the entire file is in my Emacs macros directory.

After that, I finally did manage to write some code. I feel so uncomfortable when my tools aren't working.

In the evening to an “industry night” arranged by Peter Griffith of the ACS to give students an idea of what lies ahead in the Real World. I was wearing multiple hats: AUUG, FreeBSD, MySQL and NetBSD. I had assumed that the main hat would be AUUG, but it proved to be MySQL. People showed quite a bit of interest; possibly we'll even end up hiring some of them.


Saturday, 15 July 2006 Echunga
Top of page
previous day
next day
last day

Quiet day. I've got plenty of things to do, but relaxing is also one of them; spent some time gradually following up on things I've been putting off for a while.

One of them was to review Will Backman's Podcast interview. I still don't know how to listen to MP3s on firefox; the attempt to set it up to use mpg123 (sorry, /usr/local/bin/mpg123, and not /home/grog/mpg123 which it seems to think is a good place to look) was successful in the sense that it has added it to the configuration, but when I try to listen to an MP3 via firefox, nothing happens, and it's not clear that the process got started. Somebody suggested xmms2 instead, so I installed that. Attempting to run it, I got:

=== grog@wantadilla (/dev/ttypi) ~ 51 -> xmms2
ERROR: Could not create default configfile!!

Does that look familiar? No error indication and repeated exclamation marks. Investigated with ktrace, and later during an IRC discussion with truss, which proves to be more useful in such cases:

access("/home/grog/.xmms2/clients/cli.conf",0)   ERR#2 'No such file or directory'
mkdir("/home/grog/.xmms2/clients",0755)          ERR#2 'No such file or directory'
open("/home/grog/.xmms2/clients/cli.conf",0x602,0666) ERR#2 'No such file or directory'

So xmms2 stores its configuration information in a directory hierarchy ~/.xmms2, but it doesn't create it. It checks for the existence of the file cli.conf in the subdirectory .xmms/clients. It's clever enough to recognize—after the event—that accessing the file fails because the directory doesn't exist, so it tries to create it. That's where the cleverness stops: it doesn't check if the parent directory exists, and in this case it doesn't, so the mkdir fails. It doesn't notice and tries to create cli.conf, which still fails. Then it dies without giving any information. Giving the information is as simple as this:

--- main.c~     Tue Mar 28 06:04:45 2006
+++ main.c      Sun Jul 16 11:01:37 2006
@@ -108,7 +108,10 @@
@@ -108,7 +108,10 @@

                fp = fopen (file, "w+");
                if (!fp) {
-                       print_error ("Could not create default configfile!!");
+                       fprintf (stderr,
+                                "Could not create default config file: %s
",
+                                strerror (errno);
+                                exit (-1);
                }

                fwrite (defaultconfig, strlen (defaultconfig), 1, fp);

This also gets rid of the completely useless function print_error, which simply makes it difficult to pass error information.

So why don't I submit this patch? There are a number of reasons:

If anybody from the xmms2 team reads this: prove me wrong; I'll gladly listen. On the positive side, it's worth noting that xmms2 does have man pages, so maybe there is still hope.

In any case, after creating the directory, I got the following message:

=== grog@wantadilla (/dev/ttypi) ~ 65 -> mkdir .xmms2
=== grog@wantadilla (/dev/ttypi) ~ 66 -> xmms2
ERROR: Could not connect to xmms2d: xmms2d is not running.

In other words, I would need to set up a server. This isn't the level of effort I want to go to just to play an MP3, and there's no reason to believe that xmms2 would work better with firefox. Instead, downloaded the MP3 and listened to it manually.

As if I hadn't had enough pain already, turned to look at the MythTV setup, first updating my log. That proved to be well worth while; the next step was setting up the remote control, involving another run-in with LIRC, which have in bad memory. Decided to avoid the LIRC web site and try the MythTV instructions instead.

The chapter Setting up a remote control spends most of its time telling you how to install the LIRC software. Maybe that's necessary, but my recollection is that it wasn't that difficult. The real information starts at “Completing the lirc install”, which tells you where to download remote control definitions. Mine's a DVICO, with no model number of course, but it comes with the “Fusion HDTV” card. Searched for a couple of different names, but found nothing. The MythTV instructions tell you what to do; after starting lircd, you should then run irrecord, which doesn't work:

=== root@ceeveear (/dev/ttyp5) ~ 9 -> /usr/local/sbin/lircd
=== root@ceeveear (/dev/ttyp5) ~ 10 -> irrecord myremote

irrecord -  application for recording IR-codes for usage with lirc

Copyright (C) 1998,1999 Christoph Bartelmus(lirc@bartelmus.de)

irrecord: could not init hardware (lircd running ? --> close it, check permissions)

But we've seen this before, and this insistence on ignoring error codes is the same as with xmms2. In this case, strace shows:

open("/dev/lirc", O_RDWR)               = -1 ENODEV (No such device)

So in this case it's not because lircd is running: irrecord is looking in the wrong place.

Interestingly, on the LIRC web site found a table that showed that the remote control is supported; I forget how I got there. Elsewhere it shows the same information that was in the file I downloaded. Elsewhere again discovered that there's a script called lirc-reconfig.sh that should do the installation for you. Ran that and was prompted for the model number, including dvico. Selected that and got the message:

#############
### DVICO ###
#############

WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING

   I should have been able to auto-detect this remote.  If you're sure it's plugged
   in, please send the contents of /var/log/mythtv/lirc-reconfig.log to knoppmyth@gmail.com.
   Otherwise, please be sure to plug the remote in and re-run this script.

WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING

Setting up dvico ...
 System startup links for /etc/init.d/lirc already exist.
Stopping lirc daemon: lircmd lircd.
Starting lirc daemon: lircd.

Plugged in the sensor and tried again; this time it found it OK. So at least it seems to be recognized. But how do I test it?


Sunday, 16 July 2006 Echunga
Top of page
previous day
next day
last day

Somehow didn't get much done today. This multimedia stuff really gets on my nerves. Spent a bit more time checking the remote control, and took a look at the remote control definition (in /etc/lirc/lircd.conf) and discovered that there are in fact two definitions; mine is the second (DVICO_MCE). Also discovered the lircmd daemon, which allows you to use the remote control as a mouse-like pointer device. Tried that:

=== root@ceeveear (/dev/ttyp5) /etc/lirc 62 -> ps aux|grep lirc
root     17825  0.0  0.0   2696   544 ?        Ss   11:51   0:00 lircd
root     17831  0.0  0.0   1644   500 ttyp5    R+   11:51   0:00 grep lirc
=== root@ceeveear (/dev/ttyp5) /etc/lirc 63 -> lircmd
=== root@ceeveear (/dev/ttyp5) /etc/lirc 64 -> echo $?
0
=== root@ceeveear (/dev/ttyp5) /etc/lirc 65 -> ps aux|grep lirc
root     17835  0.0  0.0   1640   504 ttyp5    S+   11:51   0:00 grep lirc
=== root@ceeveear (/dev/ttyp5) /etc/lirc 66 -> tail /var/log/daemon.log
Jul 16 12:06:05 ceeveear lircd-0.8.0-CVS[16478]: accepted new client on /dev/lircd
Jul 16 12:06:05 ceeveear lircd-0.8.0-CVS[16478]: could not open /dev/lirc
Jul 16 12:06:05 ceeveear lircd-0.8.0-CVS[16478]: default_init(): No such device
Jul 16 12:06:05 ceeveear lircd-0.8.0-CVS[16478]: caught signal

More useless error reporting! What signal did the daemon catch? Why did it exit as a result? My guess was that it was a SIGBUS or SIGSEGV, and that the last message should really be:

Jul 16 12:06:05 ceeveear lircd-0.8.0-CVS[16478]: Crashed due to programming error

With strace, however, I see:

unlink("/var/run/lircd.pid")            = 0
close(5)                                = 0
rt_sigaction(SIGTERM, {SIG_DFL}, {0x804a740, ~[KILL STOP RTMIN RT_1], SA_RESTART}, 8) = 0
tgkill(17865, 17865, SIGTERM)           = 0
--- SIGTERM (Terminated) @ 0 (0) ---

This looks very much like a planned shutdown; but why? Because it couldn't open /dev/lirc? Why should it need to do that? The configuration file confirms my recent suspicions that that's the wrong device. The correct one is in /etc/lirc/hardware.conf:

DEVICE="/dev/usb/hiddev0"

It was already open, so what is this nonsense? lircmd doesn't have the ability to specify a device; maybe it automatically assumes /dev/lirc.


Monday, 17 July 2006 Echunga
Top of page
previous day
next day
last day

As almost every year, when the rains set in, the phone lines fail. Over the weekend we had particularly heavy rains: the Bureau of Meteorology reported 76 mm of rain from 9 am 16 July to 9 am 17 July. No doubt this is the background for the subsequent failure:

ADSL line statistics

See the DSL statistics page for a description of these values.

Compared to the previous failure, though, there are some significant differences:

My guess is that another line went down first, and that they shut down the backhaul for the entire cable at this point.

This had to happen, of course, when I was the only person in my department (MySQL replication and backup) who wasn't on vacation. Murphy strikes again. Called Telstra, and for once managed to find my way through their horrible voice menu system, which spent 5 minutes trying to test the line before transferring me to Alison with no information (neither the fact that I was reporting a fault, nor the results of the test, nor even the phone number against which it was reported). She logged fault number 120947258 and got John to test the line and call me back, saying, of course, that the line was defective; less plausible is his statement that they should have it working tomorrow. Doubtless they'll send the elite troop of flying pigs.

Towards the Internet age

How I hate line outages! They bring back to me how dependent we are on the Internet. People complain—justifiably—about the electricity service, but the longest outage we have ever had was about 5 hours, and that was for previously scheduled work; most others last less than 2½ hours. By contrast, the shortest phone outage we have ever had was also in the order of 5 hours, and that too was an exception: apart from that, they have always taken more than the 2 full calendar working days legislated as the limit by the Customer Service Guarantee.

What's the problem? In each case, the predominant cause of problems is the mind set. The fact that the Customer Service Guarantee proscribes two full working days is one indication of this; the other is that the penalties for not keeping to it are meaningless. When the power is down, I can generate my own electricity (admittedly I need to improve on the equipment I have, but I can do it). When the phone lines (and thus the Internet connection) are down, I basically can't work, and there's no workaround. If I can't find non-Internet related work to do, I lose about $700 a day. The Customer Service Guarantee reimburses exactly $12 of this sum, which The Hon. Alexander Downer [at the time our Federal MP] has stated to be sufficient in his opinion.

South Australia has a lot of things which make it an ideal location for the Internet Age: well-trained native English speakers, good infrastructure, relatively low cost of living. But as long as long as this mentality prevails with our politicians, it won't happen.


Tuesday, 18 July 2006 Echunga Images for 18 July 2006
Top of page
previous day
next day
last day

Spring is on its way! The first daffodils are on their way, only 3 or 4 months after Northern Europe:


https://lemis.nyc3.digitaloceanspaces.com/grog/Photos/20060718/big/daffodil.jpeg

Image title: daffodil          Dimensions:          2112 x 2816, 576 kB
Make a single page with this image Hide this image
Make this image a thumbnail Make thumbnails of all images on this page
Make this image small again Display small version of all images on this page
All images taken on Tuesday, 18 July 2006, thumbnails          All images taken on Tuesday, 18 July 2006, small
Diary entry for Tuesday, 18 July 2006 Complete exposure details

 

What a pain it is to not have a network! Almost everything that I do needs some network access. Yes, in most cases you can work around it, but that's really irritating. Instead, chose to do something that really didn't need network access, tidying up my office. It was sorely in need of it:


https://lemis.nyc3.digitaloceanspaces.com/grog/Photos/20060718/big/office-before-1.jpeg

Image title: office before 1          Dimensions:          2816 x 2112, 560 kB
Make a single page with this image Hide this image
Make this image a thumbnail Make thumbnails of all images on this page
Make this image small again Display small version of all images on this page
All images taken on Tuesday, 18 July 2006, thumbnails          All images taken on Tuesday, 18 July 2006, small
Diary entry for Tuesday, 18 July 2006 Complete exposure details

 
https://lemis.nyc3.digitaloceanspaces.com/grog/Photos/20060718/big/office-after-2.jpeg
Image title: office after 2          Dimensions:          2816 x 2112, 528 kB
Make a single page with this image Hide this image
Make this image a thumbnail Make thumbnails of all images on this page
Make this image small again Display small version of all images on this page
All images taken on Tuesday, 18 July 2006, thumbnails          All images taken on Tuesday, 18 July 2006, small
Diary entry for Tuesday, 18 July 2006 Complete exposure details

 

Tidying up an office is something like archaeology; in the course of the dig I found things up to 3½ years old, half the time I've been in this office.

In the afternoon, the network came back, but only with greatly reduced margins; instead of the standard 1500 kb/s down and 256 kb/s up and the more typical noise margins of 12 dB and 23 dB respectively, it only had 448 kb/s down and 64 kb/s up, and noise margins of 6 dB and 4 dB.

ADSL line statistics

Went down the road to tell the Telstra crew of the problem, but there was no crew; further investigation revealed that the phones were still down. The return of the ADSL line must have been due to the cable drying out. It would have been a bit of a surprise to see the Telstra people there in less than 24 hours, anyway.


Wednesday, 19 July 2006 Echunga Images for 19 July 2006
Top of page
previous day
next day
last day

The DSL line stayed up all night and went down just before I got up, at 08:10; after it had been down for some hours, decided that it probably meant that the Telstra technicians were working on it, so down to take a look. Not surprisingly, they were about 50 metres from last year:


https://lemis.nyc3.digitaloceanspaces.com/grog/Photos/20060719/big/cable-repairs-3.jpeg

Image title: cable repairs 3          Dimensions:          2816 x 2112, 816 kB
Make a single page with this image Hide this image
Make this image a thumbnail Make thumbnails of all images on this page
Make this image small again Display small version of all images on this page
All images taken on Wednesday, 19 July 2006, thumbnails          All images taken on Wednesday, 19 July 2006, small
Diary entry for Wednesday, 19 July 2006 Complete exposure details

 

The fenced off area on the left is the site of the fault in February 2003. They put the “fence” up after the repair; I don't know why, but there's nothing visible inside.

The cable was particularly badly mangled this time. This is the way it came out of the ground:


https://lemis.nyc3.digitaloceanspaces.com/grog/Photos/20060719/big/cable-1.jpeg

Image title: cable 1          Dimensions:          2112 x 2816, 1232 kB
Make a single page with this image Hide this image
Make this image a thumbnail Make thumbnails of all images on this page
Make this image small again Display small version of all images on this page
All images taken on Wednesday, 19 July 2006, thumbnails          All images taken on Wednesday, 19 July 2006, small
Diary entry for Wednesday, 19 July 2006 Complete exposure details

 

The linesmen weren't in any hurry to get the cable fixed, and it didn't come back until 15:00:

ADSL line statistics

Still, it's the first time ever that Telstra has repaired this kind of cable fault within the time legislated by the Customer Service Guarantee. About the only thing I can complain about is that it's ridiculous to leave people without network connectivity for over 2 days; but it'll be years before that becomes a general opinion.

Spent most of the day writing online backup code; as usual, it's taking longer than I had hoped. Christiane Yeardley along in the evening with a pair of Apple PowerMac G4s that we had bought on eBay. More stuff to play with.


Thursday, 20 July 2006 Echunga Images for 20 July 2006
Top of page
previous day
next day
last day

On with the backup server code today, making good progress. The core of the code has been complete for some time, but I need to ensure that it fits into the server framework. Spent some time looking at the plethora of mutexes used to synchronize replication; more below.

Also spent some time looking at the PowerMacs that Chris brought with her yesterday. Got them set up surprisingly quickly; I'm on record as saying that I don't like all of Apple's eye candy, and that remains, but it's true that it was relatively easy to get some things to work; others leave me confused, like for example how to get rid of the text that claims that this machine belongs to Trent Richardson (presumably the previous owner). Also managed to get the machine to set the display resolution to 1600x1200, though the refresh rate dropped to only 60 Hz, presumably because the monitor can't do any better.

In the process, had to open the box. After removing eight Allen key screws, the thing still wouldn't open; then I discovered the handle on the other side:


https://lemis.nyc3.digitaloceanspaces.com/grog/Photos/20060720/big/powermac-1.jpeg

Image title: powermac 1          Dimensions:          2048 x 1536, 304 kB
Make a single page with this image Hide this image
Make this image a thumbnail Make thumbnails of all images on this page
Make this image small again Display small version of all images on this page
All images taken on Thursday, 20 July 2006, thumbnails          All images taken on Thursday, 20 July 2006, small
Diary entry for Thursday, 20 July 2006 Complete exposure details

 

It actually looks quite nice and tidy inside:


https://lemis.nyc3.digitaloceanspaces.com/grog/Photos/20060720/big/powermac-2.jpeg

Image title: powermac 2          Dimensions:          1536 x 2048, 592 kB
Make a single page with this image Hide this image
Make this image a thumbnail Make thumbnails of all images on this page
Make this image small again Display small version of all images on this page
All images taken on Thursday, 20 July 2006, thumbnails          All images taken on Thursday, 20 July 2006, small
Diary entry for Thursday, 20 July 2006 Complete exposure details

 

In the course of messing around, discovered:

  1. The boxes take PCI100 memory, which is almost unobtainable. Obviously they would also take PCI133 memory, which I had handy; in the process, updated my box (boskopp.lemis.com) from 512 MB to 896 MB, which is quite respectable, and Chris' machine (which she plans to call delicious) from 192 MB to 448 MB.

  2. Clearly these memory sizes indicate a combination of memory chips of different size (3x256 and 1x128 for me, 3x128 and 1x64 for Chris). There is some sensitivity: the first time round in delicious, I put a combination of sizes on the left, and the machine didn't get out of the self test. Putting the 64 MB chip on the right worked fine.

  3. Chris' machine came with a floppy in the drive. How to get it out? No idea. Being an Apple, it didn't have an ejector. After taking it apart, discovered:

    • The power connector had been removed for some reason.

    • It wasn't a floppy, it was a Zip drive.

    Fixed that, and all was well: I could eject from the “Finder” menu.

    Round here I appear to have omitted content, including a new item. I added the following item and the brown text 15 years later, and it's my guess that that's what I meant.
  4. Next, mount NFS file systems. I know the answer to this one: you can't. On my other systems, I have mounts like this:

    === grog@wantadilla (/dev/ttyp9) ~ 167 -> df
    Filesystem     1048576-blocks   Used Avail Capacity  Mounted on
    /dev/ad0s2a              9912   1384  7735    15%    /
    devfs                       0      0     0   100%    /dev
    /dev/ad0s1a              9912   5304  3815    58%    /destdir
    /dev/ad0s3h             51895  42615  5129    89%    /home
    /dev/ad2h               76285  61320  8861    87%    /dumpa
    procfs                      0      0     0   100%    /proc
    battunga:/               3969   3032   619    83%    /battunga
    battunga:/home           5267   1974  2872    41%    /battunga/home
    echunga:/               14873   9459  4224    69%    /echunga
    echunga:/home            9916   5301  3821    58%    /echunga/home
    echunga:/src           188356 122377 50911    71%    /src
    echunga:/dump          122037 109507  2766    98%    /dump
    devfs                       0      0     0   100%    /var/named/dev
    /dev/ad3s1h            187780 180764  5138    97%    /dumpb

    Note particularly /echunga/home and /battunga/home in that list. boskopp should also mount /wantadilla/home (mounted on /home in the list above). The way Apple does that is to mount everything under /Volumes, so “Finder” shows me three file systems called home. It doesn't even display more information in the screen title bar, so there's no way of telling which is which. What a pain!

  5. In addition, if I mount just these three file systems, they get mounted as /Volumes/battunga, /Volumes/echunga and /Volumes/wantadilla. If I then try to mount the parent file system (not a good idea, admittedly), battunga:/, the connection fails, presumably because it is trying to mount it on /Volumes/battunga and doesn't notice that the mount point is already in use.

In the evening, had a long-overdue talk on the phone with Calvin Sun, the head of MySQL's storage engine team. Had our usual problems with VoIP in the process: I could call him, but the call got dropped immediately; he was able to call me easily enough, though. But like many people at MySQL, he prefers Skype because it “just works”. Not I:

Why I prefer straight VoIP to Skype

Lots of people at MySQL like Skype, because they find it easier to use than straight VoIP. I'm not so happy:

The Skype advocates have justification, though: running VoIP is fraught with difficulties. Why is it so difficult? I get the impression that it's a combination of extremely poor documentation and the “appliance mentality”, where people think of things in isolation rather than in the context of networks.


Friday, 21 July 2006 Echunga Images for 21 July 2006
Top of page
previous day
next day
last day

More work on the backup code today. It's tiring work, and I can really only do a few hours at a time. I think I'll continue over the weekend; I have a deadline on Monday morning (i.e. evening) that I'd like to meet.

Chris and Yvonne went into town to buy lots of hardware: Chris got a couple of 300 GB disks with Firewire/USB enclosures and a DVD writer; they also bought some stuff for Jorge de Moya, and I got a replacement tuner card for the one I got from Ben Kramer last month. The only thing for me personally was a USB extension cable.

In the evening, set to replacing the DVD drive in delicious, which wasn't as obvious as it might have been. In practice, though, it was easy enough: remove the face plate and slide the drives in and out from the front. The face place has flanges on one side, and they just need to be pushed aside:


https://lemis.nyc3.digitaloceanspaces.com/grog/Photos/20060721/big/cover-1.jpeg

Image title: cover 1          Dimensions:          2816 x 2112, 416 kB
Make a single page with this image Hide this image
Make this image a thumbnail Make thumbnails of all images on this page
Make this image small again Display small version of all images on this page
All images taken on Friday, 21 July 2006, thumbnails          All images taken on Friday, 21 July 2006, small
Diary entry for Friday, 21 July 2006 Complete exposure details

 

Did that, started the machine, and it panicked (which on an Apple means you get a message in English, French and German saying “you must restart this computer”). Disconnected the drive again, sent a message to Apple (maybe; I wonder if mail is working, and it didn't offer me the option of copying me). After a bit of investigation, discovered that both the DVD drive and the Zip drive were jumpered as slave. Still, that shouldn't cause a panic (this is the latest version of MacOS X, 10.4.7).

While playing around with the Firewire/USB enclosures, decided to format one under Microsoft with pain.lemis.com, and ran into the same problem that I had last week, but appear not to have noted: I had tried to mount a USB stick (actually a flash memory in a USB adaptor), and the system hadn't found it. On that occasion people on IRC had helped me, but I had forgotten how to fix it. The problem is that there are two drives on the machine: hard disk (C:), DVD-ROM (D:), and a number of network (Samba) mounts, starting with E:. And that's the problem: the automount code didn't notice the network mount and tried to mount the USB stick as drive E:. This time Chris had a laptop with her, so we gave up and used that instead. It's also interesting to see the relative speed of building file systems under MacOS X and Microsoft XP. With identical drives, the former went in a couple of seconds, while the Microsoft format took hours; by that time, Chris had long backed up her entire Apple system.

Got a number of messages about how to do some of the things that I couldn't work out; thanks in particular to James Andrewartha (not for the first time), Ian Donaldson and “Venture37”. Clearly I need write these things down. I've made a start for MacOS X and Microsoft.


Saturday, 22 July 2006 Echunga Images for 22 July 2006
Top of page
previous day
next day
last day

Had lots of plans for things to do today, including further work on my online backup, but somehow they didn't eventuate. In the morning played around with the Mac for a while and discovered a few things; at least I now have my user ID set correctly, though there are so many things to read about NFS that I've deferred that.

Then out riding with Chris and Diane Saunders (and Yvonne, of course) in Kuitpo Forest. Somewhat slow ride, but quite pleasant. Saw a surprising number of kangaroos (in fact, I think they're really wallabies):


https://lemis.nyc3.digitaloceanspaces.com/grog/Photos/20060722/big/Kangaroos-2.jpeg

Image title: Kangaroos 2          Dimensions:          2816 x 2112, 1120 kB
Make a single page with this image Hide this image
Make this image a thumbnail Make thumbnails of all images on this page
Make this image small again Display small version of all images on this page
All images taken on Saturday, 22 July 2006, thumbnails          All images taken on Saturday, 22 July 2006, small
Diary entry for Saturday, 22 July 2006 Complete exposure details

 

Back home, more playing around with computers. In the evening, Yvonne had planned to cook Tournedos Rossini, but apart from the lack of truffles (are they really not available in Australia?), the piece of beef fillet that she was sold yesterday was not up to the task. Found a different recipe, Filet de Boeuf en brochette, aux Anchois, that was very good, though the name is somewhat misleading: the anchovies have just walked past the meat.


Sunday, 23 July 2006 Echunga
Top of page
previous day
next day
last day

More playing around with the Apples today, and got a lot more software installed, including—after several attempts—X. For reasons I don't understand, the build of X.Org 6.9 failed with a missing documentation directory. Finally ended up installing Fink, a Debian apt-get lookalike, which also set up configurations for loading most software I need. Ended up with the rather pleasant discovery that I can run X coexisting with Quartz, with the slight problem that this meant there was no root window, and thus no way to access my standard menus. I can run it with root mode, but then it's a separate display, and I don't find that as useful. I'll have to think out an alternative. The emulation even works so well that I could connect to the display from wantadilla using x2x, though (not surprisingly) only the X windows reacted.

Finally got round to continuing my work on online backup. It's really surprisingly tiring, and after a few hours I ended up in C++ type hell:

backup.cc: In function 'void spawn_thread(void* (*)(backup_thread*), backup_thread*)':
backup.cc:423: error: invalid conversion from 'void*' to 'void* (*)(void*)'
backup.cc:423: error:   initializing argument 3 of 'int pthread_create(pthread_t*, const p
thread_attr_t*, void* (*)(void*), void*)'

That seemed to be a good point to give up, so I did.

All is still not well with the ADSL line. Late on Friday night the downstream noise margin (the red line) suddenly decreased by 10 dB:

ADSL line statistics

Hopefully they'll sort that out.


Monday, 24 July 2006 Echunga
Top of page
previous day
next day
last day

Today was the day I was due to have the first cut of the backup code running. Well, I sort of made it: as I said on IRC:

* groggy celebrates.                                     [16:16]
<groggy> My backup code compiles!
* groggy ships.

Installing software on the Apple is much easier than on FreeBSD, but there are still some issues. After resetting my user ID to the same value as on my other systems, I had to change permissions on all the files belonging to me. That's straightforward; or it should be. What I got was:

=== root@boskopp (/dev/ttyp3) ~ 2 -> find ~grog | xargs chown grog
chown: /Users/grog/.Trash/Recovered: No such file or directory
chown: files: No such file or directory
chown: /Users/grog/.Trash/Recovered: No such file or directory
chown: files/SoftwareUpdateCheck.pid: No such file or directory

This went on for hundreds of lines. The problem? This BRAIN DEAD idea of putting spaces into file and directory names, which semi-literate people are now introducing into the UNIX space as well. Yes, spaces are allowed in UNIX file names, but they cause untold pain—not just for UNIX users. Microsoft's COMMAND.COM can't handle them either. I suppose I can see a reason for spaces in descriptive names, but putting them in file names is a short-sighted concept that only really works for mouse-pushers, and which causes pain in so many places. In this case, I had to cd manually into the directories—a problem for my bash macros as well:

=== root@boskopp (/dev/ttyp3) ~ 4 -> cd /Users/grog/Library/Application\ Support/
su: cd: /Users/grog/Library/Application: No such file or directory
=== root@boskopp (/dev/ttyp3) ~ 5 -> command cd /Users/grog/Library/Application\ Support/
=== root@boskopp (/dev/ttyp3) /Users/grog/Library/Application Support 6 ->

The problem here is that when the cd macro gets its parameter, the \ has been stripped, so what it sees is two parameters. I could join them together, and maybe I should, but the bash command command gets around it in the few cases where it bites me. The problem isn't just with the shell; while writing this page, the Emacs macro that I use to set the prompt colours failed on the last prompt, because it had a space in it. These names break all sorts of things.


Tuesday, 25 July 2006 Echunga
Top of page
previous day
next day
last day

Got into the office this morning to discover that a showstopper bug was still open, and today or tomorrow was the deadline for release 5.0.24 of MySQL. We had committed a patch a couple of weeks ago, but only in the 5.1 tree. Spent a bit of time looking at that, but had to wait until the evening to have it committed, since the release engineers were all sleeping. The sun may never set on MySQL, but it does on individuals and even departments.

Apart from that, continued work on my backup code. Yesterday I got it to compile, but not to link. Spent far too much time doing that today; C++ casts are the work of the devil, and POSIX threads don't help either. Got mysqld to link, but I'm still fighting the Makefiles to get the rest of the build to complete; currently mysqltest is the problem.

Got a message from Tim Bishop about my problems with file names with spaces in them; he pointed out that you can use the print0 predicate for find:

find -print0 ~grog | xargs -0 chown grog

This is correct; I had already found it while looking at the man pages—for the FreeBSD version of find. The MacOS man page looks almost the same, but the following paragraph is missing:

             However, you may wish to consider the -print0 primary in conjunc-
             tion with “xargs -0” as an effective alternative.

Following up the differences, it seems that this change was added over 4 years ago:

revision 1.43
date: 2002/04/10 16:39:22;  author: trhodes;  state: Exp;  lines: +7 -0
find.1 does not encourage users to DTRT when piping to xargs(1)

My guess is that Apple doesn't track the man pages well enough.


Wednesday, 26 July 2006 Echunga Images for 26 July 2006
Top of page
previous day
next day
last day

Finally fixed the Makefile and got the server to build completely today; it ran the test suite as far as backup (a test that I didn't know about, and that will definitely need to be rewritten), where the server crashed, not surprising for the state it's in.

Spent the rest of the day looking at bugs; strangely, lack of disk space is becoming an issue. It's easy to clone a tree for every bug, and in some cases I have two for the same issue, but it eats into disk space. Ended up using the old 30 GB external USB drive that Chris left behind over the weekend. It was really slow under MacOS X, and I was surprised to find it working pretty much normally under Linux. Maybe the Mac only supports USB version 1.


Thursday, 27 July 2006 Echunga Images for 27 July 2006
Top of page
previous day
next day
last day

Spent the morning looking at a bug that had an associated patch, but which hadn't been pushed. Merging it proved to be more difficult than I thought: it was supposed to have been reviewed, but I can't imagine that the review was completed. Certainly it's not the quick fix that I was hoping for; it'll need some considerable review.

In the afternoon to the monthly board meeting of the ICT Council for South Australia. Today Janet Hawtin came and talked to us about the proposed legislation for implementing DMCA compliance in Australia. The board had given me the impression initially that they grudgingly allowed her to talk, but certainly she got a great deal of attention. I think it was worthwhile; it makes people think about the issues. One of the biggest dangers I see with the DMCA is that many people think it won't affect them. By the time they find out that it will, it can be too late.


Friday, 28 July 2006 Echunga
Top of page
previous day
next day
last day

Bug fixing day today. Spent more time looking at yesterday's bug before deciding that I was by no means convinced that the approach is right, and that it would take a lot of work to fix. Deferred that and moved on to another bug, also with a patch; that was trivial by comparison. It's upsetting how much of my work is not fixing bugs but deciding on the correct fix; and for that, I frequently find that my experience in the area is inadequate.

SQL editors

About six months ago, on the recommendation of my accountants, I bought an accounting system, Mind Your Own Business. It runs under Microsoft, and I find it painful in the extreme. In the end, I decided not to use it.

On the other hand, what I was using before doesn't really cut it: as a died-in-the-wool UNIX man, I used a set of home-grown awk scripts to manipulate records which look like this:

TA#20060129#4##(2) Transfer in Brisbane
M#20060129#15.95##(3) Dinner
A#20060129#25##(4) Departure tax##NZD#0.928
M#20060129#41##(5) Lunch (Arjen, Grog)##NZD#0.928
M#20060128#25.95##(6) Dinner##NZD#0.928
M#20060126#9.5##(8) Lunch##NZD#0.928

UNIX people will recognize “intuitively” that these records consist of fields separated by a hash mark, # (and not a pound sign, £, which would be very confusing in this context). It's a pain to write, and to a MySQL employee it doesn't exactly sound like “own dog food”. So I've started migrating the database to MySQL. That's not really difficult; with a small Emacs script I can convert the example above into:

insert into expenses (rowkey, date, sum, gst, comment, currency, exchange_rate) values
('TA', '20060129', '4', '', '(2) Transfer in Brisbane', '', ''),
('M', '20060129', '15.95', '', '(3) Dinner', '', ''),
('A', '20060129', '25', '', '(4) Departure tax', '', 'NZD', '0.928'),
('M', '20060129', '41', '', '(5) Lunch (Arjen, Grog)', '', 'NZD', '0.928'),
('M', '20060128', '25.95', '', '(6) Dinner', '', 'NZD', '0.928'),
('M', '20060126', '9.5', '', '(8) Lunch', '', 'NZD', '0.928');

That's enough to convert it, but entering it in this form is, if anything, even more painful. What I need is simple: a screen for entering data into a MySQL table. I wrote something similar myself for my still-born Monkey database years ago. There must be dozens out there. So I went looking.

What I found surprised me. I didn't find nearly as many as I expected. MySQL supplies a program called MySQL Query Browser, which also offers the ability to edit tables. I downloaded it and tried it out. Yes, I could display my tables in typical tabular format, and there's an Edit button at the bottom of the screen—but greyed out.

I discussed this matter on one of our IRC lists, and got the information that the edit function only works if the table has a primary key. Primary keys are an issue which cause hot debate within MySQL. In Monkey, like in many systems, they were mandatory; if the table definition didn't have one, it created one automatically. The example above is a typical one where a primary key is of no use whatsoever to the application; the only reason would be to satisfy the system.

But why do we need a primary key? From the IRC discussion:

* groggy plays with MySQL Query Browser.
<groggy> Anybody know their way around it?
<groggy> Somebody said that you could use it to input data, but the only way I
     can see is to do an INSERT.
<Shawn> groggy: try a select * from a table and see if the results are editable
<groggy> Shawn: Yes, that's what I thought too.  Doesn't look like it :-(
* Shawn hasn't touched his in weeks.  Can't remember much about it
<Shawn> groggy: from the 5.0 cert manual, if the table has a pk and the
    results are from a single table, then the results are editable (look
    for an edit button)
<groggy> Shawn: Ah.  Must have a PK?  That could be it.
<groggy> Shawn: There's an edit button, but it's greyed out.
<groggy> Shawn: I don't have a primary key on the table, and it's not obvious
     that you need one.
<Shawn> without a PK, you have to hope that every row is different (across all
    fields), otherwise you can't update just one.
<Shawn> just one row*.
<groggy> Shawn: Hmm.
<groggy> You could delete one row with that content.
<Shawn> not if it weren't unique in the table! Try this: create two identical
    rows of data in a table without a PK.  Now modify just one of them to
    be different than the other.
<Shawn> groggy: you can't do it because you have no way to distinguish one row
    from the other.
<groggy> Shawn: DELETE ...  LIMIT 1?
<Shawn> ok, the limit statement will allow you to change one of them but you
    can't pick which.  Score it as 1/2 a point for you
<groggy> Shawn: By definition you don't care which one.
* groggy grabs his other half point.
<groggy> Shawn: Seriously, I'm not suggesting it's a good idea.  But it looks
     as if it can be done.

The more I think of it, the more I think that it should be possible to edit a table without a primary key. But Query Browser is still really not the tool I'm looking for: it's not intended to be a table editor. After some googling, came up with phpMyEdit, which looked like it would be suited. The documentation, which is reachable via the Support tab from the home page, is a little bare-bones:

phpMyEdit is available for download from Platon.SK. Extract, or unzip, the program files to your computer's hard disk. The download file includes icons and various language files which should be extracted into sub-folders below the file named phpMyEdit.class.php.

What's a “sub-folder” below a file? In any case, guessed that I just needed to extract the archive somewhere into my web server data directory, so did that, and it worked (thus also proving that I must have installed PHP on the system at some time in the past). Well, almost. It still needs a primary key (though the documentation claims that you don't need one; but the option it mentions seems to be no longer available). Anyway, it's something to play with, and if all else fails, I'll capitulate and add a primary key.


Saturday, 29 July 2006 Echunga Images for 29 July 2006
Top of page
previous day
next day
last day

I had intended to get back to the pain of my CVR project today, but somehow I couldn't summon up the energy. Instead read a bit, watched TV and in general behaved like a normal human being on a weekend. It's a strange feeling.


Sunday, 30 July 2006 Echunga
Top of page
previous day
next day
last day

Another quiet day today. Finally got round to investigating alternative ways of mounting NFS file systems on Mac OS X. They don't show the system from its best side.

Basically the alternatives involve automount(8), which in some cases is possibly useful. For NFS, it's a pain. To summarize these instructions, there are two ways: the very difficult way and the difficult way. First, the GUI way (i.e. the very difficult way):

Why do people have to make things so complicated? One of the lessons of UNIX was that there is a great advantage in keeping things in plain text files that can be edited easily. If it has to be stored in a database, then it should be in a standard database (dare I say MySQL?) means, but even the simplest editor (in fact, something like the SQL editor that I've been looking for) would do that easily.

The instructions go on to say that you can use the program nicl(8) to do it as well. For the same commands, this would do the job:

nicl .  -create /mounts/battunga:\/home
nicl .  -append /mounts/battunga:\/home opts rw
nicl .  -append /mounts/battunga:\/home type nfs
nicl .  -append /mounts/battunga:\/home dir /battunga/home

That's a lot to type, but of course Emacs can help. The following LISP function converts a standard /etc/fstab into a script that will do the job:

(query-replace-regexp
"^\([^:]+:\)\([^     ]+\)\([   ]+\)\([^  ]+\).*nfs[     ]+\([^     ]+\).*"
 "nicl .  -create /mounts/\1\\\\\2
nicl .  -append /mounts/\1\\\\\2 opts \4
nicl .  -append /mounts/\1\\\\\2 type nfs
nicl .  -append /mounts/\1\\\\\2 dir \3" nil nil nil)

After that, I was able to mount the file systems. Well, almost. I discovered that HUPping automount caused it to first umount all NFS file systems and then try to remount them. It also doesn't mount them where it is told:

=== root@boskopp (/dev/ttyp1) ~ 159 -> df /battunga
Filesystem 1048576-blocks Used Avail Capacity  Mounted on
battunga:/           3969 3032   619    83%    /private/var/automount/battunga

Instead, it creates a symlink:

=== root@boskopp (/dev/ttyp1) ~ 161 -> ls -l /battunga
lrwxr-xr-x   1 root  admin  26 Jul 30 13:37 /battunga -> /automount/static/battunga

Presumably there's a second (and maybe further) symlink involved.

That's all fine and good, but what happens for, say, /wantadilla/home? It's mounted on /wantadilla, a remote system. automount can't go and replace that with a symlink, though it tries. Instead, it says nothing except in the log file (/var/log/asl.log; I wonder what that stands for):

[Time 2006.07.30 04:41:30 UTC] [Facility daemon] [Sender automount] [PID 151] [Message Cannot unlink existing /wantadilla/home: Resource busy] [Level 3] [UID -2] [GID -2] [Host boskopp]

That's really a showstopper. There's no way I can use automount to mount my NFS file systems in the places I want them. But why?

So what to do? Basically, there's no reason to go along with this kludge. The man page mount(8) states, just like it does in FreeBSD (the text is identical except for the word “command”, which is called “utility” in the FreeBSD version):

     The mount command calls the mount(2) system call to prepare and graft a
     special device or the remote node (rhost:path) on to the file system tree
     at the point node.  If either special or node are not provided, the
     appropriate information is taken from the fstab(5) file.

That's not surprising, given that the command came from FreeBSD. But it's wrong:

=== root@boskopp (/dev/ttyp1) ~ 171 -> mount /wantadilla/home
mount: /wantadilla/home: unknown special file or file system.

ktrace shows that it doesn't even try to access /etc/fstab.

Still, the easiest way to solve the problem is to ignore the MacOS framework and just do it manually at startup:

=== root@boskopp (/dev/ttyp1) /var/tmp 179 -> grep -v '^ *#' /etc/fstab | \
    awk '{print "mount " $1 " " $2}' | sh

Why do people have to break things?


Monday, 31 July 2006 Echunga
Top of page
previous day

Monday mornings always have a lot of mail, but today was particularly bad. One was obviously spam, and I looked at the headers (here an excerpt):

From abuse@abuse.earthlink.net  Sun Jul 30 23:49:04 2006
Return-Path: <abuse@abuse.earthlink.net>
Received: from dillinger.admin.atl.earthlink.net (dillinger.admin.atl.earthlink.net [207.69.200.23])
        by wantadilla.lemis.com (Postfix) with ESMTP id 46B6F9BD8D
        for <grog@lemis.com>; Sun, 30 Jul 2006 23:49:00 +0930 (CST)
Received: (from abuse@localhost)
        by dillinger.admin.atl.earthlink.net (8.9.0/8.9.0) id DAA16863;
        Wed, 7 Dec 2005 03:28:26 -0500 (EST)
Date: Wed, 7 Dec 2005 03:28:26 -0500 (EST)
From: Earthlink Abuse Department <abuse@abuse.earthlink.net>
Message-Id: <200512070828.DAA16863@dillinger.admin.atl.earthlink.net>
To: grog@lemis.com
References: <d716c2d45ee280265e7e301a83ce73d6@lury>
In-Reply-To: <d716c2d45ee280265e7e301a83ce73d6@lury>
Subject:  [AUUGN]: Art for the New Assets Institute

>This is a multi-part message in MIME format.

>--===============0766769222==
>Content-Type: multipart/related;
>       boundary="----=NextMime00A_000_10458078D"

>This is a multi-part message in MIME format.

>------=NextMime00A_000_10458078D
>Content-Type: text/html;
>       charset="windows-1252"
>Content-Transfer-Encoding: quoted-printable

><!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4=2E0 Transitional//EN">
><HTML>
><HEAD>
><META http-equiv=3DContent-Type content=3D"text/html; charset=3D"></HEAD>

That looked surprisingly genuine, though the mess itself made no sense (it was completely plain text). I sent it back to abuse@earthlink.net, and got it sent back to me again:

From abuse@abuse.earthlink.net  Mon Jul 31 10:17:25 2006
Return-Path: <abuse@abuse.earthlink.net>
X-Original-To: grog@lemis.com
Delivered-To: grog@lemis.com
Received: from dillinger.admin.atl.earthlink.net (dillinger.admin.atl.earthlink.net [207.69.200.23])
        by wantadilla.lemis.com (Postfix) with ESMTP id 5DF539BD85
        for <grog@lemis.com>; Mon, 31 Jul 2006 10:17:21 +0930 (CST)
...
References: <d716c2d45ee280265e7e301a83ce73d6@lury> <200512070828.DAA16863@dillinger.admin.atl.earthlink.net>
+<20060731003259.GV88193@wantadilla.lemis.com>
In-Reply-To: <20060731003259.GV88193@wantadilla.lemis.com>
Subject:  Spam from earthlink abuse department? (was: [AUUGN]: Art for the New Assets Institute)

What is Earthlink doing? As I asked,

> Can you tell me what the purpose of this is?  From the headers, it
> clearly comes from earthlink.  It has nothing that remotely suggests a
> connection with me.  If you think it has, you should at least state
> your reasoning.  Or have you joined the ranks of the spammers?

Usual Monday, spent mainly catching up with things.


Do you have a comment about something I have written? This is a diary, not a “blog”, and there is deliberately no provision for directly adding comments. It's also not a vehicle for third-party content. But I welcome feedback and try to reply to all messages I receive. See the diary overview for more details. If you do send me a message relating to something I have written, please indicate whether you'd prefer me not to mention your name. Otherwise I'll assume that it's OK to do so.


Top of page Previous month Greg's home page Today's diary entry Next month Greg's photos Copyright information

Valid XHTML 1.0!

$Id: diary-jul2006.php,v 1.54 2021/07/30 05:22:59 grog Exp $