Image of grog
Greg's computer video recorder project
Greg's diary
Photo index
Greg's home page
Network link stats
Greg's other links
Copyright information
Main multimedia page
Multimedia packages page

This page describes the fun I have had setting up a computer as a video recorder, covering the time period April to December 2006. I also have older information which hopefully won't be needed except to document the pain, and the current log. See the main page for an overview.

Inevitably, I make mistakes. As I gain experience, I realize these mistakes; rather than just correcting the text, I prefer to leave the original statement and add a clarifying comment: if I have made a mistake, others might do the same.

Yes, this page is done in chronological order. I hate reverse chronological order. The latest entry is here.

Sunday, 2 April 2006

Discussions on Australian lists suggest that I won't find a better alternative to computer video than MythTV or mplayer, so in the afternoon spent more time trying to get things to work. It's still like pulling teeth: spent all afternoon trying to work out how to get my tuner card set up. I've done this before with FreeBSD, but with Linux and even help from people on the list, it still took all afternoon. People on the lists suggested using xawtv to test the tuner; I've already commented on how bad that program's documentation is. It now comes with a new program called scantv. It prompts for standard and country and then does a scan of all channels. The problem is, it looks for a device called /dev/vbi, which doesn't exist. In FreeBSD, I can find out about devices from the man pages: for example, for a /dev/bktr I'll find a man page bktr(4). Linux, at least as I have installed it, has nothing corresponding. Google didn't help much either; the suggestion of creating a device with mknod based on major device numbers of several years ago didn't seem to be the right way to go. Finally found that there's a device called /dev/vbi0, so specified that instead, and it seemed to work. I still have no idea what /dev/vbi0 is. The name shows up in files like drivers/media/video/bttv-driver.c in the kernel, but it has almost no comments, certainly nothing to explain what vbi means.

The output of scantv was not surprising: I had no antenna cable connected, so at least for that reason it didn't find anything. I need to move it to where there's an antenna, but by the time I had got that far, I had no more time. Next week.

Tuesday, 4 April 2006

One useful source of information that I shouldn't discount is this diary; people read it and reply to me. Today on IRC, Olli Lehmann told me about the meaning of /dev/vbi. pointing to an article on zapping VBI. It explained the meaning of the name: “Vertical Blanking Interval”. Makes sense (the vertical blanking interval is often used to carry additional information). But what a way to find out.

Saturday, 8 April 2006

More work on trying to get my TV tuners to work on ceeveear, the Ubuntu machine. It's still not easy. One of the strangest things I've seen has been that NFS mounting file systems from FreeBSD to Ubuntu takes 104 seconds per file system. Once mounted, access to the file system works fine. I suspected an incompatibility between the implementations, so spent some time running ethereal to try to work out what was going wrong, but that just showed a normal NFS mount operation followed by 104 seconds of nothing. The local (Ubuntu) /var/log/messages was more informative, but I no longer have the messages (see the end of today's entry); something about “unable to connect to localhost”. I suspect that it's a problem with the initial installation, but it happens on both machines (eucla, the laptop, and ceeveear, the TV machine).

The tuner was easier. I connected it to an antenna and answered the questions, and was gratified by:

=== root@ceeveear (/dev/pts/0)  color="red">/home/grog   color="blue">22 -> scantv -C /dev/vbi0
6    (175.25 MHz): no station
7    (182.25 MHz): ???
[unknown (7)]
channel = 7

8    (189.25 MHz): no station
9    (196.25 MHz): ???
[unknown (9)]
channel = 9

10   (209.25 MHz): ???
[unknown (10)]
channel = 10

11   (216.25 MHz): no station

That all looks correct: we do indeed have channels 7, 9 and 10. But according to the man page,

       scantv  scans  a  v4l  device  for  available  TV stations and writes a
       xawtv/fbtv config file.

So where's xawtv/fbtv? Not in the cwd, nor in the home directory. Since it didn't mention the pathname, decided that it might be somewhere else, and tried running xawtv anyway. Nothing, but the man page said that the configuration file was called .xxawtv, and not xawtv/fbtv. That wasn't there either. Ran scantv again and confirmed that it didn't write any output file. Then I read more of the man page:

       -o outfile
              specify output file.  If none is specified, scantv write to std-

It turns out that the text in the example above is in fact the configuration information:

[unknown (7)]
channel = 7

Not that you can do anything with it like that; both the defaults and the man page are less than suitable.

Finally created a configuration file, and it selected channel 61 (conveniently, not occupied, so just noise on the screen), and discovered that I could not change channel: neither mouse nor keyboard worked. This seemed to be related to the messages it printed on startup:

This is xawtv-3.94, running on Linux/i686 (2.6.12-8-386)
Warning: translation table syntax error: Unknown keysym name:  XF86AudioRaiseVolume
Warning: ...  found while parsing '<Key>XF86AudioRaiseVolume: Command(volume,inc)                '
Warning: String to TranslationTable conversion encountered errors
Warning: translation table syntax error: Unknown keysym name:  XF86AudioRaiseVolume
Warning: ...  found while parsing '<Key>XF86AudioRaiseVolume: Command(volume,inc)                '
Warning: String to TranslationTable conversion encountered errors
Warning: Cannot convert string "-*-ledfixed-medium-r-*--39-*-*-*-c-*-*-*" to type FontStruct
Warning: translation table syntax error: Unknown keysym name:  XF86AudioRaiseVolume
Warning: ...  found while parsing '<Key>XF86AudioRaiseVolume: Command(volume,inc)                '
Warning: String to TranslationTable conversion encountered errors

On another attempt, it found a different channel, one that was in use, and showed that it was, indeed, receiving TV. But it still didn't accept any mouse or keyboard input. So, possibly a forgotten dependency? That seems common in Linux—see the installation of mplayer. Looking for related packages, I found:

=== root@ceeveear (/dev/pts/0) color="red">/home/grog color="blue">26 -> apt-cache search xawtv
liblircclient0 - LIRC client library
came - Rewrite of the xawtv webcam app using imlib2
libvideo-capture-v4l-perl - Perl interface to the Video4linux framegrabber interface
pia - movie player
scantv - scan TV channels for stations
v4l-conf - tool to configure video4linux drivers
xawtv - X11 TV application
xawtv-plugin-qt - quicktime plugin for xawtv and motv
xawtv-plugins - plugins for xawtv and motv
xawtv-tools - Miscellaneous tools distributed with xawtv
libmjpegtools-dev - MJPEG video capture/editting/playback MPEG encoding
libmjpegtools0 - MJPEG video capture/editting/playback MPEG encoding
mjpegtools - MJPEG video capture/editting/playback MPEG encoding

So, which were installed and which not? What I wanted was the Linux equivalent of FreeBSD's pkg_info. Looking through the documentation for apt-get and apt-cache, that didn't seem right. Neither did the documentation for dpkg. So I went to the Ubuntu support site, which pointed me to a quick tour and a start guide, neither of which I wanted: I wanted a user guide. Looking on Google brought me 102,000 hits for "Ubuntu User Guide". The first one looked good, but despite calling itself an “unofficial user guide” (where's the official one?), it's a forum. Most of the other hits looked less than useful, so I limited my search to PDF documents. One single hit, not from the official Ubuntu site, and only 31 pages long, including table of contents, introductions and disclaimers. It had some stuff on software installation, but still nothing on how to find out which packages were installed.

OK, this system is laid out to work under GNOME, and there's a graphical program called synaptic. That produced its own error messages:

(synaptic:7099): Gtk-CRITICAL **: gtk_accel_label_set_accel_closure: assertion `gtk_accel_group_from_accel_closure (accel_closure) != NULL' failed
(synaptic:7099): Gtk-CRITICAL **: gtk_accel_label_set_accel_closure: assertion `gtk_accel_group_from_accel_closure (accel_closure) != NULL' failed
(synaptic:7099): Gtk-CRITICAL **: gtk_tree_view_unref_tree_helper: assertion `node != NULL' failed
(synaptic:7099): Gtk-CRITICAL **: gtk_tree_view_unref_tree_helper: assertion `node != NULL' failed

It also produced a ridiculously badly proportioned display, showing only the first third of the package category names, and that on a full screen 1600x1200 display. Fixed that, selected multimedia and looked for xawtv. Nothing. Also nothing in multimedia-multiverse and multimedia-universe. Only when I did a search for xawtv did I get a display that matched the list from apt-cache above, only alphabetically sorted. It showed that everything I could reasonably expect to install was already installed. So where is the problem? An hour of searching useless and non-existent documentation, and no closer to the solution.

At about this point, and not for the first time, I gave up. I decided that the hardware was working, but the software was not. I've had at least these problems with Ubuntu, and they shouldn't happen in a production release:

Not all of these have to be Ubuntu's fault, of course, and I'm reserving judgement on one other issue, which may be a configuration problem: when I open an X terminal on one of the FreeBSD machines that comprise my office desktop, the shell doesn't handle the Meta- characters correctly. For example, ~-F should move to the end of the following word. It does on FreeBSD, and it also does so on the default Ubuntu display; but with an Ubuntu X-term on one of my FreeBSD X servers, it capitalizes the word. Other keys are even less use. But that may be because of a discrepancy in character sets (UTF-8 on Linux, ISO-8859-1 (8 bit) on FreeBSD).

Anyway, I had just downloaded Fedora Core 5 from the Internode mirror server, so decided to try that on the spare partition (I always put two root partitions of 4 or 5 GB on my machines, and I only put system-related stuff on them, so that I can move from one release to another relatively easily. eucla even has three, two FreeBSD and a Linux, all of which share the same /home file system). Installed Fedora on ceeveear and was gratified by a much cleaner install. No problems with the meta-characters, NFS mounts Just Work, I now have a /dev/vbi, and I even found a MythTV HOWTO that seems to make sense, so followed that. Hopefully I won't run into Yet Another dead end.

Another interesting thing about GNOME is that I was able to get the top menu bar to display on wantadilla, my main machine, which runs fvwm2 as its standard window manager; this could be a good compromise between the GNOME menus and the flexibility of fvwm2. Today, though, it confused things: for some reason, it slowed down the repeat rate of the keyboard, and also disabled kbdcontrol, which should set the repeat rate. It also changed the input line editing on my local (wantadilla) firefox from Emacs-like to something Microsoft-like. But I know about that, and hopefully I'll be able to find a solution that is more convenient than anything I have so far.

Sunday, 9 April 2006

Up early this morning with the intention to finally make progress with the Computer Video Recorder. I did make quite a bit of progress, but mainly towards getting functional Linux installations on ceeveear and eucla:

Following For the SiS drivers for ceeveear, execute the following as root:

mkdir -p ~mythtv/sources/sis_drv
cd ~mythtv/sources/sis_drv
tar -xzvf sis_drv.o_xorg_gcc3_current.tar.gz
mv /usr/X11R6/lib/modules/drivers/sis_drv.o /usr/X11R6/lib/modules/drivers/sis_drv.o.orig
cp sis_drv.o /usr/X11R6/lib/modules/drivers/sis_drv.o

The directory contains a very large number of similarly named files, which firefox conveniently truncates. sis_drv.o_xorg_gcc3_current.tar.gz appears to be the latest version (i.e. not a static content).

At this point, things went wrong. /usr/X11R6/ has only one directory in it, bin. Fortunately, Fedora seems to do the right thing by locate (or maybe I was just lucky that it gets done Saturday night; I can't work out the format of the periodic cron scripts). It shows:

=== root@ceeveear (/dev/pts/2) color="red">/home/mythtv/sources/sis_drv color="blue">35 -> locate sis_drv

OK, rewrite that script:

mkdir -p ~mythtv/sources/sis_drv
cd ~mythtv/sources/sis_drv
tar -xzvf sis_drv.o_xorg_gcc3_current.tar.gz
mv /usr/lib/xorg/modules/drivers/sis_drv.o /usr/lib/xorg/modules/drivers/sis_drv.o.orig
cp sis_drv.o /usr/lib/xorg/modules/drivers/sis_drv.o

Interestingly, though, that's still not the same name:

=== root@ceeveear (/dev/pts/2) color="red">/home/mythtv/sources/sis_drv color="blue">41 -> l /usr/lib/xorg/modules/drivers/sis_drv*
-rw-r--r-- 1 root root 597402 Apr  9 12:03 /usr/lib/xorg/modules/drivers/sis_drv.o
-rwxr-xr-x 1 root root 609444 Feb 13 05:53 /usr/lib/xorg/modules/drivers/
=== root@ceeveear (/dev/pts/2) color="red">/home/mythtv/sources/sis_drv color="blue">42 -> file /usr/lib/xorg/modules/drivers/sis_drv*
/usr/lib/xorg/modules/drivers/sis_drv.o:  ELF 32-bit LSB relocatable, Intel 80386, version 1 (SYSV), not stripped
/usr/lib/xorg/modules/drivers/ ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), stripped

So which do we take? Took a look at Thomas Winischhofer's SiS chipset overview, which was instructive. ceeveear has:

02:00.0 VGA compatible controller: Silicon Integrated Systems [SiS] 315PRO PCI/AGP VGA Display Adapter (prog-if 00 [VGA]
        Subsystem: Silicon Integrated Systems [SiS] 315PRO PCI/AGP VGA Display Adapter
        Flags: bus master, 66MHz, medium devsel, latency 39, IRQ 5
        BIST result: 00
        Memory at d0000000 (32-bit, prefetchable) [size=256M]
        Memory at e5000000 (32-bit, non-prefetchable) [size=256K]
        I/O ports at c000 [size=128]
        [virtual] Expansion ROM at e4000000 [disabled] [size=64K]
        Capabilities: [40] Power Management version 2
        Capabilities: [50] AGP version 2.0

On further investigation, contains the file /usr/lib/xorg/modules/drivers/, and it's the correct version of Xorg, so I'll go for that.

While doing that, decided to set up the environment a bit better. One thing I definitely need is a window manager that I'm happy with, and that's pretty much fvwm2. Found the RPM and downloaded it, and was then forcibly reminded of why I didn't like Fedora:

=== root@ceeveear (/dev/pts/2) color="red">/home/mythtv/sources color="blue">102 -> rpm -i fvwm-2.5.13-1.i386.rpm
error: Failed dependencies: is needed by fvwm-2.5.13-1.i386 is needed by fvwm-2.5.13-1.i386 is needed by fvwm-2.5.13-1.i386 is needed by fvwm-2.5.13-1.i386

Spent a bit of time following up these references before it dawned on my that I was being silly. Some of these were missing simply because they were out of date. What I should have been doing was building from source—after all, when it comes to porting, I wrote the book. That worked relatively simply. Getting it to actually work was more complicated; I need to read more about GNOME.

At this point, the instructions became quite complicated, and I first need to read both them and the MythTV documentation to understand what needs to be done. The good news appears to be that there's a generic IR remote control application, which will make things much easier. But that's for another day. Tried installing xawtv, but again ended up with a maze of twisty little unfulfilled dependencies, all different. An attempt to build xawtv from source also failed, since I first needed to install the Xorg sources. Put that in the “too hard” basket.

Monday, 10 April 2006

Input from James Andrewartha on the topic of my Ubuntu woes. More in my diary, but this relates to CVRs:

Due to the way Linux development works, a lot of device information is only located in Documentation/ in the kernel source. DocBook/videobook.tmpl contains the following:

        The VBI devices capture the hidden lines on a television picture
        that carry further information like closed caption data, teletext
        (primarily in Europe) and now Intercast and the ATVEC internet
        television encodings.</entry>

Sunday, 16 April 2006

Back to looking at CVR software. Last Sunday I had got to the stage where I couldn't install xawtv on Fedora Core 5: the binary package had a missing dependency on, and the source didn't build with some missing header files.

Investigated both. For, the obvious choice was to find a yum repository with the library. It seems, though, that there are no lists of yum repositories—some of the web pages I found even seemed proud of the fact. Found a watch4tv page, which doesn't say so but appears to be a list of instructions to install a TV application on an operating system that uses yum. That was helpful, but didn't get me past the basic problem:

=== root@ceeveear (/dev/pts/1) color="red">/src/FreeBSD/ports/distfiles color="blue">48 -> rpm --import
=== root@ceeveear (/dev/pts/1) color="red">/src/FreeBSD/ports/distfiles color="blue">49 -> rpm --import
=== root@ceeveear (/dev/pts/1) color="red">/src/FreeBSD/ports/distfiles color="blue">50 -> yum install xawtv
Loading "installonlyn" plugin
--> Running transaction check
--> Processing Dependency: for package: xawtv
--> Finished Dependency Resolution
Error: Missing Dependency: is needed by package xawtv
=== root@ceeveear (/dev/pts/1) color="red">/src/FreeBSD/ports/distfiles 51 ->

Confirmed that I had a /usr/lib/, but symlinking that to /usr/lib/ didn't get me past the problem. Went back to the issue of building from source; further investigation showed that the missing header file belonged to the X font server, and that xawtv, though it uses autoconf, didn't seem to notice that the X libraries were not installed. Installing that made it build and brought instant gratification:

Click on the picture to see a medium-size version in the index

That's the TV image in centre left: it's tiny by itself:

Click on the picture to see a medium-size version in the index

It is indeed smaller than it should be, only 394x314. That's something to investigate, but the real issue was to get it to run on ceeveear. That didn't work as well:

=== grog@ceeveear (/dev/pts/11) ~ color="blue">3 -> DISPLAY=:0 xawtv
This is xawtv-3.95, running on Linux/i686 (2.6.16-1.2080_FC5)
/dev/video0 [v4l2]: ioctl VIDIOC_G_FBUF: Invalid argument
v4l-conf had some trouble, trying to continue anyway
ioctl: VIDIOC_G_FBUF(capability=0x0 [];flags=0x0 [];base=(nil);fmt.width=0;fmt.height=0;fmt.pixelformat=0x00000000 [....];fmt.field=ANY;fmt.bytesperline=0;fmt.sizeimage=0;fmt.colorspace=unknown;fmt.priv=0): Invalid argument

That looks like a problem with the X configuration; maybe the HOWTOs will help there. Also the program scantv that I used last week under Ubuntu is no longer built by default; the sources are there, but when I try to compile them, I get lots of errors. More stuff to investigate. This whole exercise reminds me of when I was writing Porting UNIX Software well over ten years ago. Why do these problems still exist?

Yvonne back with more mushrooms today, along with the information that she had met some Polish women in the forest who were gathering large quantities. They offered to tell her the name in Polish, but she didn't bother. I sent a message out to an internal MySQL list and got answers from Mårten Mickos and Boel Larsen, who both apparently eat them and tell me, as did Alexander Keremidarski, that they're not boletus pinophilus, but suillus luteus, called smörsopp in Swedish and maslovka in Bulgarian (also Butterpilz in German), all of which mean the same thing. Certainly the photo looks convincing enough. They need peeling, apparently, and there are some indications of people who get stomach upsets from them, possibly only if the skin is not removed.

Tuesday, 18 April 2006


Today continued my attempts to install MythTV. First continued with xawtv, though, and looked for scantv, which I have used before. It wasn't there. There's no target for scantv in the top-level directory, but there's a file console/scantv.c that won't compile in that directory:

=== root@ceeveear (/dev/pts/1) color="red">/usr/src/ports/xawtv-3.95/console color="blue">14 -> make scantv
cc     scantv.c   -o scantv
scantv.c:5:20: error: config.h: No such file or directory
scantv.c:19:25: error: frequencies.h: No such file or directory

Presumably config.h was in the top-level directory, so tried it there; it got a bit further:

=== root@ceeveear (/dev/pts/1) color="red">/usr/src/ports/xawtv-3.95 color="blue">16 -> make console/scantv
  CC      console/scantv.o
console/scantv.c:47: warning: 'struct vbi_event' declared inside parameter list
console/scantv.c:47: warning: its scope is only this definition or declaration, which is probably not what you want

With a bit of investigation (including far too much guesswork), established that this related to libzvbi, and that the corresponding package was called zvbi. Installed that and zvbi-devel, but scantv still didn't compile. It looked as if the header files weren't being found; but which header files? Spent some time trying to get yum to tell me which files were associated with the packages that were installed (something like FreeBSD's pkg_info -f), but that doesn't seem to exist. It was also strange that scantv.c didn't complain about missing header files. Then it dawned on me: configure was run without the libraries being present, so it decided not to build the programs. Re-ran configure and all got built.

Next was the problem with saving the config file. I had already discovered that scantv writes its output to stdout; but what's the real config file called? xatwv(1) refers to the file several times, but refuses to divulge its name. That's in xawtvrc(1): it's called ~/.xawtv, a funny name considering the name of its man page.

Finally moved on to lirc, the Linux Infrared Remote Control software. That was a new level of incomplete, inaccurate, incorrect and confusing documentation. According to one page, I should try irrecord to find out what the control said. That resulted in:

=== root@ceeveear (/dev/pts/0) ~ color="blue">219 -> irrecord foo

irrecord -  application for recording IR-codes for usage with lirc

Copyright (C) 1998,1999 Christoph Bartelmus(

irrecord: could not init hardware (lircd running ? --> close it, check permissions)

Well, lircd wasn't running, so I tried strace:

=== root@ceeveear (/dev/pts/0) ~ color="blue">220 -> strace irrecord foo 2>&1 | less
open("/dev/lirc", O_RDWR)               = -1 ENODEV (No such device)

What kind of programming is that? It doesn't even check the return value from open! Of course, I don't have the source, so I can't do the obvious two-line fix, and getting it is liable to cause other problems.

In the middle of this, ran into another problem: the dreaded NFS hangs, this time between ceeveear and wantadilla. I had to reboot ceeveear to get out of it, and then X wouldn't start: the font server kept crashing. strace didn't show anything of interest, so modified the X configuration to load the fonts directly (why are they stored in /usr/share/X11/fonts and not in the usual /usr/X11R6/lib/X11/fonts? Is this a change that came with X11R7?).

After starting X again, discovered that I also no longer had a window manager. That was GNOME anyway, something I don't need, so changed the init level to 3 and started fvwm2 manually, the way I'm used to do it.

Then continued for a while, discovering that my tuner card (MSI “TV@nywhere Master”—where do they get these names from?) probably required a kernel module (I was left to guess which one), and that lircd require a “driver” (again, though there's an entry for my remote control, there's no information about which driver to use). Finally went through with a brute force search and found that a number of them worked, including at least bte and dsp. Of course, they only understood some of the keys; for tomorrow I need to investigate how to decipher the keys.

For the fun of it, tried to run xawtv again. The results were startling: the X server crashed with a SIGSEGV and a backtrace:

0: X(xf86SigHandler+0x87) [0x80b86c7]
1: [0x675420]
2: [0x5]
3: /usr/X11R6/lib/ [0x14e9ed]
4: /usr/X11R6/lib/ [0x121ecb]
5: X(doListFontsWithInfo+0x24c) [0x808b5fc]
6: X(StartListFontsWithInfo+0x145) [0x808bbd5]
7: X(ProcListFontsWithInfo+0x6b) [0x808705b]
8: X(Dispatch+0x19b) [0x80888eb]
9: X(main+0x487) [0x80701d7]
10: /lib/ [0x6a87e4]
11: X(FontFileCompleteXLFD+0xb1) [0x806f511]

I wonder if the NFS crash somehow ended up corrupting some of the font files. I've never seen that on FreeBSD, but I've heard of it on Linux, and recently I've been experiencing lots of things I have never seen before. The overall fragility of this system is horrifying, a far cry from the claims of absolute stability that the die-hard “Open Source” advocates make.

Wednesday, 19 April 2006

Tried more stuff with lirc, but it didn't behave as described in the “manual” (not the man pages; that would be too simple. Instead it's on the web). After several readings of this document, I still don't know what to do.

Finally downloaded the source, which also required the kernel sources. ./configure was a surprise: I got a menu asking me which tuner card to compile for (and of course my TV@nywhere wasn't on the list). Somehow it's possible to generate the programs with all “drivers”, but it's not made clear how. It's also full of references to other documentation, which in turn leads on to yet other documents, none of which seem relevant. After following the instructions, I ended up with the Makefile configured only for a serial connection. After a lot of investigation, I found:

  1. The FAQ states:
    1. Which kernel module do I have to load for my hardware?

    The documentation contains a detailed list how to setup the various devices supported by LIRC.

  2. That list does nothing of the sort. Instead, it specifies pretty much the same information that the configure menu does. If your card isn't on the list, it's useless.
  3. Somehow I then found my way to the installation instructions at, which state:

    See ./configure --help for a detailed description of the possible parameters. You will have to at least choose a driver with the --with-driver=X option.
    That worked: ./configure finished with the message
    All kernel modules will be built.

    Of course, it didn't compile:

    /usr/src/ports/lirc-0.8.0/drivers/lirc_atiusb/lirc_atiusb.c:1183: error: unknown field 'owner' specified in initializer
    /usr/src/ports/lirc-0.8.0/drivers/lirc_atiusb/lirc_atiusb.c:1183: warning: initialization from incompatible pointer type
    make[5]: *** [/usr/src/ports/lirc-0.8.0/drivers/lirc_atiusb/lirc_atiusb.o] Error 1

Round about here something strange occurred to me: the system was responding to the IRC control even when there was no lirc program running! Further investigation showed that there was a kernel module ir_common, and trawling the kernel sources showed that the driver does indeed understand the remote control interface; found some useful information at, which enabled me to—finally!—configure the thing.

The next step was easy: install MySQL. Including setting up the MythTV databases, it took about a minute. The next step, mythtvsetup, looked complicated enough that I decided that it would be better to put it off until tomorrow.

Thursday, 20 April 2006

On with the pain that is MythTV. Started writing a HOWTO so that I'll find my way better next time. That might be sooner than I expect: I got as far as the step Run mythtvsetup. It's not a very clever program:

Setup finally ground to a halt with the message: Could not open '/dev/viso0' to probe its inputs. Stopped it to run strace, and saw, amongst other irrelevant messages:

=== mythtv@ceeveear (/dev/pts/2) ~ color="blue">3 -> mythtvsetup
mythtv: could not open config file /home/mythtv/.mythtv/lircrc
mythtv: No such file or directory
Failed to read lirc config /home/mythtv/.mythtv/lircrc for mythtv
2006-04-20 13:17:48.215 Joystick disabled.
2006-04-20 13:23:51.491 DB Error (simpledbstorage update):
Query was:

No error type from QSqlError?  Strange...
2006-04-20 13:27:35.911 RecordFilePrefix is not set?

I wonder what all that means. It's the first I've seen of a ~/.mythtv/lircrc.

Running strace showed what you might expect:

open("/dev/video0", O_RDWR|O_LARGEFILE) = -1 EACCES (Permission denied)

I feel like screaming! Why can't people report errors any more? But even after I fixed the permissions on all likely devices, mythtvsetup could not access the card. Tried to run xawtv again. It still crashes the local X server, though (modulo network speed) it works fine on wantadilla. Discovered that even on wantadilla, xawtv didn't work for user mythtv; it just displayed noise. Copying ~grog/.xawtv to ~mythtv helped, but only after an explicit channel change. This software is so buggy!

Decided to test my installation instructions and possibly get rid of some crashes by installing Fedora Core 5 again on a new (well, very old, but overwriteable) disk. In the process, various niggles became apparent:

While waiting for that, also downloaded a CD with the of KnoppMyth, which doesn't make it easy. Most of these documents look really confused, and neither the home page nor the FAQ delivered an obvious answer to the obvious question: where can I get it from? The answer is in, starting with the line “R5B7 is available from:”; that's the name of the latest version, spelt differently from the previous time it was mentioned.

KnoppMyth states at several occasions on the way that it's experimental beta software; most versions of Knoppix (from which it is derived) do something similar. In each case, they're wrong; Knoppix is surprisingly stable and, bar the horrible eye candy, not at all a bad installation. KnoppMyth is barely alpha quality. The installation scripts forgot the passwords I entered and chopped the first characters off each IP address. They then put me back in the same position I was earlier in the day; the only advantage was that failure came more quickly.

Very frustrated, gave it up for the day. This is not merely a question of compatibility, as some people say; it's a question of really badly written software. Not checking system call return values is a sure course for disaster.

Friday, 21 April 2006

On trying to debug ceeveear today. There's definitely something wrong with the installation for Fedora Core 5: I went to some trouble to specify that the X11 sources should be installed, but they weren't. Emacs was also missing, but it's possible that the packager is a vi bigot and left Emacs out of the default editor installation.

Finally got it installed, along with all the bits and pieces needed to build xawtv. It didn't crash the X server, but it did the same thing as KnoppMyth—blank screen and lots of ioctl error messages:

in.clipcount=0; Invalid argument
.clipcount=0; Invalid argument
ioctl: VIDIOC_OVERLAY(int=0): Invalid argument

It did this at each attempt to change channel; paradoxically, though there was no image in the main window, it displayed an image in the channel window, suggesting that there's some issue with the phone.

That left the question why the X server crashed before. Installed the new, corrected xorg.conf file and the new driver and—it crashed! So it wasn't something I did with the installation on the “old” version of ceeveear.

In the meantime I had downloaded a CD of FreeBSD 6.1-RC1, so installed that on a spare partition, in the process finding out again how to boot FreeBSD from grub:

title FreeBSD 6.1-RC1
        root (hd0,0,a)
        kernel /boot/loader

The important thing to remember here is that “hd0, 0, a” corresponds to the FreeBSD name ad0s1a; the BIOS partition numbers are off by 1. This isn't what Fedora puts in by default, and it had caused me problems booting eucla, my laptop.

It ran (of course), but didn't detect the tuner, as I expected, so I still need to continue with Linux.

Rebooted the “old” ceeveear and played around with xorg.config. The old one didn't work at all, of course, because it relied on xfs (font server; what a confusing name), which kept crashing. After much trial and error, discovered that the crashes occurred only when Type 1 fonts were used, this entry:

        FontPath   "/usr/share/X11/fonts/Type1/"

Without that, it ran fine and displayed the same ioctl problems as KnoppMyth and the “new” Fedora installation. So it looks like it's a display card problem. Spent some time investigating that, without coming to a conclusion.

Funnily, there seem to be multiple issues with xawtv. It continues to run across the network on echunga and wantadilla, my main machines in the office; but it won't run on teevee, the machine that (currently) drives the projector, also a FreeBSD machine. Another thing that I don't understand.

Saturday, 22 April 2006

Spent more time searching for the ioctl errors that had been annoying me yesterday, and got quite a few results on Google. Many reported the bug and got no solution. Some reported fixes that were really workarounds (use a different tuner card, for example). Michael Krufky reported that it was a kernel bug in Linux 2.6.15-rc2 and before; unfortunately, I'm still getting it in kernel 2.6.16. Finally, Hermann Pitton came up with a plausible explanation: the driver doesn't support overlay with PCI/PCI transfers to the graphics card. That would explain why it only happened when displaying on the same machine. But why can't there be a better error message?

Hermann recommended to use xawtv grabdisplay and referred to a file called README.cx88 without saying where to find it. It wasn't on ceeveear, and it's not in the xawtv distribution; nor is there more than a passing reference to it in the documentation. More googling suggested that it's in the Linux kernel sources, and indeed I found it there on echunga. But there's no mention of grabdisplay in that file, just the comment “For now only capture, overlay support isn't completed yet.”.

Found a reference to a program called tvtime, and installed that. Oh wonder! It just worked. Well, I had to read through the config file that was supplied, and it got a bit confused about the channel numbers, but it worked, modulo sound, which doesn't seem to be an tvtime problem: it doesn't work at all yet. About the only thing wrong with it is that it only displays TV. If only more software were of this quality.

Spent some time investigating the sound issue. According to man -k, there isn't much choice in mixer software:

=== root@ceeveear (/dev/pts/3) ~ color="blue">46 -> man -k mixer
alsamixer            (1)  - soundcard mixer for ALSA soundcard driver, with ncurses interface
alsamixergui        (rpm) - GUI mixer for ALSA sound devices
amixer               (1)  - command-line mixer for ALSA soundcard driver
dump-mixers          (1)  - dump OSS mixer settings to standard output

I suppose there's more, but how do you find it? They could at least have extended man to show what other documentation is available.

Running alsamixer brought a message that I've seen before:

=== grog@ceeveear (/dev/pts/6) ~ color="blue">6 -> alsamixer
alsamixer: function snd_ctl_open failed for default: No such device

What device? Are people deliberately trying to make it more difficult? strace revealed:

access("/home/grog/.asoundrc", R_OK)    = -1 ENOENT (No such file or directory)
open("/dev/snd/controlC0", O_RDONLY)    = -1 EACCES (Permission denied)
open("/dev/aloadC0", O_RDONLY)          = -1 ENOENT (No such file or directory)
fstat64(1, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f69000
write(2, "alsamixer: function snd_ctl_open"..., 68alsamixer: function snd_ctl_open failed for default: No such device
) = 68

After fixing the permissions, it seemed to work, but it wasn't very easy to understand. Tried the HELP menu, which told me:

Return  return to main screen

That doesn't work, so I of the help menu. tried ESC. That got me out quickly:

=== grog@ceeveear (/dev/pts/5) ~ color="blue">2 -> alsamixergui &
[2] 5465
=== grog@ceeveear (/dev/pts/5) ~ color="blue">3 ->
Segmentation fault.

I think I'm going to modify that message from bash to

Segmentation fault (too lame to dump core).

By default, a SIGSEGV creates a core image; if it doesn't, it's because the program has explicitly decided not to. Considering that much of this software is of early alpha quality, this just makes it even more difficult to debug. Decided to read the documentation again before doing anything else, and returned to trying mythtvsetup again.

After 5 minutes of swearing at the completely broken interface (right arrow moves left, left arrow moves right, cursor not visible), gave up and looked for a simpler way, like writing a program to do it or encoding it in hex. I didn't find one, so here's a list of things to remember:

Round about this point, my cousin Sandy and her husband Dean showed up. They live just down the road, but we haven't seen them since December 2004. I suppose we should stay in better contact. At least it gave me a chance to get away from this multimedia frustration.

In parallel with the other work, started downloading the latest (DVD) version of Knoppix from the Internode mirror server; contrary to my normal use of command lines, I this time allowed firefox to do it. That always annoys me, because firefox is just completely stupid about path names; it started downloading to Desktop, which on this machine proved to equate to /var/tmp. There was enough space there, so I let it go. Some hours later I found it had stopped:

=== grog@echunga (/dev/ttypl) color="red">/var/tmp color="blue">57 -> l KNOPPIX_V4.0.2DVD-2005-09-23-EN.iso
-rw-r--r--  1 grog  wheel  2147483647 Apr 22 15:07 KNOPPIX_V4.0.2DVD-2005-09-23-EN.iso

That wasn't the real size, of course; the download had stopped at the 2 GB limit. Grr. And because I was loading from a web server, I couldn't use REGET. Looked around on the web and found alternatives, and started that transfer, somewhat slower (50 kB/s instead of 150 kB/s).

Sunday, 23 April 2006

Came into the office to discover that my Knoppix download was still going:

ftp> reget KNOPPIX_V4.0.2DVD-2005-09-23-EN.iso
local: KNOPPIX_V4.0.2DVD-2005-09-23-EN.iso remote: KNOPPIX_V4.0.2DVD-2005-09-23-EN.iso
229 Entering Extended Passive Mode (|||61749|)
350 Restart position accepted (2147483647).
150 Opening BINARY mode data connection for KNOPPIX_V4.0.2DVD-2005-09-23-EN.iso (3344640000 bytes).
 91% |**********************************************************************       |  2918 MB   13.16 KB/s  5:51:43 ETA

That's nearly 24 hours. Network transfer speeds may have increased, but so have download volumes. Did some investigation and discovered that the Internode mirror server also has an ftp interface, so finished off the transfer from there.

Today was the last day of a ten day period during which I had intended to do a number of things; just about all of it was spent fighting Linux multimedia, and I'm still far from finished. The current biggest problems are:

Tuesday, 25 April 2006


Another holiday today, so spent more time torturing myself with multimedia setup. A package that I haven't tried so far is VLC, so set to trying that out. Found a detailed (and somewhat opinionated) description by Brent Norris, which required a complete kernel upgrade. That didn't go well. Once again was not accessible. It's a real pain that you can't tell yum to ignore it; it causes the entire update to fail, even if you tell it to tolerate errors. I had to move the file away (yes, I know you can also disable it, but it's easier and less error prone to move it). Even then, I got messages like this:

--> Running transaction check
--> Processing Dependency: for package: libgdiplus
--> Processing Dependency: for package: emacs
--> Processing Dependency: for package: gfontview
--> Finished Dependency Resolution
Error: Missing Dependency: is needed by package libgdiplus
Error: Missing Dependency: is needed by package emacs
Error: Missing Dependency: is needed by package gfontview

But Emacs was installed, for example, and so was While experimenting, removed Emacs:

=== root@ceeveear (/dev/pts/2) color="red">/ubuntu/home/grog color="blue">64 -> yum remove emacs
  Removing  : emacs                        ######################### [1/1]
Removed: emacs.i386 0:21.4-14
=== root@ceeveear (/dev/pts/2) color="red">/ubuntu/home/grog color="blue">65 -> yum install emacs
 emacs                   i386       21.4-14          core              1.6 M
Installed: emacs.i386 0:21.4-14

Then, later, it happens again:

--> Finished Dependency Resolution
Error: Missing Dependency: is needed by package libgdiplus
Error: Missing Dependency: is needed by package emacs
Error: Missing Dependency: is needed by package gfontview

There's obviously something very wrong here, but it's not clear what I can do about it. Later I got, after downloading 230 MB of packages:

(162/165): firefox-1.5.0.  100% |=========================|  17 MB    02:42
(163/165): mythvideo-0.19 100% |=========================| 470 kB    00:04
(164/165): libksba-0.9.13 100% |=========================| 2.2 kB    00:00
(165/165): mythweb-0.19-7 100% |=========================| 602 kB    00:05
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
[('installing package firefox- needs 17MB on the / filesystem', (9, '/', 17469440L)),
('installing package needs 17MB on the / filesystem', (9, '/', 17485824L))]

No idea what that is. Checked things, but it wasn't obvious:

=== root@ceeveear (/dev/pts/2) color="red">/ubuntu/home/grog color="blue">71 -> echo $?
=== root@ceeveear (/dev/pts/2) color="red">/ubuntu/home/grog 73 -> df
Filesystem           1M-blocks      Used Available Use% Mounted on
/dev/hda2                 5540      4865       389  93% /
tmpfs                      506         0       506   0% /dev/shm
/dev/hda1                 5629      2487      2856  47% /ubuntu
/dev/hda4               175495      9212    157369   6% /spool

So it wasn't a lack of space. The annoying thing is that yum appears to insist on downloading every package every time; presumably they're already on the system. With 230 MB at a time, it takes a while, and it could become expensive. Ended up installing some of the larger packages individually; I think in future I'll always do it that way.

That still didn't help too much, of course, since was still not accessible. Tried some other things and with a lot of effort more or less got mplayer to display on the screen, something that tvtime had done out of the box.

=== grog@ceeveear (/dev/pts/1) ~ color="blue">3 -> mplayer -tv driver=v4l2:normid=2:width=960:height=720:chanlist=australia:\
channels=2-ABC,7-Seven,9-Nine,10-Ten,28-SBS:outfmt=yuy2:brightness=70:saturation=90:hue=50  -vo x11 tv://5

Unfortunately, getting the companion mencoder to make MPEG files out of that didn't work: mplayer claimed that the result was not a valid MPEG stream. What a crock!

May 2006

Yes, I did nothing on CVRs in May.

Saturday, 3 June 2006

Other events included installing the DVB-T tuner that I borrowed from Ben Kramer on Thursday, along with c't magazine's Debian “VDR” software. I think this stands for “Video Disk Recorder”, a term which doesn't make much sense to me: where's the video disk? I prefer to use the more descriptive term “CVR” (“Computer Video Recorder”). That hung up a surprising number of times; I wonder if the motherboard is getting even flakier. And why does Debian load the floppy driver three times in the course of installing on a system that doesn't have a floppy drive?

Sunday, 4 June 2006

Spent the morning writing up some recipes, then continued trying to install c't magazine's “VDR” software. I think there must be a conspiracy amongst authors of multimedia software to avoid documentation; once again, and unexpectedly for c't, the documentation was almost completely missing. During the installation I had to choose a transmitter profile, and of course they were only available for Germany (I didn't even see Switzerland or Austria). The on-screen instructions (all that I ever found) said that they could be changed afterwards, so chose one at random. At the end of the installation, the terminal under Alt-F8 was supposed to have the application; but it just produced a text-mode message saying (roughly) “this screen reserved for DVR”.

Spent a lot of time investigating and discovered:

On the positive side, I did find out how to override DHCP and use manual setup when installing a Debian system: let it do its thing, then press the back button.

Also tried KnoppMyth again. This time the installation failed consistently with an error in an installation script. I'm not sure what I did differently, but given my previous experiences, I couldn't be bothered trying more than 4 times. Gave up in disgust. This is all far more complicated than it needs to be.

Saturday, 17 June 2006

Put the new motherboard in ceeveear. It's not at all new; in fact, it's so old that it can't correctly identify the Athlon XP+ 2500 processor that I'm using, and it runs at half speed. I've also had to scrape up some SDRAM because it doesn't take DDR. But it runs, and the hangs I had before no longer occur. I still couldn't get the “VDR” software to do anything useful, though, and this brain-dead setup is such a pain that I gave up after installing X and discovering that the configuration routines couldn't recognize my nVidia 6200-based board and wanted all the horrible configuration rigmarole that you used to have to supply years ago.

Tried KnoppMyth, which installed this time (another difference from two weeks ago), but it forgot to install the boot loader. What a pain this stuff is! Installed FreeBSD 6.1-PRERELEASE and started building -STABLE: at the speed this processor is running, it took the rest of the day before hanging on NFS access to echunga. That never used to happen.

Sunday, 18 June 2006

More work on ceeveear, the computer video recorder that I'm trying to build. Got a fractionally newer version of KnoppMyth, which refused to exit the partition menu during a manual install. Gave up on that, grabbed another (“empty”) disk and installed on that using the “automatic” install. That worked, and it even recognized my nVidia 6200-based card and installed a driver on it which set contrast so flat that I could barely recognize anything. Put in an nVidia 4000-based card and it worked. MythTV still has one of the strangest user interfaces I have ever seen, and I still wasn't able to get it to “open” the video card. xawtv found it (though at the time I didn't have an antenna connected), so it looks like my next job is to learn how to set up MythTV.

Saturday, 8 July 2006

Daniel O'Connor along in the afternoon to pick up a tuner, some Weihenstephan W68 yeast and the old iiyama monitor that I retired on Thursday. In turn he brought an AMD Duron 800 processor, still mounted on a motherboard of dubious quality:

Click on the picture to see a medium-size version in the index

After that, finally got the motivation I needed to continue with the work on the video recorder project. The last time I tried this was only barely less frustrating than the previous attempts, though there were signs that it might be more promising. On IRC and the LinuxSA mailing list I had got some suggestions to try tzap, which was typical of the documentation standards for multimedia programs:

=== root@ceeveear (/dev/ttyp2) color="red">/myth/Images 11 -> tzap
       tzap [options] <channel_name>
         zap to channel channel_name (case insensitive)
=== root@ceeveear (/dev/ttyp2) color="red">/myth/Images 12 -> man tzap
No manual entry for tzap
=== root@ceeveear (/dev/ttyp2) color="red">/myth/Images 13 ->

So I went googling for tzap and finally found a wiki page describing it; it seems that I couldn't use it until I had run dvbscan, which of course also had no documentation, and even the wiki page was pretty cursory. It did tell me that I needed a list of local transmitters, though.

Based on the information, extrapolated a file name au-Adelaide, went googling again, and to my surprise discovered indications that I should already have the file on the machine; and I did:

=== root@ceeveear (/dev/ttyp1) ~ color="blue">26 -> locate au-Adelaide | xargs ls -lrt
-rw-r--r--  1 root root  363 Apr 29  2005 /usr/local/share/dvb/dvb-t/au-Adelaide
-rw-r--r--  1 root root 4000 Sep 24  2005 /usr/local/share/dvb/zap/dvb-t/au-Adelaide
-rw-r--r--  1 root root  363 Sep 24  2005 /usr/local/share/dvb/scan/dvb-t/au-Adelaide

With a bit of playing round, got the tuner to work and copy MPEG data to disk. To get that far, I needed:

After this impressive success story, went back to the MythTV wiki and updated the documentation for dvbscan and tzap, which are now hopefully more useful (and indirectly show their origins). Tried to display the MPEG on, my projector machine, but that failed, apparently because one of the myriad parameters didn't match. Left that investigation for another day; but what a relief! Finally, after not quite two years, I'm able to save broadcast TV as MPEG. Even if MythTV doesn't do what I want, I can live with this alone.

Sunday, 9 July 2006

Today messed around trying to get MythTV working. As expected, it wasn't as simple as it should have been. Although they're mentioned on the web site, tzap and friends have nothing to do with MythTV, and the files they use are of no use in setting up MythTV. In addition, the documentation is very weak; it doesn't really tell you what you need to know to configure the tuners. From mythtv-setup, fortunately started from an xterm, I discovered that the cause of the hang was that it started a command on the xterm. I wonder where that would have gone if it had been started from the window manager.

Grab listings for which region?
0: ACT - Canberra
1: NSW - Regional
2: NSW - Sydney
3: NT - Darwin
4: NT - Regional
5: Queensland - Brisbane
6: Queensland - Regional
7: SA - Adelaide
8: SA - Regional
9: Tasmania - Hobart
10: Tasmania - Regional
11: Victoria - Melbourne
12: Victoria - Regional
13: WA - Perth
14: WA - Regional
Select one: [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14 (default=3)] 7
Grab listings for which service?
0: Austar Analogue
1: Austar Digital
2: Foxtel Analogue
3: Foxtel Digital
4: Free to Air
5: Free to Air Digital
6: Free to Air Digital (High Definition)
7: Optus
Select one: [0,1,2,3,4,5,6,7 (default=4)] 6

ERROR: can't find DataSet for Adelaide - freehd
2006-07-09 16:45:54.198 tv_grab_au --config-file '/root/.mythtv/Tuner-1.xmltv' --configure
2006-07-09 16:45:54.198 exited with status 65280

All I managed to do was 4 (“Free to Air”), which gave me a file ~/.mythtv/Tuner-1.xmltv with the following contents:

  # tv_grab_au 0.6.2 config file
  region Adelaide
  service free
  # "channel name" and "frequency" can be changed to suit
  # [+yes/-no] [channel ID] [channel name] [frequency]
  +channel free.Adelaide.10 "Network TEN" 10
  +channel free.Adelaide.2 "ABC SA" 2
  +channel free.Adelaide.7 "Channel Seven" 7
  +channel free.Adelaide.9 "Channel Nine" 9
  +channel free.Adelaide.SBS "SBS" 28

That's clearly analogue. I couldn't find any way to get the digital signals. The same happened with “Free to Air Digital (High Definition)”. Gave up and sent a message to the local mailing lists asking for help.

Trying to configure mplayer to display the data I copied yesterday was no better. mplayer did completely different things depending on where it was started from and where it displayed. In all cases, I got the error message:

VDec: vo config request - 720 x 576 (preferred csp: Mpeg PES)
Could not find matching colorspace - retrying with -vf scale...
Opening video filter: [scale]
The selected video_out device is incompatible with this codec.
VDecoder init failed :(

This, though, is just “ha ha, only joking”; it doesn't stop mplayer from working. What I got depended on where I ran the program and where I directed the output:

What a pain!

Monday, 10 July 2006

As usual when writing up my diary entry for the previous day, I tried reproducing some of the problems I had, in this case including the inability to set the channels. To do so, I started mythtv-setup on wantadilla:0.0, the 2048x1536 display. This takes forever! So did key presses, with the result that I accidentally selected the wrong entry, didn't notice it, and found myself in the channel scan menu. Despite all the error messages from yesterday, it seems that it had found the information it needed, and it was able to perform a (surprisingly slow) channel scan and locate the channels:

Click on the picture to see a medium-size version in the index

Went looking for the ~/.mythtv/Tuner-1.xmltv file, but there was none. After a bit of searching, discovered that Chris Yeoh was right: the channel information is stored in a MySQL database, specifically the table channel in database mythconverg:

=== root@ceeveear (/dev/ttyp6) color="red">/myth/Images color="blue">31 -> mysql -D mythconverg
mysql> select callsign, freqid, serviceid from channel;
| callsign        | freqid | serviceid |
| 7 Digital       | NULL   |      1360 |
| 7 Digital 1     | NULL   |      1361 |
| ABC HDTV        | 7      |       592 |
| ABC TV Adelaide | 7      |       593 |
| SBS DIGITAL 2   | 5      |       834 |
| SBS EPG         | 5      |       835 |
24 rows in set (0.00 sec)

Getting closer to work, made a dump of the table with mysqldump; presumably it's the same for all installations in Adelaide, though notably the frequency information is missing.

In the evening, confirmed that the sound problems yesterday were caused by the audio language settings of mplayer, so finally was able to play the MPEG stream with sound. Even then, though, the problems aren't over. Trying to play the recorded HDTV, the recorded audio failed again:

PROBING UP TO 2000000, PROG: 0
VIDEO MPEG2(pid=2314)...NO AUDIO!  NO SUBS (yet)!  PROGRAM N.  0
Opened TS demuxer, audio: ffffffff(pid -1), video: 10000002(pid 2314)...POS=99452
Audio: no sound

So what's going on there? It really looks as if there is no audio in the feed. I'm not going to worry about it too much until I get MythTV working and it shows up there too.

As if to add to the general instability I've been seeing lately, tvremote, the laptop I use to control teevee, died on me. Investigation showed a loose mains plug adaptor. sigh I think that machine had been up for 6 months.

Sunday, 6 August 2006

Today finally found both the time and the courage to look at the question of recording video from my DVB-T card. As I had mentioned last month, tzap offers me the opportunity to record without further pain installing undocumented software. But it would be nice to have a wrapper program to control tuning, start and stop. So I wrote a little program called record. Most of the code is, not surprisingly, in the time routines—why is UNIX timekeeping such a mess? Ended up writing about 800 lines, very little of which had to do with actually starting and stopping the tzap process. And it worked. Well, almost. Stopping the process on occasion caused the hardware to get confused:

Aug  6 17:38:38 ceeveear kernel: cx88[0]/2: queue is empty - first active
Aug  6 17:38:38 ceeveear kernel: cx88[0]/2: cx8802_start_dma w: 0, h: 0, f: 2
Aug  6 17:38:38 ceeveear kernel: cx88[0]/2: setting the interrupt mask
Aug  6 17:38:38 ceeveear kernel: cx88[0]/2: [f3cbc0c0/0] cx8802_buf_queue - first active
Aug  6 17:39:09 ceeveear kernel: cx88[0]/2: cx8802_restart_queue
Aug  6 17:39:09 ceeveear kernel: cx88[0]/2: cx8802_restart_queue: queue is empty

This wasn't the first time; the log files showed that it had happened before in March. I didn't find a way round that beyond rebooting; hopefully it won't happen if I don't kill the tzap process.

In addition, the recording stopped after 2 GB:

00:35:24 of 01:00:00 (169%)  2047 MB testfoo
File size limit exceeded

Why? There's nothing in this file system that has a 2 GB sensitivity. Tried looking in the man pages:

=== root@ceeveear (/dev/pts/0) color="red">/var/log 46 -> man 2 open
No manual entry for open in section 2
See 'man 7 undocumented' for help when manual pages are not available.
=== root@ceeveear (/dev/pts/0) color="red">/var/log 47 ->

I could scream! Why do Debian consider documentation optional? On eucla, my Fedora machine, I found:

              (LFS)  Allow files whose sizes cannot be represented in an off_t
              (but can be represented in an off64_t) to be opened.

But that's only for LFS, and a test confirmed that the flag wasn't even known.

Still, despite all, the day's work was a success. It's a sad recognition that it's easier to write your own program than install somebody else's. Documentation standards seem to have reached an all-time low.

Monday, 7 August 2006

Chasing up the failure of record to write beyond 2 GB was interesting. First, I found the Debian development man pages and installed them. man 2 open still gave me nothing; the man pages was installed as /usr/share/man/man3/open.3posix.gz, which leaves me wondering if it's even the right man page. It says nothing about O_LARGEFILE (not surprising; it has nothing to do with POSIX.1). Looking for the flags in the required header files sys/stat.h and fcntl.h brought nothing: the header files are part of the incredible mess that is the Linux system header files. The actual definitions are hidden in a maze of #ifdefs that ultimately include /usr/include/asm/stat.h, which defines three incompatible structs, __old_kernel_stat, stat and stat64. struct stat has offsets that are unsigned long (i.e. 32 bits on i386), so that can't work.

Spent some time talking with Linux people, and finally came up with the fact that I needed to define _GNU_SOURCE to get the correct definitions, and then add O_LARGEFILE after all, and replace calls to fstat with calls to fstat64, and define the receiving structure as struct stat64. And then it finally worked. But what a mess! I don't think even the most convoluted System V headers were this bad.

More importantly, though, the man pages say nothing about this mess, neither a definition of the structures, nor a mention of the _GNU_SOURCE requirement, nor the restrictions on file size. By contrast FreeBSD is a model of cleanliness. About the only excuse I can find for Linux (and a feeble one at that) is that this is the price of compatibility, exacerbated in Linux' case by the fact that the kernel and userland come from different places.

Tuesday, 8 August 2006

Mail from James Andrewartha today, pointing out the mistake that I had made yesterday: I had installed the wrong Debian man pages (manpages-posix-dev instead of manpages-dev). Under those circumstances, it's not surprising that they're in section 3 and don't refer to Linux specifics. But it's easy to make this mistake: apt-cache search manpages gives me 65 different packages, and it's easy to miss this:

manpages-de - German manpages
manpages-de-dev - German development manpages
manpages-dev - Manual pages about using GNU/Linux for development
manpages-es - Spanish man pages

The real issue is that man pages shouldn't be something you have to go out and get. If you install development software, the man pages should be there too. That was one of the great new ideas with UNIX over 30 years ago. In those days, the man pages took up a significant amount of the available disk space, but people obviously thought it was worth it; nowadays it's just noise.

The manpages-dev pages appear identical to the man pages on my Fedora system, as you'd hope. As I've noted, they mention O_LARGEFILE, but nothing else. James also found a reference to LFS on Andreas Jaeger's web pages. It seems that LFS stands for Large File Support, not Log-structured File System. Somehow it's all not very satisfying: first, there's no need for this stuff, and secondly if it's needed, it should be documented in the system, not somebody's web site.

Wednesday, 9 August 2006

Spent some time playing around with record, the DVB recording program I wrote on Sunday, making it more user-friendly (for command line users, of course). One of the interesting things is that Linux doesn't have a setproctitle() function, which changes the text displayed by ps(1). In the end, found a way to fake one by overwriting the argv and envp parameters at the top of the stack:

int main (int argc, char *argv [], char *envp [])
  for (i = 0;; i++)
    if (envp [i])
      env = envp [i];
  statusline = argv [0];                        /* display in ps output */
  statuslength = env + strlen (env) - argv [0];
    snprintf (statusline,
              "record: start recording %s to %s at %s",

The results are visible in the ps output:

=== root@ceeveear (/dev/pts/1) color="red">/myth/Images color="blue">172 -> ps aww | grep record
15144 pts/0    S+     0:00 record: start recording SBS HD to koeter-rex-2006-08-10
   at 10 August 2006 19:30:00

Thursday, 10 August 2006

More investigation of the sound problems with HDTV, with some interesting non-results. I need to learn more about MPEG-2, but it's clear that multiple data streams are multiplexed in packets, and that each packet header contains a package ID or (confusingly) pid defining the stream to which it belongs. There seems to be no particular standardization of the pids for audio and video. mplayer thus investigates the beginning of the stream to determine the pids involved in the stream; somehow it also manages to determine their nature, by a means I haven't investigated yet. With the -v option, it shows what it chooses. For Channel 10 TV (normal definition), I get:

Checking for PVA
Checking for MPEG-TS...
TRIED UP TO POSITION 0, FOUND 47, packet_size= 188, SEEMS A TS? 1
GOOD CC: 32, BAD CC: 0
TS file format detected.
Checking for MPEG-TS...
TRIED UP TO POSITION 0, FOUND 47, packet_size= 188, SEEMS A TS? 1
GOOD CC: 32, BAD CC: 0
PROBING UP TO 2000000, PROG: 0
VIDEO MPEG2(pid=512)AUDIO MPA(pid=650) NO SUBS (yet)!  PROGRAM N.  0

By contrast, 10 HD gives:

VIDEO MPEG2(pid=514)NO AUDIO!  NO SUBS (yet)!  PROGRAM N.  0

So it didn't find the audio pid. But the video pid is different too; it's different on other channels as well:

Channel           Video pid      Audio pid
7 1281 1282
9 512 650
ABC 512 650
ABC2 2307 2308
SBS 161 81
SBS 2 162 83
SBS HD 102 103

Clearly it's difficult to guess the pid, especially since it can get quite big. But ~tzap/channels.conf contains lots of information. Could it include this stuff? Looking at it (and removing a lot of stuff in the middle), I found:

ABC HDTV ...  :2314:0:592
ABC2 ...  :2307:2308:594
ABC TV ...  :512:650:595
7 Digital ...  :1281:1282:1360
7 HD Digital ...  :1345:0:1364
NINE Digital ...  :512:650:1105
NINE HD ...  :513:0:1112
TEN Digital ...  :512:650:1617
TEN HD ...  :514:0:1624
SBS HD ...  :102:103:832
SBS DIGITAL 1 ...  :161:81:833
SBS DIGITAL 2 ...  :162:83:834
SBS RADIO 1 ...  :0:201:846

Clearly the third last and second last columns are the video and audio pids. But what's the last column? It looks like a pid too, especially since exactly those channels which I can't get audio for don't have an audio pid, and SBS radio (the last line) has no video (not surprisingly) and the last two columns both set; in fact, all channels, including the ones I don't have set here, have the last column set.

So what is this? Some kind of digital encoding, MP3 maybe? The abbreviation MPA in the successful case may be a clue. The obvious thing to do is to read the tzap documentation. But, as I have complained in the past, that doesn't exist. OK, find the source. That's also more easily said than done, and by the evening I hadn't foudn anything up-to-date. apt-get said:

=== root@ceeveear (/dev/pts/1) ~ color="blue">265 -> apt-get source dvbutils
E: You must put some 'source' URIs in your sources.list

That will have to wait for another day.

Saturday, 12 August 2006

In the afternoon, more investigation of the “no sound” problem on HDTV. Spent quite some time determining that yes, it seems that I did have the latest version of dvb-utils. Documentation is horrendous: even the name of the package is not clear. Debian calls it dvb-utils, but elsewhere it's called dvb-apps or dvbtools, and the package itself is called linuxtv-dvbtools. After a lot of searching, discovered the cause of my problems: the audio modulation for HDTV in Australia is different, but the information I need is not present at all in the channels.conf file: the last field is the service ID, whatever that may mean. Instead, HDTV uses (only) AC3 audio encoding (the other programmes also apparently do so), and dvbscan doesn't look for that when run in the “build channels.conf” mode. Instead, you need to tune to the stream with tzap and then run it in a different mode:

=== root@ceeveear (/dev/pts/0) color="red">/myth/Images color="blue">262 -> tzap -r  'ABC HDTV' -S &
reading channels from file '/root/.tzap/channels.conf'
[1] 23737
=== root@ceeveear (/dev/pts/0) color="red">/myth/Images color="blue">263 -> dvbscan -c
using '/dev/dvb/adapter0/frontend0' and '/dev/dvb/adapter0/demux0'
0x0000 0x0250: pmt_pid 0x0102 ABC -- ABC HDTV (running)
0x0000 0x0251: pmt_pid 0x0100 ABC -- ABC TV Adelaide (running)
0x0000 0x0252: pmt_pid 0x0101 ABC -- ABC2 (running)
0x0000 0x0253: pmt_pid 0x0103 ABC -- ABC TV (running)
0x0000 0x0256: pmt_pid 0x0105 ABC -- ABC DiG Radio (running)
0x0000 0x0257: pmt_pid 0x0106 ABC -- ABC DiG Jazz (running)
dumping lists (6 services)
ABC HDTV                 (0x0250) 01: PCR 0x0905 V 0x090a TT 0x090c AC3 0x090b
ABC TV Adelaide          (0x0251) 01: PCR 0x0080 V 0x0200 A 0x028a (eng) TT 0x0240 AC3 0x0294
ABC2                     (0x0252) 01: PCR 0x0902 V 0x0903 A 0x0904 (eng) TT 0x0906
ABC TV                   (0x0253) 01: PCR 0x0080 V 0x0200 A 0x028a (eng) TT 0x0240
ABC DiG Radio            (0x0256) 02: PCR 0x0901          A 0x090d (eng)
ABC DiG Jazz             (0x0257) 02: PCR 0x0901          A 0x090e (eng)

Conveniently, this output is also in hex; presumably somebody else wrote it. Putting the AC3 PID into channels.conf finally gives audio. But what a pain to find that out!

Sunday, 13 August 2006

More work on the stuff I learnt yesterday, mainly updating the LinuxTV wiki with pages for dvbscan and tzap. In the process discovered that we're not out of the woods yet; sometimes mplayer still doesn't find the audio in HDTV streams.

Playing HDTV also brought me up against another issue: it looks like my machine isn't fast enough, and I get lots of dropouts:

a52: CRC check failed!   -0.354 ct:  0.002   4/  4 ??% ??% ??,?% 0 0
a52: error at resampling
a52: CRC check failed!   -0.108 ct: -0.050  30/ 30 59% 27%  6.5% 3 0
a52: error at resampling
a52: CRC check failed!   -0.147 ct: -0.162  89/ 89 49% 13%  5.1% 12 0
a52: error at resampling

Tried running it on eucla under Fedora Core 5, where it didn't have any problems with dropouts—but exactly every ten seconds it hung for about 5 seconds, and then continued as if nothing had happened. I suspect that's some artefact of GNOME, so rebooted the machine into FreeBSD to try it there—but of course mplayer wasn't installed. Spent some time trying to do that, but didn't finish.

Monday, 14 August 2006

Finally got mplayer installed on the FreeBSD incarnation of eucla and confirmed that it didn't have any difficulties with the HDTV MPEG stream. So the simple answer seems to be: throw more CPU at the problem. eucla has a 2.0 GHz Pentium M; teevee has a (1.466 GHz) Athlon XP 1700+.

Friday, 18 August 2006

(Not directly related, but potentially of interest):

Playing around with the cxm driver for Hauppauge PVR-350: we found that if you pipe it directly into mplayer, it suffered severe buffer overflows, giving really bad images. Modifying the number of buffers in the ring significantly improved that; we'll have to look at what's really needed.

Also looked at the mplayer port, about which I have said less than friendly things in the past. I have only just discovered that the version in the Ports Collection is version, which is so old that I can't even find a date for it; but 1.0pre7 came out well over a year ago, and 1.0pre8 came out recently. Did some attempts to build that, with the usual problems of port dependencies getting in my way. Put it on, a test machine with (almost) no ports on it, and left it go (it's a quad CPU 200MHz Pentium Pro).

Saturday, 19 August 2006

Writing up my diary entry for yesterday gave cause to investigate the versions of mplayer I was running. Looking at the build log file, I found:

install -o root -g wheel -m 555 /src/FreeBSD/ports/multimedia/mplayer/work/MPlayer-1.0pre8/
      TOOLS/ /usr/local/share/mplayer/tools
install -o root -g wheel -m 555 /src/FreeBSD/ports/multimedia/mplayer/work/MPlayer-1.0pre8/
      TOOLS/ /usr/local/share/mplayer/tools
===>   Compressing manual pages for mplayer-0.99.8_2
===>   Registering installation for mplayer-0.99.8_2

So it is the latest version, only the reported version number is wrong!

Following up on my HDTV sound problems of a few days ago, discovered that a programme I had recorded from 9 HDTV still claimed to not have any sound. I'd seen intermittent problems in this area, so kept the program for investigation. As it turned out, it worked fine on wantadilla, but not on teevee, which is running 5.3-RELEASE. It also has problems with a bad sector on the main disk, so it's a clear candidate for an upgrade. I don't have a spare machine, and I don't want to take teevee down for an extended period, so chose ceeveear instead—the only thing that it's doing is recording the occasional programme, and I can always reboot it for that. Chose a new disk and installed 6.1-RELEASE, then upgraded to 6.1-STABLE. In the process it came home to me just how little I have achieved in the last two year in my attempts to create a good system upgrade procedure.

Sunday, 20 August 2006

Planned a quiet day today, but somehow it didn't quite happen that way. Instead, continued with the rebuild of teevee, which proceeded slowly but steadily and took up most of the day. In the end had swapped the hardware of ceeveear and teevee. As a result, I can now watch HDTV with audio and only use about 50% of the CPU. In combination A number of things have probably contributed to this:

Also managed to get rid of the annoying noise at the bottom of the display: turned out to be a mode line issue: I didn't wait long enough at the bottom of the display. With the help of xvidtune got rid of it completely, so I'm now running 1280x720 as intended, and not the 1280x736 that I had been running for over a year. Here's the mode line for my Panasonic PT-AE700E:
     ModeLine  "1280x720"  99   1280 1324 1420 1656    720  725  728  772 +hsync +vsync

Thursday, 24 August 2006

Yvonne into town this afternoon to pick up some pieces for a new, real CVR, based on a “bare bones” system from MSI. Amusingly, this relatively low-end machine is my first own AMD64 machine (I had a dual Opteron machine last year, but it belonged to Rocksoft). The current box looks interesting; it's almost the size to fit into a home entertainment tower, though I fear it might be a little deep. Hopefully it won't be too noisy.

Friday, 25 August 2006

Spent most of the day today installing the new CVR machine. Its internal layout is quite interesting: there's a removable cage at the front for disk and DVD drive (and, puzzlingly, for a floppy disk, made in such a way that you can't use it for a second disk). The remainder looks quite well laid out too:

Click on the picture to see a medium-size version in the index Click on the picture to see a medium-size version in the index

Putting the machine together wasn't without its surprises. First, I ended up with the wrong memory (DDR-2 instead of DDR, so I had to take some memory out of teevee), and secondly I found two fans, one that came with the CPU and one that came with the system:

Click on the picture to see a medium-size version in the index

The (larger) one on the left is just sitting on the PCI slots for comparison: that's the one that came with the CPU, but it doesn't fit properly on the CPU (too loose). I wonder how many people have lost their CPU as a result, and why MSI made such an incompatible board.

Installing FreeBSD on the machine brought a couple of surprises, too: read errors from the installation DVD, and later a panic with corrupted data. Guessed at memory incompatibilities, but the SIMM I put in was a PC 400 SIMM, and that's what the BIOS was set for. Turned the memory speed down to 166 MHz and had no further stability problems.

Setting up X was the next issue: the board has its own display chip, but doesn't recognize it. It managed to set it up with a really unusual resolution, something like 1800x1350, and mplayer wasn't able to render correctly on it. Put in an SiS 315 board, set to 1280x720, and it worked.

But, not for the first time, it seems that 64 bit machines are slow! This is an Athlon 64 2800+, presumably intended to be faster than the Athlon (32) 2500+ in teevee; but tests with a specific HDTV file showed much more CPU usage. Of course, there are a number of things to compare: display card, display drivers, memory timing and 64 bit mode. I'll do them in sequence.

Saturday, 26 August 2006

I had intended to go riding today, but it was cool, and somehow I managed to find excuses not to do so. Instead carried on looking at the new CVR machine, and did numerous tests with mplayer, making interesting discoveries on the way:

In summary, nothing I did significantly improved the situation. The next step was to install a 64 bit kernel; but that failed miserably. The instability I had noticed before became impossibly bad with the 64 bit kernel. Both FreeBSD and Ubuntu didn't make it through the installation process. Spent a lot of time playing with BIOS settings (the “restore default settings” and “set optimized defaults” both seemed to do nothing), and finally removed all boards, used the new disk I had bought for the machine (previous attempts were done with disks about 3 to 5 years old, but still in good condition) and tried again—and it worked! So: is this a problem with the BIOS settings, some of which I might have reset after all, with the boards, or with the disk? Who knows? Still more work to do.

Sunday, 27 August 2006

Brew day today, but also found some time to work on the CVR machine. Spent a lot of time working on the system upgrade scripts, but also installed X—strangely, it created a configuration file very different from the one it created on the same hardware in 32 bit mode. Did some preliminary tests with mplayer suggested that it was a lot faster than in 32 bit mode.

But before I could start some real tests, it crashed. Not once, but repeatedly; it took several attempts before it would stay up long enough to finish the boot and fsck. And the only way I could get it to stay up longer than about 5 minutes was to turn all the timings down to their minimum values, which also slowed the machine down unacceptably. Finally, frustrated, gave up. What a waste of time!

Monday, 28 August 2006

What a waste of a day! Spent most of it double and triple checking what's wrong with the new machine, without any firm result except to confirm that it's not working correctly. That's a particular nuisance because it's holding me up doing other things, notably video performance testing.

Tuesday, 29 August 2006

Into town with my defective computer today and spent quite some time at the warranty/service department, where I was first told that the processor that I had been given was a 32 bit Athlon—this after I had said that it had run for some time in 64 bit mode—and then that it was not the chip that I had ordered. In the end it eventuated that the motherboard BIOS didn't know about the processor—which of course was an Athlon 64, as the photos show—and thus couldn't set the operating parameters correctly. It seems that Socket 754 chips are pretty much unobtainable nowadays, so in the end we agreed to return the machine and I started again with the same cabinet, an MSI Socket AM2 motherboard and a faster processor (Athlon 64 3500+).

Home to put the new machine together. That wasn't without its surprises either. It worked fine in 64 bit mode, but X didn't load the driver for the on-board video chip (part of the VIA chip set), and the sound driver didn't detect anything. Some research showed that the via driver isn't 64 bit clean. Damn.

Putting in an AGP card wasn't an option, either: this board has PCI Express slots, two of them, but I don't have any PCI Express boards. Finally removed a PCI card from wantadilla and used that. Result: I can now finally display the test HDTV image without dropping frames. Now to investigate the other issues.

Wednesday, 30 August 2006

Spent most of the day playing around with the new motherboard. Far too much of it isn't supported. Booted back into 32 bit mode and found that I still couldn't load the via driver: it seems that this chip set is too new. I was also unable to find a FreeBSD driver that would recognize the sound chip, which seems to be intimately related to the VGA chip. pciconf -lv reports:

none1@pci1:0:0: class=0x030000 card=0x72531462 chip=0x32301106 rev=0x11 hdr=0x00
    vendor   = 'VIA Technologies Inc'
    class    = display
    subclass = VGA
none2@pci4:1:0: class=0x040300 card=0x72531462 chip=0x32881106 rev=0x10 hdr=0x00
    vendor   = 'VIA Technologies Inc'
    class    = multimedia

Did get some testing done, which showed that the difference in mplayer performance between 32 bit and 64 bit versions of X was less than the margin of experimental error: so for the moment it doesn't seem worthwhile taking in the disadvantages of 64 bit mode (mainly less driver support).

After that, decided to install Linux, the latest version of Ubuntu (“Dapper Drake”). They've managed to dumb down the installation to a point where it's not possible to set anything except possibly the partition size. How I hate these “modern” GUIs!

Thursday, 31 August 2006

As the result of a suggestion by Stephen Rothwell, went looking on the VIA web site and found sources for VGA drivers for the chip set I'm using, released under what is effectively a BSD license. That's in marked contrast to the claims on the web that VIA hasn't understood “Open Source” and provide no support for them. I wonder why the drivers aren't included in the distribution, and whether they're in the XFree86 distribution.

Tried installing the latest version of KnoppMyth. The installation process is no more refined than the previous version which I tried a couple of months ago . The “automatic” install fails with:

/usr/local/bin/KnoppMyth-auto: line 609: / 10000000: syntax error: operand expected (error token is "/ 1000000").

Looking at the code, it's in partition_autosize() trying to calculate the disk size based on a variable $SIZE that obviously didn't get set. It's set by the output of fdisk, and it seems to assume that the disk in question is /dev/hda; in this case, for some reason probably related to SATA, it's /dev/hdc. But I want to install, not debug installation procedures.

Tried the custom install and on leaving the Partition-Menu (sic), got

Fatal dialog error:
Error: Expected at least 7 tokens for --radiolist, have 4.

I should be used to this by now, but somehow I still find it incredibly frustrating.

Friday, 1 September 2006

On advice from Ben Herrenschmidt, downloaded the Ubuntu “alternate” CD image, but that didn't give me much more control over the installation. In particular, no choice of what software to install, certainly not whether to start GNOME or not.

The manual disk partition menu (text mode) is marvellous: it has to have a frame, which makes it one line shorter than it should be. It scrolls, but there's nothing to tell you that. Instead, you're left with a menu that looks complete, but is missing the “Next” button: it looks as if there's no way to continue.

Somehow didn't have the courage to try to continue beyond the basic installation. First I need to fix my “new system” scripts to work with Linux as well, then I'll be able to do things much more quickly.

Instead, back looking at the drivers for the VIA chip. What they supply is supposedly tailored for Fedora Core 4 (now happily obsolete), but part of it at least (marked “kernel”) is intended to be incorporated in a build of as a replacement for the via driver. Started that, but ended up with no accessible header files and even less desire to continue, so put it off until next week. I should take a break from computers over the weekend.

Thursday, 7 September 2006

Tried Yet Again to install a version of MythTV on the new CVR box, which I'm calling tv2 for the moment. First I started with the version of Ubuntu that I installed last week. I hate Ubuntu! It installs nothing of value, and it doesn't even have a valid apt/sources.list: it appears to have only the CD-ROM, and that doesn't include everything. In particular, I wasn't even able to apt-get Emacs. I also couldn't find a menu item for mounting NFS file systems. To make matters worse, the xterm replacement in the menu misinterprets all Meta-key functions, so I couldn't even type into it. There is an xterm, but I didn't find it in the menus; and like always (it seems) in Linux today, the normal pointers had been replaced by a symbolized hand with truncated fingers. The whole thing looks more and more like a poor copy of Microsoft.

In addition, had more NFS mount hangs. I can't necessarily blame them on Ubuntu, though in this case the lack of correct DNS configuration might have helped. Possibly I should be installing Debian, but there seems to be no way to install the unstable version directly, and nobody I know uses the stable version.

So I went back to installing KnoppMyth again, having first ensured that my disk was in the right place. That worked better, though still not correctly. On reboot, it asks for the root password which I had had to enter, and then rejects it. On further investigation discovered that the password hadn't been set—and that the login in that particular window wouldn't accept an empty password. So ended up logging in on /dev/tty2, setting the password, and returning. The next step is to configure MythTV, which I'll leave until tomorrow.

Friday, 8 September 2006

On with the KnoppMyth configuration. The “automatic installation” asks very few questions: user name and password, “administrator” password, time zone. As I saw yesterday, It doesn't set the administrator password, and today during setup I discovered it doesn't set the user password or the time zone either (though it does create the user name). It also had the same mutilation of IP addresses in the network setup screens that I had noticed (but not reported) earlier (dropped the first digit, so 192 became 92).

Setup is interesting. Here's a detailed description of what I did.

Saturday, 9 September 2006

Lots of work on MythTV, and made some progress; most of the information is in the log. It's not clear whether KnoppMyth thinks the installation is complete or not; rebooting the machine still brings the setup menu. This time I didn't kill it quickly enough, and... it killed my configuration! It completely blew away the mythconverg database, and I had to start again! It's a good thing that I've been writing down what I did.

Sunday, 10 September 2006

One seemingly unrelated problem I had yesterday was poor picture quality when recording from the new DViCO card. Spent some time playing around and found that it didn't even complete initialization in ceeveear:

hiddev96: USB HID v1.10 Device [DVICO DVICO USB HID Remocon V1.00] on usb-0000:00:10.0-2
usbcore: registered new driver yealink
drivers/usb/input/yealink.c: Yealink phone driver:yld-20050816
usbcore: registered new driver usbhid
drivers/usb/input/hid-core.c: v2.6:USB HID core driver
Linux video capture interface: v1.00
cx2388x dvb driver version 0.0.5 loaded
CORE cx88[0]: subsystem: 18ac:db10, board: DViCO FusionHDTV DVB-T Plus [card=21,autodetected]
TV tuner 4 at 0x1fe, Radio tuner -1 at 0x1fe
ACPI: PCI Interrupt 0000:00:06.2[A] -> GSI 17 (level, low) -> IRQ 17
cx88[0]/2: found at 0000:00:06.2, rev: 5, irq: 17, latency: 32, mmio: 0xdf000000
cx88[0]/2: cx2388x based dvb card
DVB: registering new adapter (cx88[0]).
DVB: registering frontend 0 (Zarlink MT352 DVB-T)...
CORE cx88[1]: subsystem: 18ac:db10, board: DViCO FusionHDTV DVB-T Plus [card=21,autodetected]
TV tuner 4 at 0x1fe, Radio tuner -1 at 0x1fe
ACPI: PCI Interrupt 0000:00:0c.2[A] -> GSI 18 (level, low) -> IRQ 18
cx88[1]/2: found at 0000:00:0c.2, rev: 5, irq: 18, latency: 32, mmio: 0xdc000000
cx88[1]/2: cx2388x based dvb card
cx88[1]: frontend initialization failed
cx88[1]/2: cx8802_stop_dma
ACPI: PCI interrupt for device 0000:00:0c.2 disabled
cx88-dvb: probe of 0000:00:0c.2 failed with error -1

Suspecting a newer hardware revision, I compared the PCI parameters. They were identical for both cards, both under Linux and FreeBSD . Linux says:

0000:00:05.0 Multimedia video controller: Conexant Winfast TV2000 XP (rev 05)
        Subsystem: Unknown device 18ac:db10
        Flags: bus master, medium devsel, latency 32, IRQ 17
        Memory at de000000 (32-bit, non-prefetchable) [size=16M]
        Capabilities: [44] Vital Product Data
        Capabilities: [4c] Power Management version 2

0000:00:05.2 Multimedia controller: Conexant: Unknown device 8802 (rev 05)
        Subsystem: Unknown device 18ac:db10
        Flags: bus master, medium devsel, latency 32, IRQ 17
        Memory at df000000 (32-bit, non-prefetchable) [size=16M]
        Capabilities: [4c] Power Management version 2

FreeBSD says:

none4@pci5:4:0: class=0x040000 card=0xdb1018ac chip=0x880014f1 rev=0x05 hdr=0x00
    vendor   = 'Conexant Systems, Inc'
    device   = 'CX2388x TV Capture Card'
    class    = multimedia
    subclass = video
none5@pci5:4:2: class=0x048000 card=0xdb1018ac chip=0x880214f1 rev=0x05 hdr=0x00
    vendor   = 'Conexant Systems, Inc'
    device   = 'CX2388x TV Capture Card'
    class    = multimedia

So do I have a dud card, or is there some undocumented change to the cards that the current drivers can't handle? It would be nice to know what frontend initialization failed means.

While investigating the problem, tried to install FreeBSD on the machine; it failed consistently with disk access problems. Presumably it doesn't understand the new hardware. I've seldom had so many compatibility problems with a motherboard.

On the suggestion of Jan Jones, decided to install MythDora, a Fedora-based MythTV system, on the machine. As usual, finding the documentation is not easy, and managed to download an ancient ISO image (release 2.1; current is 2.32) at a snail's pace (took nearly 3 hours). By that time I had found the correct version and started another download, but it wasn't ready by the evening. The documentation promises a step-by-step pictorial illustration of the installation process, but in fact only gets as far as the end of the standard Fedora installation. I'll have to wait until tomorrow to find out what the installation really looks like.

While waiting for the MythDora download, also tried installing Microsoft “Windows XP Media Center Edition” from the MSDN DVD set. It got about half way and asked for CD 2, whatever that may be. Looks like a bug in the packaging.

Monday, 11 September 2006

Finally burnt an ISO for MythDora and booted it. The results weren't encouraging. It did confirm that the sound hardware is working, the first distribution to do so, but that was about the only good news:

So currently I'm dead in the water. There doesn't seem to be any reason why the X server won't start—on booting it recognized the vesa driver, and this particular image is designed for the VIA chip sets anyway. As I said: not very encouraging.

It's becoming clear that I need to split this problem into hardware and software issues; probably the best is to swap the old and new machine and get the new machine to do the video recordings until I can get MythTV running on the old one, and then address the display hardware issues on the new one.

Tuesday, 12 September 2006

More work on the black box today, and discovered that there's still a surprising amount of work to do on the build infrastructure. Got that pretty well sorted out, but there's still the can of worms of the startup scripts to look at, and there's plenty to keep me busy there.

In the afternoon, swapped the disks and tuners between ceeveear and tv2 and did a test recording. A good thing too; the “old” card, which works perfectly in ceeveear, produced numerous error messages in the VIA motherboard:

Sep 12 16:09:47 ceeveear kernel: cx88[0]/2: cx8802_restart_queue
Sep 12 16:09:47 ceeveear kernel: cx88[0]/2: cx8802_restart_queue: queue is empty
Sep 12 16:09:47 ceeveear kernel: cx88[0]/2: queue is empty - first active
Sep 12 16:09:47 ceeveear kernel: cx88[0]/2: cx8802_start_dma w: 0, h: 0, f: 2
Sep 12 16:09:47 ceeveear kernel: cx88[0]/2: setting the interrupt mask
Sep 12 16:09:47 ceeveear kernel: cx88[0]/2: [d375cb80/4] cx8802_buf_queue - first active

The resultant recording was full of artefacts that roughly paralleled the error messages, so I put the hardware back again and had no further problems.

Somehow I've not had luck with this board (an MSI K9VGM-V with the Via K8M890 chip set):

Looks like time to investigate alternatives. It seems that the nVidia nForce 550 would be a good choice, but I haven't found any in MicroATX form factor.

Thursday, 21 September 2006

Spent most of the day looking at DVD authoring software and confirmed my past observations that the coding standard in this area is terrible. Beyond the function name, I have no idea what this code is supposed to do (no comments, of course), but it must be possible to make it more readable:

int readscr(const unsigned char *buf)
    return ((buf[0]&0x18)<<27)|

Doubtless a little documentation of the underlying data structure would work wonders.

Friday, 22 September 2006

Spent most of the day learning about DVD mastering. Pioneer has some relatively useful documentation on the subject, in particular a Technical guide that is unfortunately somewhat out of date and doesn't mention DVD+R at all (only DVD-R). But somehow none of these documents give me the kind of overview I want. The way I see it, there are two different structures to understand:

  1. The logical structure:
    • The DVD is divided into titles. You don't need to investigate the data format to know that, but it's useful to know that the data structures include a single Video Manager Group (VMG) and several Video Title Sets (VTS). The latter represent the individual titles.

    • The payload data for each of these components is stored in VOB (video object) files, which as far as I can tell are always a specific kind of MPEG-2.

    • Metadata for each of these components is stored in Video Title Set Info (VTSI) files. They contain things like chapter markers (currently of particular interest to me), subtitles, camera angles and things.

    • The VMG is divided into the Video Manager Information (VMGI) and the Video Object Set for the VMG menu (VMGM_VOBS). At least, that's what the Pioneer docco says; it looks pretty clumsy to me. Based on my current understanding, these are basically the DVD menus.

  2. The file system structure. DVDs have both UDF and ISO 9660 file systems (they use the term UDF bridge in a manner which is not defined clearly enough). There's a relatively strict hierarchy, since these files need to be readable by DVD players and other systems with low intelligence. There are up to three top-level directories:

    • audio_ts contains audio-only data.
    • video_ts contains video data.
    • video_rm appears to only be on DVD-R and DVD+R media. I haven't found out what it does yet.

    Here's an example from a DVD and a DVD+RW, taken from a mount of the DVDs as an ISO 9660 file system under FreeBSD. First, the DVD:

      total 1
      dr-xr-xr-x  1 root  wheel  2048 Jan  1  1970 audio_ts
      dr-xr-xr-x  1 root  wheel  2048 Jan  1  1970 video_ts

      total 0

      total 7736
      -r-xr-xr-x  1 root  wheel       14336 Jan  1  1970 video_ts.bup
      -r-xr-xr-x  1 root  wheel       14336 Jan  1  1970 video_ts.ifo
      -r-xr-xr-x  1 root  wheel       36864 Jan  1  1970 video_ts.vob
      -r-xr-xr-x  1 root  wheel       18432 Jan  1  1970 vts_01_0.bup
      -r-xr-xr-x  1 root  wheel       18432 Jan  1  1970 vts_01_0.ifo
      -r-xr-xr-x  1 root  wheel    52074496 Jan  1  1970 vts_01_0.vob
      -r-xr-xr-x  1 root  wheel    24506368 Jan  1  1970 vts_01_1.vob
      -r-xr-xr-x  1 root  wheel       49152 Jan  1  1970 vts_02_0.bup
      -r-xr-xr-x  1 root  wheel       49152 Jan  1  1970 vts_02_0.ifo
      -r-xr-xr-x  1 root  wheel    51853312 Jan  1  1970 vts_02_0.vob
      -r-xr-xr-x  1 root  wheel  1073709056 Jan  1  1970 vts_02_1.vob
      -r-xr-xr-x  1 root  wheel  1073709056 Jan  1  1970 vts_02_2.vob
      -r-xr-xr-x  1 root  wheel  1073709056 Jan  1  1970 vts_02_3.vob
      -r-xr-xr-x  1 root  wheel   821295104 Jan  1  1970 vts_02_4.vob
      -r-xr-xr-x  1 root  wheel       51200 Jan  1  1970 vts_03_0.bup
      -r-xr-xr-x  1 root  wheel       51200 Jan  1  1970 vts_03_0.ifo
      -r-xr-xr-x  1 root  wheel    63410176 Jan  1  1970 vts_03_0.vob
      -r-xr-xr-x  1 root  wheel  1073709056 Jan  1  1970 vts_03_1.vob
      -r-xr-xr-x  1 root  wheel  1073709056 Jan  1  1970 vts_03_2.vob
      -r-xr-xr-x  1 root  wheel  1073709056 Jan  1  1970 vts_03_3.vob
      -r-xr-xr-x  1 root  wheel   655984640 Jan  1  1970 vts_03_4.vob

    There are a number of things to note here:

    1. video_ts.ifo is a single file containing the Video Manager Information (VMGI).
    2. video_ts.vob contains the Video Object Set for the VMG menu (VMGM_VOBS).
    3. The vts_nn* files represent individual titles, which are identified by the nn. In this example there are three titles.
    4. The files with names ending in .vob contain the image data (both video and audio). They're limited in size by the standard to 1 GB, but logically they're concatenated. So the first title has a total of about 73 MB (don't know why it was divided into two files, probably part of the mastering), the second title is about 3.8 GB, and the third title is about 3.6 GB. Clearly the first title is one of these funny DVD menus, and the other two are the payload.
    5. The files with names ending in .ifo contain the Video Title Set Info (VTSI).
    6. The files with names ending in .bup are backups of the .ifo files.
    7. This particular DVD has an empty audio_ts directory. I don't think that this is necessary.
    8. The time stamp fields are obviously empty. This again seems to be specific to this DVD.
  3. DVD+Rs (and presumably also DVD-Rs) also have a directory video_rm:

      Example 2: DVD+RW


       dr-xr-xr-x  1 root  wheel  2048 Sep  4 02:46 video_rm
       dr-xr-xr-x  1 root  wheel  2048 Sep  4 02:46 video_ts

       total 1
       -r-xr-xr-x  1 root  wheel  116736 Sep  4 02:46 video_rm.bup
       -r-xr-xr-x  1 root  wheel    8192 Sep  4 02:46 video_rm.dat
       -r-xr-xr-x  1 root  wheel  116736 Sep  4 02:46 video_rm.ifo


    I haven't found out what it's for yet, though clearly it follows the same file naming convention, and from that naming it's presumably global in scope.

Tuesday, 26 September 2006

Spent the afternoon looking at DVD formats. It's a real can of worms. The following is an attempt to structure it in a way that's easy for me to understand. It may contain errors; please point out any you see.

Saturday, 4 November 2006

In the evening, addressed an issue that has annoyed me about mplayer for some time: the on-screen time display bears no relationship to reality when displaying MPEGs (recorded or directly) from digital TV. It seems that the time information embedded in the stream does not get converted, so the “elapsed time” seems to be randomly distributed in the range 0 to 30 hours. Made a change to at least partially fix that—simply subtract the time of the first frame from the elapsed time, and things work almost correctly. That can definitely be improved on, though, so I won't publish yet. In passing I'm once again amazed by how simple it is to make this kind of fix to mplayer, despite a desperate lack of comments. Given that there's no other technical documentation, that's a particular challenge.

Saturday, 11 November 2006

Had my first attempt at running freevo. The documentation is of the usual quality, or maybe a little better. It starts off giving all sorts of details of how to install it, all done automatically for me by the FreeBSD Ports collection, and then doesn't tell you how to start the program. It does tell you how to configure it, which is fortunate. It's relatively straightforward if you want the defaults (NTSC, 800x600 display). Tried with my setup and entered:
=== root@teevee (/dev/ttypb) ~ color="blue">31 -> freevo setup --geometry=1280x720 --tv=pal --chanlist=australia
System path first=No
checking for mplayer...    /usr/local/bin/mplayer
checking for mencoder...   /usr/local/bin/mencoder
...  many more of these
geometry must be one of: 800x600 768x576 640x480

Wow! That makes it pretty useless. I suppose I need to go back and try my luck with MythTV again. Or maybe I should learn python and fix it. Continued anyway and was able to point it at my Maybe directory (for stuff that I may look at if nothing else comes up), and found:

Freevo image

That scared me—did it maybe first delete the contents? No, they're still there:

=== root@teevee (/dev/ttypb) color="red">/spool/Images 43 -> l Maybe/
total 40652
-rw-r--r--  1 grog  wheel  4700372992 Jul 25 19:45 W26-Ideal-Husband
-rw-r--r--  1 grog  wheel  6675234816 Sep  5 10:34 englar-anheimsins
-rw-r--r--  1 grog  wheel  6068109312 Oct 11 14:40 gycklarnas-afton
-rw-r--r--  1 grog  lemis  6958350336 Sep 23 00:54 l-idole
-rw-r--r--  1 grog  wheel  6978273280 Sep 18 00:55 pasir-bersibik
-rw-r--r--  1 grog  wheel  3641704448 Sep 19 02:00 stupeur-et-tremblements
-rw-r--r--  1 grog  wheel  7583301632 Sep 12 15:05 vertical-de-lete

The problem is, of course, that the thing insists on “extensions”. Once given that, it works, but I still can't display full screen.

Wednesday, 22 November 2006

Spent most of today looking at remote controls under FreeBSD. I have this remote control that came with the DVICO tuner card; it's unclear what the real name is: the unit itself bears the name “FusionREMOTE”, but the Linux dmesg output states:

hiddev96: USB HID v1.10 Device [DVICO DVICO USB HID Remocon V1.00] on usb-0000:00:10.0-2

It's also not clear which letters are written in capitals and which in lower case; I've also seen “DViCO” and possibly others.

I've already had considerable pain trying to get lirc, the Linux Infrared Remote Control software, to work. There's a partial port for it in the FreeBSD Ports Collection, so decided to play around with that. Discovered that my particular remote control uses the HID (Human Interface Device) interface, called /dev/usb/hiddev in Linux. The corresponding FreeBSD device is /dev/uhid. Played around with uhidctl(1) and added a hex dump feature that showed that repeated data sets the high-order bit of the (16 bit) data word:

Microsoft:0x0001.Button:Button_1=1 [0x1]
Microsoft:0x0001.Generic_Desktop:Vno=21502 [0x53fe]

Microsoft:0x0001.Button:Button_1=1 [0x1]
Microsoft:0x0001.Generic_Desktop:Vno=54270 [0xd3fe]

The data is delivered in bits 1 to 7 from the left; the last 8 bits are always 0xfe. Looking at the data in lircd.conf, this corresponds pretty well. Clearly this was the down button:

      begin codes
          ok                       0x5e
          up                       0x51
          down                     0x53
          left                     0x5B
          right                    0x5F

What about the kernel API? It's described in uhid(4) for FreeBSD. Linux doesn't seem to have kernel man pages; instead, the interfaces are described in the directory /usr/src/linux/Documentation, in this case in /usr/src/linux/Documentation/usb/hiddev.txt. The interface is very different and I spent some time looking at the LIRC code in daemon/hw_hiddev.c, where I found obscenities like:

    rd = read(hw.fd, &event, sizeof event);
    if (rd != sizeof event) {
        logprintf(LOG_ERR, "error reading '%s'", hw.device);
        return 0;

Looking at the interface API, this probably does represent an error, but where's the analysis? Is the value returned greater than, equal to or less than 0? Where's errno when you need it?

In addition, this code seems to pass parameters via global variables:

    pre_code = event.hid;
    main_code = event.value;

     * This stuff is probably dvico specific.
     * I don't have any other hid devices to test...
    if (event.hid == 0x10046) {
        repeat_flag = (main_code & dvico_repeat_mask);
        main_code = (main_code & ~dvico_repeat_mask);
        return decode_all(remotes);

Round about here, I got sick of even looking. lirc has proven to be a real pain at the best of times. Do I ignore the problem, fix it or write a different interface? Spent some time thinking about that and came to the conclusion that I had to live with the problem, at least in the short term, but boy, is that painful! This code also showed something that could be a problem: Linux refers a struct hiddev_event with the members hid and value. value is clearly the value displayed by usbhidctl, but I don't see anything corresponding to the hid value of 0x10046. Where does that come from?

Thursday, 23 November 2006

More work on remote controls today, and made reasonable progress. The more I look at lirc, the less I like it. Starting the daemon, got the message

lircd 0.7.2: could not assign address to socket
lircd 0.7.2: No such file or directory

Why? Went through the code and checked: it seems that the directory for the socket didn't exist. How much easier it is with this patch:

        if(bind(sockfd,(struct sockaddr *)&serv_addr,sizeof(serv_addr))==-1)
-               fprintf(stderr,"%s: could not assign address to socket
-                       progname);
-               perror(progname);
+               fprintf(stderr,
+                        "%s: could not assign address to socket %s: %s (%d)
+                       progname,
+                        lircdfile,
+                        strerror (errno),
+                        errno );
                goto start_server_failed1;

Now we get:

lircd 0.7.2: could not assign address to socket /var/lirc/lircd: No such file or directory (2)

Is that really so much work? How many people have scratched their heads about this one?

lircd 0.7.2: accepted new client on /var/lirc/lircd
lircd 0.7.2: initializing '/dev/uhid'
lircd 0.7.2: unable to open '/dev/uhid'
lircd 0.7.2: caught signal
Terminated: 15

Huh? Why can't it open /dev/uhid? Why did it “catch a signal”? With some patches:

        if ((hw.fd = open(hw.device, O_RDONLY)) < 0) {
-               logprintf(LOG_ERR, "unable to open '%s'", hw.device);
+               logprintf(LOG_ERR,
+                          "unable to open '%s': %s (%d)",
+                          hw.device,
+                          strerror (errno),
+                          errno );

it became blindingly obvious:

lircd 0.7.2: unable to open '/dev/uhid': No such file or directory (2)

Why couldn't that be put in as well?

Presumably the “caught a signal” is lircd's own way of stopping (or committing suicide). I didn't bother to fix that one.

In the process of testing this code, discovered another problem, not related to the software: the DVICO remote control suffers from premature repetition. It's almost impossible to press a button without repeating, and presumably it would blow the mind of the designer of my brain-dead Motorola L6 mobile phone: during the time you have to hold down the “off” button to turn on the phone, the remote control repeats 27 times. This isn't a FreeBSD-related problem, nor an isolated case: it happens on both of the controls I have, and on both FreeBSD and Linux. Ended up writing code to ignore the first two repeats, which still makes it very responsive.

Next was to actually use the functionality with mplayer. That required going back to the inaccurate, incorrect, incomplete and contradictory LIRC documentation. It seems I needed a .lircrc file. Format? Who knows? I found multiple examples, but no documentation. One of the examples was the DVICO/LIRC HOWTO, which included a file which I copied as discussed. No reaction. Why? Well, an obvious reason is that the button names mentioned in that file don't correspond to the button names in the /etc/lircd.conf file. The format appears to be straightforward, but who knows what gremlins lurk? Spent some time looking at that. The format of the file appears to be (caution: once again this is an example, not the (missing) documentation):

    button = Right
    prog = mplayer
    config = pt_step 1
    repeat = 0

Here the names on the right of the = sign are clearly strings. Right is presumably intended to the name of the “right” button (called right in my lircd.conf), mplayer the name of the program, pt_step 1 an entry in the ~/.mplayer/input.conf file, and the parameter to repeat might mean that further repeats of this button should be ignored (though in this case that seems a bad choice).

This clearly gives me the option of controlling multiple programs from the same remote control, a very useful function. But it also seems to imply that specific buttons relate to specific programs, which is far less useful. The top of the control has four buttons marked “DTV”, “MP3”, “DVD” and “CPF” (whatever that may mean). Clearly the intention is to divert the functionality of the other buttons to one of four programs. How do you do that? I don't know yet.

Friday, 24 November 2006

In the afternoon, back to the remote control stuff, and found yesterday's assumptions confirmed, including the concerns: mplayer reacts to the remote control even when it's iconified, so if you have multiple mplayers running (something I do frequently), presumably all will react to the remote controls. I need to look at doing something there. Also had permission problems with lirc. By default, the socket is writable by root only:

=== grog@teevee (/dev/ttyp0) ~ color="blue">4 -> ls -l /var/lirc/
total 0
srw-r--r--  1 root  wheel  0 Nov 24 17:10 lircd
prw-r--r--  1 root  wheel  0 Aug 23 09:16 lircm

It's not clear why mplayer needs to open this file with write permissions, but it stops it from working. I also don't know wat /var/lirc/lircm is. Clearly it's been there for a while, but I didn't mention any work on lirc on 23 August.

Sunday, 26 November 2006

More work on mplayer today, still trying to get the on-screen display showing correct values. This time I was concentrating on getting the total time correct; frequently it would show values less than half the correct value. I had suspected some kind of overflow, but after tracing through some particularly emetic code, found that the input stream was either supplying incorrect bit rate data, or was being interpreted incorrectly. After that, found that it also gets reported accordingly on mplayer startup. The values seem to have no relationship to the real information; do they come from the stream, or is mplayer misinterpreting them? Here's what I get for the Adelaide TV channels:

  VIDEO:  MPEG2  1280x720   (aspect 3)  50.000 fps  9600.0 kbps  (1200.0 kbyte/s)
  VIDEO:  MPEG2  720x576    (aspect 3)  25.000 fps  15000.0 kbps (1875.0 kbyte/s)
  VIDEO:  MPEG2  720x576    (aspect 3)  25.000 fps  15000.0 kbps (1875.0 kbyte/s)
7 Digital:
  VIDEO:  MPEG2  720x576    (aspect 3)  25.000 fps  15000.0 kbps (1875.0 kbyte/s)
7 HD Digital:
  VIDEO:  MPEG2  720x576    (aspect 3)  50.000 fps  9500.0 kbps  (1187.5 kbyte/s)
NINE Digital:
  VIDEO:  MPEG2  720x576    (aspect 3)  25.000 fps  15000.0 kbps (1875.0 kbyte/s)
  VIDEO:  MPEG2  1440x1088  (aspect 3)  25.000 fps  15200.0 kbps (1900.0 kbyte/s)
TEN Digital:
  VIDEO:  MPEG2  704x576    (aspect 3)  25.000 fps  12000.0 kbps (1500.0 kbyte/s)
  VIDEO:  MPEG2  1440x1088  (aspect 3)  25.000 fps  12800.0 kbps (1600.0 kbyte/s)
  VIDEO:  MPEG2  720x576    (aspect 3)  50.000 fps  7650.0 kbps  (956.2 kbyte/s)
  VIDEO:  MPEG2  720x576    (aspect 3)  25.000 fps  5000.0 kbps  (625.0 kbyte/s)
  VIDEO:  MPEG2  720x576    (aspect 3)  25.000 fps  4500.0 kbps  (562.5 kbyte/s)

There seems to be no relationship between the bit rate and the resolution, and in addition some “HD” streams are nothing of the sort. In many cases, the HDTV streams have a slower specified bit rate than the normal resolution. Seven HDTV is a special case: it seems to be very weak, and frequently enough I get nothing at all.

What does this mean for mplayer? Clearly it can't always relate on the specified or calculated bit rate. I think the best choice would be to look at the file size and calculate from there. That would also help where the timer wraps around in the middle.

Also did some work on our project; in this case, mplayer is reading from a pipe, and we somehow need to find a way to get it to flush the pipe when it starts so that it's showing current data and not the data that had collected in the pipe. That's straightforward enough: a keyboard input would do it, but in our case it looks like we'll send the command down another pipe:

=== grog@ceeveear (/dev/pts/2) ~ color="blue">121 -> mkfifo /var/tmp/mplayer-command
=== grog@ceeveear (/dev/pts/3) ~ color="blue">36 -> mplayer /dev/dvb/adapter0/dvr0 -really-quiet \
                                        -framedrop -input file=/var/tmp/mplayer-command
=== grog@ceeveear (/dev/pts/2) ~ color="blue">122 -> echo "seek 10" > /var/tmp/mplayer-command

This example was done on Linux, but it works exactly the same way under FreeBSD. Note the two terminals: mplayer writes to the terminal.

Monday, 27 November 2006

More work on mplayer status lines. Calculating the bit rate isn't nearly as easy as I expected it to be.

Tuesday, 28 November 2006

How do you get mplayer to catch up with an input stream? You can create a command pipe with input file=/pipename, but if you send a seek beyond the end of the current input stream, it hangs for a while and then crashes. In our prototype, that then brings down the entire application. Spent some time trying various options, but didn't find a solution.

Friday, 1 December 2006

Spent most of the day looking at the cxm driver for the Hauppauge PVR250. It could do with a lot of improvement; I've already updated the man page for the companion pvr-setchannel program, but there's much more to look at. How do resolution and data rate compare? Checked some DVDs and discovered that a commercial DVD (or at least the one I tested) runs with a resolution of 720x576 (PAL) with 25 fps and 9.0 Mb/s. By contrast, data recorded from TV with my impossible Digitrex GKX-9000 DVD recorder at “standard” resolution (2 hours, 7 minutes on a single-layer 4.5 GB DVD+RW) had a resolution of 352x576,) 25 frames per second and 9.396 Mb/s.

Part of our requirements are also to put 2 hours onto a standard DVD. We're (currently) doing it at full 720x576 resolution, but are dropping the bit rate to 4.52 Mb/s. That works, but is it better than half resolution and full data rate? The way we're doing it also requires patching the driver, and what I'm looking at at the moment is to specify the data rate separately from the “profile” (which is hidden behind screen resolutions and implicit standards).

Saturday, 9 December 2006

Started importing the cxm driver for the Hauppauge PVR-250 into the FreeBSD kernel tree; it's currently in the Ports Collection. No great surprises, but enough work.

Sunday, 10 December 2006

Spent too much time trying to build a kernel with the cxm driver. Building the module works fine, but the sources require a header file iicbb_if.h that is (apparently) only built when the modules are built. I wish I understood the build process better.

Monday, 11 December 2006

In the evening, tried to look at an MPEG-2 stream recorded on ABC “High Definition” TV at 1280x720 non-interlaced (“progressive”).

VIDEO MPEG2(pid=2314)AUDIO A52(pid=2315) NO SUBS (yet)!  PROGRAM N.  0
Opened TS demuxer, audio: 2000(pid 2315), video: 10000002(pid 2314)...POS=195896, PROBE=2000000
VIDEO:  MPEG2  1280x720  (aspect 3)  50.000 fps  9600.0 kbps (1200.0 kbyte/s)

For some reason, it flat-lined the CPU and couldn't display properly. By contrast, the other night I had watched “9 HD”, with the rather adventurous resolution of 1440x1088 and more than 50% higher data rate:

VIDEO MPEG2(pid=513)AUDIO A52(pid=651) NO SUBS (yet)!  PROGRAM N.  0
Opened TS demuxer, audio: 2000(pid 651), video: 10000002(pid 513)...POS=194392, PROBE=2000000
VIDEO:  MPEG2  1440x1088  (aspect 3)  25.000 fps  15200.0 kbps (1900.0 kbyte/s)

On that occasion, it had used up about 75% of the CPU. Trying again today, it flat-lined the CPU, though it displayed better than ABC.

What's going on here? There are a number of things that need clarification:

In summary, I'm left wondering whether there's something else involved, possibly the scheduler or elsewhere in the kernel. More investigation needed.

Tuesday, 26 December 2006

For some time I've been meaning to investigate software for creating DVDs from broadcast MPEG streams, and I've been putting it off because I feared the amount of work it would take to put it right. That's all the funnier because that's almost exactly what we do with our Black Box project, but maybe the pain we've had there explains some of the fear.

Today was the day; went through the Ports Collection multimedia section once again and came out with a bewildering number of programs, all described in enormous fixed-width fonts. How I hate this broken web site! In the end, asked on IRC, where Jürgen Lock was very helpful in describing how he does it. First, though, I tried replex.


We're using replex at work, and we're doing our best to replace it there. As usual, there is almost no documentation, just a README that says:
Usage: ./replex [options] <input files>

  --type,             -t:  set output type (MPEG2, DVD, HDTV)
  --of,               -o:  set output file

A typical call would be
replex -t DVD -o mynewps.mpg myoldts.ts

Trying that gave the following output on the screen:

=== grog@teevee (/dev/ttyp4) color="red">/spool/Images/Already color="blue">8 -> replex -t DVD -o 3suns.mpg tre-solar
Reading from tre-solar
Input file length: 6655.00 MB
Output File is: 3suns.mpg
Checking for TS: confirmed
Trying to find PIDs
vpid 0x0066
apid 0x0067
Audiostream: layer: 2  BRate: 224 kb/s  Freq: 48.0 kHz frame size: 672 ( 0:00:00.024 ) starting
audio PTS: 13:22:14.750
Wrong audio frame size: 976
Wrong audio frame size: 1040
Video: aspect ratio: 16:9  size = 720x576  frame rate: 50.000 fps  bit rate: 7.65 Mbit/s
  vbvbuffer 3817472
Sequence Extension: chroma 4:2:0   size = 720x576  bit rate: 7.65 Mbit/s  vbvbuffer 3817472
frame rate: 50.000
starting with video PTS: 13:22:15.172
video PTS inconsistent: 13:22:15.212  0:00:00.040 13:22:15.172  0:00:00.000  diff:  0:01:03.945
video DTS inconsistent: 13:22:15.152 26:30:43.697 13:22:15.152 26:30:43.697 diff:  0:01:03.905
video PTS inconsistent: 13:22:15.212  0:00:00.000 13:22:15.192  0:00:00.020  diff:  0:01:03.885
(many more)

They were matched by the following output on the console, which indicates that it's trying to write to a DVD. There's nothing in the “documentation” that explains why it's trying to do this.

Dec 26 13:07:56 teevee kernel: (cd0:ata1:0:0:0): READ(10).  CDB: 28 0 0 0 1 3 0 0 1 0
Dec 26 13:07:56 teevee kernel: (cd0:ata1:0:0:0): CAM Status: SCSI Status Error
Dec 26 13:07:56 teevee kernel: (cd0:ata1:0:0:0): SCSI Status: Check Condition
Dec 26 13:07:56 teevee kernel: (cd0:ata1:0:0:0): NOT READY asc:3a,0
Dec 26 13:07:56 teevee kernel: (cd0:ata1:0:0:0): Medium not present
Dec 26 13:07:56 teevee kernel: (cd0:ata1:0:0:0): Unretryable error
Dec 26 13:07:56 teevee kernel: (cd0:ata1:0:0:0): cddone: got error 0x6 back

Finally, replex died with the message:

ringbuffer overflow 124<184 629145
ring buffer overflow 629145

Possibly I'm doing something wrong, but without any documentation, like so much multimedia stuff, it's pretty much useless.

Burning DVDs: a workable solution

Jürgen's recommendation is Project X to disassemble the file into its component streams, then dvdstyler to put them back together again. That doesn't seem to make sense: the VOB files are MPEG-2 program streams, and the input files are MPEG-2 transport streams, which are related, so it seems unnecessary to take them apart and put them back together again; replex doesn't need that. On the other hand, replex also doesn't seem to work. Also, dvdstyler is really a program for creating menus for a DVD, something that I don't want. But can you create a DVD without a menu? I don't know. So I'm guessing that Jürgen's method is one “that works” rather than an optimal procedure.

Project X

Project X is based on Java, which doesn't make things any easier. Installing the port failed with:
===>  diablo-jdk- :
 Because of licensing restrictions, you must fetch the distribution
 manually.  Please access



 with a web browser and "Accept" the End User License Agreement for
 "Caffe Diablo 1.5.0".  Please place the downloaded
 diablo-caffe-freebsd6-i386-1.5.0_07-b01.tar.bz2 in /usr/ports/distfiles.
.*** Error code 1

After doing that, it installed, but there was no documentation. The initial screen looks pretty, but is completely confusing:


I was about to give up on it until Jürgen came along and helped me through it, step by step. In the past I've always refused such offers of help: they shouldn't be necessary. But, sad as it seems, they are necessary, because people write programs without any documentation. Basically, the procedure is:


After tearing the stream apart with Project X, use dvdstyler to create the DVD itself. dvdstyler is unusual for a couple of reasons: it has nicely formatted documentation, and the error messages it produces are intelligible:


To be fair to the program, I didn't read the documentation, since Jürgen was navigating. The first thing you'd expect to do with the initial screen would be to select something from the Files tab:


In fact, that's just for loading project files (and has this same stupid, brain-dead ignorance of the working directory that all “modern” programs seem to have nowadays). To get at your files, use the Directories tab on the side. Then drag the files shown on the right in specific sequence:

  1. First, drag the video (.m2v) file to the bottom. This appears here as “Title 1”.
  2. Next, drag the audio (.mp2) file on top of the title. The text at the bottom is the only indication that anything has happened, and it disappears when you move the cursor off the icon.
  3. Don't do anything with the _log.txt file. It's really just the log of the conversion, and it's a little strange that it's there in the first place.

At the end, it should look something like this:


Next, create a menu:

Wednesday, 27 December 2006

Spent a lot of time today writing up yesterday's experiences with DVD mastering, and continued investigating the reasons why some of the resultant DVDs wouldn't work in my brain-dead Digitrex DVD recorder, though mplayer had no problems with it. With the help of mplayer, established that there were format differences:

VIDEO:  MPEG2  720x576  (aspect 3)  50.000 fps  9000.0 kbps (1125.0 kbyte/s)
VIDEO:  MPEG2  704x576  (aspect 2)  25.000 fps  9000.0 kbps (1125.0 kbyte/s)

The first was Astérix DVD that I burnt was “progressive scan” (i.e. non-interlaced) and “aspect 3” (i.e. 16x9 aspect ratio), and the second, the one that did work, was interlaced with “aspect 2” (i.e. 4x3). It also had a different resolution: on closer examination, none of the resolutions are 4x3. 720x576 is 5x4, while 704x576 is 11x9. To make matters worse, Project X offered different ratios again: 0.6735 (2000x1437), which it calls “4:3”, and 0.7031 (really 10000x7031), which it calls “16:9”. Clearly there's something to be learnt here. It also uses the term DAR, which apparently means Display Aspect Ratio. What other kind of aspect ratio is there? I suppose I might find out if I can make sense of the ratios I've seen. Tried a small test after setting the horizontal resolution and aspect ratio to match the image that did work. This is the top right of the Project X screen after the changes:

  href="/grog/Photos-20061227.html#project-x-options">Imagetitle="Click on the picture to see a medium-size version in the index"
src="/grog/Photos/20061227/tiny/project-x-options.gif" width="177" height="265" />

Indeed, the result worked. Which change was it? My guess is the aspect ratio. Now to learn how to chop off leaders and trailers. The image contained nearly 400 MB of junk at the beginning because the TV stations never start broadcasting on time. I thought I had cut it off, but it ended up on the DVD anyway.

Thursday, 28 December 2006

Did some more work on DVD mastering, and confirmed yesterday's suspicion that it was the aspect ratio that caused the Digitrex DVD recorder to refuse previous DVDs. But this is still so slow! Read up on the quite considerable documentation in the Transcode wiki, discovering in the process a script called any2DVD, which may have the distinction of being the largest shell script I have ever seen:

  -rwxr-xr-x  1 root  wheel  441726 Sep 13 20:44 any2dvd

Set to porting that—it doesn't have any installation environment, so I had to do it manually. Wasn't finished by the evening.

Mail from James Andrewartha telling me that DVD pixels are not square, thus explaining the strange aspect ratios I had noticed yesterday. What a can of worms!

Friday, 29 December 2006

Still more DVD authoring stuff. Fixed the Makefile for any2DVD, maybe a little early: despite being a shell script, it had a number of surprises: it claims to run with /bin/sh (it doesn't, but in Linux /bin/sh is a link to /bin/bash, with which it does run); it uses a flavour of sed incompatible with BSD sed; and it makes this horrible assumption that the terminal background is black, then produces illegible messages:
Image title: any2dvd
Dimensions: 732 x 267, 8 kB
Dimensions of original: 732 x 267, 8 kB
Display this image:
thumbnail    hidden   alone on page
Display all images on this page as:
thumbnails    this size
Show for Friday, 29 December 2006:
thumbnails    small images    diary entry

In the evening, our Digitrex DVD recorder gave up the ghost in a slightly different way: the front panel on/off switch got stuck in the housing and wouldn't come out:


It's not clear what caused it to work in the first place; possibly a coil spring that has since got lost. In any case, I can't complain. It's by far the worst piece of electronics I've bought in the last few years, but since the manufacturer refunded the purchase price and didn't want it back, I suppose I've got my money's worth. Took it apart and investigated, taking some photos in the process. There might be something salvageable in there (at least the DVD+RW drive). Put it back together without the button; now we'll have to use a cotton stick to locate the real switch on the PCB board behind the front panel on those occasions where we still use it.

Saturday, 30 December 2006

Spent a lot of the day looking at any2DVD, gradually coming to the conclusion that I had committed it too hastily. Shell scripts are a pain to debug at the best of times, and when they're this size, it's hell. I seem to have left behind a trail of half-completed ports, so this might be one that I should just forget.

Sunday, 31 December 2006

For some time I've been worrying about displaying HDTV on, an AMD Athlon XP 2500+ (1833 MHz) running FreeBSD 6.1. Basically, earlier this month I had established that, try as I might, I couldn't get the CPU grunt to render 720p effectively with any driver available to me. Earlier, in August I had noted:

The nVidia driver does not improve performance; in fact, it decreases it.
But what about hardware MPEG rendition? Did some investigation and came up with a reference on Installing the nVidia drivers isn't enough: you also need to use XvMC, X video motion compensation, which you invoke with:
mplayer -vo xvmc -vc ffmpeg12mc image-file-name

The -vo specifies the video driver, and -vc specifies the video codec.

Installed the latest nVidia driver (/usr/ports/x11/nvidia-driver) and the configuration utility (/usr/ports/x11/nvidia-xconfig) and ran the latter, which did nothing useful apart from introduce gratuitous changes to the xorg.conf file. The real diffs are:

RCS file: RCS/xorg.conf,v
retrieving revision 1.11
diff -wu -r1.11 xorg.conf
--- xorg.conf   2006/12/30 07:07:01     1.11
+++ xorg.conf   2006/12/30 07:12:24
@@ -19,7 +20,6 @@


 Section "Module"
        Load  "dbe"
-       Load  "dri"
        Load  "extmod"
        Load  "glx"
        Load  "record"
@@ -119,7 +119,7 @@
         #Option     "FPScale"                  # [<bool>]
         #Option     "FPTweak"                  # <i>
        Identifier  "Control"
-       Driver      "nv"
+       Driver      "nvidia"
        VendorName  "nVidia Corporation"
        BoardName   "GeForce 6200"
        BusID       "PCI:1:0:0"

I don't know why the config program removed dri; possibly it doesn't support it. I suppose I should try replacing it and seeing what happens.

With those changes, mplayer worked much better. Instead of dropping frames like crazy at 100% CPU usage, it worked fine with 75% idle. top shows that the 25% was all used by mplayer; X didn't even register. So it looks as if I can stick with X.

Greg's home page Greg's diary Greg's photos Copyright

Valid XHTML 1.0!

$Id: log-2006.php,v 1.24 2017/12/21 03:40:41 grog Exp $