| Greg's computer video recorder project |
|
||||||||
|
|
|
|
|||||||||||||
This page describes the fun I have had setting up a computer as a video recorder, starting in January 2007. It's an extract from my online diary. There are also older logs: information going back to April 2006 and a even older information which hopefully won't be needed except to document the pain. See the main page for an overview.
Inevitably, I make mistakes. As I gain experience, I realize these mistakes; rather than just correcting the text, I prefer to leave the original statement and add a clarifying comment: if I have made a mistake, others might do the same.
Yes, this page is done in chronological order. I hate reverse chronological order. The latest entry is here.
A bit more work on multimedia today, mainly trying to get firefox to display the videos at SBS Food Safari. Attempts to display on wantadilla come up with the message Unknown Plugin (application/x-mplayer2), and the Manual Install takes me to Microsoft's Download Center [link now defunct]. Why do they think that they can help? Interestingly, I have exactly the same problems with MacOS X Safari. I'm intrigued by the name mplayer2. It's probably an abbreviation for some Microsoft product name.
Finally “solved the problem” (a multimedia expression for “found the stepping stones to work around the crocodiles”) and re-installed firefox on teevee.lemis.com for the umpteenth time, this time version 2.0 with these horrible, not-turn-offable tabs and keymap that steals several keys, but with the ports www/linux-flashplugin7 and www/linux-mplayer-plugin, and it worked as well as SBS intended, which includes being forced to watch commercials with violent content and being unable to skip over them. Grr.
Spent an inordinate amount of time trying to find out how to use the S/VHS video input on my DVICO tuner card, without success.
Multimedia's straightforward, right? It's all a matter of moving data from one place to another; about the only complication is that it might need to be reformatted in the process. So why is it so difficult to find out how to move data around? Got a couple of replies to yesterday's message on S/VHS inputs to DVB cards. Neither was a real answer, just “you could try that”. Played around with the suggestions, in the process becoming more than ever aware of the deep divide made by Linux between analogue and digital tuners. No success.
Yesterday I had found a pretty picture of the dependencies between the plethora of devices presented by the V4L interface, but today I couldn't even find it again. Somehow the structure of the Wiki is deficient, probably a general problem with wikis.
Gave up on that and played around with MythTV, which I had installed as KnoppMyth on ceeveear a few months ago and run into problems. Today discovered that one of those problems was that the MySQL my.cnf file included the line
skip-networking
That's particularly clever when the server is addressed via an IP address. Fixed that (commented out the line) and then ran into further trouble. After starting mythfrontend and selecting “Watch TV”, the screen went blank, nothing happened for 20 seconds, and then the same menu returned. The only information of any utility was in the log file:
2007-01-07 10:37:17.194 New DB connection, total: 2 2007-01-07 10:37:17.196 Connected to database 'mythconverg' at host: localhost 2007-01-07 10:37:17.289 Connecting to backend server: 192.109.197.140:6543 (try 1 of 5) 2007-01-07 10:37:37.295 ReadStringList timeout (quick). 2007-01-07 10:37:37.295 Unexpected response to MYTH_PROTO_VERSION: 2007-01-07 10:37:37.297 TV: Attempting to change from None to None
What does that mean? My guess that the “unexpected response” was none at all (thus the 20 second delay). But why no response? The backend seems to be running quite happily.
Foolishly, decided to try reinstallation on a new disk, and ended up back in the maze of twisty little tuner setup scripts, all different. The thing that really takes the cake is when mythtv-setup starts a script underneath its own window, and there's no obvious way of moving it away. And all that for a script that I don't need anyway (it's for setting up analogue tuners). Decided to give up and try again from scratch with the latest version of Ubuntu. Downloading the DVD for that takes about 6½ hours at full ADSL speed:
=== grog@echunga (/dev/ttyp6) /src/ISOs 19 -> ftp http://mirror.internode.on.net/pub/ubuntu/releases-DVD/
edgy/release/ubuntu-6.10-dvd-i386.iso
Requesting http://mirror.internode.on.net/pub/ubuntu/releases-DVD/edgy/release/ubuntu-6.10-dvd-i386.iso
100% |******************************************************| 3552 MB 156.85 KB/s 00:00 ETA
3725318144 bytes retrieved in 6:26:32 (156.85 KB/s)
So that's a job for another day.
Had planned to install the latest version of Ubuntu this morning, but two things stopped me: first, I need a way to know what packages to install on the new machine, and secondly I needed the machine for recording a TV programme. The first proved as difficult in Ubuntu as in FreeBSD: the information about installed packages does not include information about whether the port was installed explicitly, or whether it was installed implicitly as a dependency. Given that there are nearly 1000 packages installed, that's an issue, though of course it would be possible to just install them all sequentially.
Spent some time looking for ways of skipping commercials automatically when recording TV programmes; I could have sworn that I had seen some reference that transcode could do it, but couldn't find any further reference. Instead, spent some time making my mplayer wrapper scripts cleverer. I can now save JPEG images (“snapshots”), but only when running without XvMC, which also only works with MPEG-2 streams; the script now recognizes that, too, and only sets XvMC for MPEG-2s.
Our project has slowed down again, so I had time to do other work with my “black box” machine—in fact, the work for which I had bought it, to turn it into a low-cost media PC. Finally found time to install the Ubuntu DVD that I downloaded on Saturday.
The results were impressive: it failed completely. The installer couldn't start X properly. I thought that this might be because of the strange on-board display chip on this motherboard, but a comparable installation of FreeBSD worked without a hitch, and Ubuntu failed even when I inserted an nVidia 4000 series board.. I thought I had also installed the previous version of Ubuntu on this box. Not a good result for Ubuntu. The character mode installation didn't work much better: most of the characters were corrupted. Left that for later and wondered what I should do next.
Started looking at multimedia stuff again. Took another look at the MythTV User Manual, which looks a lot better than I recall it being. Recalled that I still have a half-finished port in the FreeBSD Ports Collection, so set to working on that. In the meantime, tried to follow up on the Australian TV Guide information at tuhs.org, but ran into trouble accessing some pages. After a lot of searching, discovered that the culprit was the Internode web proxy server: other people round the world could access the site, and I could access it by ssh (I used to have an account). This is the first time I've run into problems of this nature, but it's certainly a show-stopper.
Did some more work on the MythTV port, and got it to the point where it would install the database correctly if it wasn't already there. Running mythtv-setup was another matter, though: it came up in incorrect colours, as if it were running in PseudoColor; certainly the colours changed with focus, but they were always wrong. That could be due to the fact that I had selected a remote display, though it works fine from Linux to the same displays, so decided to try it by installing on eucla first. That took several hours, after which the build failed with a dependency issue. Mañana.
Somehow didn't get much done today. Got MythTV installed on eucla, and this time the colours were correct, but without a tuner card there doesn't seem to be much that you can do with it.
Over to Hahndorf for a project meeting this morning; the Peters have other work to do for the moment, so I brought the prototype home for testing.
In the afternoon, got the prototype up and running. The current testing relates to the IFO files, so set out looking for some tools to analyse the content. There's one for Microsoft, called ifoedit, but it took a while to find PgcEdit, which is written in tcl. No installation instructions, but there is a Linux executable which seemed to work—except that the “open” window (which conveniently starts in the root file system rather than the current directory) didn't work. Took quite some time to discover that it insists on having the file names in capital letters, so a name like vts_01_0.ifo wasn't recognized, and there was also no error message. Changing the names to VTS_01_0.IFO worked. Now how do I fix that in the tcl code?
Spent most of the day working on the DVD generation, and made reasonable progress: at least I was able to generate a DVD, though the format still leaves something to be desired.
Into Echunga in the afternoon to have my hair cut. Last week's bushfire is still the talk of the town.
More work on the DVD data today. I am now writing .ifo files, but how do I check if they're correct? Spent more time looking at PgcEdit, and discovered it needs a mounted /proc file system to work—otherwise it does the typical multimedia thing and just stops, returning a 0 status. But it's far too complicated for what I want, and it outputs to the screen, so I can't put the output through scripts. Instead, took a look at a small proprietary program that Peter Denton had given me. It was written for Microsoft, but in C++, so it's portable, right? Wrong.
I suppose I shouldn't be amazed at the quality of this software; it's closed source, and thus not subject to the same scrutiny that open source software is. In addition, it's multimedia software, and written for Microsoft. Still, this is what I had to change:
This program doesn't have a main() function! Instead, it has a function called WinMain(), presumably to display the results on the screen—unnecessary in this case, since the only output is a file.
Presumably because of this interface, it doesn't pass the file name to the program (though it looks as if one of the myriad parameters of WinMain is designed to do that). Instead, it picks a fixed name.
It contains the following source line:
#include "stdafx.h"
This file wasn't found in my source tree. After some searching, found a file called StdAfx.h. It's stupid to spell names differently (even Microsoft can distinguish, right, but just falls back if it doesn't find the exact match?). So if somebody were to add a file stdafx.h in the same directory as StdAfx.h, it would be chosen instead. What a mess!
Most of the variables are of type BYTE, WORD and DWORD. This seems to be a Microsoft thing; I couldn't find any definition in the source tree. I had assumed that they correspond to char, int32_t and int64_t respectively, but Peter tells me that the latter ones are more likely to be int16_t and int32_t
If anything goes wrong, they don't say anything; they just return an error status, which isn't even unique within the program:
nbytes = fread (&vts_vobu_adm[i],1,VOBU_ADMAP_SIZE,fin);
if (nbytes != VOBU_ADMAP_SIZE) {
fclose (fin);
return DVD_IFO_READ;
}
Under these circumstances I find it particularly amusing that they first close the file; I wonder if Microsoft doesn't do it for you when you exit a program. But then, they open many files and don't close them again unless something goes wrong, so I'd assume it's just another indication of the quality of the program.
At the end, I had the program compiling with no warnings, but it still didn't run. Presumably I didn't convert the structure definitions exactly.
Spent more time looking at TV programme software. Internode has now bypassed their “transparent” proxy, and I can now get the xmltv listings from the OzTivo TV guide. But what software should I use to look at it? The obvious thing would be a web browser. Strangely, I didn't find anything useful. OzTivo has web pages giving an overview, but they're not easy to use. I'd prefer something more like the SBS programme, which seems to be the only useful guide left in Australia, now that ABC have deliberately mutilated theirs.
Went out looking for Finally came up with XSLTv, which purports to do that, but after following the instructions, nothing happened. It's not clear whether it was a missing dependency or incorrect instructions, but my desire to continue was somewhat curbed by the warning that it's very slow, and of course at there's no guarantee that it will do what I want.
Surely there must be something out there to do the most obvious presentation of the programme data.
Did some large recordings today, including a programme on Channel 9 (what's their real name? ninemsn, written in lower case? Nomen est omen, I suppose), which in high definition came to a total of 22 GB for a single programme. But it wasn't broadcast in high definition; when I finally got to look at it, all I saw were aerial landscapes of Australian cities. Why do they do that? Did they only broadcast it in normal definition? Or did they drop it altogether? Tried calling up to ask them, and was flabbergasted:
Spent more time looking at programs to convert programme guide data today, and finally came to the conclusion that none of them did what I want. Instead, set to writing an i>Emacs macro to do the conversion instead. It's pretty ugly, but it works. The disappointing thing was that, once I had done the conversion, I discovered that the original data is not as detailed at least as the SBS programme.
More work on the DVD structure analysis program today; made progress, but I'm always left wondering if I shouldn't be attending to the task at hand rather than the infrastructure.
Our machine is back from Brisbane again, so over to Hahndorf to take a look at it. We still seem to have transport problems. I'm amazed how badly people treat delicate electronic equipment.
Most of the day testing the IFO generation code. Well, it's more like “reading and understanding”. At least it's now becoming clearer what the structures are. I wonder if better documentation would have helped; for once, we seem to have enough.
Continued work on the DVD stuff today. I can now generate a VTS_01_0.IFO file that looks almost correct, but there are some issues in the location of the first VTS_01_1.VOB file. I wish the documentation were clearer.
Continued work on the DVDs today, and I think I now have the VTSI files (VTS_01_0.IFO and friends) correct. It's amazing how difficult it is to get the correct information, not in the slightest helped by the fact that just about all the software I see doesn't understand the concept of structures. For one example (there are many), here's code from dvdifo.c, part of dvdauthor, for creating a vtsi_mat structure, the beginning of the VTSI file:
// sect 0: VTS toplevel
memset(buf,0,2048);
memcpy(buf,"DVDVIDEO-VTS",12);
buf[33]=0x11;
write4(buf+128,0x7ff);
i=1;
buf[0xCB]=i; // VTS_PTT_SRPT
i+=Create_PTT_SRPT(0,ws->titles);
buf[0xCF]=i; // VTS_PGCI
i+=CreatePGC(0,ws,0);
if( jumppad || forcemenus ) {
buf[0xD3]=i; // VTSM_PGCI
i+=CreatePGC(0,ws,1);
}
What does that 0x11 mean? It's the contents of the specification version field, and it means that the file complies to version 1.1 of the DVD specification. The following line (write4) sets the value of vtsi_last_byte, the end address of the VTSI, which in fact must always be 2047. But what ugly code! And it's no exception: all multimedia code seems to look like this.
Quiet day today, spent reading and playing around with Internet radio. Why is it so difficult to get clear URLs for radio feeds? Discovered Realplayer's typically broken list of radio stations. Selecting them is a different matter; somehow I ended up with an mplayer plugin for the browser, which didn't work. It's not in the list of “download actions”, so I can't remove it. Moved it away instead, which at least gave me the option of browsing my home directory for likely programs to play the stream. Selected realplayer, but even then it couldn't handle all the URLs. There seem to be at least three different ways to pass the URL to realplayer:
<asx version="3.0">
<entry>
<title>Klassik</title>
<ref href="http://www.tv1.de/tv1/cms/_vm100/53864/asx" />
</entry>
</asx>
More work on DVDs today, and finally got something that would play in my Digitrex recorder—sort of. Still a number of things need to be done. On Peter Denton's recommendation, finally gave up on Open Source analysis programs and downloaded IFOEdit, which runs only on Microsoft. It's free, but for some reason the author doesn't want to release the source. In any case, it's a lot better than anything that I could find in the Open Source space. I fear I'm going to have to continue to use Microsoft for this purpose.
More work on the code today, without making much progress in the debugging. Discovered that one of the programs, rtdvd, no longer worked:
rtdvd -d /dev/cd1 -v 1 -t 1 -s 0 -i /scratch/DVD/temp -f -n Inappropriate ioctl for device
Checking with gdb brought me to this code:
/* Open the device file and ensure it exists */
if ((device->fd = open (device->name, O_RDONLY)) < 0)
...
if (ioctl (device->fd, CAMGETPASSTHRU, &ccb) < 0)
{
/*
* We can assume the device will be ready with valid media so if it failed
* it really is a problem.
*/
syslog (LOG_ERR,
"unable to CAMGETPASSTHRU for %s: ", device->name);
perror (NULL);
exit (FATAL_START (errno));
}
Nothing much looked wrong with that, except maybe the missing parameters to perror, and the log entry confirmed that the device was /dev/cd1. Running the old version of the program on the same file worked.
Scratched my head and ran ktrace, which showed nothing unusual:
89424 rtdvd CALL open(0x805a760,0,0) 89424 rtdvd NAMI "/dev/cd1" 89424 rtdvd RET open 7 89424 rtdvd CALL fstat(0x7,0xbfbfe150) 89424 rtdvd RET fstat 0 89424 rtdvd CALL ioctl(0x7,0xc25e1503 ,0xbfbfdef0) 89424 rtdvd RET ioctl -1 errno 25 Inappropriate ioctl for device
Ran it again with the old program and got:
89510 rtdvd NAMI "/dev/cd1" 89510 rtdvd RET open 7 89510 rtdvd CALL fstat(0x7,0xbfbfe240) 89510 rtdvd RET fstat 0 89510 rtdvd CALL ioctl(0x7,CAMGETPASSTHRU,0xbfbfdfe0) 89510 rtdvd RET ioctl 0
The only obvious difference there was the name of the ioctl: in the previous ktrace it was in hex. Running kdump with the -n option showed the hex value for CAMGETPASSTHRU:
89510 rtdvd CALL ioctl(0x7,0xc2601503,0xbfbfdfe0)
So the correct value for the ioctl is 0xc2601503, but my version was using 0xc25e1503. That looked like I was using the wrong header files. Ran the preprocessor to see what was going on, using the trick described in chapter 6 of Porting UNIX Software (page 85 of this particular version). In that chapter I used -E; since then, I've taken to adding the compiler flags -C -dD -E, which give more information. But in this case, they didn't: I was using the correct system header files:
# 1 "/usr/include/cam/cam_ccb.h" 1 3 4 /*- * Data structures and definitions for CAM Control Blocks (CCBs). ... # 34 "/usr/include/cam/scsi/scsi_pass.h" 2 3 4 /* * Convert to using a pointer to a ccb in the next major version. * This should allow us to avoid an extra copy of the CCB data. */ #define CAMIOCOMMAND _IOWR(CAM_VERSION, 2, union ccb) #define CAMGETPASSTHRU _IOWR(CAM_VERSION, 3, union ccb)
The value for ioctl calls is quite structured; I described that, too, in Chapter 15 of “Porting UNIX Software” (page 252). There are four parts:
So somehow the size of my union had changed! That made more sense than it sounds: in an effort to ensure correct struct definitions for DVDs, I had decided to pack all structures. If it hadn't been for structures returned from the kernel, that would have worked. So clearly the -fpacked compiler option is a bad choice. Fixed things by defining the individual structures to be packed.
How I hate working on multimedia code! Somewhere there's something wrong with the navigation information that I'm outputting, but how do I find out where? The obvious thing to do would be to use gdb to look at the structures as they're output. But what structures? DVDs are full of them, but most of the code doesn't use any structure declarations; instead you end up with things like:
result = (buf [0] & 0x18) << (24 + 3); result |= (buf [0] & 0x03) << (24 + 4); result |= (buf [1] & 0xFF) << (16 + 4);
What does that mean? Good code has comments, but that doesn't help gdb much. When I look at this kind of code, I'm torn between “fixing” it to use proper structure declarations and just testing it. Today did a bit of both, very half-heartedly, and as a result had no results by the evening.
More playing around copying MPEG-2 streams to DVD. One of the disappointing discoveries last week was that DVDs are not compatible with HDTV: the highest (PAL) resolution is 720x576. Looks like the best way to archive things is by simply splitting the MPEG stream into 1 GB chunks and burning a file system to DVD. Even so, most films won't fit onto a single layer DVD, but since they need to be copied back to disk to be played, I don't suppose it makes much difference. Started writing a chapter for a potential future book on how to burn multimedia data to DVD.
Today turned my attention towards some issues I have with mplayer:
Did a bit of work on a more general on-screen display routine and tried to compile it. There's been a new release of mplayer since the last time I worked on it, and it no longer compiles at all out of the box: it seems that something has changed in the FreeBSD Ports Collection, and mplayer no longer compiles under releases as recent as 6.2-PRERELEASE. Frustration.
Finally had time to look at my mplayer mods today.. It took me all day:
It tried to re-install lirc, something I really didn't want to do. On investigation, discovered:
.if defined(WITH_LIRC)
LIB_DEPENDS+= lirc_client.1:${PORTSDIR}/comms/lirc
CONFIGURE_ARGS+= --enable-lirc
Is this really a man page? I was later told that it was a dependency on liblirc_client.so.1, but that's really a strange way to spell it.
So what do the changes do?
More work on the mplayer mods today. The problems with lirc proved to be due to the fact that they seem to have made some far-reaching changes in the new version of lirc, so clients that talked to the old version can no longer talk to the new one. They haven't fixed any of the problems I have seen, however; my patches applied with no problems.
Finally got as far as to build the mythtv port again; I've had a number of updates to make, but I first wanted a repeatable platform to do it on. At least I now have that for the software; to get it running properly, I need to put in a tuner as well. I have plenty of these—at the moment a total of 5, of which 2 run under FreeBSD. But they don't fit in the low-profile machine I have, so I'll have to do some hardware rearrangement first.
Strangely, I now have five TV tuner cards available: two el-cheapo analogue ones, the Hauppauge PVR-250 from the Black box, and two DVICO DVB-T cards. Of these, I can currently only use one, one of the DVICO cards. For reasons already documented, the second identical DVICO card doesn't work properly. And FreeBSD doesn't have support for them, nor for one of the analogue cards. Decided to kill two birds with one stone and put the Hauppauge and the other “functional” analogue card, a BT 878 based card apparently called “TV Excel” which I thought I had got to work, after a fashion, about 2 years ago. FreeBSD's bt878 driver has improved over the years. Last time I tried, I had to find the tuner type manually; now the probe finds it.
That's about all that worked, though. Try as I might, I wasn't able to find any TV stations, no matter what card type I tried. It's not clear from my logs whether I was able to do so at the time, though I had thought so. In any case, I also have a project to move the pvr250 driver into the kernel, so tried installing that from the Ports Collection. That didn't work either: in the meantime, module builds have become more pedantic, and the code contains some const variables that get passed as parameters to system functions defined without const. Spent far too much time trying to find how the Makefiles decide to issue a -Werror flag to the compiler, but finally gave up and appended a -W-no-cast-qual flag to the Makefile. Then it built, and I just needed a new kernel, with the result that I didn't make any further progress today. Pedanticism makes for good code, but it can also certainly waste a lot of time.
More playing around with the analogue tuners. Loaded the cxm driver for the Hauppauge card and ran scantv against it; again nothing was found. Tried various alternatives and got nothing, until I tried:
=== grog@tv2 (/dev/ttyp1) ~ 4 -> pvr250-setchannel -m 8 7=== grog@tv2 (/dev/ttyp1) ~ 5 -> cat /dev/cxm0 > foo^C=== grog@tv2 (/dev/ttyp1) ~ 6 -> l foo-rw-r--r-- 1 grog wheel 65638336 Feb 23 10:05 foo=== grog@tv2 (/dev/ttyp1) ~ 7 -> file foofoo: MPEG sequence, v2, program multiplex
mplayer confirmed that the content was a broadcast TV stream. So it seems that scantv doesn't understand the cxm driver, and in true multimedia tradition ignores any problems.
Moved on to trying to set up MythTV with the PVR 250. I never cease to be amazed. In the course of testing, discovered:
When I finally got to the card configuration, it produced an error message
Could not open '/dev/cxm0' to probe its inputs.
What error? Who cares, we're multimedia.
Running ktrace, found:
4233 mythtv-setup NAMI "/dev/cxm0" 4233 mythtv-setup RET open -1 errno 13 Permission denied
=== grog@tv2 (/dev/ttyp1) ~ 8 -> ls -l /dev/cxm0
cr--r--r-- 1 root wheel 0, 105 Feb 23 09:20 /dev/cxm0
QStringList CardUtil::probeV4LInputs(QString device)
{
bool ok;
QStringList ret;
int videofd = open(device.ascii(), O_RDWR);
if (videofd < 0)
{
ret += QObject::tr("Could not open '%1' "
"to probe its inputs.").arg(device);
return ret;
}
There are so many things wrong here:
It's trying to open the card O_RDWR. This is an input device, and its permissions are set accordingly. Of course the open must fail.
When it fails, it doesn't report the cause of the error. We've seen this already, of course.
The value returned by probeV4LInputs is some obscure QObject. Looking at the code, it looks as if it might only take one parameter. Certainly I wouldn't be sure to know how to add the text of the error (apart, I suppose, from a sprintf to a local buffer to return). C++ may have some advantages, though I'm seeing fewer and fewer as time goes on, but this kind of code is unnecessarily obfuscatory.
I wonder if there are cases where the card needs to be opened for writing. This code seems to be intended only for probing, so I suspect that it's not the case.
So I set the permissions on /dev/cx0 to rw-rw-rw- and tried again. Same message. Well, almost:
Could not open '/dev/cxm0' to pr
But then, truncation, both vertical and horizontal, seems to be another hallmark of multimedia software.
Back to ktrace, which confirmed that the open now succeeded, but a subsequent ioctl failed:
4233 mythtv-setup NAMI "/dev/cxm0" 4233 mythtv-setup RET open 9 4233 mythtv-setup CALL ioctl(0x9,0x40685600 ,0xbfbfc890) 4233 mythtv-setup RET ioctl -1 errno 25 Inappropriate ioctl for device
In fact, during the probe the device was opened no less than 13 times. Came to the conclusion that mythtv was too lame to report the failed ioctl call, and it had failed before, so it had an error message to return, after first mutilating it.
But why was the ioctl failing? Possibly because the cxm driver doesn't conform to the standard ioctls. That's something for me to check out; first, though, tried recompiling without optimization to make using the debugger easier. That alone took an hour—why does C++ compile so slowly?—by which time the day was over.
Discussed the matter on IRC; we're all agreed that most multimedia software is a complete pain. It's so tempting to write your own; but there's a lot of good stuff in mythtv, so it's difficult to know which way to go. As the Germans say, außen hui, innen pfui.
Did a bit of investigating the ioctl calls in the cxm driver, and quickly came to the recognition that they were, indeed, incompatible with MythTV. Investigated the possibility of adding them, and found the list (without comments, of course) in libs/libmythtv/videodev2_myth.h—all 57 of them, many referencing a struct v4l2_foo. Decided that that was too hard and moved on to confirm the installability of the package, in the process running portlint, which is supposed to check the correctness of a port. Some of the things it does are just plain stupid. It's clear that the sequence of declarations in a Makefile can have an influence on variables, especially when using constructs like ?=, but portlint considers the following to be a fatal error:
COMMENT= MythTV is a homebrew PVR project
BUILD_DEPENDS= qmake:${PORTSDIR}/devel/qmake \
${SITE_PERL}/${PERL_ARCH}/XML/Parser/Expat.pm:${PORTSDIR}/textproc/p5-XML-SAX-Expat
LIB_DEPENDS= mp3lame.0:${PORTSDIR}/audio/lame \
freetype.9:${PORTSDIR}/print/freetype2
There must be an empty line after the COMMENT definition. It also complains here that LIB_DEPENDS must come earlier in the Makefile (before BUILD_DEPENDS). But if you do that, it complains that BUILD_DEPENDS must come earlier. There doesn't seem to be any way to make it happy.
Later in the day heard from Stacey Son, the original author of the MythTV port. He confirmed that the cxm driver in its current form doesn't work with MythTV, and sent some patches. More work to do.
Spent some time today working on Stacey Sun's patch to the cxm driver. It's a couple of years old, and although it applied cleanly, it seems that one of the structures has changed. It contains the following line:
const struct cxm_codec_profile *cpp;
...
codec->audio_bitmask = cpp->audio;
Unfortunately, there is no longer a member audio in struct cxm_codec_profile. Compiled anyway, but didn't get much tested.
Had intended to take it easy today, but somehow that didn't happen. Instead, finally got hold of a new version of the Hauppauge driver and tried to install it on tv2, my current test box. It installed without too much trouble, but then paniced out of msleep with the message sleeping without a mutex. Strange message! We used to have the opposite problem. Tried to get a dump, but the system hung. How long has it been since I've been able to get a processor dump reliably? Once upon a time the command panic out of ddb would do it for you, then we had to issue a call doadump, but today the dump hung in each case.
Decided to set up a firewire debugging session, but that doesn't work either any more unless you have firewire compiled into the kernel. Another afternoon's work bringing the system up to date and building a kernel.
In the meantime, looked at a couple of other issues: on VOB files copied from DVDs, my version of mplayer always showed the elapsed time as being equal to the total time. Tried recompiling it, again running into strange issues with asm directives that took me some time to fix, and also issues with debugging symbols. Somehow things used to be easier. Once I got that done, the problem was obvious: a simple cut-and-paste-o.
Also looked at my record program, which I had written specifically for the Linux DVB interface. Now I can use the Hauppauge card, I needed to get it to work for that as well. That took surprisingly little time and worked—almost correctly—first time.
More work on the cxm driver today, and came up with a patch that worked around this strange feature in msleep. This is almost close enough for me to be able to put it into the source tree.
Back to play around with mythtv-setup, which frequently displays in incorrect colours on remote systems. It now recognizes the tuner and can set it up; but the setup is so non-intuitive that I didn't make it. Yet another day's work.
Also tried a couple of approaches to video editing. I had tried LiVES earlier, and it didn't compile. Now I got it to compile, but on starting and trying to edit an 8 GB MPEG of a TV programme, was presented with a warning that it's not optiimized for large files. That's an understatement: it calls mplayer to convert every frame to a JPEG. I don't have enough disk for that, let alone enough time.
Then tried Jahshaka, which ran a lot faster:
=== grog@tv2 (/dev/ttyp0) /spool/Images 3 -> jahshaka bridget-jones
preferences are ok...
QLayout: Adding QLabel/unnamed (child of QFrame/unnamed) to layout for QFrame/unnamed
QLayout: Adding QProgressBar/unnamed (child of QFrame/unnamed) to layout for QFrame/unnamed
QLayout "maindesktopLayout" added to QHBox "desktop", which already has a layout
QLayout "Form1Layout" added to QHBox "unnamed", which already has a layout
(many repeats)
It ignored the file name on the command line, of course, and when I selected “Load” it was clear that it also ignored the current directory, giving me /usr/X11R6/lib/jahshaka/media, in which it apparently really intends to store media data. I couldn't change the path name, though: when I moved the mouse cursor to the window, I got:
Segmentation fault: 11 (core dumped)A gdb session shows a 43 frame back trace through the Qt libraries, starting with:
#0 0x29e6325b in parse_fontdata () from /usr/X11R6/lib/X11/locale/lib/common/xomGeneric.so.2 #1 0x29e634be in parse_vw () from /usr/X11R6/lib/X11/locale/lib/common/xomGeneric.so.2 #2 0x29e64062 in create_oc () from /usr/X11R6/lib/X11/locale/lib/common/xomGeneric.so.2 #3 0x28c1e914 in XCreateOC () from /usr/X11R6/lib/libX11.so.6 #4 0x28c1dd5c in XCreateFontSet () from /usr/X11R6/lib/libX11.so.6 #5 0x285d49f8 in getFontSet () from /usr/X11R6/lib/libqt-mt.so.3 #6 0x285d4cb5 in QInputContext::setXFontSet () from /usr/X11R6/lib/libqt-mt.so.3
Probably a configuration error, but somehow I just can't be bothered with this kind of mess.
More work on getting MythTV set up today. Tried to revive the MythTV HOWTO page that I started on for Linux nearly a year ago and then gave up because it was so much pain. That page is very much Linux-related, so decided to make a separate FreeBSD HOWTO page. The good news is that it's much easier. Got as far as the mythfilldatabase step, which produced much output and ended with:
===============================================================
| Attempting to contact the master backend for rescheduling. |
| If the master is not running, rescheduling will happen when |
| the master backend is restarted. |
===============================================================
2007-02-27 14:24:06.099 Connecting to backend server: 192.109.197.177:6543 (try 1 of 5)
2007-02-27 14:24:06.101 Connection timed out.
You probably should modify the Master Server
settings in the setup program and set the
proper IP address.
Further investigation showed that, though my port installs permissions for user mythtv on the local host, it doesn't (and shouldn't) install them for remote access, which for MySQL includes TCP access from the same machine. For security reasons you need to set that up manually. But as usual, the error message was completely wrong.
The next step was to get tuner data. The MythTV port depends on XMLTV, but for some reason that port doesn't include tv_grab_au port. Spent the rest of the afternoon making a port for that.
Still more work on MythTV today, mainly documenting what I did. I still don't really understand the internal connections; probably a diagram would help. Got as far as configuring a tuner input and confirming that it knew the correct channels, but on leaving mythtv-setup, I got a message saying “Your tuner is set to start on channel 3, which doesn't exist. Fix?“. I replied “Yes”, and it went away and did nothing for a while, but didn't fix it. I stopped mythtv-setup, restarted it and manually set the tuner to start on channel 28, which does exist. On exit, I got the same message about channel 3, and when I ran mythfrontend it also tried to set channel 3. Head-scratching time. I suspect that it's trying to use a different input.
Also had a couple of problems with the new driver:
Mar 1 10:26:25 tv2 kernel: cxm0: SAA7115 rev 1 video decoder Mar 1 10:26:25 tv2 kernel: cxm0: MSP4418G-B3 audio decoder Mar 1 10:26:25 tv2 kernel: cxm0: IR Remote Mar 1 10:26:25 tv2 kernel: cxm0: [FAST] Mar 1 10:26:27 tv2 kernel: cxm0: encoder firmware version 0x2060039 Mar 1 10:26:27 tv2 kernel: cxm0: decoder firmware version 0x2020023 Mar 1 10:27:06 tv2 kernel: cxm0: video decoder isn't locked ... Mar 1 10:27:06 tv2 kernel: Could not detect FPS Mar 1 10:27:06 tv2 kernel: could not config dec Mar 1 10:27:06 tv2 kernel: could not start encoder
More work on MythTV today. This is really like pulling teeth. In the course of the day, discovered that there's something I still don't understand about attaching “inputs”. It could be my imperfect understanding of how things fit together, but at the moment it looks more like a bug to me. I had already mentioned that mythtv-setup produces a warning about incorrect channels when exiting. I was unable to get rid of it, and ended up looking at the underlying MySQL database, which showed that yes, indeed, the information in the database was incorrect. Either mythtv-setup is setting up the channels incorrectly, or the documentation is so bad that I still can't work out what's going on.
Things were not all plain sailing after that; mythfrontend still claimed that no channels were defined. In the end, did a complete scan, which added all channels in addition to those derived from the guide data. After that, I was finally able to schedule a recording, only to discover after the event:
2007-03-01 19:30:03.471 TVRec(2): HW Tuner: 2->2 2007-03-01 19:30:03.596 TVRec(2) Error: Failed to set channel to . Reverting to kState_None 2007-03-01 19:30:03.597 TVRec(2): Changing from RecordingOnly to None 2007-03-01 19:30:03.639 Finished recording Inspector Rex "The Baby Dealers": channel 1000 2007-03-01 19:30:03.657 Started recording: Inspector Rex "The Baby Dealers": channel 1000 on cardid 2, sourceid 1 2007-03-01 19:30:04.661 Reschedule requested for id 0. 2007-03-01 19:30:04.735 Scheduled 1 items in 0.1 = 0.00 match + 0.07 place
Clearly there's something seriously wrong somewhere. Time to revisit the database.
In the process, ran into more trouble with the new cxm driver: it failed to tune to the channels. After some experimentation, including rebooting to change to the old driver (which worked perfectly), discovered that tuning would succeed if the old driver had been run previously. Further investigation shows that the old driver, but not the new, automatically selects the tuner when you try to tune to a channel. The new driver (currently) requires the -t option to select the tuner.
The work on MythTV continues. Came in this morning to investigate why the recording I had set up for last night had failed. Presumably the important message was:
2007-03-01 19:30:03.596 TVRec(2) Error: Failed to set channel to . Reverting to kState_None
That looks very much like a problem with the channel number. Further investigation in the database revealed that the programme guide had not set any channel numbers. Here a couple of channels:
+--------+---------+--------+-------------------+-------------------+ | chanid | channum | freqid | callsign | name | +--------+---------+--------+-------------------+-------------------+ | 1001 | | NULL | ABCSA | ABC SA | | 1003 | | NULL | CHASE | Channel Seven | | 1007 | 2 | 64250 | Adding Channel 2 | Adding Channel 2 | | 1013 | 7 | 182250 | Adding Channel 7 | Adding Channel 7 |
The first two were set up by the programme guide data; the second two, the same channels, were found by the “channel scan”, which doesn't seem to have done anything beyond including all possible channels in the table. Ended up having to use SQL to set the channel numbers and frequencies.
Things still didn't work too well, and I discovered that I had programme data only for SBS. Followed up on a message from Alex Wilkinson and installed a new guide data grabber, tv_grab_au_reg, which confuses the Ports Collection by being a single, uncompressed shell script. Spent a lot of time trying to work out how to explain that to the Ports Collection before giving up and just installing it. MythTV didn't like the change in grabber, and in the end decided it would be easier to remove (well, move aside) the database and start again. That wasn't that simple either: MythTV doesn't create the database if it doesn't exist, it just goes crazy. Finally got the thing installed, writing instructions in the process. tv_grab_au_reg can handle all the data it receives, or filter only those channels you specify. MythTV is not so clever: if you don't specify any channels, it doesn't recognize any of the data it gets.
Finally pulled in the data and discovered the times were off by 10½ hours; unlike the older tv_grab_au, you have to set Auto in the TimeOffset field.
Finally all was in place, and I really had guide data for all 7 channels—but I had lost my channel frequencies again. Put them back in and I was finally, for the first time, able to record a programme from TV using only MythTV. After only 2½ years!
Building on yesterday's success, spent a lot of time today documenting what I had done. There's still a lot of stuff to remove from the documents, but hopefully they'll fill the gaps where so much documentation falls down: document not just what to do, but what to look out for. So far I haven't been able to get the channel data included automatically, for example, so my tv_grab_au_reg document explains the problem and how to solve it.
Then spent some time installing MythWeb, a web-based interface to MythTV, requiring of course all the myriad bits and pieces of Apache, MySQL and PHP. After installation, nothing much displayed, except for the entries in the Apache error log:
[Sat Mar 03 16:26:56 2007] [notice] child pid 66756 exit signal Segmentation fault (11) [Sat Mar 03 16:27:08 2007] [notice] child pid 66758 exit signal Segmentation fault (11)
Nothing indicated what the child was, but a tail -f showed that it happened when I tried to load the home page. Tried it on echunga, where it worked (modulo not being able to access the database), so compared the httpd.conf files and discovered:
LoadModule php5_module libexec/apache22/libphp5.so LoadModule php5_module libexec/apache22/libphp5.so
The first entry is what I put in there. The second, with the strange spacing, was added by the port, which didn't first check that the entry was there. For some reason, this double entry was causing the SIGSEGVs: after removing it, things worked well, and displayed reasonably useful information:
There are still a number of things to fix, of course. Firstly, the colour scheme is emetic, the rendering incorrect, and there's a PHP error message at the bottom that I need to attend to. Still, a good basis for building something useful.
In view of the impending multimedia boom, decided to upgrade Yvonne's machine, battunga, which is still running FreeBSD 4.9. Things didn't work well: the first disk I found was from one of the machines that was killed in the December 2005 power surge. It became very clear that the disk had also been affected: the clunking noise on spinup was accompanied by a burning smell. I wonder how high that voltage spike was.
Continued work on MythWeb, and found the cause of yesterday's error message (easily enough, given that it was accurate and said what the problem was): it seems that MythWeb tries to save state after every display, but for some reason it opens the database at the beginning, then closes it (where? I can't find that), and then tries to write to it. Kludged around that by reopening in the function itself, but it brings home to me how little I understand about debugging PHP.
Also spent some time playing around with the CSS style sheets, something else I don't really understand, to try to fix the horrible colours. Ended up with a black on white rendition which is probably too much oversimplification. I wonder what it would be like to simply invert all colours, and how best to write a script to “fix” the style sheet.
Resolved my question about what to do next by deciding that it would make most sense to get my current recording setup converted to MythTV. That involved mainly setting it up on ceeveear, which was based on KnoppMyth. Spent some time reading the documentation, but there's very little there about multiple back ends, so just tried setting it up. The configuration seemed to work well enough, but for some reason it wanted a duplicate set of guide data; it seems that MythTV defines a master backend, but it doesn't do the logical thing and keep all the data in a single database.
Rather than setting up the guide data from scratch, copied the configuration from tv2 and renamed it to match the input. That still didn't work too well; after realizing that ceeveear didn't have a default route to the outside world, managed to do an update. But what did it really do? It's difficult to say from this output:
2007-03-08 13:06:14.346 Updating icons for sourceid: 1 removing conflicting program: ABC-SA Engineering At The Cutting Edge 20070309103000-20070309110000 conflicted with : ABC-SA Australians 20070309105500-20070309110000 removing conflicting program: SBS-SA Spasm (Sayew) 20070309230500-20070310011000 conflicted with : SBS-SA Spasm (Sayew) 20070310000000-20070310011000 Updated programs: 1081 Unchanged programs: 0 2007-03-08 13:06:21.610 Data fetching complete. 2007-03-08 13:06:21.611 Adjusting program database end times. 2007-03-08 13:06:21.624 0 replacements made 2007-03-08 13:06:21.625 Marking generic episodes. 2007-03-08 13:06:21.636 Found 39 2007-03-08 13:06:21.637 Marking repeats. 2007-03-08 13:06:21.648 Found 0 2007-03-08 13:06:21.648 Unmarking new episode rebroadcast repeats. 2007-03-08 13:06:21.657 Found 0
The 39 generic episodes suggest that it loaded something, but the numbers didn't inspire confidence. Carrying on, though, it was clear that there was something wrong:
2007-03-08 13:06:21.673 Connecting to backend server: 192.109.197.177:6543 (try 1 of 5) 2007-03-08 13:06:22.306 Protocol version mismatch (frontend=26,backend=30)
The backend server is tv2 (why does all this stuff use IP addresses instead of DNS?), and it's running MythTV version 0.20a. ceeveear was a long way behind that, but since it's a Debian box, it's easy to update to the latest version, right? Well, yes, but since it's Debian, the version installed was the latest version. Instead went off looking for the sources for MythTV, which proved more difficult than I thought, considering I've done a port for FreeBSD. In the end decided to check out from the Subversion repo.
More work on getting Linux to work on my CVR boxes today, and somehow seemed to spend all day without achieving very much. I probably own Ubuntu an apoology: KnoppMyth R5E50 (where does he get these revision numbers from?) also did so. Couldn't install Fedora Core 6: after burning the DVD, discovered that firefox (my favourite program) had stopped downloading in the middle and hadn't reported the error anywhere where I could see it, so used ftp to download the rest. For some reason that came up with an invalid checksum, so I repeated the entire download with ftp. It proved to be that I had used sha256 instead of sha1, but I didn't find that out until after downloading and confirming that both images had the same checksum.
Decided to give up on installing Linux on the Via motherboard (“black box”) and do some reshuffling. My main concern is to not be in a position where I can't record TV, so decided to put the disk for the tv2 FreeBSD box in the black box and use the other system for Linux.
That brought another problem with itself: I also needed the Linux box (which I called cvr2) in the Hi-Fi cupboard, because that's where the antenna connections are. But the switch there (also wireless access point) only had five connections, and with tv2 they were all in use (uplink, downlink to the VoIP ATAs in the old office, ceeveear, teevee and tv2). Ended up switching the switch in my old office (8 ports, only 3 in use) with the access point. And that again was complicated by the fact that the access point had a 115V power supply, so I had to move the transformer, disconnecting the other power cord, which I had thought was the old TiVo that I no longer use. It wasn't until later that I discovered that it was for the amplifiers in the old analogue video switch, meaning that the TV no longer had an input. What a pain!
In the end gave up with the display on cvr2 and left it in the black box; I only really need this as a test box for the MythTV backend. The good news is that the DVICO tuner card that had given me problems earlier now worked fine; I had suspected that this might be a driver issue, and the new version of KnoppMyth has a relatively new kernel (but very old versions of dvbscan and tzap). Spent still more time trying to set up MythTV like that, but it seems that the channel setup is still a serious issue. It seems that both Australian TV guide grabbers provide some information, but not enough, and none of the instructions I found (specifying unknown grabbers) worked for me. It should be possible to create this information from the output from dvbscan, and maybe I'll do that.
Continued with channel setup for DVB today, and got some useful help on the #mythtv-users IRC channel. As I suspected, it's all a question of documentation. All along I had expected that tv_grab_au_reg or similar should miraculously produce the tuning information, but despite the documentation this isn't the case: it just provides the channel IDs. For DVB-T, the tuning information can come from one of two places: from the internal scan or from the output of dvbscan, exactly what I hand been planning to do. The tuning information for DVB is currently (MythTV version 0.20a) shared across two tables. The channels table contains information for both digital and analogue channels, but for digital channels the freqid attribute is ignored. Instead, it is linked to the dtv_multiplex table via the attribute mplexid. dtv_multiplex contains transponder information, notably the frequency, bandwidth and modulation stuff—much the same as what dvbscan outputs. There must be other stuff as well, but so far I haven't found it.
Tried both methods and decided that I'm more comfortable with the channels.conf file method. Here's how to do each:
To scan with mythtv-setup, you first need the guide data properly installed. With tv_grab_au_reg this means that you need to exit mythtv-setup and follow the instructions to get the guide data (I think). Restart mythtv-setup if necessary, go to Input connections and select the tuner input. Select Scan for channels, then select Full Scan for Scan Type. This works, but the output worries me, since all I see are failures.
In either case, you may want to tidy up the channel information. In my case I ended up with 32 channels (6 analogue, 26 digital, including programme guides, duplicates and radio). If you're using both analogue and digital tuners, you'll need to ensure that the channel information is linked to the same programme guide data. This is done via the xmltvid column.
In addition, 32 programmes are a real pain to view on screen. MythTV only displays those channels who have the value 1 (and not, for example, 2) in the column visible. You can always update this using mysql with commands like:
mysql> update channel set visible = 1 where chanid = 3006;
It's a lot easier with MythWeb, though. One of the selections in the Settings menu is channel info, and there you can select the visible attribute and set the xmltvid directly.
In the end, you should be able to see something like this:
mysql> SELECT chanid, channum, freqid, name, xmltvid, mplexid, serviceid, visible
-> FROM channel
-> ORDER BY chanid;
+--------+---------+--------+-----------------+----------+---------+-----------+---------+
| chanid | channum | freqid | name | xmltvid | mplexid | serviceid | visible |
+--------+---------+--------+-----------------+----------+---------+-----------+---------+
| 1501 | 501 | | ABC HDTV | ABC-SA | 12 | 592 | 1 |
| 1502 | 502 | | ABC TV Adelaide | ABC-SA | 12 | 593 | 0 |
| 1503 | 503 | | ABC2 | ABC2 | 12 | 594 | 1 |
| 1504 | 504 | | ABC TV | ABC-SA | 12 | 595 | 0 |
| 1505 | 505 | | ABC DiG Radio | | 12 | 598 | 0 |
| 1506 | 506 | | ABC DiG Jazz | | 12 | 599 | 0 |
| 1507 | 507 | | 7 Digital | Seven-SA | 13 | 1360 | 1 |
| 1508 | 508 | | 7 Digital 1 | Seven-SA | 13 | 1361 | 0 |
| 1509 | 509 | | 7 Digital 2 | Seven-SA | 13 | 1362 | 0 |
| 1510 | 510 | | 7 Digital 3 | Seven-SA | 13 | 1363 | 0 |
| 1511 | 511 | | 7 HD Digital | Seven-SA | 13 | 1364 | 0 |
| 1512 | 512 | | 7 Guide | | 13 | 1366 | 0 |
| 1513 | 513 | | NINE Digital | Nine-SA | 14 | 1105 | 0 |
| 1514 | 514 | | NINE HD | Nine-SA | 14 | 1112 | 1 |
| 1515 | 515 | | TEN Digital | Ten-SA | 15 | 1617 | 0 |
| 1516 | 516 | | TEN Digital 1 | Ten-SA | 15 | 1618 | 0 |
| 1517 | 517 | | TEN Digital 2 | Ten-SA | 15 | 1619 | 0 |
| 1518 | 518 | | TEN Digital 3 | Ten-SA | 15 | 1620 | 0 |
| 1519 | 519 | | TEN Guide | | 15 | 1623 | 0 |
| 1520 | 520 | | TEN HD | Ten-SA | 15 | 1624 | 1 |
| 1521 | 521 | | SBS HD | SBS-SA | 16 | 832 | 1 |
| 1522 | 522 | | SBS DIGITAL 1 | SBS-SA | 16 | 833 | 0 |
| 1523 | 523 | | SBS DIGITAL 2 | SBS-SA | 16 | 834 | 0 |
| 1524 | 524 | | SBS EPG | | 16 | 835 | 0 |
| 1525 | 525 | | SBS RADIO 1 | | 16 | 846 | 0 |
| 1526 | 526 | | SBS RADIO 2 | | 16 | 847 | 0 |
| 3001 | 2 | 64250 | ABC-SA | ABC-SA | NULL | 0 | 0 |
| 3002 | 7 | 182250 | Seven-SA | Seven-SA | NULL | 0 | 0 |
| 3003 | 9 | 196250 | Nine-SA | Nine-SA | NULL | 0 | 0 |
| 3004 | 10 | 209250 | Ten-SA | Ten-SA | NULL | 0 | 0 |
| 3005 | 28 | 527250 | SBS-SA | SBS-SA | NULL | 0 | 0 |
| 3006 | 31 | 548250 | 31-Adl | 31-Adl | NULL | 0 | 1 |
+--------+---------+--------+-----------------+----------+---------+-----------+---------+
32 rows in set (0.01 sec)
Here the chanids in the range 15xx are digital, and the chanids in the range above 3000 are analogue.
After doing that, it's definitely a good idea to save the channel information. I've saved the information for Adelaide here. This is mysqldump output, so it can be installed (overwriting previous tables!) with:
$ mysql mythconverg < channels.sql
After all that, finally had a properly running backend with digital input. Things are looking up.
Today spent some time—far too much time—looking at how to get multiple tuners working in one machine. It's not easy, and I didn't manage completely.
I ended up adding two other tuners: the second (really the older) DVICO tuner and the MSI TV@nywhere analogue tuner. The first problem was just entering the device name. The first DVB-T tuner had the device name /dev/video0 in capturecard.videodevice, and (not surprisingly) the system came up with the devices /dev/video1 and /dev/video2 as well. But I couldn't find a way to enter /dev/video1 with mythtv-setup. Tried it with mysql, which worked, but then I ended up a non-functional mythbackend which ignored the device name and tried to open adapter 0 twice. After a lot of trial and error, two things became apparent:
My best guess is that the code is using atoi or a similar function to convert the field into binary, and not checking whether it's valid. atoi always returns 0 if pointed at a non-numeric string.
So, finally I can record two things at once. As MythWeb reports:
Encoder 1 is local on cvr2 and is recording: 'Seven News at 4.30' on 7 Digital. This recording will end at 17:00. Encoder 4 is local on cvr2 and is recording: 'National Nine Afternoon News' on NINE HD. This recording will end at 17:00.
The adapter numbers are the result of MySQL's autoincrement feature. At some time I'll fix them. First, though, I need to get the analogue tuner working too.
Somehow didn't get anything done today. Spent all the day playing around with hardware, but at the end of the day didn't have any more to show than having added a 250 GB drive to teevee. Started by putting it on wantadilla as a USB-attached disk, but for some reason it did a spontaneous reset when I tried to newfs it. it. Finally managed to shoot down teevee as well, so put the disk in on an ATA connection.
More problems recording programmes in the evening: one of the recordings didn't work correctly. Do I still have a problem with one of those DVICO cards after all?
Looking at yesterday's DVICO recording problems revealed that they only worked incorrectly on a single channel, my favourite Nine MSN. Also discovered that the two cards aren't identical after all: they have a different demodulator chip, the Zarlink ZL10353, about which I have heard of other problems. Looks like I'll have to upgrade the drivers, or, better, write a driver for FreeBSD.
That didn't help with my recording, unfortunately. In the middle of a recording in the evening, the recording just stopped. If there was an error message, it's not clear where. Spent over an hour messing around trying to find what was going on, during which the “good” DVICO card stopped producing any output, despite good signal, and mythbackend refused to start. Finally gave up in disgust.
The cause of yesterday's problems could have been in part due to the way I set up the channel table, so decided to start again from scratch today. That worked better than I had expected:
This left me with a whole lot of channels with no guide data, and a whole lot of guide data with no channels. This time I used the MythWeb channel editor to set the xmltv id for the channels in question. I wonder how it's meant to be done.
In the process, discovered that the attribute sourceid relates to the tuner number, so it is possible to restrict certain channels to certain tuners. That'll be handy with the current Nine MSN problems. It's also interesting to note that you can run mythfilldatabase several times before it finds nothing more to update, though typically the changes are things like:
removing existing program: Ten-SA The Firm 2007-03-17T21:30:00 - 2007-03-18T00:40:00 inserting new program : Ten-SA The Firm Sat Mar 17 21:30:00 2007 - Sun Mar 18 00:40:00 2007
After all that, ended up with multiple displays for each program, one per tuner card. I don't have a good idea of how to fix it; probably I'll have to hack MythWeb.
Up early today with the intention to address the issues I have with MythWeb, and discovered that it's more difficult than it seems to change. Made a horrible kludge to select only one instance of each channel, but it's specific to my current database setup, and it desperately needs improvement.
Back home and discovered that MythTV had decided to record everything from the analogue tuner, and I couldn't find a way to make it change its mind. Reading the documentation, I found:
The Master backend will always choose the first available tuner in the same order as you add cards through "mythtv-setup". In other words, the second card you add will only be used when there are two overlapping recordings, the third when there are three, and so on.
In fact, it was using tuner 8, the last of the three: the tuner number is in capturecard.cardid, an autoincrement field that leaves you with numbers reflecting the number of attempts you have made to install tuners. In my case, the numbers were 6, 7 and 8, so the master backend should have been using tuner 6, then tuner 7, but it seems that they didn't get stored physically in that order:
mysql> SELECT cardid, videodevice, cardtype FROM capturecard; +--------+-------------+----------+ | cardid | videodevice | cardtype | +--------+-------------+----------+ | 8 | /dev/video0 | V4L | | 6 | 0 | DVB | | 7 | 1 | DVB | +--------+-------------+----------+
So it seems that the order is the order in which they're stored in the MySQL table, not the order in which you add them. But the sequence in a MySQL (MyISAM) table is indeterminate, even for primary keys (in this case cardid). Rebuilt the table so that the physical sequence matched the numerical sequence, but still couldn't get it to stop. In the end removed most channel entries for the analogue tuner card, leaving only channel 31, which doesn't have digital broadcasts. Clearly that's unsatisfactory: it should be possible to specify preferences.
Even that didn't work: recording an old Bonanza sequence gave me 32 GB of junk. Maybe it's related to the problems that KnoppMyth has with the sound chip on this particular motherboard, or maybe it's the driver for this particular card, which I've never had working. Maybe I can try it with the PVR 250 next week.
Started putting together the next generation CVR, with a rather nice motherboard including firewire, gigabit Ethernet and an nVidia graphics chip, along with the AVerTV Hybrid+FM PCI card. Installation with KnoppMyth was not encouraging: it didn't recognize the graphics chip, and it didn't recognize the tuner card. Maybe they're both too new; hopefully I'll be able to sort it out, but it's still not plain sailing. Brought up FreeBSD on the machine and confirmed that it didn't recognize the graphics chip either, though at least it was able to install a VESA driver and display something.
Work on two fronts today: continued installing the “black box”, and ran into significant trouble with the Hauppauge PVR-250 card, which failed to probe. When loading the kld I got:
cxm0: <Conexant iTVC15 MPEG Coder> mem 0xe4000000-0xe7ffffff irq 17 at device 9.0 on pci0 cxm_iic0: <Conexant iTVC15 / iTVC16 I2C controller> on cxm0 iicbb0: <I2C bit-banging driver> on cxm_iic0 iicbus0: <Philips I2C bus> on iicbb0 master-only cxm0: LG Innotek TPI8PSB01N tuner cxm0: SAA7115 rev 1 video decoder cxm0: MSP4418G-A2 audio decoder cxm0: IR Remote cxm0: [GIANT-LOCKED] cxm0: could not initialize hardware iicbus0: detached iicbb0: detached cxm_iic0: detached device_attach: cxm0 attach returned 6
After much head-scratching noticed that this message was followed by:
vr0: <VIA VT3043 Rhine I 10/100BaseTX> port 0xc000-0xc07f mem 0xd9000000-0xd900007f irq 18 at device 10.0 on pci0 vr0: MII without any phy! device_attach: vr0 attach returned 6
That shouldn't have happened. Since I didn't need the card, decided to remove it, and contrary to my expectations it fixed the problem, which doesn't seem to have anything to do with interrupt sharing.
With the other machine, installed the nVidia driver, which contrary to my fears did recognize the graphics chip:
nvidia0: <Quadro NVS 210S / NVIDIA GeForce 6150LE> mem 0xfd000000-0xfdffffff, 0xd0000000-0xdfffffff,0xfc000000-0xfcffffff irq 16 at device 5.0 on pci0 nvidia0: [GIANT-LOCKED]
Created an X configuration file, started X and—it spontaneously rebooted, repeatably.
Rather than follow that track, tried installing Ubuntu Edgy, which installed correctly, the first Linux installation that has been able to start X for some time. Then tried installing the nVidia driver. For FreeBSD it's simple:
# cd /usr/ports/x11/nvidia-driver # make install
It's not quite that easy with Linux. You need to:
# sh NVIDIA-Linux-x86-1.0-9755-pkg1.run
This runs interactively, downloads a kernel interface and builds it, but only if libc6-devel is installed.
After this, the driver ran fine. I suspect that the problem with FreeBSD was that I had set the DIAGNOSTIC option in the kernel, which catches program bugs that normally go undetected. But that's for another day.
The display card, the AVerTV Hybrid+FM PCI, is another matter. After reading the details more carefully, it seems that it's not supported properly after all. Leader doesn't want it back, so it looks as if I'll be in for a wait until somebody—maybe myself—adds support for it.
In the afternoon, checking why I couldn't access MythTV database on ceeveear. The answer was simple: it was set up that way at install time, without any information. That's not the default, and the following change was all that was needed:
--- my.cnf~ 2006-11-29 13:47:04.000000000 +1030 +++ my.cnf 2007-03-23 14:35:35.000000000 +1030 @@ -53,7 +53,7 @@ # # Instead of skip-networking the default is now to listen only on # localhost which is more compatible and is not less secure. -bind-address = 127.0.0.1 +# bind-address = 127.0.0.1 #
Why do people do these things?
That wasn't enough, unfortunately. For some reason there's a protocol difference between front and back end. They're both supposed to be the same version, so that puzzles me. Still more work needed.
Set to installing a compatible version of MythTV today; fortunately Geoff Buckingham had sent me a port of the “stable” version, which is only available from the subversion repository. I wonder how we should handle this; there are at least three versions out there, and they all seem to be incompatible with each other:
Built that, and was finally able to get mythfrontend on FreeBSD communicating with a mythbackend under Linux. Now to learn how to handle the remote control, and how to tell mythfrontend about XvMC.
As planned, spent some time today trying to get MythTV working with XvMC and with LIRC support. The former was complicated by the fact that the display cards on wantadilla use nVidia GeForce MX 4000 chipsets, and they're not supported by the current drivers. Installed the “legacy” driver, which for some reason brought up wantadilla:0.0 with a maximum resolution of 1856x1392. In the log file it stated:
(WW) NVIDIA(0): Not using mode "2048x1536" (width 2048 is larger than (WW) NVIDIA(0): EDID-specified maximum 1856)
This appears to be the opinion of the monitor itself; but it's been running fine for years at 2048x1536, and ultimately it's the horizontal frequency that determines the maximum resolution. Put that monitor back onto the nv driver and continued only with the second card, but that didn't seem to want to run XvMC either. Quite possibly the card doesn't support it, so I'll have to try on teevee. Some other day.
Mail from Patrick Hess about the issues with the nVidia driver and EDID; as I suspected, you can turn it off with the following line in the DEVICES section:
Option "IgnoreEDID" "true"
That doesn't help with the fact that the card doesn't seem to handle XvMC, of course.
Finally got round to trying out MythTV with XvMC and with LIRC support on teevee. That didn't work either; the remote control failed with “unable to open socket”, and the XvMC with a more obscure error message. First, here's what happened with the standard codec:
2007-03-27 16:25:08.165 AFD: Opened codec 0x8602c10, id(MPEG2VIDEO) type(Video) 2007-03-27 16:25:08.165 AFD: Opened codec 0x861e010, id(MP3) type(Audio) 2007-03-27 16:25:09.880 TV: Attempting to change from None to WatchingPreRecorded [mpegts @ 0x28f97460]Parser not found for Codec Id: 94211 ! 0: start_time: 5909.602 duration: -9223372036854.775 1: start_time: 5909.578 duration: 674.622 2: start_time: 1961.266 duration: 674.683 stream: start_time: 21791.849 duration: 51365.929 bitrate=1187 kb/s
At this point I selected the XvMC codec with the mythfrontend setup menu. Then I got:
2007-03-27 16:25:10.715 AFD: Opened codec 0x9761010, id(MPEG2VIDEO_XVMC) type(Video)
2007-03-27 16:25:10.715 AFD: Opened codec 0x9761410, id(MP3) type(Audio)
2007-03-27 16:25:10.719 Opening OSS audio device '/dev/dsp0.0'.
2007-03-27 16:25:11.150 VideoOutputXv: XvMCTex: Init failed
2007-03-27 16:25:11.151 VideoOutputXv: XvMC Adaptor Name: 'NV17 Video Texture'
2007-03-27 16:25:11.197
XError type: 0
display: 0x8514800
serial no: 36
err code:
(BadAccess (attempt to access private resource denied))
req code:
minor code:
resource id: 290
2007-03-27 16:25:11.198
XError type: 0
display: 0x8514800
serial no: 36
err code:
(BadAccess (attempt to access private resource denied))
req code:
minor code:
resource id: 290
X Error: BadShmSeg (invalid shared segment parameter) 179
Major opcode: 149
Minor opcode: 2
Resource id: 0x220000c
2007-03-27 16:25:11.208 VideoOutputXv Error: Failed to create XvMC Buffers.
2007-03-27 16:25:11.214 VideoOutputXv: Falling back to X11 video output over a network socket.
*** May be very slow ***
This must be a MythTV issue, since XvMC works fine with mplayer. More to investigate.
Spent the afternoon following up the MythTV with XvMC problems, with no progress. The LIRC problem proved to be a naming issue: in Linux, it creates a socket /var/lirc/lircd, whereas the FreeBSD convention is to put it in /var/run/lirc/lircd. Ended up putting in a symlink; I fear there will be more such cases.
More work with MythTV and XvMC, still with no progress. Got a couple of replies to my question on the mailing list, but they were at the level of “reinstall your nVidia drivers”. Frustrating day.
Intended to take it quietly today, so didn't investigate the XvMC issues any further. Instead considered the problems I've been having with the TV guide from tvguide.org.au and considered the alternatives. One is Shepherd, which culls information from a number of places. Installation—on Linux—was not as simple as it looked. It needs a whole lot of Perl modules, and in the process of installation I discovered cpan, which sort of makes things easier. But some of them required compilation, and that failed. In the course of fixing that, I discovered:
Of course, all this probably wouldn't have been correct if the instructions had been structured correctly: after working my way through them, I found instructions on how to do it with Debian. Presumably they would have been easier.
Setting up shepherd was strange. It ran for ever, and at the end it didn't produce any output. The instructions are rather vague about what it's supposed to do, but I'm pretty sure it should populate the mythconverg.program table. But it didn't.
Another issue with shepherd is that it doesn't seem to know about Adelaide channel 31. Decided to shelve that effort and try IceTV instead. That installed better, but still required running through mythtv-setup and clobbering the channel table. Ended up writing MySQL queries to fix it again. It seems that this is a common problem. The instructions specified that you should edit the channel table in mythtv-setup, but there's no documentation on how to do that, and the screen itself is unclear. With MySQL it's clear what needs to be done.
Unfortunately, that only gave me data for two channels, probably (as occurred to me later) because of the hacks I had put in my MythWeb source to only select certain copies of the guide data. Also, channel 31 still wasn't there.
Time was coming up for new recordings, so reverted to the old configuration (aren't database backups a good idea?) and looked at the next item, remote controls.
The wiki user manual describes the default remote control key bindings, but there seems to be no configuration file to match them. There are five sample lircrc files in the source tree, but three of them don't even relate to MythTV, and of the other two, one is be for the Microsoft Media Center Edition Remote, model 1039, not exactly a typical remote control. Started writing my own, but then wondered if I had missed something. Asked on the #mythtv-users channel on Freenode and had a surprising reaction:
<groOgle> Is there really no default lircrc?
<Dagmar> There's no default remote, either.
<juski> why would there be a default lircrc?
<juski> there are like _millions_ of different remotes out there
<Dagmar> because thinking is WORK
<groOgle> Dagmar: So is guessing what commands the application wants.
<groOgle> juski: Strange that all remotes are so similar that VCR users can
use them.
<Dagmar> groOgle: Damn. You didn't buy the psychic add-on for your machine?
<Dagmar> hahaha
<Dagmar> Wow what breathtaking ignorance, er, innocence
<groOgle> Dagmar: No, just a less stupid attitude that some people show.
<kormoc> groOgle, there's defaults for some of the more common remotes
already...
<Dagmar> groogle; You're about to attempt to insult someone who knows a ton
more than you. I suggest you abandon this plan.
<groOgle> kormoc: What I find is in the configfiles directory.
<groOgle> kormoc: 5 files.
<kormoc> groOgle, that's the more common ones
<juski> groOgle: there are PLENTY of example lircrc files out there. use
google
<groOgle> Dagmar: Who might that be?
<groOgle> kormoc: 3 of them with default program irxevent.
<Dagmar> groOgle: Pretty much everyone in this channel if you think all
remotes are alike.
<Dagmar> ...and probably some of the people at the homeless shelter.
<juski> groOgle: if you knew an iota about lirc, you'd know that it'd be nigh
on impossible to have a common, likely to work out-of-the-box lircrc
file
<groOgle> juski: Not so.
<kormoc> groOgle, sure, there's standard remote layouts, play is typically
play, but uhh, it all depends on the lirc remote definition file,
which can have play as PLAY or play or Play or just PLA or just p or
so on and what not
<juski> nobody seems to call their buttons the same as what another guy called
them
<Dagmar> groOgle: hahaha
<groOgle> juski: Certainly "some aseembly required".
<Dagmar> suuure
<Dagmar> Yep. You're looking at "some assembly" with all those pre-made
config files
<juski> incidentally in the time we've been arguing the toss about it here you
could've made one already ;)
<groOgle> juski: But at the moment there's not even a layout that matches
what's in the wiki.
<groOgle> juski: No, I've been working on one for 30 minutes now.
<Dagmar> Gee, but I thought you said all remotes were the same...
<groOgle> Dagmar: No, you just didn't understand. Shall I explain it to you
again?
IRC seems to bring out the worst in people such as Dagmar, but this fruitless dialogue shows a number of other issues with projects like MythTV:
| Button | Function |
| RecTV | Alt+R |
| Guide | Alt+G |
The syntax requires that this be changed into the following entries:
begin
prog = mythtv
button = RecTV
config = Alt+R
end
begin
prog = mythtv
button = Guide
config = Alt+G
end
And as juski says, the names for the buttons vary from one remote control definition to the other. The names of the functions (“config”) also vary from program to program, which makes it particularly unhelpful to put in configurations for irxevent (which proves to be quite a useful program, one that I've been looking for). And so far I haven't found a definitive list of the names that mythfrontend is looking for.
Finally found a configuration file for a DVICO remote control by Chris Pascoe on Google, but it wasn't the same as the one I used. Set it up roughly, and it worked, but there's still plenty to be done.
Spent some time looking at why MythTV no longer compiles under FreeBSD 7-CURRENT. It proved to be a problem with the incredibly convoluted series of #include headers: a total of 137 header files and 16 levels of nesting. In the process, sys/time.h (included from the threads header file, of all places) was replaced by time.h. Somehow this can do with improving.
Didn't do much in the day, mainly watching TV. Spent a bit of time scratching my head about the problems I've been having with the remote control and XvMC in conjunction with MythTV. The former is mainly a documentation problem, but it would be nice to have an easier way of testing it: each change to the .lircrc requires stopping mythfrontend, which is horribly slow on startup (and not much better the rest of the time). Started a build on eucla, which of course took most of the afternoon.
The issues with XvMC look like more of a problem: it looks like it's going to require serious debugging in an area about which I know almost nothing. I suppose I'm going to have to bite the bullet and start investigating the X protocol.
Spent some time playing with the lircrc file for MythTV, and got some better results. One thing that I haven't seen documented anywhere is how to send multiple characters to the application: separate them with commas. I have no idea how to pass a comma instead. Here's what I ended up with for sending the digit 1 followed by the “Cursor right” key (skip 1 minute forward) to mythfrontend:
# Big (10x) skip forwards
begin
prog = mythtv
button = skip
config = 1, Right
end
Back to work on the black box today. One of the problems of the buffer rearrangement is that mplayer, instead of dropping behind the action, doesn't run at all. It seems that there is absolutely no provision in mplayer to wait for data to arrive if it gets to the end of a stream. This could happen, for example, if you skip forward while watching an incoming stream.
Spent all day looking at mplayer with an aim to get it to stop trying to seek past the end of a file, or to wait for data to arrive if it does. I was partially successful, but once again left with a sense of amazement about how this program is written. After checking the result of a read at EOF—which should return a read count of 0—I discovered I was getting -1 (error condition) instead. Of course mplayer doesn't check for error conditions—that's a wimp's way out—so I added code to print a message when this happened. The error code? 0, no error. After further searching, I found this gem in stream/stream-file.c:
static int fill_buffer(stream_t *s, char* buffer, int max_len){
int r = read(s->fd,buffer,max_len);
return (r <= 0) ? -1 : r;
}
I won't even comment about the layout of the code. All this function does is to:
There are others as well that seem more than a little dubious. In libmpdemux/demux_mpg.c, function demux_seek_mpg, I found:
while (1) {
if(newpos<demuxer->movi_start){
if(demuxer->stream->type!=STREAMTYPE_VCD) demuxer->movi_start=0; // for VCD
if(newpos<demuxer->movi_start) newpos=demuxer->movi_start;
}
Apart from the fact that this check doesn't need to be done multiple times, the comment and the code seem to contradict each other. The layout makes it not only difficult to read, but also confuses Emacs when it looks for start and end of the functions. In the end I reindented the functions I was working on to be able to read them at all.
Not feeling well again today, but struggled on with mplayer, which is still not starting the way I want it. Why is this program so fragile? I've been getting more messages that the X server doesn't support xv, but they seem to be related to some permission issue rather than lack of support; it all depends on which window I start mplayer from. Frustration.
Still more work on the black box. Sometimes our “testing” seems more like trial and error, which it most certainly should not. We now have the machine running reasonably well, but my modifications to mplayer don't seem to have been as successful as I had hoped. Surely it shouldn't be that difficult to get it to track an incoming stream without falling behind.
We still don't have mplayer tracking the camera correctly! Wrote a program to keep up with the input stream and feed it to mplayer via a pipe, but it was so spectacularly bad that it dropped 95% of the stream. Ran it against /dev/null instead with not much better results. Clearly I'm missing some important performance issue here, but it's not pipes—the same thing happens directly to /dev/null. Maybe I should hold off a bit.
The black box is finally on its way, so I can get back to the definitive MythTV installation under Ubuntu 6.10 (“Edgy Eft”) a month ago. Since then they've come out with a new release, 7.04, but generally called “Feisty Fawn”—what names! Tried installing that with less than spectacular success.
This time I tried it in graphics mode, which is effectively the default. It comes up with a GNOME screen and a couple of icons—and that's all. I suppose the GUI generation can instinctively (“intuitively”) know to double click on “Install”. Then an installation dialogue comes up, pretty much unchanged since the last release, including things like double selection of the time zone. On many screens (e.g. keyboard selection), there's no reaction to the mouse until you first move it away from where it is and then back again if necessary, a thing that I had noticed on previous versions.
I've already established that the performance of the ext3 file system is abysmal under certain circumstances: in particular, deleting large files takes for ever, up to 100 times longer than with ufs. ufs isn't a real option with Linux, of course, so selected a single XFS partition instead, along with 1.1 GB swap. After partitioning, I got a message entitled “Install the GRUB loader to XFS” and saying “The GRUB boot loader installation often fails or hangs when /boot is on a XFS file system. Using LILO in this situation is recommended. [Go Back] [Continue]”. Pressing Go back just continued. It gave no way to change the loader.
At the end of this install, the message was repeated. Again, selecting Go back did nothing. In fact, nothing at all happened. No information what to do next.
I rebooted, and it seems that LILO did get installed after all (or was just left behind from the previous installation). Saw many fatal modprobe failures during startup, along with a login prompt. While thinking about logging in, it started GNOME. The whole thing looked very flaky, and my USB mouse didn't work (it did during the installation). Rebooting didn't help. I then plugged in a PS/2 mouse, which worked immediately.
Decided to reinstall with the default parameters. The “Migrate documents and settings” stated “There were no users or operating systems suitable for importing from”. If the same version of Ubuntu isn't suitable, what is? It also completely ignored the existing layout and created a single ext3 77 GB root file system along with and 3 GB (!) swap. This time the installation completed with a message asking me to reboot. After reboot, the mouse worked.
I then installed for a third time, because I want XFS. This time, in addition to the other partition options, I was given the recommendation to resize the master partition to half the original size. Why? The previous installation was the default, and now it should be halved? What sense does that make?
I selected the “manual” partitioning again. This time, after getting the GRUB message, I answered “continue”, which didn't: I ended up in the same menu as before. I can't find any way to select the boot loader or its location. It seems that you have to press Go back to continue.
At the end of the installation, got the GRUB message again. Again I selected continue this time and got another message:
Executing 'grub-install (hd0)' failed. This is a fatal error.There was no further help; presumably the installer had crashed.
Finally, at the fourth attempt, I copped out and partitioned a 4 GB ext3 root file system and the rest as an XFS partition. That worked, but it's not what I want, and I'm only forced into it because of the limitations of the installation process.
Ubuntu is supposed to be a system for non-technical users. Does that mean that the installation process should only work properly in the default case?
XFS has given me other concerns as well. Over the last few days I've lost a number of recordings on ceeveear: the files were empty, and in a couple of cases there was no file at all. I assumed that there was some problem with initializing the tuner cards, and rebooted. That worked, but for some strange reason the second disk that I installed a week or so ago (with the XFS file system) came up running in PIO mode. That seems remarkably ancient; FreeBSD has selected DMA by default for nearly a decade. It's also not immediately clear from the hdparm(8) man page how to set DMA mode:
SYNOPSIS
hdparm [ flags ] [device] ..
...
-d Disable/enable the "using_dma" flag for this drive. ...
In fact, the -d option takes a parameter (0 to disable, 1 to enable), but this isn't described properly in the man page.
Today, on reboot, I entered the magic incantation:
# hdparm -d1 /dev/hdb
Immediately I was greeted by a stream of messages indicating a timeout on one of the tuner cards. Why? And why weren't they logged? Looks like a DMA channel conflict, but the only mention of DMA in the dmesg output relates to the parallel port, of all things. More head-scratching. Is the XFS? Probably not. This is also a pretty flaky motherboard, so maybe changing that would help.
Installed mythfrontend (display part) on Yvonne's computer (battunga) today. It wasn't as straightforward as I had hoped: although I have no intention of running a backend (tuner part) on this machine, and the database should be on a different machine, I couldn't run mythtv-setup because it wanted access to a local database. Ended up copping out and installing a MySQL server on battunga, for all the good that will do.
Did some looking to find the cause of the DMA problems I had with ceeveear yesterday, but they seem to have gone into hiding. Frustrating.
Got a message from James Andrewartha regarding my problems with the Ubuntu installation process:
The Ubuntu graphical installer is designed for non-technical people, and to only be run once. So it tries to preserve the existing filesystems by resizing them. The Migrate Documents and Settings feature is for people coming from Windows.In short, you should use the alternate install cd, which uses debian-installer and allows much more control over the process.
I still have a problem with this, and I don't completely agree with Andrew. If an installer installs a specific partitioning layout, it shouldn't want to change it when the installation program is run again, even if this is not intended use. Also, the download page doesn't mention any of an alternate install CD, and the Internode mirror server doesn't have the image. In fact, I'm not sure which image James is referring to. The only mention I see on the download page reads “Check here if you need the alternate desktop cd suited for computers with less than 256MB of RAM”. That's not the case here.
More mail from diary readers on the question of the Ubuntu installation process. Oliver Herold writes:
http://ftp.tu-chemnitz.de/pub/linux/ubuntu-releases/7.04/It looks like it's really halfway down:
just scroll down to the bottom.
Alternate install CD
The alternate install CD allows you to perform certain specialist installations of Ubuntu. It provides for the following situations:
PC (Intel x86) alternate install CD
creating pre-configured OEM systems;
setting up automated deployments;
upgrading from older installations without network access;
LVM and/or RAID partitioning;
installs on systems with less than about 256MB of RAM (although note that low-memory systems may not be able to run a full desktop environment reasonably).
For almost all PCs. This includes most machines with Intel/AMD/etc type processors and almost all computers that run Microsoft Windows, as well as newer Apple Macintosh systems based on Intel processors. Choose this if you are at all unsure.
OK, that's the correct alternate install CD, I suppose. As I mentioned earlier, none of the reasons apply to me. It's also just a CD image (696 MB) which means that a lot of the stuff on the installation DVD is missing. It's a pity that the documentation is so misleading.
Ian Donaldson writes about my problems installing mythfrontend on battunga:
You don't need to run mythtv-setup on a front end. Just run mythfrontend. When it can't connect to the default db (mythconverg on local) it should prompt you for database host, database name, user name and password.He's right; I was just misled by the continuous stream of error messages:
=== grog@echunga (/dev/ttypa) ~ 1 -> mythfrontend
2007-04-27 09:58:45.882 Using runtime prefix = /usr/local
xscreensaver-command: not found
2007-04-27 09:58:45.973 DPMS is disabled.
2007-04-27 09:58:46.012 Unable to read configuration file mysql.txt
2007-04-27 09:58:46.012 Trying to create a basic mysql.txt file
2007-04-27 09:58:46.023 Writing settings file /home/grog/.mythtv/mysql.txt
2007-04-27 09:58:46.222 New DB connection, total: 1
2007-04-27 09:58:46.515 Unable to connect to database!
2007-04-27 09:58:46.515 Driver error was [1/1045]:
QMYSQL3: Unable to connect
Database error was:
Access denied for user 'mythtv'@'localhost' (using password: YES)
QSqlQuery::exec: database not open
QSqlQuery::exec: database not open
2007-04-27 09:58:46.583 DB Error (KickDatabase):
Query was:
SELECT NULL;
No error type from QSqlError? Strange...
(new query)
2007-04-27 09:58:46.635 Unable to connect to database!
2007-04-27 09:58:46.635 Driver error was [1/1045]:
QMYSQL3: Unable to connect
Database error was:
Access denied for user 'mythtv'@'localhost' (using password: YES)
QSqlQuery::exec: database not open
I repeated the install on echunga and let it finish vomiting. 800 lines of these repetitive error messages later, it started mythfrontend normally. It's interesting to note that the first line of the error message is not the one after the empty line; that's in the middle of the message. The first line is Unable to connect to database!. I had tried to fix this nonsense earlier, but it's in a maze of twisty little qt functions, all different, so I gave up.
It's been nearly a month since I started a trial subscription to IceTV, as I mentioned at the beginning of the. I found a couple of reasons not even to use the service if it were free. As I later wrote,
People had suggested to me that the IceTV people are Internet-savvy (you'd expect that, wouldn't you, but so many are not), but I got no reply. Today I received a reminder that my trial subscription would soon expire, completely ignorant of my message. That's a pity: a commercial service needs to offer something that free services don't, and just a slightly more accurate description (and that only on average) doesn't make up for the lack of other information.
We've been discussing tuner drivers on the FreeBSD-multimedia mailing list. It's a thing I don't know much about, and so I decided to first find some documentation. This is multimedia, of course, so there's not too much. The page on the linuxtv wiki should be the place to look for that sort of thing, but it's very deficient, and so I took a look at the Wikipedia page, which was also a bit flaky, so spent some time tidying that up. It still doesn't have the kind of information I'm looking for (otherwise I wouldn't still be looking for it).
Spent some time today further researching documentation for tuner cards. I could have sworn that I had found something somewhere, but despite the fact that I normally save such links, I couldn't find anything. It seems to me that nowadays the most successful people are those who can search the web most efficiently. While that's definitely an advantage, it seems that limiting the requirement to search for everything should be taken more seriously.
In that spirit, decided to dissect and document FreeBSD's bktr driver, and to my surprise made relatively good progress. That begs its own questions, of course: what's the best way to document a driver?
More playing around with XvMC today, and established definitively that on tv2.lemis.com, the new AMD64 box I'm putting together. It works with mplayer and not with MythTV—exactly the same results as on teevee. What was different, though, was the amount of CPU time that the “standard” MythTV used: on teevee, an Athlon XP 3000+ (2.171 GHz) it maxed out the processor, while on tv2, an Athlon 64 3500+ (2.210 GHz) it used less than 50% of the CPU time. I'm more than a little puzzled by that and wonder if there's a kernel difference that accounts for it. I use INVARIANTS in many kernels, though not, it seems, in this case: teevee is using the GENERIC kernel, while tv2 is using a custom kernel (confusingly called TEEVEE; I'll rename the machine when it's running properly). Still, it looks good, so maybe I should go and look after the currently unsupported sound hardware after all.
Yesterday's success with MythTV on tv2.lemis.com had one minor downside: the sound hardware was still non-functional. Followed up on Oliver Herold's recommendations and installed the development versions of the new sound drivers and the nfe driver. Both suffer from inadequate documentation: the sound drivers come in binary form and work pretty well as described, but they're a complete replacement for all 32 sound drivers, and I had to find out by trial and error that the driver I needed was snd_hda:
pcm0: <NVidia MCP51 High Definition Audio Controller> mem 0xfead8000-0xfeadbfff irq 22 at device 16.1 on pci0 pcm0: <HDA Codec: Realtek ALC883> pcm0: <HDA Driver Revision: 20070505_0044>
The if_nfe driver was more of a problem: it came as a series of patches with unknown base directory, and it didn't include the configuration file information. For both of these I had the 7-CURRENT source tree handy, so I was able to put it into the kernel tree, but it could have been done much more easily. It also didn't build. I think it was intended to be added independently of the kernel, but that didn't make too much sense. The kld module did build, so used that instead.
The module worked well enough to find the hardware and bring it up:
miibus1: <MII bus> on nfe0 ciphy0: <VSC8601 10/100/1000TX PHY> on miibus1 ciphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto nfe0: Ethernet address: 00:19:db:33:c0:07
Unfortunately, I couldn't communicate with other systems. arp showed that the interface was sending (other machines saw its Ethernet address), but it didn't receive anything (arp didn't see their Ethernet address). The interface isn't currently important to me, since I have plenty of plug-in cards, so decided to leave this one.
In the evening, more surprises: teevee.lemis.com can now also display ABC 720p images via MythTV with less than 50% processor; previously it was maxed out and jerky. I wonder what has changed.
One thing that did change was the default encoder. Yesterday I had added SBS World News to the list of channels, in the hope of getting some more information about the French election, but for some it didn't work: MythTV was unable to extract the information from the stream. It's not a tuning issue, since it's in the same stream as the normal SBS programme. More investigation needed some other time. Today, though, I discovered that it had caused MythTV to decide to record from encoder 8, the el-cheapo analogue TV@nywhere card. That had never worked properly, so decided to delete the entry from the database.
What a pain MythTV is! Yesterday's deletion of the database entry for the analogue TV card wasn't really necessary, and to my surprise I discovered that it had already recorded a programme with the card, so it wasn't as dead as I thought. Put it back in again and started up mythbackend and—it went catatonic. It wasn't until many hours later that I made the connection with trying to make a recording with the card and the lack of reaction from the back end.
Finally found a TV programme which maxes out teevee.lemis.com and runs with less than 50% CPU on tv2.lemis.com. The big difference is in the system time (almost none on tv2, over 50% on teevee), so you'd think it was related to a kernel build parameter, but that doesn't explain why it only hits some programmes, and in particular not when displaying with XvMC.
I've been meaning to buy some new multimedia electronics for a while, and spent a lot of today looking for a new camera and a new data projector. It looks like the wrong time for both: the upcoming Olympus E-510 looks a lot better than its predecessor, the E-500, but currently the prices are still 50% higher. In six months that could be completely different.
The other interesting thing is that finally HDTV (1920x1080, or “1080p”) data projectors are appearing, presumably because Blu-Ray technology requires them. They're also still too expensive, but maybe that will change faster than the prices of current crop of 1280x720 (“720p”) projectors, whose prices have been relatively unchanged since I bought mine 2 years ago.
I suspect that both they and Blu-Ray will have an interesting effect: finally the Microsoft world will migrate away from 1024x768 resolution which has been the de-facto standard for nearly 20 years. That will exacerbate the problem with web browsers which I have ranted about from time to time: Microsoft will be forced to allow their browsers to scale text over a wider range of resolutions, and many more people will see the broken rendering that I complain about all the time.
Talking with Edwin Groothuis about that on IRC. He recommended SeaMonkey, so installed it for the fun of it. Of course it didn't do any better than any other browser—quite the contary. It immediately came up with a screen saying “updates are available” (and displayed in a way that I couldn't copy the text to the cut buffer). For the fun of it I selected “download now” and got the largest window I have ever seen:
Absolute upper-left X: -484 Absolute upper-left Y: 265 Relative upper-left X: 0 Relative upper-left Y: 0 Width: 4787 Height: 2504 ... Corners: +-484+265 --2255+265 --2255--1233 +-484--1233 -geometry 4787x2504+-489+244
That's larger than the resolution of most digital cameras. It consisted mainly of margin, and it took a while to realize that what content was present was off to the right of the screen. And yes, of course there was no way to download the updates.
| Greg's home page | Greg's diary | Greg's photos | Copyright |