Greg's Setting up a new FreeBSD system

Note: this page is not just under development, it's mainly wrong. Please don't use it in this state; I'll remove this header when it's ready for use. In particular, I've almost completely changed how I do things.

Preparations

/var subdirectories to keep on /home:
- cron
- mail
- db/mysql

Installing new version of system, second system

Install FreeBSD on second (backup) system.
Install ports (see below).
dump the backup system.
restore to second root partition on primary system
On primary system, edit /etc/fstab and /etc/rc.conf.
On primary system, ciRCS
On primary system, /destdir, check out any RCS stuff.
Reboot primary system with new root partition.

After rebooting

The following problems need attention:

Reconfigure X.

Move xearth:

# cp -p /destdir/usr/X11R6.9/bin/xearth /usr/local/bin
# cp -p /destdir/usr/lib/compat/* /usr/lib/compat

Rename firefox:

  # ln -s /usr/local/bin/linux-firefox /usr/local/bin/firefox

Create spool directories for printer.

groff: set the paper size correctly in /usr/share/groff_font/devps/DESC:

--- DESC        2006/01/18 05:42:02     1.2
+++ DESC        2008/02/10 23:38:16
@@ -10,5 +10,5 @@
 tcommand
 postpro grops
 broken 7
-papersize letter
+papersize a4
 print lpr

Remove cups conflicts

For acroread:

#  ln -s /usr/bin/lpr /usr/compat/linux/usr/bin

More old stuff

I set up enough new systems that it's becoming a pain to do all the routine things. This list is primarily for me so that I don't forget anything, but it might be useful to others as well. It's currently anything but complete, but will be updated in the course of time.

This example shows the way I use to upgrade a system wantadilla to a newer version of the system. Basically it completely replaces the /usr, /var and root file systems: it creates a new disk with the upgrade software and then installs the old configuration.

Preparations before setup

Assuming you're upgrading a system or making something very similar to another machine (in other words, cloning), you can make things easier like this:

Maintain all config files in RCS under /home/RCS. Symlink the RCS directories of each directory of interest; e.g.
```
# cd /etc
# ln -s /home/RCS/etc RCS
# ln -s /home/RCS/etc/named named/RCS
```
Pay special attention to /etc/master.passwd and /etc/group. They should be kept under RCS like other files, but to aid updating, they should only be appended to. That makes it much easier to merge.
Maintain a list of installed ports. pkg_info doesn't distinguish between explicitly installed ports (the ones you want) and implicitly installed ports (the ones installed as dependencies). This list should specify the name of the ports install directory.
```
/usr/ports/shells/bash2
/usr/ports/editors/emacs
/usr/ports/x11/XFree86-4
```

Upgrade installation, different disk

Install FreeBSD from a recent ISO image onto a disk with a root partition the same size as the destination disk.

Upgrade to latest version of FreeBSD with

# make buildworld
# make kernel
# make installworld

Meanwhile, install Ports Collection:

# mkdir /src
# mount eureka:/src /src
# ln -s /src/CVS/FreeBSD/ncvs /home
# export CVSROOT=/home/ncvs
# mv /usr/ports /usr/ports.orig      # or remove them
# cd /usr
# cvs co ports

Once the new kernel is installed, install bash and Emacs manually.
XXX think about groff fonts (/usr/share/groff/devps)

ensure that the RCS files are checked in on the old system:

# /home/Sysconfig/scripts/ciRCS      check in RCS files (on old system)

Then, on the new system:

# ln -s /src/Sysconfig/mysystem /home/Sysconfig
# /home/Sysconfig/scripts/installRCS -f create new RCS hierarchy
# cd /
# /home/Sysconfig/scripts/coRCS         and check them out

Update files in $DESTDIR/etc:

# cd $DESTDIR/etc
# rcsdiff -wu RCS/* 2>/dev/null | less

Fix any conflicts.

Special cases:

aliases: Try just adding stuff at the end. Maybe it'll DTRT.
crontab: Merge new stuff if you want.
dumpdates: Should be empty in the distro. Replace with local file, but beware if the file system hierarchy is changing.

fstab:

Check the entries for / and any other file systems on the install disk.
Comment out /compat mount point in /etc/fstab. We need to install Linux compatability first.

Then check in the updated version of /etc/fstab: Afterwards you should see:

--- fstab       2005/07/08 15:31:01     1.22
+++ fstab       2005/07/11 03:17:28
@@ -1,7 +1,7 @@
 # $Id: newsystem-old.php,v 1.13 2010/09/06 02:11:36 grog Exp $
 # Device               Mountpoint      FStype  Options         Dump    Pass#
 /dev/ad0s2b            none            swap    sw              0       0
-/dev/ad0s2a            /               ufs     rw              1       1
+/dev/ad0s1a            /               ufs     rw              1       1
 /dev/ad0s3h            /home           ufs     rw              1       2
 echunga:/cdrom         /cdrom          nfs     ro,noauto       0       0
 proc                   /proc           procfs  rw              0       0
@@ -10,13 +10,13 @@
 echunga:/              /echunga        nfs     rw              0       0
 echunga:/home          /echunga/home   nfs     rw              0       0
 echunga:/src           /src            nfs     rw              0       0
-linproc                        /compat/linux/proc  linprocfs   rw      0       0
+# linproc                      /compat/linux/proc  linprocfs   rw      0       0
 /dev/ad2h              /dumpa          ufs     rw              0       2
 /dev/ad3s1h            /dumpb          ufs     rw              0       2
 echunga:/dump          /dump           nfs     rw              0       0
 # blackwater:/         /wantadilla     nfs     rw              0       0
 # blackwater:/home     /wantadilla/home  nfs   rw              0       0
 # blackwater:/dumpa    /dumpa          nfs     rw              0       0
-/dev/ad0s1a            /destdir        ufs     rw              2       2
+/dev/ad0s2a            /destdir        ufs     rw              2       2
 teevee:/               /teevee         nfs     rw,noauto       0       0
 teevee:/spool          /teevee/spool   nfs     rw,noauto       0       0

group: Add new groups. The existing ones probably don't need change.
inetd.conf: Add new entries and check any differences for the existing ones.
master.passwd: Careful comparison. Probably this is just a matter of adding new userids, but things like home directories and shells could change for the system userids. Note that if you change master.passwd, you'll have to regenerate /etc/passwd after rebooting to the new system.
Note the alternative of building the password file at boot time
printcap: Check, but you can probably keep the local one unchanged.
rc.conf: make sure that host name and IP don't conflict with any other system in the network. This won't happen if you're upgrading a system, but it could if you're cloning.
resolv.conf: Keep old version.
shells: Keep new version. It'll get updated by the shell installs if needed.
syslog.conf: Merge.

If the root shell is in the Ports Collection, install it now, or make sure that there's a root user with Bourne shell in the password file. Normally toor will do that. To install the shell,
```
# /home/Sysconfig/scripts/installport /usr/ports/shells/bash2
```
Don't install any other ports at this stage: the ports database gets put in the wrong place.
Update databases:
```
# pwd_mkdb -p $DESTDIR/etc $DESTDIR/etc/master.passwd
# newaliases
```
This currently doesn't work: it claims cross-device links and crap. Do it after rebooting, but make sure that you will be able to log in.

Check out any files missing in /etc:

# rcsdiff -wu RCS/* 2>&1 >/dev/null | grep "No such file"

For each file, decide what to do. A brute force method is:

# rcsdiff -wu RCS/* 2>&1 >/dev/null | grep "No such file"|sed 's/rcsdiff:/co -M /; s/:.*//'|sh

For the last time, check in the current RCS-controlled files:
```
# /home/Sysconfig/scripts/ciRCS
```

Create the RCS links, and check out those which won't be in the way:

# cd /home/Sysconfig/MasterRCS
# .  ../scripts/linkRCS
# .  ../scripts/coRCS

Make mount points for file systems in /etc/fstab:
```
# .  mkmounts
```
Reboot.

Upgrade installation, same disk

For updates on the same system, add the new disk to the system, say /dev/ad2.
Partition and label disk. sysinstall is probably the easiest for this. Create a root file system of 8 GB and a swap partition big enough to be bigger than RAM for the life of the system image. 1.1 GB or 2.1 GB come to mind as sizes. Note that I don't recommend a /usr or /var file systems. If you have a good reason, you can deviate for /var, but it doesn't make sense for /usr.
Mount the file system as /destdir (for your definition of /destdir)

If the /home file system will be different, mount it and start synchronizing it now:

# mount /dev/ad2s3h $DESTDIR/home
# for i in Book Makefile OED PHRA Texts andrew core etc finanzamt grog hiten house jdunham lemis martins mikal monkey nanyang ncvs norm oldRCS oldx postgres reiterhof release rms root seren setuid var waveform wkt yana yvonne;
  do echo Syncing $i; rsync -Halv $i $DESTDIR/home; done
# rsync -Halv /wantadilla/home $DESTDIR/home
# cd /var
# rsync -Halv backups games mail rwho spool $DESTDIR/var

# export DESTDIR=/destdir
# cd /usr/src

At this point at the latest, make sure that the host system has any new users and groups that the target system will need for installation. chmod and friends always use the local system credentials, so if you don't, you'll get messages like

ERROR: Required proxy user is missing, see /usr/src/UPDATING.

Continuing,

# make world
# cp -p /boot/device.hints $DESTDIR/boot/device.hints
# make kernel
# cd /usr/src/etc
# make distrib-dirs  (until we update the Makefile)
# make distribution
This won't currently work across NFS because pwd_mkdb wants to use flock.
Find a solution.

XXX think about groff fonts (/usr/share/groff/devps)

Install most of the RCS files. If going from one disk to another, ensure that the RCS files are checked in on the old system:

# /home/Sysconfig/scripts/ciRCS      check in RCS files (on old system)

Then, on the new system:

# /home/Sysconfig/scripts/installRCS -f create new RCS hierarchy
# cd /
# /home/Sysconfig/scripts/coRCS         and check them out

Update files in $DESTDIR/etc:
```
# cd $DESTDIR/etc
# rcsdiff -wu RCS/* 2>/dev/null | less
```
Fix any conflicts, in particular /etc/group and /etc/master.passwd. If you change /etc/master.passwd, note that you'll have to regenerate /etc/passwd after rebooting to the new system.
If the root shell is in the Ports Collection, install it now, or make sure that there's a root user with Bourne shell in the password file. Normally toor will do that. To install the shell,
```
# /home/Sysconfig/scripts/installport /usr/ports/shells/bash2
```
Don't install any other ports at this stage: the ports database gets put in the wrong place.
Update databases:
```
# pwd_mkdb -p $DESTDIR/etc $DESTDIR/etc/master.passwd
# newaliases
```
This currently doesn't work: it claims cross-device links and crap. Do it after rebooting, but make sure that you will be able to log in.

Check out any files missing in /etc:

# rcsdiff -wu RCS/* 2>&1 >/dev/null | grep "No such file"|sed 's/rcsdiff:/co -M /; s/:.*//'|sh

For each file, decide what to do.

Fix /etc/fstab:

Swap the entries for $DESTDIR and /.
Comment out /compat mount point in /etc/fstab. We need to install Linux compatability first.

Then check in the updated version of /etc/fstab: Afterwards you should see:

--- fstab       2005/07/08 15:31:01     1.22
+++ fstab       2005/07/11 03:17:28
@@ -1,7 +1,7 @@
 # $Id: newsystem-old.php,v 1.13 2010/09/06 02:11:36 grog Exp $
 # Device               Mountpoint      FStype  Options         Dump    Pass#
 /dev/ad0s2b            none            swap    sw              0       0
-/dev/ad0s2a            /               ufs     rw              1       1
+/dev/ad0s1a            /               ufs     rw              1       1
 /dev/ad0s3h            /home           ufs     rw              1       2
 echunga:/cdrom         /cdrom          nfs     ro,noauto       0       0
 proc                   /proc           procfs  rw              0       0
@@ -10,13 +10,13 @@
 echunga:/              /echunga        nfs     rw              0       0
 echunga:/home          /echunga/home   nfs     rw              0       0
 echunga:/src           /src            nfs     rw              0       0
-linproc                        /compat/linux/proc  linprocfs   rw      0       0
+# linproc                      /compat/linux/proc  linprocfs   rw      0       0
 /dev/ad2h              /dumpa          ufs     rw              0       2
 /dev/ad3s1h            /dumpb          ufs     rw              0       2
 echunga:/dump          /dump           nfs     rw              0       0
 # blackwater:/         /wantadilla     nfs     rw              0       0
 # blackwater:/home     /wantadilla/home  nfs   rw              0       0
 # blackwater:/dumpa    /dumpa          nfs     rw              0       0
-/dev/ad0s1a            /destdir        ufs     rw              2       2
+/dev/ad0s2a            /destdir        ufs     rw              2       2
 teevee:/               /teevee         nfs     rw,noauto       0       0
 teevee:/spool          /teevee/spool   nfs     rw,noauto       0       0

Check rc.conf: in particular, make sure that host name and IP don't conflict with any other system in the network. This won't happen if you're upgrading a system, but it could if you're cloning.
For the last time, check in the current RCS-controlled files:
```
# /home/Sysconfig/scripts/ciRCS
```

Create the RCS links, and check out those which won't be in the way:

# cd /home/Sysconfig/MasterRCS
# .  ../scripts/linkRCS
# .  ../scripts/coRCS

Make mount points for file systems in /etc/fstab:
```
# .  mkmounts
```
Remove any DESTDIR definition in the checked-out files.
Reboot.

New installation

Install base system from CD.
Upgrade to latest -STABLE.
NFS mount environment, create /home/Sysconfig hierarchy:
```
# ln -s /src/Sysconfig/<systemname> /home
```

Reinstalling ports

Current kludge, in /home/Sysconfig:

awk < installed-ports '{printf ("cd %s; %s %s %s %s %s %s %s %s\n", $7, $8, $9, $10, $11, \
$12, $13, $15, $15); }'|uniq > installports
mailme sh installports

After rebooting

This is all wrong. To be fixed.

If you've changed your /home file system, copy it from the original now.
Copy the directories in /var that you need: Probably: backups log crash postfix rwho spool/postfix spool/lpd Probably not: db run tmp
XXX everything below here is from an earlier attempt, and most of it is wrong. It's left for comparison purposes only.
Set up DNS for the new machine.
Install a system from the latest CD-ROM, including bash and emacs.

Mount the local NFS server,

# rm -rf /usr/src
# ln -s /src/FreeBSD/5-CURRENT-WANTADILLA /usr

mergemaster -p
make buildworld
Create a config file. Involves creating a symlink from src/sys/i386/conf to /src/FreeBSD/configfiles/RCS, then check out and change the name. Don't forget to check in before building the kernel.
make kernel, reboot.
make installworkd
Copy /etc/RCS/* to new system.

# cd /etc
# mv group group.orig
# mv master.passwd master.passwd.orig
# co group master.passwd shells
# diff -wu group group.orig
Merge differences, notably new groups
# diff -wu master.passwd master.passwd.orig
Merge differences, notably new users
# pwd_mkdb -p /etc/master.passwd
# (cd /usr/ports/shells/bash2; make install)
# (cd /usr/ports/editors/emacs21; make install)
# co ntpd.conf
# cp -p /usr/share/zoneinfo/Australia/Adelaide localtime

R=/wantadilla               remote system name
cp $R/root/.bashrc $R/root/.bash_profile  $R/root/.emacs ~root
(cd $R; tar cf - usr/local/share/abbreviations usr/share/syscons/keymaps/au.emacs.kbd ) \
   | tar xvf -
cp -p /wantadilla/usr/local/share/emacs/site-lisp/* /usr/local/share/emacs/site-lisp

Run mergemaster again:

rm -rf /var/tmp/temproot/
find /var/tmp/temproot/|xargs chflags noschg
rm -rf /var/tmp/temproot/
# cd /ETC
# cp -p master.passwd group shells    /etc
# pwd_mkdb /etc/
# cd /usr
# mv src src.local
# ln -s /src/FreeBSD/5-CURRENT-ADELAIDE/src
# mailme mergemaster -ia

Maintain a list of the currently installed packages in ~/packages. To make it,

pkg_info > /var/tmp/packages

Edit out the ones you don't want (including any you don't know: they're probably dependencies). Then:

sed <packages 's:-[0-9].*$: \\; Make install:; s:^:echo cd /usr/ports/*/:' | sh >~/packages

This will give something like this:

cd /usr/ports/graphics/ImageMagick ; Make install
cd /usr/ports/x11/XFree86 ; Make install
cd /usr/ports/distfiles/acroread /usr/ports/print/acroread ; Make install
cd /usr/ports/*/apache ; Make install
cd /usr/ports/french/aspell /usr/ports/german/aspell /usr/ports/portuguese/aspell /usr/ports/russian/aspell /usr/ports/textproc/aspell /usr/ports/ukrainian/aspell ; Make install
cd /usr/ports/audio/audacity ; Make install
cd /usr/ports/*/bash ; Make install
cd /usr/ports/sysutils/cdrtools ; Make install
cd /usr/ports/misc/cpuid ; Make install
cd /usr/ports/devel/cscope ; Make install
cd /usr/ports/net/cvsup ; Make install
cd /usr/ports/textproc/docbook ; Make install
cd /usr/ports/textproc/docbook-xml ; Make install
cd /usr/ports/textproc/docbook-xsl ; Make install
cd /usr/ports/textproc/docproj ; Make install
cd /usr/ports/textproc/dsssl-docbook-modular ; Make install
cd /usr/ports/editors/emacs ; Make install
cd /usr/ports/audio/esound ; Make install
cd /usr/ports/net/ethereal ; Make install
cd /usr/ports/distfiles/exim /usr/ports/mail/exim ; Make install
cd /usr/ports/mail/fetchmail ; Make install
cd /usr/ports/misc/figlet ; Make install
cd /usr/ports/print/freetype2 ; Make install
cd /usr/ports/www/galeon ; Make install
cd /usr/ports/print/ghostscript-gnu ; Make install
cd /usr/ports/print/ghostview ; Make install
cd /usr/ports/*/gimp ; Make install
cd /usr/ports/finance/gnucash ; Make install
cd /usr/ports/security/gnupg ; Make install
cd /usr/ports/audio/gramofile ; Make install
cd /usr/ports/audio/grip ; Make install
cd /usr/ports/print/gv ; Make install
cd /usr/ports/french/ispell /usr/ports/german/ispell /usr/ports/hungarian/ispell /usr/ports/textproc/ispell ; Make install
cd /usr/ports/textproc/jade ; Make install
cd /usr/ports/*/kdebase ; Make install
cd /usr/ports/audio/lame ; Make install
cd /usr/ports/chinese/links /usr/ports/www/links ; Make install
cd /usr/ports/multimedia/linux-realplayer ; Make install
cd /usr/ports/emulators/linux_base ; Make install
cd /usr/ports/*/mplayer-gtk ; Make install
cd /usr/ports/chinese/mutt /usr/ports/distfiles/mutt /usr/ports/japanese/mutt /usr/ports/mail/mutt ; Make install
cd /usr/ports/distfiles/openoffice /usr/ports/editors/openoffice /usr/ports/french/openoffice /usr/ports/german/openoffice /usr/ports/japanese/openoffice /usr/ports/korean/openoffice /usr/ports/portuguese/openoffice /usr/ports/russian/openoffice ; Make install
cd /usr/ports/*/perl ; Make install
cd /usr/ports/sysutils/portupgrade ; Make install
cd /usr/ports/distfiles/postfix /usr/ports/mail/postfix ; Make install
cd /usr/ports/distfiles/postgresql /usr/ports/japanese/postgresql ; Make install
cd /usr/ports/mail/procmail ; Make install
cd /usr/ports/net/rsync ; Make install
cd /usr/ports/japanese/samba /usr/ports/net/samba ; Make install
cd /usr/ports/audio/sweep ; Make install
cd /usr/ports/mail/sylpheed ; Make install
cd /usr/ports/www/tidy
cd /usr/ports/converters/unix2dos ; Make install
cd /usr/ports/archivers/unzip /usr/ports/korean/unzip ; Make install
cd /usr/ports/textproc/urlview ; Make install
cd /usr/ports/x11-servers/x2x ; Make install
cd /usr/ports/sysutils/x86info ; Make install
cd /usr/ports/x11-clocks/xalarm ; Make install
cd /usr/ports/sysutils/xcpustate ; Make install
cd /usr/ports/astro/xearth ; Make install
cd /usr/ports/astro/xephem ; Make install
cd /usr/ports/sysutils/xmbmon ; Make install
cd /usr/ports/audio/xmixer ; Make install
cd /usr/ports/chinese/xmms /usr/ports/multimedia/xmms ; Make install
cd /usr/ports/chinese/xpdf /usr/ports/graphics/xpdf /usr/ports/japanese/xpdf ; Make install
cd /usr/ports/sysutils/xsysstats ; Make install
cd /usr/ports/x11/xtset ; Make install
cd /usr/ports/distfiles/xv /usr/ports/graphics/xv /usr/ports/japanese/xv ; Make install
cd /usr/ports/archivers/zip ; Make install

You need the space before the ; character so that the wild card expansion works correctly. Where more than one occurs, you need to work out which one is correct and remove the rest. Where a * remains, you need to go looking for the port.

Finally,

.  ~/packages