electronics/soekris

Notes on the Soekris net6501 router; see also the official wiki.

The Hardware

The onboad gigabit ethernet controllers are the Intel 82574L, which use the e1000e driver on linux (not e1000).

More info about the Atom e6xx CPU and co-processor here and here.

Firmware updates are available from the soekris website.

The miniPCIe WiFi card I have for use with this device is the Intel 4965AGN.

When specifying the boot device on the bootloader command line (accessible over the serial port), the ordering is:

0x80 = primary drive
0x81 = secondary drive
0x82 = tertiary drive
0x83 = quaternary drive
0x84 = quinary drive (etc.)
0xF0 = PXE boot (netboot)
0xFF = stop and go to comBIOS prompt

The connector ordering (labeled on the PCB) is:

1. SATA Port 0 (J1)
2. SATA Port 1 (J2)
3. PCI Express connector (J3)
4. PCI Express riser (J4)
5. mSATA 0 socket (J5)
6. mSATA 1 socket (J6)
7. external USB connector (JP3)
8. internal USB connector (JP5)

Installing Debian wheezy on a SATA disk

See old debian 5.0 directions on the Soekris wiki.

The procedure is to boot from a debian stable USB stick, partition the SATA disk and install stable on to that, then boot into the stable image and do a distupgrade to testing (wheezy).

To get the syslinux-based USB stick to boot correctly, need to edit txt.cfg and set the console and partition settings:

default install
label install
    menu label ^Install
    menu default
    kernel linux
    append vga=normal initrd=initrd.gz -- quiet console=ttyS0,38400 earlyprint=serial,ttyS0,38400

Boot with serial console set to 38400 baud. In the soekris bootloader, select USB stick device:

> boot 81

Start up the installer. Wait for it to detect all drives and load components; ignore the kernel module warning. Ignore the no ethernet card warning. At that point “Go Back” which brings up the system menu.

Installing pfSense on a SATA disk

See also FreeBSD 8 on Soekris net6501

These directions assume your host/work machine is linux and that you have a serial console connection to the soekris (eg, USB adapter and minicom).

Grab a “memstick-serial” snapshot image from snapshots.pfsense.org, flash it to a USB stick:

$ dd if=pfSense-memstick-serial-2.1-DEVELOPMENT-i386-20120720-0129.img of=/dev/sdb

Boot with serial console set to 38400 baud. In the soekris bootloader, select USB stick device:

> boot 81

Switch serial console to 9600 baud. Terminal output from the pfSense bootloader will be clunky/garbled, wait for it to try to draw, maybe press enter a couple times. When the kernel actually loads the output will be clearer.

After kernel loads, system will ask if you want to do install or boot. I recommend that you do a boot, with basic configuration, then enable SSH and complete the installation over a network session, as the ncurses installer interface will be much easier to read. You can enable SSH and start the installer from the terminal prompt that appears on every terminal session.

Hopefully the install is self-explanitory. I created a 4GB “low-level”/slice partition for pfSense (which got split into a 3GB partition and a 1GB swap partition), a 16GB Linux/ext2 kFreeBSD partition (didn’t create filesystem yet), and the rest of the space as another Linux/ext2 partition; depending on your use case and disk type you probably want to do something different. I selected embedded kernel.

Install kFreeBSD in pfSense

To get debian/kFreeBSD running in a jail within pfSense, first the pfSense userland needs to be updated to a full FreeBSD 8.3 install.

Starting from a functional pfSense 2.1 install, login to the web console and enable SSH access. We will copy over sysinstall and dependancies from a FreeBSD 8.3 LiveCD. From this point on DO NOT reboot until the end.

Mount and copy over all the libraries required for sysinstall from FreeBSD 8.3 .iso (try ldd /usr/sbin/sysinstall to get a list):

scp libdialog* libncurses* libutil* libftpio* libdevinfo* root@192.168.1.1:/usr/lib/
scp sysinstall root@192.168.1.1:sysinstall8

Run this sysinstall. Go to options and set “Release Name 8.3-RELEASE” (not -p3). Install the base, ports collection, and system kernel source (/src/sys).

Before rebooting, use the pfSense web interface to upgrade to the most recent development image; this will revert to the pfSense version of the kernel and configuration, but doesn’t remove the vanilla FreeBSD userland stuff.

Now, reboot and hope everything comes back up with no disk errors.

A few modules need to be installed to support kFreeBSD. Go to /usr/src/sys/modules, and for each of the following, enter the directory and make && make install:

linux
linprocfs
fdescfs
linsysfs
tmpfs
ext2fs

pkg_add -r any desired packages. I also installed a linux_base, not sure if it was necessary:

pkg_add -r linux_base-f10

Inspect /boot/loader.conf and make sure it’s sane (?).

Reboot again and hope everything comes back up with no disk errors. Then we are ready for jail configuration.

If you want the jail filesystem to be on a seperate partition, create a linux ext2 filesystem:

pkg_add -r e2fsprogs
mke2fs /dev/ad6s2
mount -t ext2fs /dev/ad6s2 /jail/debian/

Following directions from blog.vx.sk:

mkdir -p /jail/debian
debootstrap wheezy /jail/debian http://cdn.debian.net/debian

The probably took a long time.

Add rc.conf.debian to /root/:

jail_enable="YES"
jail_list="debian"
jail_debian_rootdir="/jail/debian"
jail_debian_hostname="guest0.rooter.is"
jail_debian_ip="127.0.0.1"
jail_debian_devfs_enable="YES"
jail_debian_exec_start="/etc/init.d/rc 3"
jail_debian_flags="-l -u root"

Add start_debian.sh to /root/ to get the jail up and running:

#/bin/sh
kldload linux fdescfs linprocfs linsysfs tmpfs
mount -t ext2fs /dev/ad6s2 /jail/debian/
mount -t linprocfs linprocfs /jail/debian/proc
mount -t linsysfs linsysfs /jail/debian/sys
mount -t tmpfs tmpfs /jail/debian/lib/init/rw
ifconfig em1 alias 192.168.1.201/32
cp /root/rc.conf.debian /etc/rc.conf
/etc/rc.d/jail start debian
jls

Run that script, then try running bash in the jail:

[2.1-BETA0][root@rooter0.rooter.is]/usr/src/sys/modules/tmpfs(42): jexec 1 /bin/bash
root@debian0:/# uname -a
GNU/kFreeBSD debian0.rooter.is 8.3-RELEASE-p3 FreeBSD 8.3-RELEASE-p3 #1: Wed Jul 18 19:29:09 EDT 2012     root@FreeBSD_8.3_pfSense_2.1.snaps.pfsense.org:/usr/obj./usr/pfSensesrc/src/sys/pfSense_wrap.8.i386 i386 i386 Genuine Intel(R) CPU        @ 1.00GHz GNU/kFreeBSD

Horray!

To allow ping from inside kFreeBSD, add “security.jail.allow_raw_sockets=1” to /etc/sysctl.conf in pfSense. There are some other tips and gotchas on the FreeBSD wiki.