Having installed DSL on a Neoware CA5 thin client with its 200MHz SiS processor I was surprised by the responsiveness of the desktop environment. It was perfectly usable and far better than I remembered PuppyServer to be on a 300MHz GX1. DSL seemed to be worth exploring more deeply than I had in the past.
This is not intended as a straight 'how to', it's more of a 'why, what and how to'. The first bit of describes how DSL works (at least how I think it works!) so that we can decide how we want to install it. In building up my understanding of exactly how DSL works I decided that I'd end up doing this project in three phases:
Read on to find out why, what and how.
The driver for this particular project was the fact that I was interested in benchmarking the performance of a number of the thin clients I have. As I often use these as small web servers it seemed a good idea to use that application as a metric and measure throughput: pages/second, bytes/second served and so on. I found a simple perl script that I altered to suit my needs. (See the Tools page). I already had a number of installations I could check, but there were other variants - some ran under DSL, some under Tiny Core, some Compact Flash based, one disk based. I decided it would be best to build a single Compact Flash based solution that I could move across the various bits of hardware. By selecting DSL it would also provide an interesting comparison with my DSL hard disk based version.
Rather than just throwing something together I thought I'd be a bit more methodical about it - hence this article. (Writing it down does the focus the mind!)
DSL supports two methods of installation; hard disk and frugal. The former is a conventional installation whilst the latter is more suited to Compact Flash as all the write operations occur in RAM. It is only when you come to shut the system down that you have the option of committing any changes to the Compact Flash card.
I have several IDE-to-CF adaptors, so my media of choice was a Compact Flash card on the IDE interface. A lot of what you find here would be equally applicable to a pen drive on a USB interface. The IDE-to-CF adaptors come in various flavours - I have single ones and also double ones as well as a number of Compact Flash cards left over from my early days of digital photography. These range in size from 32MB to 2GB. One possibility might be to use a dual-socket adaptor, use one 64MB card for the frugal DSL install, and use the second socket for the persistent storage. In any case, whether it is one large CF card partitioned into two or two smaller CF cards, it makes no difference other than in the references to /dev/hdxx for the persistent storage.
What we're interested in here is the 'frugal' install method as that is better suited to Compact Flash based systems. The best way to explain how a 'frugal' DSL is structured is to have a look at a running system. Here I've taken a 128MB Compact Flash card (/dev/hda), created two partitions on it, installed DSL to /dev/hda1 and formatted /dev/hda2 as an ext2 file system to use for persistent storage.
If we start by seeing what the installation has put into /dev/hda1...
KNOPPIX: -rwxr-xr-x 1 root root 50569003 Aug 23 21:01 KNOPPIX boot: drwxr-xr-x 2 root root 1024 Aug 23 21:02 grub -rwxr-xr-x 1 root root 1005209 Aug 23 21:01 linux24 -rwxr-xr-x 1 root root 860643 Aug 23 21:01 minirt24.gz /lost+found:
So we have the usual /boot directory which contains our kernel (linux24), initrd (minirt24.gz) and grub. We also have a directory /KNOPPIX which holds a single 50MB file of the same name. This file is actually the base DSL file system. It is saved in a compressed format known as a cloop file. It is ~50MB on disk, but, when mounted, appears to have a size of ~122MB. Note this is actually a read-only file system. What ever we do on our running system - adding/deleting system files, doesn't actually change anything in this file. Changes are all handled through the union file system (see on).
In the grub directory we have the usual menu.lst file which tells grub where to find our kernel & initrd, and passes a variety of additional information on the command line to the various startup scripts.
When DSL boots, various scripts effectively put the system together from several components. A good way to see what has happened is to run the ls command on the root directory to see what we have there.....
drwxrwxr-x 19 root root 4096 Jul 9 2008 KNOPPIX lrwxrwxrwx 1 root root 12 May 17 2004 bin -> /KNOPPIX/bin lrwxrwxrwx 1 root root 13 May 17 2004 boot -> /KNOPPIX/boot drwxr-xr-x 5 root root 1024 Aug 23 21:01 cdrom drwxr-xr-x 3 root root 100 Aug 26 11:36 dev drwxr-xr-x 54 root root 340 Aug 26 11:42 etc lrwxrwxrwx 1 root root 13 Aug 26 2010 home -> /ramdisk/home lrwxrwxrwx 1 root root 12 May 17 2004 lib -> /KNOPPIX/lib drwx------ 2 root root 1024 Mar 24 2006 lost+found drwxr-xr-x 11 root root 1024 Aug 26 11:36 mnt lrwxrwxrwx 1 root root 12 Aug 26 11:35 opt -> /ramdisk/opt dr-xr-xr-x 36 root root 0 Aug 26 2010 proc drwxrwxrwt 12 root root 240 Aug 26 11:35 ramdisk drwxr-xr-x 2 root root 1024 Aug 26 11:35 root lrwxrwxrwx 1 root root 13 Aug 26 2010 sbin -> /KNOPPIX/sbin drwxr-xr-x 2 root root 1024 Jan 12 2004 sys lrwxrwxrwx 1 root root 12 Aug 26 11:35 tmp -> /ramdisk/tmp lrwxrwxrwx 1 root root 12 May 17 2004 usr -> /KNOPPIX/usr lrwxrwxrwx 1 root root 12 Aug 26 2010 var -> /ramdisk/var....and the mount command to see what file systems we have and how/where they are mounted.....
/dev/root on / type ext2 (rw) /dev/hda1 on /cdrom type ext2 (rw) /dev/cloop on /KNOPPIX type iso9660 (ro) /ramdisk on /ramdisk type tmpfs (rw,size=355416k,size=351100k) /proc/bus/usb on /proc/bus/usb type usbdevfs (rw,devmode=0666) unionfs on /KNOPPIX/bin type unionfs (rw,dirs=/ramdisk/bin=rw:/bin=ro) unionfs on /dev type unionfs (rw,dirs=/ramdisk/dev=rw:/dev=ro) unionfs on /etc type unionfs (rw,dirs=/ramdisk/etc=rw:/etc=rw) unionfs on /KNOPPIX/lib type unionfs (rw,dirs=/ramdisk/lib=rw:/lib=ro) unionfs on /KNOPPIX/sbin type unionfs (rw,dirs=/ramdisk/sbin=rw:/sbin=ro) unionfs on /KNOPPIX/usr type unionfs (rw,dirs=/ramdisk/usr=rw:/usr=ro) unionfs on /ramdisk/var type unionfs (rw,dirs=/ramdisk/var=rw) /dev/hda2 on /mnt/hda2 type ext2 (rw,nosuid,nodev)
Interpreting this is not easy. Glossing over some of the complexities and taking a simplistic view....
We know the KNOPPIX file (which we know to be read-only) has been mounted as our root file system. The /dev/cloop entry (line 3) shows it was mounted ro on /KNOPPIX. We also have a ramdisk (/ramdisk/ - line 4). The ls listing shows that the standard system directories: /bin /boot /lib /sbin /usr are links to the corresponding directories in the KNOPPIX file. The mount entries for all of these show that these are union file systems - a combination of the respective directories in the KNOPPIX file and ones in the /ramdisk/. This is how our read-only base file system now becomes read-write. For example any changes we make to /lib happen in /ramdisk/lib as that is read/write. We don't actually change anything in /lib directory in the KNOPPIX file as that is read-only. Whenever we access /lib the unionfs presents us with a view made up from the the files in the KNOPPIX /lib directory and those in /ramdisk/lib.
The partition holding our system (/dev/hda1) has been mounted on /cdrom.
If we look back at the ls listing we can see that other standard directories - /home /opt /tmp /var - have all been linked to identically named directories in the ramdisk.
So, overall, almost all 'writes' in our file system happen in the ramdisk. The only exceptions are if we access the root partition via the /cdrom mount point, or access our persistent storage that is on another partition (hda2).
As things stand, when we poweroff, all changes are lost and we have to start from scratch the next time we power up. (Depending on your point of view this is either a good thing or a bad thing). DSL gets round this with the concept of the mydsl directory on some persistent storage - such as another partition on the Compact Flash (hda2 in this example). It is in that directory that DSL stores the necessary information that it needs to recreate the environment that you're currently running. For example, if you've installed several applications, you'll find the xxx.tar.gz, xxx.dsl, xxx.unc or xxx.uci files stored here. Every time you boot the system, the DSL startup scripts will automatically integrate these applications into the base system.
DSL applications come packed in four ways:
The first two can consume large amounts of memory whilst the .uci and .unc types don't. i.e. With the first two types the application is 'unpacked' and all its files are copied into the ram disk. With the last two types the application's files appear as a mounted file system and nothing is copied to the ram disk. Obviously, with both types, (more) ram is consumed when the application is loaded into memory and run.
The reason for working through this simple explanation is to point out installed applications can consume RAM as the various binaries and libraries may end up in the ramdisk after they are unpacked. This may not matter, but if you are running on a Thin Client with limited RAM, then there is a lot to be said for remastering your copy of DSL if they only exist in tar.gz or .dsl formats.
With remastering you expand the KNOPPIX file into a conventional file system, add/remove/change what you want, and then recreate the KNOPPIX file. Having done that the application's files are now in the KNOPPIX file on /dev/hda1 and no longer appear in the ramdisk.
For example, to illustrate the issue of actual memory use, I used the 'MyDSL' to install the samba.dsl and gcc1-with-libs.dsl packages. Having done this, if we look in the persistent /mnt/hda2/mydsl directory we can see:
-rwxr-xr-x 1 dsl staff 26 Aug 26 11:35 MyDSL -rwxrwxr-x 1 dsl staff 15 Aug 26 11:35 Samba -rw-rw-r-- 1 dsl staff 20352409 Jun 5 2007 gcc1-with-libs.dsl -rw-rw-r-- 1 dsl staff 53 Jun 5 2007 gcc1-with-libs.dsl.md5.txt -rw-rw-r-- 1 dsl staff 143535 Nov 29 2008 mydslinfo.bz2 -rw-rw-r-- 1 dsl staff 143535 Nov 29 2008 mydslinfo.bz2.1 -rw-rw-r-- 1 dsl staff 4419536 Oct 15 2007 samba.dsl -rw-rw-r-- 1 dsl staff 44 Oct 16 2007 samba.dsl.md5.txt
So that's where the packages sit. As mentioned before, on installation or power-up DSL unpacks these and puts the various files in the right places - /usr/bin /usr/lib etc.
If we look explicitly at the ramdisk (du -s -h /ramdisk/*) to see how much memory the ramdisk is using we get:
0 /ramdisk/bin 0 /ramdisk/dev 76.0k /ramdisk/etc 496.0k /ramdisk/home 2.0M /ramdisk/lib 396.0k /ramdisk/opt 0 /ramdisk/sbin 24.0k /ramdisk/tmp 86.3M /ramdisk/usr 12.0k /ramdisk/var
We can see that the ramdisk part of /usr is using 86.3MB of RAM!
Now if we 'uninstall' gcc by deleting it from the mydsl directory and reboot:
0 /ramdisk/bin 0 /ramdisk/dev 72.0k /ramdisk/etc 492.0k /ramdisk/home 0 /ramdisk/lib 396.0k /ramdisk/opt 0 /ramdisk/sbin 24.0k /ramdisk/tmp 15.1M /ramdisk/usr 12.0k /ramdisk/var
The /ramdisk/usr usage has dropped by 71MB. Also another 2MB has gone from /ramdisk/lib. (Ok, the gcc example was a bit extreme, but it does illustrate the point!).
Getting rid of the samba installation as well gets us to:
0 /ramdisk/bin 0 /ramdisk/dev 76.0k /ramdisk/etc 484.0k /ramdisk/home 0 /ramdisk/lib 388.0k /ramdisk/opt 0 /ramdisk/sbin 8.0k /ramdisk/tmp 1.8M /ramdisk/usr 12.0k /ramdisk/var
...another 14MB saving!
So from this we can see that, if you are intending to use DSL on systems with limited RAM and applications that don't exist in uci or unc format, you either need to convert them that format or remaster your system so they are part of the core KNOPPIX file system.
Next we'll look at installing DSL onto Compact Flash and then we'll move on to remastering.
Any comments? email me. Last update August 2010