New Release! Fixed Bugs!

Hi All!

The latest and greatest One2One2Go image can be downloaded here:
http://people.sugarlabs.org/~it/One2One2Go.iso.

By modifying casper.conf during the image creation process, I resolved the autologin errors.

There are still major modifications to be made to facilitate fast user switching and to speed up the boot process, but this release is a solid backbone off of which we can build.

Best,
--
Ian

Shared Workstation Image

To address boot-speed issues associated with older hardware, Luke Faraone and I have designed what we call the "shared workstation image" or SWI for short.

The intended result is a continuously-running system which, upon the insertion of a flash drive, links /home and /etc/passwd to the /home and /etc/passwd contained in the persistence file. This will allow a user to log into the system and have access to all their user-specific configuration files and documents in a near seamless fashion.

I have not yet been able to test the system we designed, as I've been preoccupied with boot speed. As a result, I'd sincerely appreciate technical assistance with this project. With this in mind, here's what we have


as the contents of /usr/local/bin/fastswap...

mountPoint=$(mount | grep $0 | awk '{print $3}')

if [ -a $mountPoint/casper-rw]; then

    mount -o loop $mountPoint/casper-rw /mnt

    mount --bind /mnt/home /home

    mount --bind /mnt/etc/passwd /etc/passwd

else

    echo "Already mounted"

fi

contents of /etc/udev/rules.d/fastswap.rules

ACTION=="add",KERNEL=="sdb*", RUN+="/usr/local/bin/fastswap %k"

ACTION=="remove", KERNEL=="sdb*", RUN+="killall gdm-session-worker"

contents of /etc/gdm/PostSession/Default

#!/bin/sh

umount /etc/passwd

umount /home

umount /mnt

DISPLAY=:0.0  zenity --info --timeout=10 \

          --text="Your removable storage device can now be safely removed from the system."

# TODO: use the device name in the message above

exit 0

Speedy Boot

A live CD or USB key starts up using syslinux, casper, then upstart. My efforts to speed up the boot process have been centered around grapling with these tools.

Syslinux is the first-stage bootloader, starting a Linux kernel from a FAT filesystem like our sticks. Casper is a set of scripts packaged in the initial ramdisk file(initrd) which facilitate booting off of live media. Upstart is the designed-for-Ubuntu startup manager. 

All of these software systems have significant room for improvement and specialization, especially Casper. 

Out of the box, casper is designed to provide the best possible user experience for an Ubuntu livecd. This, however, does not mean that it is well-documented or efficient. In this case, it is neither. I have been unable to turn up official or unofficial documentation on the internals of casper.

The Casper scripts are located in a folder aptly titled "scripts" in the root of the initrd. They are split up by category, with the scripts located in "casper-bottom" doing most of the work, and the rest serving largely as hooks to other boot steps.

In the "scripts/casper-bottom" folder, there are scripts for everything from checking the integrity of the boot media to configuring x. Many of the scripts located in casper-bottom are single-purpose. I'm having a hard time figure out what to keep and what's acting as an unnecessary load on the startup process.

In addition, there are a few scripts in casper-bottom that are actively interfering with progress. Scripts "10adduser," "15autologin," and "25configure_init" appear to be responsible for a bug I've observed. Upon boot, Linux TTYs 0-6 are locked down with security errors. These scripts appear to be attempting to automatically log in the [non-existent] user "ubuntu" to these terminals. For deployment and testing purposes, we most definitely do not want third parties have access to user's account. I've been working on figuring out how[and where!] to disable the autologin without breaking casper or weakening some other aspect of the system.

Another significant time-suck in both the boot and development process is the upstart system. De-crufting the startup tasks requires more knowledge of upstart than I have right now, and would greatly improve the startup speed and user experience. Many of programs started on boot are only occasionally used by the average user. On a modern computer, this results in a baseline increase in RAM usage. On a live system, this can result in an unacceptable slowdown.

Tool for Comparing Two Directories

I'm struggling with initrd patches to improve boot-speed and durability and have found myself needing to compare two directories.
For future reference, this command, specifically for initrd images, will do so.

diff -qr $DIR1 $DIR2 | grep -v modules | sed -e 's/ and / /g' -e 's/Files //g' -e 's/ differ//g' | xargs -L 1 diff

Floppy Drive Errors

Often, on boot of a livecd, one gets errors related to '/dev/fd0.'

The solution for this error is to blacklist the floppy-drive module, preventing its use during boot.

To do this, extract the initial ramdisk file and run the command

"echo 'blacklist floppy' >> etc/modprobe.d/blacklist.conf "

and rebuild.

Technical Tour-de-Stick

If you are not interested in the technical underbelly of this project, it would be best if you skipped this post. Also, use the information below at your own risk. I've done my best to make sure that it is correct and up-to-date, but there may be places where I'm off-base.

Before I delve into initrd and casper, I figure I'd best post an explanation of the files on the stick. To do this, I'll explain what's on my development stick at present.

I ran the command "find -maxdepth 3" from the root of my stick and pasted the result below. 
I annotated the paste with comments above the file/folder they describe.


#used by syslinux for first-stage booting
./ldlinux.sys
./casper/
#a list of the installed packages
./casper/filesystem.manifest
#a list of the installed packages discounting those used for installation
./casper/filesystem.manifest-desktop
#a compressed, read-only filesystem containing a copy of the O121 image
./casper/filesystem.squashfs
#the initial ramdisk image, compressed. more about this below
./casper/initrd.gz
#the linux kernel
./casper/vmlinuz


#a folder containing an extracted copy of the the initial ramdisk
./casper/initrdMod/
#contains shared files and application libraries used in the boot process
./casper/initrdMod/usr/
#contains the casper boot scripts. This folder deserves(and will get!) its own blog post.
./casper/initrdMod/scripts/
#contains the binaries used in the boot process
./casper/initrdMod/bin/
#contains more esoteric binaries used in the boot process
./casper/initrdMod/sbin/
#contains low-level drivers, libraries, and kernel modules
./casper/initrdMod/lib/
#contains configuration files for the boot process
./casper/initrdMod/conf/
#contains configuration files for the kernel and programs used in the boot process
./casper/initrdMod/etc/
#the main boot script
./casper/initrdMod/init


#the scripts used to extract and rebuild the initrd image
./casper/modScripts/
#gzip -dc ../initrd.gz | cpio -id

./casper/modScripts/extract
#mv ../initrd.gz ../initrd.$(md5sum ../initrd.gz | awk '{print $1}').gz


#find . | cpio --quiet --dereference -o -H newc | gzip -9 > ../initrd.gz
./casper/modScripts/rebuild
#the initial initrd file, kept around for good luck and occasional versioning
./casper/initrd.da1cb8c239c5130882ffa888e128639d.gz
#a previous version of the initrd file
./casper/initrd.5c6912e1d862a9cb67e3e0a833dcc260.gz


./install/
#a version of the Linux kernel designed to test ram
./install/memtest
#smart boot manager - http://linux.simple.be/tools/sbm
./install/sbm.img
#a large file containing an ext3 filesystem, used as a boot-time overlay for persistence 
./casper-rw
#contains misc metadata files
./.disk/
./.disk/base_installable
./.disk/cd_type
./.disk/info
./.disk/release_notes_url
./md5sum.txt
./README.diskdefines
#empty file used to claim this image as an ubuntu derivative
./ubuntu


./syslinux/
./syslinux/boot.cat
#the isolinux bootloader
./syslinux/isolinux.bin
#the isolinux config file - isolinux is called "syslinux" when used on a USB drive
./syslinux/syslinux.cfg


For more information, a full "find" tree of my development drive can be found here. 

Best!

--

Ian

Progress!

Made tons of progress yesterday.

The Main Stick Image

• now boots fully on the Haitian Coalition machines with wonky graphic cards
• doesn't block login on consoles 1-6
• shutsdown cleanly
• doesn't probe the floppy drive on boot

All of these changes were made by modifying the initrd.gz image, except for the graphics issue, which was resolved by disabling 3d hardware acceleration.

I'll be posting the casper hacks I made to the comments section of this blog in the next few days.

The Fastswitch Image

• needs work, but progress is no longer being blocked by the above

The Boothelper Image

• no longer requires userinput to boot, this eliminating any need for USB keyboards to work 

Backup

• was proof-of-concept'ed yesterday as I programatically installed dropbox on a One2One2Go drive with the sync directory set to /home

Keep an eye out for an ISO containing the main-image fixes in the next few days!

Summary of Technical Work

The software for the upcoming pilot with the Haitian Coalition has four main pieces.

 

The Main Stick Image

The software for the stick is largely completed, as the build scripts to create an image from the open1to1 harddrive files have been written. However, there are bugs with our implementation of casper and with the video drivers for the donated computers. The casper bugs are related to casper's autologin and autohardware check. Both of these interfere with a normal use case and slow boot time. 

There are four possible kernel arguments to resolve the driver issues, and the best approach to fixing this problem appears to be a syslinux.conf file which poses all of the options as selectable items. This means I won't have to rebuild every time I want to try a new approach.

The casper problems are more challenging. I haven't found good documentation for the purpose and contents of the scripts involved, and while they're written in BASH, this does not mean they are easy to read. Fortunately, other individuals on the internet have encountered similar problems, and there are some solutions posted. However, due to casper's role in the initial ramdisk image, modifying casper necessitates a lengthy rebuild from scratch.

 

The Fast Switch image.

Luke Faraone and I worked together to design a solution to allow for students to switch between computers in a lab very quickly. The basic premise is that as the students don't have access to modify the stick past their /home directory, a simple over-mounting of the /home from the persistent file should function to allow students to access all the files on their stick in a transparent manner.

For more information, see my notes at on SVN at http://open1to1.org/svn/trunk/build-scripts/fastswitch_notes.txt.

 

The Boothelper image.

As the BIOS of older machines, like many of the P4s we're using in our pilot, don't support booting from USB, some form of alternate bootloader is needed. For this purpose, we're using the "PLOP" Bootmanager, customized to detect USB devices and attempt to boot off them in a mostly-silent manner. The ISO image for this is available here.

One issue with this is that the boothelper does not support USB keyboards. As, in normal operation, no keyboard entry should be needed, this isn't really a problem. However, this makes debugging difficult.

Backup.

The backup solution designed for this pilot is dropbox-based. Keep an eye out for a blog post about that soon!

In other news, the most recent image can be downloaded as an ISO from my sugarlabs page. This file has both Spanish, French, and Haitian creole language packs, as well as a plethora of educational software. The md5sum for this file is f6d8724693c90ac9b2b758b7730a5604.

Best!

--

Ian