The leaves are falling, or, what is coming in 0.2

The leaves are changing colors, so I've decided to make a progress post. Work on evoke has been on and off, mainly due to this being effectively a one man show. When I get busy, the development simply halts, even though I continue to use the cds I generate on a daily basis. There were also some bugs with nexusd (more on that later) and the general build environment, that was drawing the remaining free time away. Hopefully I will be able to work on evoke more in the coming months, as the holidays are approaching.

I'd also like to take this time to thank John (jeev), for donating two new mirrors for evoke. The mirrors are hosted at USColo and Lomag Internet Services. The update system has already been bumped, so the update system should already be using them. They can be found here, and here.

I've been working on portions of what will become 0.2. We've replaced init with nexusd (a hybrid of init, watchdogd, and eventually powerd and devd). This means that now evoke has a 'single user mode', to bypass systart. You will probably never need to do this, however, now if you accidentally select it, the system won't panic, it will just drop you into a shell.

I've also added 8.0 to the image, and added a 'kernel only' option, so 0.2 will be released with 8.0 and 7.2 as kernels, but will share the 7.2 userland. There are also some cluster related additions in the works, but they will probably not be usable for this release.

Autologin commands, which finally allow evoke to be used outside the 'administrator's toolkit', are now supported. in your user directory, just add an 'autologin' file, chmod +x it, and sysconfig commit current /mem/sysconfig a few times (yes, I'll fix that before release). Right now evoke can be used as an evoke bootserver, if you set up the autologin file by hand. Before 0.2 release, we will make this part 'automagic' (within reason).

As for X.org, well, with recent changes to Xorg (and some changes, which are not in ports, but will also be a massive shakeup), it is difficult to build X into the image without overflowing the boot time size limit for memory disks. Unfortunately, Linux based systems have the luxory of KDrive/TinyX, which on FreeBSD, does not work too well. Unless someone wants to assist with finding a solution to either problem (getting kdrive Xvesa working, or fixing the memory disk overflow), I don't see X being on 0.2. One of our biggest strengths come from using a memory disk in all environments for the base system. While we could cheat and mount the cd for X support, it's something I'd rather not do. We would lose a lot, just to get a GUI environment. This is considering that in all cases, we would be running unaccelerated, it's just not worth it.

If you'd like to take a sneak peak at these changes, I've uploaded the iso image to the mirrors, here. Please test, and report any regressions under the issue tracker. Remember, you can't fix a bug you don't know about!

As always, we are looking for developers, testers and users, for assistance, feedback, and any resources you may think we'd have a use for. Even if you just occasionally use evoke, we'd love to hear your comments, complaints, and suggestions!

Announcing Evoke 0.1R1

Description

0.1 is the first official 'release' of Evoke, It represents the first step into our ultimate goal of improving the UNIX userland. This release contains scaffolding and general structural improvements, including the binary updater that will support releases in the future, the sysconfig utility that provides a backing store for key data, and various other build time improvements to streamline the development.

While 0.1 may be underwhelming, it was a necessary step to lead the way to 0.2, which will begin the changes on user and data management, including defining what a 'datastore' is, what functions the librarian will have, and what api is exported to userland.

Please report any bugs discovered running this or any other release to the online issue tracker, available here:

• http://code.google.com/p/evoke/issues/list

It is recommended that if you use evoke in any capacity, to join the following lowtraffic mailing list:

• http://groups.google.com/group/evoke-security-advisories

We also invite you to join the mailing list for technical discussions, and suggest any features or discuss any issues you may have:

• http://groups.google.com/group/evoke-tech

Highlights:

• Collision free. Evoke can be installed alongside FreeBSD, PCBSD, and DesktopBSD, and can be booted by adding a single line to /boot.config. (Evoke's updater can also activate different releases).
• Many versions of Evoke may be installed on the same hard drive, and chosen by the mechanism mentioned previously.
• update - Bandwidth efficient binary updater, that supports full file fallback and ensures that only 'correct' releases get committed to disk.
• installer - a beta-level utility, that is capable of installing both FreeBSD and Evoke, and supports GPT partitioning.
• verify - a file verification utility that supports writing and signing a single file describing a filesystem hierarchy, and is used by the updater to verify releases to detect malicious and accidental file tampering.
• doc - a basic documentation utiity that supports a search interface.
• sysconfig - a robust system for storing key data, verifies all data for consistancy and supports fallback to previous records in case of silent data corruption.
• mounter - Flexible replacement for 'mount', including a transparent URL-like interface to native filesystems, fuse filesystems, and automatic creation of md device nodes when files are specified.
• Ganglia Monitor Daemon preinstalled, activated by an 'evoke.monitor=yes' in /evoke/misc/site.conf
• Preinstalled venti utilities, including the venti server, vbackup, vcat, vac, unvac, and vacfs.
• Many filesystems, including ufs, cd9660, udf, ext2, xfs, reiserfs, tmpfs, http/https, ftp, ntfs, fat16/fat32, cryptofs, ssh/sftp, vac, and 9p,
• Preinstalled programs, including ntpd, netcat, openssh, bsdiff, bspatch, fuse, pv, nano, nload, makefs, mkisofs, cdrecord, dmidecode, and many more.

Mirrors:

• HTTP:
  • http://www.damnsmallbsd.org/pub/evoke/misc/ISO-IMAGES/0.1/r1/

ISO Image Checksums:

• MD5 (evoke.iso) = 0e5929d9ede0c5963b5c956c9be9844e
• SHA256 (evoke.iso) = e2e1db1bbc0026694c2be8a6fc5fe065d6365b77d502d1ddb4497635da621586
  
  

Call For Testers: installer/update

This is a general call for testers for the installer/updater. Please download this iso, and run 'installer'. It is highly recommended you do this on a spare machine, with no important drives plugged in. Also take heed of the warning, the installer only supports dedicated installations. It will gladly overwrite anything on the drive you choose.

Any bugs, please post them in the issue tracker. Barring any serious bugs, 0.1R1 will go live two weeks from today, April 7th 2009. Please test this image!

(Edit: The iso suffered from a recent bug in the bootloader, which prevented cds from being booted. I've replaced it with an iso with working bootcode. Please use this one (r913) instead.)

Down to the wire

It always seems like when you're getting closer to releasing something, a ton of interesting ideas flood into your brain. However, since we need to get 0.1 out soon so it will at least get tested by some people, they have to get filed away for the next release.

Right now the installer is the big ticket item that is preventing the release,  an ssh server bug (introduced recently), an update bug that causes it to hammer the update server, and the lack of proper timezone support round things out.

My ideas for the next release are:

• Clean up our idea of boot server, and per-host configuration. Right now you can configure via sysconfig and site.conf, but the site.conf is only intended to store /site-wide/ configuration. I actually have a solution, which is to use the freebsd 'loader.conf' dhcp option, and have loader use that if it's available, and if it's not pxe0, use /evoke/local.conf. In this way, we will support group-wide or host-specific local.conf's, to configure machines. This will also separate the boot server config from the hosts, and allow hard drive boots to work as expected.
• Research using llvm to lower the per-platform footprint. Right now adding another platform doubles the size of the release. It should really be something other then O(n).
• A GUI, be it Xorg, Kdrive, or another windowing system entirely. Unfortunately, Xorg seems to suck up space and can be ... difficult to use in a heterogeneous environment.

I will update this page with other ideas to be filed away for later.

SSHD re-enabled

SSHD has been reenabled in evoke. After the last issue with loader.conf, I disabled sshd. I've decided to enable it if and only if an ssh key is specified in site.conf, with the evoke.sshkey kernel environment variable is specified. Note, that the old behaviour of dynamically generating a host key will be retained if either a userconfig and sysconfig partition is not specified, or useonly is not chosen. The host key is generated if it does not exist, and is then copied to the userconfig and sysconfig partititions, before starting the daemon. Then the user can commit the sysconfig partition and store the host key for each boot.

Due to a limitation in loader, there is only one key that can be specified by this means. In the future, it will merge the contents of this environment variable, and the keys specified by the userconfig partition, at least for the administrative account.

The missing piece

Update has been coming along nicely, it now has a menu, activation support (and it prompts you after an update install if you would like to activate). The two big missing pieces, are supporting installing from filesystems mounted by mounter (effectively multiplexing getfile), and installer creating the evoke partitions. And this second part, I've been leaning more towards using GPT rather then MBR style labels by default. The only issue is the gpart utility, however, gpt is available on 7.x, so I may just use gpt until the time where the gpart utility grows the functionality to replace gpt.

So, my question to you (users and potential users alike), is if you had a choice, would you rather use GPT starting fresh, or have the default MBR? Note, that both will always be available, the evoke system is flexible enough to handle both transparently. This is just the default, preferred method.

And here is a gratuitous screenshot of the new update menu.

The update utility has liftoff.

The grunt work for the update utility is now complete. It aggressively tries to keep the amount of data that has to be downloaded to the bare minimum necessary, as I detailed in my previous post. Unfortunately, that means it takes a longer time to do the update (every step of the way, it verifies the file against the trackfile, if it fails, it keeps going, stubbornly trying to get the correct files for the release)

Here's a screenshot:

There are a few missing features, there is no menu interface for example (that will come later), but the update functionality is complete, and I will be testing it, on every build I queue, to make sure it works as intended.

Updating Evoke

Recently, with the changes to the build system in place, I've been looking at another of the big ticket items: the installer. Now, I added another bit to that entry; before we have a working installer in the image, I would like to have an 'update' command, for system administration. I've seen first hand what not having a binary update facility in place at the start can do to a project (the obsessive buildworld/buildkernel path many people still take with freebsd), so I'd rather handle that case first.

I've been researching using bspatch/bsdiff (the same mechanism in use by freebsd-update), with a fallback mechanism to grab the files directly. Since we support multiple concurrent versions, we don't have to worry about a lot of the headaches that freebsd-update (and installworld) has to do fancy footwork to avoid.

Glossary:

• trackfile - a file containing a list of files for the release, and their associated sha256 hashes. People wondered why I did that when we didn't even have a working kernel. My mind is about 2 years ahead of the current system right now.

Workflow:

1. update fetches the trackfile for the target release from the mirrors.
2. We use the sha256 hash of the destination file, to build a url like so:  http://www.damnsmallbsd.org/pub/evoke/BIN-UPDATES/${HASH}/
3. We go through the trackfiles for any currently installed releases, starting with the most recent timestamped directory (as it's most likely to match), and attempt to fetch it, based on the source hash, for example: http://www.damnsmallbsd.org/pub/evoke/BIN-UPDATES/${HASH}/${SOURCEHASH}.
4. If we can fetch the file, we check to see if the destination or source has a .gz, or .bz2 extension. If it does, we first uncompress it to the staging area.
5. We now run bspatch over the source, using the downloaded file, saving it to the staging area.
6. If the destination's extension was .gz or .bz2, we compress it.
7. Now, we sha256 the destination file, and compare it to the entry in the trackfile. If it matches, then we go to step #9.
8. If we can't get the file, or the resulting file is corrupt, we go through each available release and repeat the process until we hit the oldest one. If the oldest one fails, then we fall back, to grabbing the file directly, for example: http://www.damnsmallbsd.org/pub/evoke/0.1/R1/loader.conf. Then we go to step #4. We also set a bit to detect the end of the logic, if this method does not work, we error out and tell the user something is broken.
9. Since we've verified that the file was not corrupted in transit, and that the patch applied cleanly, put it in another part of the staging area.
10. Once all the files in the trackfile are done, move the files out of the staging area onto the boot filesystem.
11. Allow the user the option of activating the downloaded release by calling installer.
12. Follow installer's mo, sync the disks, disable all write caching, and write the /boot.config for the active filesystem. Sync. re-enable write caching, done. Rinse, Repeat.

Any feedback is more then welcome :) Please leave a comment if you see any serious holes.

Doc 2.0

After hitting some inflexibilities with google code's #pragma's, I reimplemented the search algorithm for doc. Now, attributes are stored between wiki:comment tags, and multiple names/types/author/programming-languages can be specified there, and the doc utility uses a better way of finding matching pages.

It should be identical to the previous 1.0, in regards to appearance. If there are any bugs, please let me know.

Preparing for 0.1R1

We are getting even closer to the 0.1R1 release. I've been spending time fixing a bug with building 6.4 ports on 8-CURRENT (which lacked KSE), and I've also been adding some features which have been missing up to this point. A big one is the addition of 'doc', a way of viewing some of our Wiki page documentation while running the image. Until now, we've had zero inline documentation, which in my mind is a pretty big problem. I'll continue to flesh out some of the documentation up until the release time.

There are a few more big ticket items that need to be worked on before the release. I'm considering not bothering with anything resembling a -BETA[0-9] or -RC[0-9], and just releasing an 'R1', and fix any big problems in R2, R3, etc, in addition to normal security updates.

When (if) the installer target for evoke is finished, we will also have to add an 'update' command which will simply fetch the trackfile from one of the mirrors, and fetch the most recent Revision of that branch. Until the installer works, however, this is not necessary; as installing to a hard drive or thumb drive would still be unsupported.

Since this is our first 'true' release, it's difficult to gauge where the line is, so I'm going to be revising this part up until the release is made. Since we do complete versioning, with a seperate 'activation' step, we have a lot more freedom in this regard.

Userconfig

Originally, sysconfig was going to extract everything, and to have more then one user, you would have more then one sysconfig partition. However, I realized that this was adding unnecessary inflexibility, and user's may wish to have multiple 'users', which they can log in as at will.

Therefore, I created userconfig, which allows you to create, list, and login as a user. Each user is stored in the sysconfig partition, in a directory named after the user's UUID. This was chosen to avoid the collisions inherent in using a login name, or an incrementing id number. Users may have the same login name, as long as their UUID is different, and even have the same real name (If you want, you can have 100 users in a sysconfig partition named 'John Smith', and nothing will break.).

The userconfig utility mirrors sysconfig, and has the added bonus of supporting transparent password based encryption via cryptofs or encfs. Note, that this is not 'remote login', as this will only encrypt the real 'keyring' data stored in the directory specified by N_CURUSER and soon, HOME (note, we may make them seperate, due to some assumptions made by processes using HOME, but N_CURUSER will always point to this directory).

But that will come later.

Both sysconfig and userconfig are enabled by default, and we are edging closer to a new release.

sysconfig

Conventionally, most UNIX's store their configuration information in the /etc directory on the root filesystem. This has numerous downsides:

• The system configuration is accessed like a conventional root filesystem, meaning that if the disk drive fails, or the controller/bridge, the system configuration is unusable.
• There is little to no redundancy, and rarely will you be able to recover from obvious consistency problems. As well, there's no way of rolling back to a previous version.
• The system configuration info is almost always required to exist on the same filesystem as the boot system. This means that you have to implement hacks such as unionfs to have a running system from a read-only root, for files that are normally modified, such as /etc/resolv.conf, or /etc/mtab on SYSV systems.

After musing on these fundamental weaknesses, I came up with a simple solution, which has now been personified in sysconfig. Rather then creating a filesystem which we mount conventionally, and demand load, the sysconfig utility extracts the most recent timestamped record, which is also consistent (we use a cryptographic hash algorithm for verification, due to the collision resistance), and copies the contents to ram, which is then used.

This has several benefits, one of which is that we always have a log of configuration information, and we can fall back in case a write was incomplete. We also choose the most recent based on a time stamp, so we don't have to do any dependency checking (if we were to store a 'live' field in the header, we'd have to update the header each commit.. In this case, it's unnecessary). The directory is stored in disk as effectively a gzipped tar archive, and the data within the archive is currently free form. The header stores a version number, and uses name=value pairs with newline deliminators, allowing us to store more information in the header for 'free'.

Right now, sysconfig is not run by systart. This is because of the nature of sysconfig, which will require some testing before I am confident. If you'd like to test, please download this file, chmod a+x it, and attempt to create/commit/extract a sysconfig partition. Please report any issues, especially any that do any damage to disk filesystems, to the issue tracker.