The Unity Linux Build Server

One of the new additions to the Unity Linux packaging process is the introduction of the Build Server (or buildserv for short) The main purpose of the buildserv is to provide users with the ability to package content following our established procedures without being burdened by the overhead of doing so. The buildserv really consists of two parts: a backend server and a frontend web interface.

How Does It Work?

The frontend is a secure web interface to our standard build environment. Once an account is created for the packager, they can log into the buildserv and, with a few simple clicks, be able to select any of the packages they have recently committed to SVN, and schedule them for building. The actual RPM building happens by the backend server which processes the queued up requests.  With the help of our chroot-based standard build environments it generates the i586 and x86_64 arch RPMs automatically. The success or failure of this build, as well as the details of the output are then accessible to the user from the web frontend.

This takes care of the initial packaging aspects of the buildserv. However, the full capability of the build server is far from over. If the RPMs for the requested package(s) were successfully created, the backend server then automatically transfers the files into the Test channel of our repos and cleans up any duplicate/obsoleted RPMs. Moreover, the buildserv can then promote these RPMs from the Test channel to the intended channel once the RPMs are deemed stable and tested*.  In other words, much of the burden of tedious maintenance of packages while they are in the repos is removed from the shoulders of contributors, freeing them up to do more useful work.

Why Is the Buildserv Important?

Anytime you automate tasks there are benefits to those users and developers who would normally perform those tasks manually.  In this case, the Unity Linux developers and contributors will save the tedious tasks of  RPM building by the automated buildserv.  In other words, packagers will be able to package more because of a huge time saving.  This also could translate to more RPMs and packages for end users as developers may take on more packages due to the automation.  When end users request a package it will be a few clicks of the mouse to get it into testing.  This is a huge benefit for developers which is passed directly to end users through time savings.

The future of buildserv

There is still much work to be done on the buildserv.  Currently, many of the administrative tasks cannot be done over the web interface (will only be available for users with admin privileges); the web interface itself, was done with quite primitive HTML and can use a modernized face lift; the frontend can use more cross-linking to improve usability; the backend still needs to be improved for synchronization with our repo content; the backend needs to be made PLF aware; etc.

There are many more features and improvements that will happen soon. Nevertheless, the build server is already gaining momentum and is on track to becoming the next big hit for Unity Linux.

* This capability is currently being worked on and is not yet fully functional.

Mirrors: We have found an abundance of freely hosted mirror around the world thanks to Paul’s and others work. With the addition of smart-utils and the upgrade of smart to version 1.3, our package manager problems have really been minimized.

Package Requests: This has been one of the aspects that has occupied much of my time. Since the rc1 release I have gotten a constant stream of requests from the IRC channel, forum, Mail Lists, and personal contact with others using Unity Linux.

Upgrades & Fixes: I have also done quite a few upgrades to core packages along with others devs that have corrected some off our gfx card, wireless cards and other application bugs.

Build Server: With the developments happening to the BS and our constant improvements on the pkgutils suite. All contributors builds and those without he capability to build both arches should soon start filtering throught the BS. Also, we have perfected our method for creating chroots for building or creating the livecd iso.

Open Communication and Collaboration: We have had some success in this area despite many attempt to heal old wounds. Yoper and Unity Linux have found ways to share resources on a few levels that are beginning show beneficial to each other. I will continue to try to reach out to other rpm-based community distros and see if there is not more that we can share and learn from each other.

I am finally coming around on my third pass through the svn repo and over half of the packages are now successfully built. I will probably let the autobuilder make one more pass through the svn repo before I kill it and try to reconstitute a native Unity-Linux chroot jail environment and start over on rebuilding the packages cleanly without old Mandriva build dependencies.

Needless to say, the autobuilder (caos-builder) for unity linux is now working and can be applied in the future to build both 32bit and 64bit unity-linux rpms.

In the past week or so unity linux has got the following updates:
gcc 4.4.1
glibc 2.10.1
binutils-2.19.51.0.14
rpm5 5.2.1

Very exciting indeed.

Matt

Now, we are completely free of those chains, and it’s pretty much open season, for maintaining and building your favorite applications. You still have to import packages, but they are not the long list of build dependencies that we once had.

Just to give an example. I recently built mozilla-thunder and wine for unity, and I hope to have miro ready for the repo soon too. Also, it’s nice to see new releases of applications like Firefox 3.5 make it into the repo so quickly!

I say this because, everyday Unity is becoming more useful as a desktop or a core for whatever one may want to use it for. As we are finally narrowing down our selections for the logo, that means we are getting very close of an official alpha release. We are at an exciting time of what will be an explosion on the Unity Linux platform. It will be interesting to see what the near future has for us in-store.

There are over 200 logo designs to sort out.  The GFX guys were busy.  I was hoping to have the first set of voting underway early this week but it looks more like it going to be for the end of this week.

This is the remaining major issue preventing us from having our first official alpha release so things are looking good.  There are other minor issues but those can be dealt with during the alpha release stage.

On another subject, I was able to install kde4 simply by installing “task-kde4″ from Smart package manager.  Gnome packaging has started and is shaping up nicely.  I need to go through XFCE to complete the missing packages so that it become available for installation.

We’ve been busy since Build 3 with seven releases for testing purposes (uploading as we speak). So what’s new? Bootsplash is back as is the 2.6.29.5 kernel. I’m waiting on kernel-2.6.30.1 to be released to fix a few bugs I’ve encountered.
I’ve fixed most of the issues with the livecd with the exception of the errors messages during the shutdown period but those are harmless (still on my todo list though).

I’ve started collecting all the information, and work related to the logo and graphics and should be able to present a fairly decent sample for team to start voting. I’m hoping to complete that phase within a week.

and I’ll try to keep people a bit better informed too

I finally got a handle on xorg-1.6.1 and have successfully created a first test iso.  There are still several broken elements but we need to rebuild the various DE against it so I’m pushing the changes forward even if it’s in an unfinished state to allow everyone to start updating the various DE.

From Mdawkin’s comments, it seems that the basesystem is almost complete which is really good news.

I finally got to integrate Mandriva’s fastboot and I have to admit that they did a brilliant job there.  There are still a few teething problems but there’s no denying the quality of their work.

Now we need to focus on getting the graphics together to get the first alpha release out.

I just need to clean up my work, tidy the iso and release it.

Unfortunately we lost lzma compression since it’s not yet supported on the squashfs-tools4.0.

This is hopefully last of the internal build iso and the first alpha should follow soon afterward.