Hi Rabeeh,

On Mon, Oct 01, 2007 at 01:47:14PM +0200, Rabeeh Khoury wrote:

Lately I'v tried building boost package in two different methods while measuring the time it consumes.

The first using Orion-2 system and running completely native build using the FC-8 RPMs. The second is using Orion-2 that has distcc client that calls through fast Ethernet to a Core 2 duo x86 machine that has the FC-8 cross-compiler environment with distcc daemon.

I have a mostly drop-in setup for this -- cross compiler RPMS at: http://www.wantstofly.org/~buytenh/cross/

and distcc (client and server) RPMS at: http://fedora-arm.wantstofly.org/add-ons/

The only configuration it needs is:

Server (x86) side: - Edit /etc/sysconfig/distccd: - Set arm box IP range, and add --jobs 200 or so. - Set DISTCCD_PATH to /usr/armv5tel-redhat-linux-gnueabi/bin:$PATH

Client (arm box) side: - Put IP of build machine into /etc/distcc/hosts.

You might also want to: - Hardcode %{_smp_mflags} on the ARM side to '-j 4' or so, to use multiple gcc instances in parallel for the build (to overcome network latency.)

I've used these RPMS successfully to (pseudo-natively) compile glibc, eglibc, gcc, etc.

I guess we should add this info to the wiki...

The total build time using the distcc for boost was 5X faster (complete build with archiving).

The question is can we use this as an addition to Koji build system to make building from scratch faster?

Probably. All it would need is:

- Create an ARM package that contains a suitable version of /etc/distcc/hosts (called, say, distcc-arm-stubs.) - Create a new version of buildsys-build that pulls this package in.

This way, the distcc hooks get pulled into the build root automatically.

Note that the distcc client package has /etc/profile.d/ scripts that automatically add the distcc hook directory to the path if /etc/distcc/hosts is present.

I know about the efforts to make everything cross-build, but until that becomes ready the distributed build can be very attractive.

There are a couple more differences: - Cross-building can (at least in theory! :) be used to bootstrap a distro, whereas the distcc method can not. - Cross-building does not (at least in theory! :) need actual ARM hardware.

I realise that you can do the distcc trick with a qemu-system-arm instance, so you don't really need actual physical ARM _hardware_, but it's still somewhat cumbersome to set that up.

Note that for the build I didn't need to change anything in RPM specs, or RPM macros and configuration; only installed distcc and changed the path.

ACK.

As a pure build speedup method, the distcc trick is fine (we have been using this trick for the Debian big-endian ARM port for a couple of years now, since the build hardware we have is pretty slow), but there are couple of advantages of pure cross building that you can't really duplicate with this method.

For building Fedora ARM packages, we use 14 build machines (2 real ARM boxes and 12 qemu-system-arm instances), and the speedup from distcc isn't really critical, so we don't use it.

cheers, Lennert