Build Server

Revision as of 14:43, 22 August 2014 by Tiziano (Talk | contribs)


Overview

  • Base chroot environment for the creation of most binary packages which will be copied to the Binary Package Server.
  • Packages, which don't need to be built, for example Oracle's Java SE Development Kit (JDK), can be fetched from the Mirror Server.
  • Software stacks built upon specialized binary packages, require their own chroot environment. If possible, avoid this.
  • For reproducibility, Portage and Profiles are frozen with the help of version control system (git in our case). This is only done on the Mirror Server.
  • When building packages in a chroot environment, the portage and overlay versions need to set.

Portage Snapshotting

To be able to reliably rebuild packages, we are working with a snapshotted/versioned portage tree.

Updating the snapshot to latest

We are using emerge-webrsync since it features an already determined version, checksum- and gpg-validation.

user $ git clone $YOURGITREPO portage
user $
cd portage
user $
PORTAGE_RSYNC_EXTRA_OPTS="--exclude=/.git" PORTDIR="${PWD}" emerge-webrsync
Trying to retrieve 20140819 snapshot from http://mirror.switch.ch/mirror/gentoo ...
Fetching file portage-20140819.tar.xz.md5sum ...
Fetching file portage-20140819.tar.xz.gpgsig ...
Fetching file portage-20140819.tar.xz ...
Checking digest ...
Getting snapshot timestamp ...
Syncing local tree ...

If you instead get the following then there is no newer snapshot available:

user $ PORTDIR="${PWD}/portage" emerge-webrsync
Fetching most recent snapshot ...
 * Latest snapshot date: 20140821
 * 
 * Approximate snapshot timestamp: 1408668300
 *        Current local timestamp: 1408667701
 * 
 * The current local timestamp is possibly identical to the
 * timestamp of the latest snapshot. In order to force sync, use
 * the --revert option or remove the timestamp file located at
 * '/run/user/1000/portage/metadata/timestamp.x'.

Set the tag and push if there was an update:

user $ cd portage/
user $
git add -A
user $
snapshot_date=$(date --date=@$(cut -f 1 -d " " portage/metadata/timestamp.x) +%Y-%m-%d)
user $
git commit -m "Update tree to ${snapshot_date}"
user $
git tag -a -m "Tagging ${snapshot_date}" ${snapshot_date}
user $
git push && git push --tags
user $
echo "snapshot date: ${snapshot_date}"


Chroot Setup

Preparation

At least the following is required beforehand to ensure reproducible building:

  • a versioned portage tree
  • versioned overlays (if any)

Setup

  • Create a staging root and extract a stage3 or stage4 tarball
  • Mount proc, sys and dev (required for portage to control the tty, determine available storage space, etc.)
  • Copy /etc/resolv.conf
  • Enter the chroot
root # stagingRoot=/var/staging/staging-base
root #
mkdir -p "${stagingRoot}"
root #
tar -xjpf /var/tmp/your-stage4.tbz2 -C "${stagingRoot}"
root #
mount -t proc none "${stagingRoot}/proc"
root #
mount --rbind /dev/ "${stagingRoot}/dev/"
root #
mount --rbind /sys/ "${stagingRoot}/sys/"
root #
cp -af /etc/resolv.conf "${stagingRoot}/etc/"
root #
chroot "${stagingRoot}" /usr/bin/env -i HOME="/root" TERM="${TERM}" /bin/bash --login
Note
We are deliberately using a special command to enter the chroot to avoid leaking environment variables
Last modified on 22 August 2014, at 14:43