Difference between revisions of "Build Server"
From stoney cloud
| [unchecked revision] | [unchecked revision] |
(→Chroot Setup) |
|||
| (One intermediate revision by one other user not shown) | |||
| Line 17: | Line 17: | ||
|git clone "${PORTAGE_GIT_REPO}" portage | |git clone "${PORTAGE_GIT_REPO}" portage | ||
|cd portage | |cd portage | ||
| − | |PORTAGE_RSYNC_EXTRA_OPTS{{=}}"--exclude{{=}}/.git" PORTDIR{{=}}"${PWD}" emerge-webrsync | + | |PORTAGE_RSYNC_EXTRA_OPTS{{=}}"--exclude{{=}}/.git --exclude{{=}}/.gitignore" PORTDIR{{=}}"${PWD}" emerge-webrsync |
|output=<pre> | |output=<pre> | ||
Fetching most recent snapshot ... | Fetching most recent snapshot ... | ||
| Line 33: | Line 33: | ||
{{Cmd | {{Cmd | ||
| − | |PORTAGE_RSYNC_EXTRA_OPTS{{=}}"--exclude{{=}}/.git" PORTDIR{{=}}"${PWD}" emerge-webrsync | + | |PORTAGE_RSYNC_EXTRA_OPTS{{=}}"--exclude{{=}}/.git --exclude{{=}}/.gitignore" PORTDIR{{=}}"${PWD}" emerge-webrsync |
|output=<pre> | |output=<pre> | ||
Fetching most recent snapshot ... | Fetching most recent snapshot ... | ||
| Line 96: | Line 96: | ||
* Install {{Package|app-portage/layman}} and probably {{Package|dev-vcs/git}} to install overlays | * Install {{Package|app-portage/layman}} and probably {{Package|dev-vcs/git}} to install overlays | ||
* You may want to add a mirror configuration if you have local mirrors: | * You may want to add a mirror configuration if you have local mirrors: | ||
| − | <source lang='bash'> | + | {{File|/etc/portage/make.conf||<source lang='bash'> |
| − | + | ||
GENTOO_MIRRORS="https://mirror.example.com/public/gentoo" | GENTOO_MIRRORS="https://mirror.example.com/public/gentoo" | ||
| − | + | </source>}} | |
| − | + | * If needed, put the mirror for fetch restricted ebuilds in a local mirror definition: | |
| − | + | {{File|/etc/portage/mirrors||<pre> | |
local https://mirror.example.com/restricted/gentoo | local https://mirror.example.com/restricted/gentoo | ||
| − | + | </pre>}} | |
| − | </ | + | |
| − | + | ||
* Add the following to generate binary packages during installation: | * Add the following to generate binary packages during installation: | ||
| − | + | {{File|/etc/portage/make.conf||<source lang='bash'> | |
| − | + | FEATURES="${FEATURES} buildpkg cgroup ipc-sandbox network-sandbox parallel-fetch" | |
| − | FEATURES=" | + | </source>}} |
| − | + | ||
| − | </source> | + | * Make sure that the repository configuration points to your git mirror as well: |
| + | {{File|/etc/portage/repos.conf/gentoo.conf||<source lang='ini'> | ||
| + | [DEFAULT] | ||
| + | main-repo = gentoo | ||
| + | |||
| + | [gentoo] | ||
| + | location = /usr/portage | ||
| + | sync-type = git | ||
| + | sync-uri = https://mirror.example.com/versioned/portage.git | ||
| + | </source>}} | ||
| + | |||
* Rebuild everything: | * Rebuild everything: | ||
{{RootCmd|emerge -e world}} | {{RootCmd|emerge -e world}} | ||
[[Category:Infrastructure]] | [[Category:Infrastructure]] | ||
Latest revision as of 08:51, 1 June 2015
Contents
Overview
- Base chroot environment for the creation of most binary packages which will be copied to the Binary Package Server.
- Packages, which don't need to be built, for example Oracle's Java SE Development Kit (JDK), can be fetched from the Mirror Server.
- Software stacks built upon specialized binary packages, require their own chroot environment. If possible, avoid this.
- For reproducibility, Portage and Profiles are frozen with the help of version control system (git in our case). This is only done on the Mirror Server.
- When building packages in a chroot environment, the portage and overlay versions need to set.
Portage Snapshotting
To be able to reliably rebuild packages, we are working with a snapshotted/versioned portage tree.
Updating the snapshot to latest
We are using emerge-webrsync since it features an already determined version, checksum- and gpg-validation.
Warning
If you get the following instead then there is no newer snapshot available
Set the tag and push if there was an update:
The subtraction of 86400 (1 day in seconds) above is to account for the fact that the snapshot for a given day is taken at 00:45 UTC on the following day and the timestamp contained in the tarball therefore is one day ahead (see /usr/bin/emerge-webrsync).
The meaning of the tag is therefore: contains at least all changes from that day (and possibly a bit more).
Chroot Setup
Preparation
At least the following is required beforehand to ensure reproducible building:
- a versioned portage tree
- versioned overlays (if any)
Setup
- Create a staging root directory and extract a stage3 or stage4 tarball
- Mount proc, sys and dev (required for portage to control the tty, determine available storage space, etc.)
- Copy /etc/resolv.conf
- Checkout the versioned portage tree (since you probably won't have git yet in the tree)
- Enter the chroot
Note
We are deliberately using a special command for entering the chroot to avoid leaking environment variables
- Install app-portage/layman and probably dev-vcs/git to install overlays
- You may want to add a mirror configuration if you have local mirrors:
File/etc/portage/make.conf
GENTOO_MIRRORS="https://mirror.example.com/public/gentoo"
- If needed, put the mirror for fetch restricted ebuilds in a local mirror definition:
File/etc/portage/mirrors
local https://mirror.example.com/restricted/gentoo
- Add the following to generate binary packages during installation:
File/etc/portage/make.conf
FEATURES="${FEATURES} buildpkg cgroup ipc-sandbox network-sandbox parallel-fetch"
- Make sure that the repository configuration points to your git mirror as well:
File/etc/portage/repos.conf/gentoo.conf
[DEFAULT] main-repo = gentoo [gentoo] location = /usr/portage sync-type = git sync-uri = https://mirror.example.com/versioned/portage.git
- Rebuild everything:
