Mirror Server

Requirements

• Hosts all the files required to build a package (GENTOO_MIRRORS=mirror.example.com/public/gentoo/distfiles)
  • Acts as a caching mirror for already downloaded packages from an official mirror, example caching mirror URL https://mirror.example.com/mirror
  • Serves fetch-restricted files (dev-java/oracle-jdk-bin for example), to authorized clients
• Files are served via HTTPS
• Distinguishes between three groups of files
  • public: Files which are available to all clients (theoretically even to the entire internet, added to clients via GENTOO_MIRRORS)
  • site-restricted: Files which are only available to authenticated clients belonging to the same infrastructure (for example those which would put us into legal troubles if available to the public, used by clients via a local entry in /etc/portage/mirrors)
    • A subdirectory named after a site will be created below site-local, for example site-local/example-site.
  • role-restricted: Files which are only available to authenticated clients belonging to the same infrastructure and the software stack group (private files of a specific customer, used by clients via a local entry in /etc/portage/mirrors)
    • A subdirectory per role will be created below role-restricted, for example role-restricted/mediawiki_mysql.
• Provides an easy way to let an administrator manually upload new files, for example via WebDAV-CGI, SFTP or a similar mechanism.
• Possibility to authenticate clients either via HTTP basic auth or client certificates.
• Old or no longer supported files will be removed automatically
• Can be implemented on the build host

Directory structure

The visible structure starting from the root directory will be as follows:

├── public
│   ├── gentoo
│   │   └── distfiles
│   └── stepping-stone.ch
│       └── sepiola
├── role-restricted
│   ├── crm_sugarcrm_mysql
│   └── wiki_mediawiki_mysql
└── site-restricted
    └── stepping-stone.ch
        └── gentoo
            └── distfiles

Packaging notes

Fetch restriction

To avoid passive information disclosure all packages requiring source files from at least one of the restricted locations must be fetch restricted (if possible). Otherwise a client may in some circumstances try to fetch the package from a public mirror, leaving traces in the respective access log.

For Gentoo ebuilds this means:

RESTRICT="fetch"