View unanswered posts | View active topics It is currently Mon Aug 21, 2017 4:56 pm



Reply to topic  [ 2 posts ] 
Building debs on one machine with Pbuilder 
Author Message
LQFP112 - Up with the play

Joined: Sun Apr 06, 2008 6:30 pm
Posts: 100
This mini-tutorial is copyright David J. Andruczyk Jan 1st 2012 based uponvarious howto's scattered across the internet with a little personal experienced tossed in.

Goal: To be able to build debian/ubuntu packages for multiple releases without needed a machine for each release.

Tools used:
1. apt-cacher-ng (to save from having to download the same stuff over and over again)
2. pbuilder with appropriate configuration and setup (see below)
3. Debian or ubuntu machine. This tutorial is based on Ubuntu 10.04 running on an i686 32 bit CPU.

Requirements:
1. Linux machine (64 bit required if you want to make 64 bit packages), this tutorial is based on Ubuntu 10.04, other version/os your mileage will vary.
2. Plenty of space under /var/cache. apt-cache-ng will cache every pkg downloaded into /var/cache/apt-cacher-ng, and pbuilder defaults to using /var/cache/pbuilder as it's root.

NOTE: if you want to build 64 bit packages you MUST be on a 64 bit capable machine with a 64 bit OS, otherwise you're restricted to making 32 bit packages only.

First off install the tools:
Code:
apt-get install apt-cacher-ng pbuilder debootstrap quilt


apt-cacher-ng acts like a debian or ubuntu mirror/proxy listening on port 3142. So you can take an /etc/apt/sources entry and change the hostname to "localhost:3142", or setup apt to use a http proxy of http://localhost:3142 and run apt-get update and it should work, when a request for a file comes in, it will serve it from the cache if present, otherwise it'll download and stream it to the client at the same time populating the cache, thus hte next time around it just uses the cache saving lots of time. The cache is intelligent enough to always give you current stuff.

pbuilder is used to make clean pristine chroot's for each distro and architecture target (for 32/64 bit pkgs), and hence it requires some configuration so things go into the right place as well as handling the special case of sequential package creation where a latter package depends on the former package being available within the buildroot to work (case in point the FreeEMS toolchain compiler depends on binutils which isn't available in the normal repository), so some magic needs to happen in order to make the appropriate stuff available in the build root.

Special steps:
Code:
mkdir /usr/lib/pbuilder/hooks
mkdir /var/cache/pbuilder/repo


create /usr/lib/pbuilder/hooks/D05deps with the following content
Code:
#!/bin/sh
(cd /var/cache/pbuilder/repo; apt-ftparchive packages . > Packages && gzip -c Packages >Packages.gz)
apt-get update


make it executable
Code:
chmod +x /usr/lib/pbuilder/hooks/D05deps


If you are on an "older" linux distro (I'm on 11.04) you might need to create a symlink or two in /usr/share/deboostrap/scripts so that debootstrap knows what to do.

Copy and paste the following into a terminal
Code:
cd /usr/share/debootstrap/scripts
for i in `echo wheezy squeeze lenny` ; do if test ! -L ${i} ; then echo "Creating missing link for ${i}" ; sudo ln -s sid ${i} ; fi ; done
for i in `echo precise oneiric natty maverick lucid hardy` ; do if test ! -L ${i} ; then echo "Creating missing link for ${i}" ; sudo ln -s gutsy ${i} ; fi ; done


Create /etc/pbuilderrc which is the main pbuilder system wide config

Code:
# this is your configuration file for pbuilder.
# the file in /usr/share/pbuilder/pbuilderrc is the default template.
# /etc/pbuilderrc is the one meant for editing.
#
# read pbuilderrc.5 document for notes on specific options.

BASETGZ=/var/cache/pbuilder/base.tgz
BUILDPLACE=/var/cache/pbuilder/build/
OTHERMIRROR="deb file:///var/cache/pbuilder/repo ./"
USEPROC=yes
USEDEVPTS=yes
USEDEVFS=no
BUILDRESULT=/var/cache/pbuilder/result/

REMOVEPACKAGES="lilo"
HOOKDIR="/usr/lib/pbuilder/hooks"
# make debconf not interact with user
export DEBIAN_FRONTEND="noninteractive"
DEBEMAIL=""
# for pbuilder debuild (sudo -E keeps the environment as-is)
BUILDSOURCEROOTCMD="fakeroot"
PBUILDERROOTCMD="sudo -E"

# command to satisfy build-dependencies; the default is an internal shell
# implementation which is relatively slow; there are two alternate
# implementations, the "experimental" implementation,
# "pbuilder-satisfydepends-experimental", which might be useful to pull
# packages from experimental or from repositories with a low APT Pin Priority,
# and the "aptitude" implementation, which will resolve build-dependencies and
# build-conflicts with aptitude which helps dealing with complex cases but does
# not support unsigned APT repositories
PBUILDERSATISFYDEPENDSCMD="/usr/lib/pbuilder/pbuilder-satisfydepends"

#Command-line option passed on to dpkg-buildpackage.
DEBBUILDOPTS=""

#APT configuration files directory
APTCONFDIR=""
# the username and ID used by pbuilder, inside chroot. Needs fakeroot, really
BUILDUSERID=1234
BUILDUSERNAME=pbuilder

# BINDMOUNTS is a space separated list of things to mount
# inside the chroot.
BINDMOUNTS="/var/cache/pbuilder/repo"

# Set the debootstrap variant to 'buildd' type.
# DEBOOTSTRAPOPTS[0]='--variant=buildd'
# or work around bug in debootstrap 3.0.0 (314858)
unset DEBOOTSTRAPOPTS

# Set the PATH I am going to use inside pbuilder: default is "/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin"
export PATH="/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin"

# SHELL variable is used inside pbuilder by commands like 'su'; and they need sane values
export SHELL=/bin/bash

# The name of debootstrap command.
DEBOOTSTRAP="debootstrap"

# default file extension for pkgname-logfile
PKGNAME_LOGFILE_EXTENTION="_$(dpkg --print-architecture).build"

# default PKGNAME_LOGFILE
PKGNAME_LOGFILE=""



I chose to also have a PER USER ~/.pbuilderrc script which overrides some of the defaults for some specific cases of using pbuilder, I recommend using this as a baseline and adjusting it to your needs as you see fit.

~/.pbuilderrc
Code:
# Codenames for Debian suites according to their alias. Update these when
# needed.
UNSTABLE_CODENAME="sid"
TESTING_CODENAME="squeeze"
STABLE_CODENAME="lenny"
STABLE_BACKPORTS_SUITE="$STABLE_CODENAME-backports"

# List of Debian suites.
DEBIAN_SUITES=($UNSTABLE_CODENAME $TESTING_CODENAME $STABLE_CODENAME
    "unstable" "testing" "stable")

# List of Ubuntu suites. Update these when needed.
UBUNTU_SUITES=("precise" "oneiric" "natty" "maverick" "lucid" "hardy")

# Mirrors to use. Update these to your preferred mirror.
DEBIAN_MIRROR="ftp.us.debian.org"
UBUNTU_MIRROR="mirrors.kernel.org"

# Optionally use the changelog of a package to determine the suite to use if
# none set.
if [ -z "${DIST}" ] && [ -r "debian/changelog" ]; then
    DIST=$(dpkg-parsechangelog | awk '/^Distribution: / {print $2}')
    # Use the unstable suite for certain suite values.
    if $(echo "experimental UNRELEASED" | grep -q $DIST); then
        DIST="$UNSTABLE_CODENAME"
    fi
fi

# Optionally set a default distribution if none is used. Note that you can set
# your own default (i.e. ${DIST:="unstable"}).
: ${DIST:="$(lsb_release --short --codename)"}

# Optionally change Debian release states in $DIST to their names.
case "$DIST" in
    unstable)
        DIST="$UNSTABLE_CODENAME"
        ;;
    testing)
        DIST="$TESTING_CODENAME"
        ;;
    stable)
        DIST="$STABLE_CODENAME"
        ;;
esac

# Optionally set the architecture to the host architecture if none set. Note
# that you can set your own default (i.e. ${ARCH:="i386"}).
: ${ARCH:="$(dpkg --print-architecture)"}

NAME="$DIST"
if [ -n "${ARCH}" ]; then
    NAME="$NAME-$ARCH"
    DEBOOTSTRAPOPTS=("--arch" "$ARCH" "${DEBOOTSTRAPOPTS[@]}")
fi
BASETGZ="/var/cache/pbuilder/$NAME-base.tgz"
# Optionally, set BASEPATH (and not BASETGZ) if using cowbuilder
# BASEPATH="/var/cache/pbuilder/$NAME/base.cow/"
DISTRIBUTION="$DIST"
BUILDRESULT="/var/cache/pbuilder/$NAME/result/"
APTCACHE="/var/cache/pbuilder/aptcache/$NAME/"
BUILDPLACE="/var/cache/pbuilder/build/"

if $(echo ${DEBIAN_SUITES[@]} | grep -q $DIST); then
    # Debian configuration
    MIRRORSITE="http://$DEBIAN_MIRROR/debian/"
    COMPONENTS="main contrib non-free"
    # This is for enabling backports for the Debian stable suite.
    #if $(echo "$STABLE_CODENAME stable" | grep -q $DIST); then
    #    EXTRAPACKAGES="$EXTRAPACKAGES debian-backports-keyring"
    #    OTHERMIRROR="$OTHERMIRROR | deb http://www.backports.org/debian $STABLE_BACKPORTS_SUITE $COMPONENTS"
    #fi
elif $(echo ${UBUNTU_SUITES[@]} | grep -q $DIST); then
    # Ubuntu configuration
    MIRRORSITE="http://$UBUNTU_MIRROR/ubuntu/"
    COMPONENTS="main restricted universe multiverse"
else
    echo "Unknown distribution: $DIST"
    exit 1
fi



OK, now that pbuilder is configured you will need to init the base build roots to start with. This takes some time.

For 32 bit machines
Code:
for dist in `echo wheezy squeeze lenny precise oneiric natty maverick lucid hardy` ; do sudo DIST=${dist} pbuilder --create --architecture i386 --http-proxy http://localhost:3142 --distribution ${dist}  --basetgz /var/cache/pbuilder/${dist}-i386-base.tgz ; done


For 64 bit machines where you want to be able to make both 32 and 64 bit packages:

Code:
for arch in `echo i386 amd64` ; do for dist in `echo wheezy squeeze lenny precise oneiric natty maverick lucid hardy` ; do sudo DIST=${dist} ARCH=${arch} pbuilder --create --architecture ${arch} --distribution ${dist}  --http-proxy http://localhost:3142 --basetgz /var/cache/pbuilder/${dist}-${arch}-base.tgz ; done ; done


To build something for real:
If you're within a package that has a "debian" or "DEBIAN" dir its a simple command:
Code:
pdebuild --architecture <i386|amd64> --buildresult /tmp --pbuilderroot "sudo DIST=<lenny|squeeze|wheezy|precise|oneiric|natty|maverick|lucid|hardy> ARCH=<i386|amd64>"


This will do all the debian magic and complain as needed if you don't have everything perfectly setup.

When complete the output files will be in /tmp as per the "--buildresult" argument.

See the manual page for pbuilder and pdebuild for additional methods of use.

_________________
-- David
http://www.megatunix.com
http://megatunix.sourceforge.net


Last edited by dandruczyk on Tue Feb 28, 2012 2:48 am, edited 5 times in total.



Sun Jan 01, 2012 8:55 pm
Profile
Moderator
User avatar

Joined: Tue Jan 15, 2008 2:31 pm
Posts: 14715
Location: Home sweet home!
Awesome work, Dave! Comments: viewtopic.php?f=35&t=1506

_________________
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!


Mon Jan 02, 2012 12:59 am
Profile WWW
Display posts from previous:  Sort by  
Reply to topic   [ 2 posts ] 

Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group
Designed by ST Software for PTF. ColorizeIt.