pkgjam: Moving Away from the One True Tree

James K. Lowden

FreeTDS maintainer & pkgsrc whiner

What is pkgjam?

Organization of Talk

Motivation

As a novice at package creation, I found the guide adequate for dealing with ordinary situations, but lacking on how do deal with the weirder stuff. Worse yet, there's almost always something weird.

— user-pkg@ 21 April 2007

Welcome to pkgjam

Dependency Independence

Objective

Means

Dependency Independence (2)

Why hard links?

Database Diagram

Complexity is inherent in the problem domain. We manage this complexity now, but without the benefit of a diagram or a relational database.

Interpolated strings are not a complexity management tool.

Three Databases

bin and site are locally managed. src is built from CVS-controlled sources and is downloaded by users.

The build tool, pkg, consults all three databases to determine what to build and how to build it. (Dependencies may be satistified from bin and options driven by site.)

Interesting Database Columns

id
integer representing a version of a package, managed by a central registry
option
user-settable package choice akin to PKG_OPTIONS
canover
canonical version, a lexically sortable package version number
knob
any build-time package setting, large or small

bin Database

site Database

Three kinds of Options

  1. Global options that apply to many packages use id 0
  2. Global options affecting a single package use that package's id
  3. Package-specific options

User can discover options by quering the database.

src Database

Main tables

Packages
one id per name+version
Dependencies
define a “package mask”, a range of package versions that would satisify the package's requirement
Alternatives
describe cases in which one of several options satisfy a dependency
Knobs
are any build-time setting, including e.g. PREFIX
Options
control knobs, sets of knobs, and other options

Canover

Describing Dependencies as Relations

Loading src Database

src Database Benefits

The database answers ad hoc queries instantly that either can't be answered today, or that take overnight to answer. For example:

Note: no src.Plist. Instead, we derive a plist (for bin.Plist) by watching what files are installed by the package into its directory.

Exempli Gratia

sqlite> select p.name as pkg, dp.name as 'depends on' 
        from Packages p 
        join Dependencies d on p.id = d.id 
        join Packages dp on d.pkgname = dp.name 
        where dp.name = 'libxml2';

        pkg         depends on
        ----------  ----------
        libxslt     libxml2   
        py24-libxm  libxml2   
        xmlto       libxml2   

SQLite

We chose SQLite because:

Other features:

Build tool: pkg

Johnny Lam discusses pkg later.

Status

Coming Attractions

Choose pkgjam

[any material that should appear in print but not on the slide]