What pkgsrc needs most
Amitai Schlair
The NetBSD Project
Where's agc@?
Where's AGC?
"Al's Gargantuan Catalog":
- What pkgsrc does well
- How and when it got that way
- What we wish pkgsrc did better
The point of this talk
Make agc@'s AGC obsolete.
Make agc@'s AGC obsolete.
Make agc@'s AGC actionable.
Not as much new this year: Reasons (1)
- pkgsrc infrastructure does what it does reasonably well
- jlam@ has retired
Not as much new this year: Reasons (2)
Complicated code
- mix of make/sh/awk/sed, none suited to all the tasks
- ordering constraints (some nicely hidden)
- odd couplings
- hard to find (all of the) relevant code
Not as much new this year: Reasons (3)
Change-resistant code
- obvious approach often doesn't quite work
- seems to work, bulk build shows otherwise
- being careful means having to run a bulk build
- two bulk builds, really
Not as much new this year: Reasons (4)
Unclear, undescribed code
- internal behaviors and interfaces not clearly defined
- my 2008 GSoC project could have improved the situation
joerg@ seems to do okay
But what about the rest of us?
- small, finite number of people
- small, finite amount of time
- volunteer project: people do what they can when they can
How do we get pkgsrc unstuck?
Fun ideas (1): completely unhelpful
- persuade volunteers to try harder
- rewrite package system from scratch
- la la la la la la la I can't hear you
Fun ideas (2): somewhat helpful
- targeted development
- Google Summer of Code
Fun ideas (3): my idea
- pkgsrc is a large, functional, featureful system used in production around the world.
- We're sitting on a gold mine. pkgsrc makes our lives easier. Let's keep it.
- Piece by piece, make pkgsrc easier and safer to change.
- Then the intersection of "features we want" and "developers willing and able to touch pkgsrc internals" will grow.
- We might start getting more new features.
- After the next AGC, you yourself might go implement a feature you care about.
- Eventually, Al will have to talk about something else at pkgsrcCon.
My very specific one idea
- Let's write automated tests for pkgsrc internals.
- ...
- $ pkg_profit # !
Automated tests for pkgsrc internals?
- "integration" tests: high-level pkgsrc behaviors
- "functional" tests: behaviors of individual components
- "unit" tests: behaviors of fine-grained bits of code
But it doesn't matter what they're called, it matters that they're called.
Why tests?
- There is not now, nor has there ever been, a specification.
- To get one, we'll need to delve into pkgsrc internals.
- For each area someone comes to grok, we'll want thorough documentation.
- comments, text files, wiki pages, graphs.
- The best documentation for developers: a bit more code.
A spec that runs
- Comments are lies.
- Specification documents are lies. (Either general and useless, or detailed and wrong.)
- Lying is rude.
- We're all friends here.
- Executable specifications are not only friendly, they also stand a chance of being helpful.
Why automated tests?
- No special knowledge or setup needed.
- Unambiguous description of any state required.
- Glance at the tests, know what the code should do.
- Run easily == run frequently.
Hang on, I've heard this somewhere before
- Yes, gavan@ wrote pkg_regress gave a talk.
- rillig@ and others wrote tests (under regress/*).
- We all liked the idea. Why'd it get forgotten?
Part of the build
TEST_TARGET for individual packages. What about pkgsrc itself?
- stick the tests in an installable package
- encourage users to run the tests and report results
- suggest running tests from cron (like audit-packages)
- run it early in the bulk build
- BUILD_DEPENDS ~everything on it (like digest)
- but only when PKG_DEVELOPER
- strongly encourage developers to run the tests when modifying infrastructure
More implementation details
- New tool, suited to the task: ATF.
- Let's convert regress/* to ATF, move them into pkg_autotest, and retire pkg_regress.
- Anyone, anytime, can write new pkg_autotest tests. Just bump VERSION.
- Running tests manually is easy. Just invoke pkg_autotest.
- Running tests automatically... yes. Who and when?
The one thing pkgsrc needs most
- Your help building a comprehensive test suite for pkgsrc internals.
If we make it easier to work on, pkgsrc will get better faster.
If we work on this together, it will get easier sooner.