Our code creates a new pdo instance for every test. It looks like none of these instances ever disconnect from the database, resulting in the number of simultaneous connections equaling to the number of tests we have.
On postgres the number of allowed connections out of the box appears to be fairly small (or maybe it's just my system) and the test suite runs out of it.
Apparently pdo has no provision to disconnect from the database. You are meant to set the pdo handle to null at which point the object will be garbage collected (php uses refcounting? I suppose you better not have any cycles involving pdo objects) and that's when the connection will be closed.
Since we at some point hand the pdo object to phpunit the fun task of determining whether phpunit properly unreferences pdo it is given is left as an exercise to the reader.