Cakephp 2.x: failing Phing using CakePHP testsuites

Digging todays deployment scripts on CI (Continuous integration) machine, I’ve noted that no matter how many Unit tests fail, phing still thought that builds were successful. Teetering on the brink of a heart attack, I’ve started checking stage machines, and production systems, to get the prove of concept. Thankfully, the number of failed tests wasn’t too big, and patches were added shortly, but the issue remained.

It appeared that ExecTask of Phing (no matter how the script is ran), will return success, unless you start comparing the values outside of it. Solution was pretty obvious, but took some time to go through the documentation of Phing/PHPUnit/CakePHP: