Tom Newby

Speed up heavy PhpUnit test suites with this 1 weird trick!

26 Nov 2018

Because you got beyond the clickbaity title, you’re rewarded with a TLDR:

  • PHPUnit doesn’t clean up your test cases after they run
  • This leads to memory leaks in large/heavy test suites
  • You can use an event listener to clean up after yourself
  • It cuts our test suite time from 11 minutes to 3 minutes

I don’t care, just take me straight to the code

Background

When writing PHPUnit tests, it’s not uncommon to have a common setUp() method that loads fixtures for each of your tests:

<?php

declare(strict_types=1); // You are doing this, right?

namespace Tests\Foo;

use PHPUnit\Framework\TestCase;
use App\Entity\Bar;

class BarTest extends TestCase
{
	/**
	* @var Bar 
	*/
	protected $someObject;
	
	protected function setUp(): void
	{
		$this->someObject = FixtureLoader::loadSomehow();
	}
	
	public function testItDoesTheThing(): void
	{
		// Arrange
		// Act
		// Assert
	}
	
	public function testItDoesNotDoAnotherThing(): void
	{
		// Arrange
		// Act
		// Assert
	}
	
	public function testAnExceptionIsThrownIfSomethingBadHappens(): void
	{
		// Arrange
		// Act
		// Assert
	}
	
}

Consider in this circumstance that $this->someObject is a heavy Doctrine object or similar. For each test method (there are three in this example), PHPUnit instantiates your test case class (i.e. BarTest), calls setUp and runs one test method. In the above example, the BarTest class get instantiated three times, to execute each test method.

As a result, this means loading the fixtures three times. This prompts obvious steps to cache and optimise the fixture loading but we should also pay attention to how PHPUnit works.

After the test method has been completed, PHPUnit will retain this instance of the TestCase until the end of all the test executions, where it can then produce summaries/reports/output. This means that $this->someObject is retained in memory the whole time, which can be quite expensive.

PHPUnit does natively support a solution here, you can implement a tearDown method in your TestCase that clears out these properties:

<?php

//...

class BarTest extends TestCase

    protected function tearDown(): void
    {
    	$this->someObject = null;
    }

This certainly solves the problem technically speaking, but it still requires developers to manually de-alloc each object which is no doubt fraught with error. As you update, rename and move things around it becomes more and more likely you’ll forget to update the tearDown method (or even include one in the first place).

Evolution

Back at the start of the year, I stumbled upon Kris Wallsmith’s blog post about speeding up PHPUnit - they had found great success from writing a base test class which all other tests then extended. This works by using Reflection to strip the class of any ‘magic’ (read: important) PHPUnit data (like did the test pass, timing, etc.).

Fearful of inheritance, we opted for a trait instead:

<?php

declare(strict_types=1);

namespace App\Tests\Helper;

/**
 * Adapted from http://kriswallsmith.net/post/18029585104/faster-phpunit.
 */
trait TidyTestTrait
{
    /**
     * This goes through the test case and unsets any properties that have been set on the class. If you need to actually
     * use tearDown in your own method, you can directly call stripProperties.
     */
    protected function tearDown(): void
    {
        static::stripProperties($this);
        parent::tearDown();
    }
    /**
     * @param \object $target
     */
    public static function stripProperties($target): void
    {
        $refl = new \ReflectionObject($target);
        foreach ($refl->getProperties() as $prop) {
            if (!$prop->isStatic() && 0 !== \strncmp($prop->getDeclaringClass()->getName(), 'PHPUnit_', 8)) {
                $prop->setAccessible(true);
                $prop->setValue($target, null);
            }
        }
    }

This worked fine for us for a few months, but we were still noticing we were obviously forgetting to add it to many tests, so with a bit more digging I discovered that you can extend PHPUnit with event listeners.

With this information in hand, we ended up with using an event listener to automatically perform this data stripping on all of our test cases. We’ve now been using this for a few months and haven’t found any issues with it, so feel free to try it out for yourself.

The code

<?php

declare(strict_types=1);

namespace Tests\Listener;

use PHPUnit\Framework\Test;
use PHPUnit\Framework\TestListener;
use PHPUnit\Framework\TestListenerDefaultImplementation;
use Symfony\Bundle\FrameworkBundle\Test\WebTestCase;

/**
 * Adapted from http://kriswallsmith.net/post/18029585104/faster-phpunit.
 * and packaged into an event listener:
 * https://tomnewby.net/posts/speed-up-phpunit-1-weird-trick/
 */
final class TestTidierListener implements TestListener
{
    use TestListenerDefaultImplementation;

    /**
     * This goes through the test case and unsets any properties that have been set on the class.
     */
    public function endTest(Test $test, $time): void
    {
        if (false === ($test instanceof WebTestCase)) {
            // We only care about inspecting if this is a WebTestCase test
            return;
        }

        self::stripProperties($test);
    }

    /**
     * @param \object $target
     */
    public static function stripProperties($target): void
    {
        $refl = new \ReflectionObject($target);
        foreach ($refl->getProperties() as $prop) {
            if (!$prop->isStatic() && 0 !== \strncmp($prop->getDeclaringClass()->getName(), 'PHPUnit_', 8)) {
                $prop->setAccessible(true);
                $prop->setValue($target, null);
            }
        }
    }
}

And then register your class in phpunit.xml:

<phpunit>
    ...

    <listeners>
        <listener class="Tests\Listener\TestTidierListener" />
    </listeners>
</phpunit>

Results

Across 1117 tests (largely functional/integration tests with fixtures), this cuts the time from 11.42 minutes to 3.16 minutes as a result of reducing the memory from 2744MB to 488MB.

Results comparison

This helped us, maybe it’ll help you!

Follow me on Twitter: @tomnewbyau