Dynamically generating FlexUnit test suite

I have been working with Flex recently, which is a framework for building Flash applications. The standard testing framework for Flex is FlexUnit. A good example of FlexUnit can be found at How to use FlexUnit with FlexBuilder 2

One limitation of FlexUnit is that it does not have a way to dynamically build a test suite at runtime. All of the examples online have a manually created and maintained test suite that looks like:

package {
  import flexunit.framework.TestSuite;
  import com.foo.FooTest;
  import com.bar.BarTest;

  public class Suite {

    public static function createTestSuite() : TestSuite {

      var ts : TestSuite = new TestSuite();
      ts.addTestSuite(FooTest);
      ts.addTestSuite(BarTest);
      // Many more additions
      return ts;
    }
  }
}

We believe in constantly writing and refactoring tests, and we did not want to manually maintain this test suite. We first investigated reading the filesystem to get the list of available tests at runtime. However, since the tests run inside of flash (in a browser or standalone player), there is no access to the filesystem. Kent Spillner and I brainstormed and came up with a build time solution instead.

First, we wrote a ruby script that would read the filesystem and generate the test suite file. We called the script test_suite_generator.rb, and it looks something like:

#!/usr/bin/env ruby

require 'find'

search_path = File.dirname(__FILE__) + '/test/'

test_cases = []
Find.find search_path do |path|
  filename = File.basename(path)
  Find.prune if filename =~ /^\./
  test_cases << path.gsub("#{search_path}", '').gsub('.as', '').gsub('/', '.') if filename =~ /Test\.as$/
end

test_cases.sort!

File.open('test/Suite.as', 'w') do |file|
  file.puts <<EOF
// This file is generated by #{File.basename(__FILE__)}.
package {
  import flexunit.framework.TestSuite;
EOF
  test_cases.each {|tc| file.puts "  import #{tc};"}
  file.puts <<EOF

  public class Suite {

    public static function createTestSuite() : TestSuite {

      var ts : TestSuite = new TestSuite();
EOF
  test_cases.each {|tc| file.puts "      ts.addTestSuite(#{tc});"}
  file.puts <<EOF
      return ts;
    }
  }
}
EOF
end

This script uses Find to get all of the test files in the test folder (files that end in Test.as). For each file, it turns the path into a package (com/foo/HelloWorldTest.as -> com.foo.HelloWorldTest). Then, it writes a file called Suite.as into the test directory which imports each of these packages and then adds them to a TestSuite object.

We use Flex Builder for flex development, which is based on eclipse. This allows us to add a custom builder which will run our ruby script.

In Project -> Properties -> Builders, choose New -> Program. Type TestSuiteGenerator for the name, and browse for the ruby script in the Location field:

Then, on the “Build Options” tab, select “During auto builds” and “Specify working set of relevant resources:”

Finally, click on “Specify Resources…” and choose the folder where your tests live. Make sure that you don’t select the folder that includes Suite.as, or the generation of the file will kick off the builder again. If your tests are nested inside a “com” folder, then choose “com.”

Now, whenever we add or rename a test, Flex Builder runs test_suite_generator.rb and regenerates the Suite.as file. We don’t have to manually maintain anything.