What Changed

The introduction of multiple database support touched many parts of Django, and the testing framework is no exception.

Django Developers

Django's tests now require a more detailed settings file in order to pass. If you run the test suite with your old settings file, which simply defined a single database, the MultiDB tests will fail (22 of them, to be exact). An example of the minimum settings file needed is supplied below. As of the Django Pycon sprint, this file is actually shipped with Django itself. So in order to run the Django test suite on SQLite, you don't have to touch your filesystem!

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
    },
    'other': {
        'ENGINE': 'django.db.backends.sqlite3',
        'TEST_NAME': 'other_db',
    }
}

To run the Test Suite, cd into the tests directory of your Django checkout, and run the following command:

./runtests.py --settings=test_sqlite

This will run with the supplied test_sqlite.py file, which is identical to the above settings file.

App Developers

This only really affects you if your application expects to be deployed against a multiple database environment. The Django documentation on testing has specific topics about Testing Master/Slave setups, which allows you to use the TEST_MIRROR setting on a slave database and have it mirror the master for tests. The Django TestCase also added the multidb attribute, which will flush all databases before running the test cases inside. By default, Django only flushes the default database, as an optimization.

However, unless you are specifically writing against a multible database environment, you should be able to safely ignore these changes.

What Changed

As of Revision 11843, Django added a --failfast option to the test runner. When this option is selected, Django will exit the test run once it encounters a failure. This is useful for certain work flows, including some forms of Test Driven Development, where you expect tests to fail often, and don't want to run through the entire test to see what has failed.

In Django 1.1 and before, if you were to <Ctrl+C> and cancel a test run, you would be treated with a nice ugly traceback with no information.

............^CTraceback (most recent call last):
  File "./manage.py", line 11, in <module>
    execute_manager(settings)
  File "/Users/eric/svn-clones/django/template/__init__.py", line 285, in parse
    compiled_result = compile_func(self, token)
KeyboardInterrupt

When you try the same trick as of Revision 12034, the current test will finish running and exit gracefully.

.....................^C <Test run halted by Ctrl-C> .
----------------------------------------------------------------------
Ran 22 tests in 1.308s

OK

This allows you to get the same benefits from the --failfast option, even if you forget to set it at the beginning. Once you see a test fail, you can simply Ctrl+C, and you will get your failing test output back out.

Django Developers

This is important for developing against the Django source tree because the Django test suite takes a long time to run. The full suite can take upwards of 10 minutes, especially for bigger databases like Oracle. Previously, if you saw a failure you would just have to wait until the tests had finished running to be able to see the actual error. Breaking out early would result in the loss of the data you were after! Now you will be able to easily bail out on the first failure, or if you're forgotten the --failfast option, a simple Ctrl+C will do it for you.

App Developers

This is important for integrating testing into your work flow. One of the main ideas behind testing is that you should be running your tests while you are developing, and see if you are making errors while you are writing the code.

Adding the --failfast argument to whatever mechanism you have to run your tests after changes, will allow you to work faster when things break. You will be alerted right away to breakages in your code, and be able to fix them.

There are a number of other features that would be nice to have in this realm as well. Being able to run your full test suite, and then only run the failing tests from the previous run is another optimization here. This will hopefully make it into Django 1.3, or be realized as a stand-alone app before then.

There are also a number of efforts to integrate your Django tests with nose. So if you haven't started writing your test suite, looking at django-nose or django-sane-testing might make sense, as nose already has a lot of these features built in.

What Changed

As of Revision 12255, the default test runner for Django is now a class. There is code that will detect if you are trying to pass in a 1.1-style function based runner, and do the appropriate thing. However, you should try to update custom test runners to the new style, because it will result in being less code and more maintanable.

This is only really applicable to application developers, because if you are running the Django test suite, you should be using the official Django test runner.

An example

In my Pony Utils project, I wanted to create my own test runner, and pre-1.2, that required me to copy 60 lines of code, only to edit a few (line 60). The run_tests function in Django is now 7 lines of code, which calls out to other methods. These other methods are easy to subclass in your own test runner, overriding only the parts needed to get the desired behavior.

The diff of the new and old show the changed lines of code. However, this really should be done in the suite_result function on the test runner class, since I only want to modify how the results of the tests are sent. This shows a place where the design could be improved.