|Date||Fri, 14 Sep 2007 11:44:41 +1200|
|Subject||Re: [greenstone-devel] Testing greenstone3|
At the moment only Greenstone2 undergoes nightly regression tests. The regression testing code is available via svn from http://svn.greenstone.org/test-greenstone.
The regression tests are fairly minimal, but we have plans to make them much more rigourous. There is a program written in perl, called diffcol, which compares collections. Wrapped around this program is a script which checks out the latest gsdl, compiles it, and builds the demo collection with it, then compares the newly built collection with a model version of the same collection. The model collection was built when testing first began (13 Jun 2007).
To run a regression test, you need to add a directory called model-collect to the root of your working copy and put a built collection(s) in it. We can't keep the model collections in the repository because for some reason the regression test will fail if the model collections were built on a different machine than the one you run the test on. (Finding the cause of this phenomenon and deciding what to do about it is one of the things on the todo list.)
Like I say, there are some plans to improve our regression testing, including:
- make the diffcol do a more meaningful comparison, not just a file to file diff comparison
- start testing on a number of different platforms, not just linux
- test the collection serving parts of greenstone, not just the collection building parts
- keep track of the time taken (or better yet, number of cpu cycles) to build and serv certain model collections and pages
- test greenstone3 as well as greenstone2
- use more model collections, including some very large ones
By the way, there is a page with all the regression testing reports on it at http://www.greenstone.org/test-reports
If you have any other questions, please write back. I have most likely missed something important, so feel free to let me know what it is!
Ifi Okoye wrote: