Re: [greenstone-devel] Incremental rebuild question

From Richard Managh
DateMon, 09 Jan 2006 13:02:10 +1300
Subject Re: [greenstone-devel] Incremental rebuild question
In-Reply-To (f06230905bfe44eef5f93-172-16-1-51)
Hi Rich,

I reread your question and noticed I had missed out answering some of it.

A little background first, to aid the explanation:
Greenstones full process for creating a completely build collection
involves a import process, which takes files to be added to the
collection in a "import" directory, and converts them into greenstones
internal xml form (Greenstone Archive Format) with associated files in a
"archives" directory. This is followed by a build process which converts
the contents of the "archives" directory into an "index" directory
containing the built collection ready to be served.

Regarding adding a few items for importing:

To clarify: In most cases, if you add a few items for importing, during
the import process greenstone only imports the few items you added from
the import directory into the archives directory.

The removeold option completely removes the contents of the archives
directory, before starting the import process. This means that instead
of a few new items been added to the archives directory, the entire
contents of the import directory would be reimported into the archives

So, again, the Incremental Rebuild option is a feature of GLI that tries
to do nothing, or only a build, or an import and a build, depending on
what the user has changed in the collection, the removeold option forces
a complete fresh import during greenstones import process.

Here are some (not exhaustive) senarios for Incremental Rebuild:

When it does nothing to rebuild the collection:
The user has only changed a format statement. This does not require the
collection to be rebuilt for the change to come into effect.

When it only "builds" the collection:
The user has only changed options for a classifier. This only affects
the build process, as this only affects the way that classifer is
constructed, and this happens during the build process.

When it imports, and builds the collection:
The user has either added new documents to the collection, or changed
metadata for a document in the collection. This requires a full import
and build because those changes need to be available in the archives
directory (previously mentioned) to be available to the build process,
to build the collection with the new information.

Hope this is helpful,

DL Consulting
Greenstone Digital Library and Digitisation Specialists

Richard Managh wrote:

Hi Rich,

The Incremental Rebuild option tries to skip the import phase or the
build phase if it decides they are unnecessary based on the changes you
have made in the GLI to your collection. If you want to ensure that the
GLI is building your collection properly, choose the Complete Rebuild
option, but in most cases Incremental Rebuild should be fine.

Hope this helps,

DL Consulting
Greenstone Digital Library and Digitisation Specialists

Rich Robinson wrote:

> Hi, in Greenstone 2.62 there are Complete Rebuild and Incremental
> Rebuild options in the GLI. Exactly when should Incremental be used?
> I thought from previous discussions that in the GLI, you can only do a
> full import and full build, even if you are adding only a few items,
> so what does Incremental do?
> Also how does the removeold option apply, vis a vis Incremental? Are
> they related in any way?
> Rich
> _______________________________________________
> greenstone-devel mailing list