[greenstone-users] Large CDS/ISIS database

From John Rose
DateWed Dec 17 22:37:52 2008
Subject [greenstone-users] Large CDS/ISIS database
In-Reply-To (20081207220020-A697A3F20EB-mail-alumni-caltech-edu)
Dear Renate,

In the CDS/ISIS to Greenstone guide
(http://wiki.greenstone.org/wiki/gsdoc/others/CDS-ISIS_to_DL.doc),
the limitations to exploding are explained in chapter 4 and several
solutions are given. If adjusting the "records per folder" parameter
does not help, please try exploding and also building from the
command line and inform us if the problem has been solved.

There is also the possibility of splitting the CDS/ISIS database into
several sub-databases (exporting different MFN ranges to separate ISO
files in CDS/ISIS, and reimporting them to CDS/ISIS databases with
different names). You can then build separate Greenstone collections
to be searched with the cross-collection search facility (to be set
in the GLI Format panel). This has the disadvantage that browsing
across more than one the sub-databases at one time would not be possible.

Modifying GLI to handle the large amounts of metadata associated with
large bibliographic databases would be a major project which could be
considered if there is a sufficiently broad user demand.

Best regards, John

>From: rmorgenstern@unam.na
>To: greenstone-users@list.scms.waikato.ac.nz
>Date: Sat, 06 Dec 2008 15:53:09 +0000
>Subject: [greenstone-users] Large CDS/ISIS database
>
>Good day,
>
>I want to explode a CDS/ISIS database with 32146 records. Is there a
>limit to the number of records that can be exploded? I used 100
>records per folder. It just hangs for a long while and then actually
>explodes some records, but not all the 32146.
>Here is the error message: For testing purposes I entered 3 file
>names of tiff files in the last 3 records of the database with the
>corresponding files in the prefix folder. If necessary I can send the
>database master file, the FDT and XRF off list.
>
>Thanks very much
>Renate Morgenstern
>
> Number of records: 32146
> Obtaining document file c:winisisdataair2416...
> doc file full = c:winisisdataair2416
> WARNING: Could not obtain document file c:winisisdataair2416
> Creating a nul document 2416.nul
> Obtaining document file c:winisisdataair2416...
> doc file full = c:winisisdataair2416
> WARNING: Could not obtain document file c:winisisdataair2416
> Creating a nul document 2416.nul
> Obtaining document file c:winisisdataair2416...
> doc file full = c:winisisdataair2416
> WARNING: Could not obtain document file c:winisisdataair2416
> Creating a nul document 2416.nul
> Obtaining document file c:winisisdataair2416...
> doc file full = c:winisisdataair2416
> WARNING: Could not obtain document file c:winisisdataair2416
> Creating a nul document 2416.nul
> Obtaining document file c:winisisdataair2416...
> doc file full = c:winisisdataair2416
> WARNING: Could not obtain document file c:winisisdataair2416
> Creating a nul document 2416.nul
> Obtaining document file c:winisisdataair2416...
> doc file full = c:winisisdataair2416
> WARNING: Could not obtain document file c:winisisdataair2416
> Creating a nul document 2416.nul
> Obtaining document file c:winisisdataair2416...
> doc file full = c:winisisdataair2416
> WARNING: Could not obtain document file c:winisisdataair2416
> Creating a nul document 2416.nul
> Obtaining document file c:winisisdataair2416...
> doc file full = c:winisisdataair2416