RE: [greenstone-users] Out of Memory Error

From Nathan Einwechter
DateTue, 18 Oct 2005 17:38:51 -0400
Subject RE: [greenstone-users] Out of Memory Error
In-Reply-To (13960BD5-1394-47BF-974B-B66191E2811C-dlconsulting-co-nz)
The problem is that this is ALL custom. I'm actually using it for
hierarchies of security log information (IP addresses, ports,
protocols). This is all generated by a program I have created.

I'm thinking about just exporting to a self designed XML repository at
this stage though. Greenstone has been good so far, but we're looking at
scaling up even farther and I don't know how it can be done using GSDL.

-- Nathan

-----Original Message-----
From: greenstone-users-bounces@list.scms.waikato.ac.nz
[mailto:greenstone-users-bounces@list.scms.waikato.ac.nz] On Behalf Of
John Rowe
Sent: October 18, 2005 5:28 PM
To: greenstone-users@list.scms.waikato.ac.nz
Subject: Fwd: [greenstone-users] Out of Memory Error

What is contained in your metadata.xml? What filetypes are in
your collection? Is there any way you could build an equivalent
collection without using the metadata.xml? Have you experimented with
a custom import plugin?

Cheers,
John Rowe

DL Consulting
Greenstone Digital Library and Digitisation Specialists
contact@dlconsulting.co.nz
www.dlconsulting.co.nz


Begin forwarded message:

> From: "Nathan Einwechter" <nathan@inorb.com>
> Date: 19 October 2005 10:04:14 AM
> To: <greenstone-users@list.scms.waikato.ac.nz>
> Subject: RE: [greenstone-users] Out of Memory Error
>
>
> Does anyone know the maximum size I can have for a metadata.xml
> file for
> a given amount of ram etc.?
>
> -- Nathan
>
> -----Original Message-----
> From: greenstone-users-bounces@list.scms.waikato.ac.nz
> [mailto:greenstone-users-bounces@list.scms.waikato.ac.nz] On Behalf Of
> John Rowe
> Sent: October 17, 2005 10:03 PM
> To: greenstone-users@list.scms.waikato.ac.nz
> Subject: Fwd: [greenstone-users] Out of Memory Error
>
> it seems that the Greenstone import process is simply not built
> to handle a metadata file that large. The easiest solution would be
> to split your import directory (and metadata.xml file) up into
> smaller directories under the main import directory. The filesystem
> should end up looking something like the following:
> import/
> import/001/metadata.xml
> import/001/file1
> import/001/file2
> import/001/.....
> import/001/file9
> import/002/metadata.xml
> import/002/file10
> import/002/file11
> import/002/...
> import/002/file19
> import/003/metadata.xml
> import/003/file20
> import/001/file21
> .............
>
> Hope this helps.
>
> Cheers,
> John Rowe
>
> DL Consulting
> Greenstone Digital Library and Digitisation Specialists
> contact@dlconsulting.co.nz
> www.dlconsulting.co.nz
>
>
>
> _______________________________________________
> greenstone-users mailing list
> greenstone-users@list.scms.waikato.ac.nz
> https://list.scms.waikato.ac.nz/mailman/listinfo/greenstone-users
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
https://list.scms.waikato.ac.nz/mailman/private/greenstone-users/attachm
ents/20051019/05ec692b/attachment.html
_______________________________________________
greenstone-users mailing list
greenstone-users@list.scms.waikato.ac.nz
https://list.scms.waikato.ac.nz/mailman/listinfo/greenstone-users