[greenstone-users] RE: [greenstone-devel] distributed storage for objects?

From Mehrling, Martin
DateSat Aug 15 04:03:57 2009
Subject [greenstone-users] RE: [greenstone-devel] distributed storage for objects?
In-Reply-To (005c01ca1c27$c6250e60$526f2b20$-gov-ar)
Diego,

Very helpful! Thanks!

It works well on the same drive but when I try different drives (Windows Server 2003) it kind of works but not as well. I'm getting errors like this:

import.pl> util::hard_link: unable to create hard link. Attempting to copy file: C:Program FilesGreenstone2 mpF53.gif -> D:GstoneColls2archivesHASHfb50.dirmoonflag_thumb.gif

Should I not use drive letters? I also want to read/write to different machines. Do you know why this is raising an error? Could you let me know what values you are using for 'importdir' and 'archivedir'?

Thanks a lot,
Martin

************************************
Martin Mehrling
Digital Systems Specialist
m.mehrling@neu.edu<mailto:m.mehrling@neu.edu>
617.373.5885
========================
301 Snell Library
Northeastern University
Boston, Massachusetts 02115

From: Diego Spano [mailto:dspano@orsna.gov.ar]
Sent: Thursday, August 13, 2009 11:07 AM
To: Mehrling, Martin
Cc: greenstone-users@list.scms.waikato.ac.nz
Subject: RE: [greenstone-devel] distributed storage for objects?

Martin,

In GS2 you have 3 folders that requieres disk space: import, archive and index. The index folder MUST be inside the collectionâ–ˇs root folder. You canâ–ˇt separate it.

But the other two yes. The import process has two important options:

-importdir: indicates from where to take the source documents. It can be another disk or another machine.
-archivedir: indicates the path to archive folder where to save the converted documents. It can be another disk or another machine.

The build process has the option -archivedir too, and shoul be set to the same path that you use in import process.

Let me comment you about a digital library I set here:

GS was on installed on a linux server and I mounted a Windows file system over a linux filesystem.

The server has 2 disks (not two partitions but two separate disks). When I run the import process the source documents are read from the windows file system mounted locally (so, no space wasted on linux disk) and the archives are generated on the second disk.

Then the buildprocess take documents from that disk and generates indexes on the maind hard disk. This way you vae separate the 3 parts of the process, and even more, having the archives and indexes in differents disks is a way to get a better performance.

Hope this helps you.

Diego Spano

De: greenstone-devel-bounces@list.scms.waikato.ac.nz [mailto:greenstone-devel-bounces@list.scms.waikato.ac.nz] En nombre de Mehrling, Martin
Enviado el: Jueves, 13 de Agosto de 2009 10:49 a.m.
Para: greenstone-users@list.scms.waikato.ac.nz; greenstone-devel@list.scms.waikato.ac.nz
Asunto: [greenstone-devel] distributed storage for objects?

Hi Folks,

We are running out of space on our Greenstone server and were wondering how other Greenstone sites deal with this problem. I know Greenstone3 has a more distributed model, but we aren't ready to move to that version just yet. Have any of you figured out a way to store objects in multiple locations?

Thanks a lot for any ideas!

Martin

************************************
Martin Mehrling
Digital Systems Specialist
m.mehrling@neu.edu<mailto:m.mehrling@neu.edu>
617.373.5885
========================
301 Snell Library
Northeastern University
Boston, Massachusetts 02115


-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://list.scms.waikato.ac.nz/mailman/private/greenstone-users/attachments/20090814/24bba5bd/attachment.html