Re: [greenstone-users] Re: greenstone-users Digest, Vol 53, Issue 21

From John Rowe
DateTue, 28 Aug 2007 12:38:02 +1200
Subject Re: [greenstone-users] Re: greenstone-users Digest, Vol 53, Issue 21
In-Reply-To (F93D6D24-6E65-48D3-82E1-CFDC97392EE6-uiuc-edu)
Hello Jerome,

To convert them to the right byte order you need access to both a
big endian and little endian machine. You need to dump the databases
to text and then re-import that text into the databases. On the
source machine run (make sure you've ran . setup.bash first):

db2txt users.db > users.db.txt

Then copy the text file across to the destination machine. On the
destination machine run:

txt2db users.db < users.db.txt

That'll build a GDBM database of the correct endianness for the
destination platform. This process worked for me on a Mac OS X
system. The key.db just stores the keys for the currently
authenticated users, those keys are only fresh for half an hour on
standard Greenstone. I suggest just removing the keys.db and creating
a new one with write permissions for Greenstone by running the
following command in the gsdl/etc/ directory:

mv keys.db keys.db.old && touch keys.db && chmod a+rw keys.db

Indexes also have the same problem with GDBM. To convert the index
across, either run:

buildcol.pl [collection_name]

Or run through the same process as above with the gsdl/collect/
[collection_name]/index/text/[collection_name].ldb or bdb.
I did have some unresolved problems with MG endianness on the Intel
Mac that I have yet to track down. For us here, it ended up being
easier to do a full rebuild. I think the problems may stem from
running an older version of Greenstone - specifically GSDL 2.5. You
probably won't see the problems I had, but I'd love to hear whether
that process I outlined above worked for your collections on a more
recent Greenstone installation.

Cheers,
John Rowe

DL Consulting
Greenstone Digital Library and Digitisation Specialists
contact@dlconsulting.com
www.dlconsulting.com


On 28/08/2007, at 12:01 PM, Jerome McDonough wrote:

> OK, I can answer part of my own question below now. The key.db and
> users.db files distributed
> with the OS X have Intel machine byte order, which means that they
> can't be read by the software
> compiled for use on a PowerPC. I managed to create a functioning
> users.db file by simply deleting
> the existing one and trying to access a password protected section
> of the Administration Page, which
> caused Greenstone to create a new users.db file with the correct
> byte order. That specific problem is
> solved, but I'm wondering what problems will be resulting from
> key.db and history.db not being readable,
> and whether there's a simple way to convert them to the proper byte
> order.
>
> Any help on question 2 still appreciated.
>
> On Aug 27, 2007, at 5:07 PM, greenstone-users-
> request@list.scms.waikato.ac.nz wrote:
>> Message: 6
>> Date: Mon, 27 Aug 2007 16:33:29 -0500
>> From: Jerome McDonough <jmcdonou@uiuc.edu>
>> Subject: [greenstone-users] GLI errors on OS X
>> To: greenstone-users@list.scms.waikato.ac.nz
>> Message-ID: <C9C4A944-3FF4-4A63-8739-6B3D8770CBED@uiuc.edu>
>> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
>>
>> Howdy,
>>
>> I'm trying to setup Greenstone 2.74 on a PowerBook G4 (a powerpc-
>> based machine) running OS X 10.4.10.
>> I've done a clean install and have been able to access Greenstone via
>> my machine's apache server without
>> any problem. However, I'm experiencing two difficulties I can't
>> resolve. Any thoughts on either of these appreciated.
>>
>> 1. adminstrative login -- I can't login to the adminstrator account
>> through the web interface at all.
>> I've tried different passwords during the installation, using the
>> default 'admin' password, no matter
>> what I do, trying to login to the admin account draws the error
>> message "Either your username or password was incorrect. "
>> Looking at my apache error log, I'm seeing the following:
>>
>> database open failed on: /Library/WebServer/Documents/gsdl/etc/
>> users.db
>> database open failed on: /Library/WebServer/Documents/gsdl/etc/
>> users.db
>> database open failed on: /Library/WebServer/Documents/gsdl/etc/key.db
>> database open failed on: /Library/WebServer/Documents/gsdl/etc/key.db
>>
>> Both the users.db and the key.db file are owned by me, in the admin
>> group, and have -rw-rw-rw-
>> permissions set.
>>
>> 2. GLI -- when I run sli.sh, it starts out reasonably well, and I do
>> get the initial user interface displayed,
>> but I'm also seeing these error messages:
>>
>> Greenstone Librarian Interface (GLI)
>> Copyright (C) 2006, New Zealand Digital Library Project, University
>> Of Waikato
>> GLI comes with ABSOLUTELY NO WARRANTY; for details see LICENSE.txt
>> This is free software, and you are welcome to redistribute it
>>
>> Checking GSDL: /Library/WebServer/Documents/gsdl
>> Your environment has successfully been set up to run Greenstone
>> Checking Perl: /usr/bin/perl
>> Checking Java: /usr/bin/java
>>
>> Running the Greenstone Librarian Interface...
>> Version: v2.74
>>
>> Java vendor: Apple Computer, Inc.
>> Note that the Librarian Interface has been developed and tested with
>> Java from Sun Microsystems.
>> SAX exception: The markup in the document preceding the root element
>> must be well-formed.
>> org.xml.sax.SAXParseException: The markup in the document preceding
>> the root element must be well-formed.
>> at org.apache.xerces.framework.XMLParser.reportError
>> (XMLParser.java:1196)
>> at
>> org.apache.xerces.framework.XMLDocumentScanner.reportFatalXMLError
>> (XMLDocumentScanner.java:626)
>> at org.apache.xerces.framework.XMLDocumentScanner
>> $XMLDeclDispatcher.dispatch(XMLDocumentScanner.java:809)
>> at org.apache.xerces.framework.XMLDocumentScanner.parseSome
>> (XMLDocumentScanner.java:381)
>> at org.apache.xerces.framework.XMLParser.parse
>> (XMLParser.java:1081)
>> at org.greenstone.gatherer.util.XMLTools.parseXML
>> (XMLTools.java:436)
>> at org.greenstone.gatherer.util.XMLTools.parseXML
>> (XMLTools.java:412)
>> at org.greenstone.gatherer.gui.DownloadPane.loadDownload
>> (DownloadPane.java:256)
>> at org.greenstone.gatherer.gui.DownloadPane.<init>
>> (DownloadPane.java:106)
>> at org.greenstone.gatherer.gui.GUIManager.display
>> (GUIManager.java:363)
>> at org.greenstone.gatherer.Gatherer.openGUI(Gatherer.java:
>> 431)
>> at org.greenstone.gatherer.GathererProg.main
>> (GathererProg.java:80)
>> SAX exception: The element type "PluginList" must be terminated by
>> the matching end-tag "</PluginList>".
>> org.xml.sax.SAXParseException: The element type "PluginList" must be
>> terminated by the matching end-tag "</PluginList>".
>> at org.apache.xerces.framework.XMLParser.reportError
>> (XMLParser.java:1196)
>> at
>> org.apache.xerces.framework.XMLDocumentScanner.reportFatalXMLError
>> (XMLDocumentScanner.java:635)
>> at org.apache.xerces.framework.XMLDocumentScanner
>> $ContentDispatcher.endOfInput(XMLDocumentScanner.java:1462)
>> at org.apache.xerces.framework.XMLDocumentScanner.endOfInput
>> (XMLDocumentScanner.java:418)
>> at
>> org.apache.xerces.validators.common.XMLValidator.sendEndOfInputNotifi
>> cat
>> ions(XMLValidator.java:694)
>> at
>> org.apache.xerces.readers.DefaultEntityHandler.changeReaders
>> (DefaultEntityHandler.java:1026)
>> at org.apache.xerces.readers.XMLEntityReader.changeReaders
>> (XMLEntityReader.java:168)
>> at
>> org.apache.xerces.readers.AbstractCharReader.changeReaders
>> (AbstractCharReader.java:150)
>> at org.apache.xerces.readers.AbstractCharReader.scanContent
>> (AbstractCharReader.java:1097)
>> at org.apache.xerces.framework.XMLDocumentScanner
>> $ContentDispatcher.dispatch(XMLDocumentScanner.java:1150)
>> at org.apache.xerces.framework.XMLDocumentScanner.parseSome
>> (XMLDocumentScanner.java:381)
>> at org.apache.xerces.framework.XMLParser.parse
>> (XMLParser.java:1081)
>> at org.greenstone.gatherer.util.XMLTools.parseXML
>> (XMLTools.java:436)
>> at
>> org.greenstone.gatherer.greenstone.Plugins.parsePluginsListXML
>> (Plugins.java:302)
>> at
>> org.greenstone.gatherer.greenstone.Plugins.loadPluginsList
>> (Plugins.java:232)
>> at org.greenstone.gatherer.Gatherer.openGUI(Gatherer.java:
>> 458)
>> at org.greenstone.gatherer.GathererProg.main
>> (GathererProg.java:80)
>> java.lang.NullPointerException
>> at
>> org.greenstone.gatherer.greenstone.Plugins.parsePluginsListXML
>> (Plugins.java:303)
>> at
>> org.greenstone.gatherer.greenstone.Plugins.loadPluginsList
>> (Plugins.java:232)
>> at org.greenstone.gatherer.Gatherer.openGUI(Gatherer.java:
>> 458)
>> at org.greenstone.gatherer.GathererProg.main
>> (GathererProg.java:80)
>>
>> I can't open the demo collection; when I try, I get a warning popup
>> saying "the presence of a lock file
>> suggests this collection is already open" and asking if I want to
>> steal control of the lock. If I say OK,
>> it says the collection cannot be opened, and I get the following
>> error messages in the terminal window:
>>
>> SAX exception: The markup in the document preceding the root element
>> must be well-formed.
>> org.xml.sax.SAXParseException: The markup in the document preceding
>> the root element must be well-formed.
>> at org.apache.xerces.framework.XMLParser.reportError
>> (XMLParser.java:1196)
>> at
>> org.apache.xerces.framework.XMLDocumentScanner.reportFatalXMLError
>> (XMLDocumentScanner.java:626)
>> at org.apache.xerces.framework.XMLDocumentScanner
>> $XMLDeclDispatcher.dispatch(XMLDocumentScanner.java:809)
>> at org.apache.xerces.framework.XMLDocumentScanner.parseSome
>> (XMLDocumentScanner.java:381)
>> at org.apache.xerces.framework.XMLParser.parse
>> (XMLParser.java:1081)
>> at org.greenstone.gatherer.util.XMLTools.parseXML
>> (XMLTools.java:436)
>> at org.greenstone.gatherer.util.XMLTools.parseXML
>> (XMLTools.java:412)
>> at
>> org.greenstone.gatherer.collection.ScriptOptions.loadArguments
>> (ScriptOptions.java:252)
>> at org.greenstone.gatherer.collection.ScriptOptions.<init>
>> (ScriptOptions.java:43)
>> at org.greenstone.gatherer.collection.Collection.<init>
>> (Collection.java:113)
>> at
>> org.greenstone.gatherer.collection.CollectionManager.loadCollectionIn
>> ter
>> nal(CollectionManager.java:969)
>> at
>> org.greenstone.gatherer.collection.CollectionManager.access$200
>> (CollectionManager.java:82)
>> at org.greenstone.gatherer.collection.CollectionManager
>> $LoadCollectionTask.run(CollectionManager.java:856)
>> java.lang.NullPointerException
>> at
>> org.greenstone.gatherer.collection.ScriptOptions.loadArguments
>> (ScriptOptions.java:255)
>> at org.greenstone.gatherer.collection.ScriptOptions.<init>
>> (ScriptOptions.java:43)
>> at org.greenstone.gatherer.collection.Collection.<init>
>> (Collection.java:113)
>> at
>> org.greenstone.gatherer.collection.CollectionManager.loadCollectionIn
>> ter
>> nal(CollectionManager.java:969)
>> at
>> org.greenstone.gatherer.collection.CollectionManager.access$200
>> (CollectionManager.java:82)
>> at org.greenstone.gatherer.collection.CollectionManager
>> $LoadCollectionTask.run(CollectionManager.java:856)
>> SAX exception: The markup in the document preceding the root element
>> must be well-formed.
>> org.xml.sax.SAXParseException: The markup in the document preceding
>> the root element must be well-formed.
>> at org.apache.xerces.framework.XMLParser.reportError
>> (XMLParser.java:1196)
>> at
>> org.apache.xerces.framework.XMLDocumentScanner.reportFatalXMLError
>> (XMLDocumentScanner.java:626)
>> at org.apache.xerces.framework.XMLDocumentScanner
>> $XMLDeclDispatcher.dispatch(XMLDocumentScanner.java:809)
>> at org.apache.xerces.framework.XMLDocumentScanner.parseSome
>> (XMLDocumentScanner.java:381)
>> at org.apache.xerces.framework.XMLParser.parse
>> (XMLParser.java:1081)
>> at org.greenstone.gatherer.util.XMLTools.parseXML
>> (XMLTools.java:436)
>> at org.greenstone.gatherer.util.XMLTools.parseXML
>> (XMLTools.java:412)
>> at
>> org.greenstone.gatherer.collection.ScriptOptions.loadArguments
>> (ScriptOptions.java:252)
>> at org.greenstone.gatherer.collection.ScriptOptions.<init>
>> (ScriptOptions.java:43)
>> at org.greenstone.gatherer.collection.Collection.<init>
>> (Collection.java:114)
>> at
>> org.greenstone.gatherer.collection.CollectionManager.loadCollectionIn
>> ter
>> nal(CollectionManager.java:969)
>> at
>> org.greenstone.gatherer.collection.CollectionManager.access$200
>> (CollectionManager.java:82)
>> at org.greenstone.gatherer.collection.CollectionManager
>> $LoadCollectionTask.run(CollectionManager.java:856)
>> java.lang.NullPointerException
>> at
>> org.greenstone.gatherer.collection.ScriptOptions.loadArguments
>> (ScriptOptions.java:255)
>> at org.greenstone.gatherer.collection.ScriptOptions.<init>
>> (ScriptOptions.java:43)
>> at org.greenstone.gatherer.collection.Collection.<init>
>> (Collection.java:114)
>> at
>> org.greenstone.gatherer.collection.CollectionManager.loadCollectionIn
>> ter
>> nal(CollectionManager.java:969)
>> at
>> org.greenstone.gatherer.collection.CollectionManager.access$200
>> (CollectionManager.java:82)
>> at org.greenstone.gatherer.collection.CollectionManager
>> $LoadCollectionTask.run(CollectionManager.java:856)
>> java.lang.NullPointerException
>> at org.greenstone.gatherer.greenstone.Plugins.getPlugin
>> (Plugins.java:74)
>> at org.greenstone.gatherer.cdm.Plugin.create(Plugin.java:75)
>> at
>> org.greenstone.gatherer.cdm.DOMProxyListModel.getElementAt
>> (DOMProxyListModel.java:221)
>> at org.greenstone.gatherer.cdm.PluginManager.<init>
>> (PluginManager.java:80)
>> at
>> org.greenstone.gatherer.cdm.CollectionDesignManager.loadDesignDetails
>> (CollectionDesignManager.java:125)
>> at
>> org.greenstone.gatherer.cdm.CollectionDesignManager.<init>
>> (CollectionDesignManager.java:100)
>> at
>> org.greenstone.gatherer.collection.CollectionManager.loadCollectionIn
>> ter
>> nal(CollectionManager.java:1008)
>> at
>> org.greenstone.gatherer.collection.CollectionManager.access$200
>> (CollectionManager.java:82)
>> at org.greenstone.gatherer.collection.CollectionManager
>> $LoadCollectionTask.run(CollectionManager.java:856)
>>
>> It looks to me like several problems seem to focus on XML parsing of
>> the PluginList, so just for fun, here's the
>> disastrous output from pluginfo.pl. Note that everything seems to be
>> going fine up until the fourth plugin it
>> processes (CONTENTdm) and then things go south. I tried removing the
>> CONTENTdmPlug.pm file to see if that
>> would make a difference, but it just blows up on the next in line.
>>
>> fiachdubh-2:/Library/WebServer/Documents/gsdl/bin/script
>> jeromemcdonough$ ./pluginfo.pl -listall
>> <?xml version="1.0" encoding="UTF-8"?>
>> <PluginList length="54">
>> <PlugInfo>
>> <Name>ArcPlug</Name>
>> <Desc>Plugin which recurses through an archives.inf file (i.e. the
>> file generated in the archives directory when an import is done),
>> processing each file it finds.</Desc>
>> <Abstract>no</Abstract>
>> <Inherits>yes</Inherits>
>> <Processes></Processes>
>> <Blocks></Blocks>
>> <Explodes>no</Explodes>
>> </PlugInfo>
>> <PlugInfo>
>> <Name>BasPlug</Name>
>> <Desc>Base class for all the import plugins.</Desc>
>> <Abstract>yes</Abstract>
>> <Inherits>no</Inherits>
>> <Processes></Processes>
>> <Blocks></Blocks>
>> <Explodes>no</Explodes>
>> </PlugInfo>
>> <PlugInfo>
>> <Name>BibTexPlug</Name>
>> <Desc>BibTexPlug reads bibliography files in BibTex format.
>> BibTexPlug creates a document object for every reference in the file.
>> It is a subclass of SplitPlug, so if there are multiple records, all
>> are read.</Desc>
>> <Abstract>no</Abstract>
>> <Inherits>yes</Inherits>
>> <Processes>(?i).bib$</Processes>
>> <Blocks></Blocks>
>> <Explodes>yes</Explodes>
>> </PlugInfo>
>> <PlugInfo>
>> <Name>BookPlug</Name>
>> <Desc>Creates multi-level document from document containing
>> &amp;lt;&amp;lt;TOC&amp;gt;&amp;gt; level tags. Metadata for each
>> section is taken from any other tags on the same line as the
>> &amp;lt;&amp;lt;TOC&amp;gt;&amp;gt;. e.g.
>> &amp;lt;&amp;lt;Title&amp;gt;&amp;gt;xxxx&amp;lt;&amp;lt;/
>> Title&amp;gt;&amp;gt; sets Title metadata. Everything else between
>> TOC tags is treated as simple html (i.e. no processing of html links
>> or any other HTMLPlug type stuff is done). Expects input files to
>> have a .hb file extension by default (this can be changed by adding a
>> -process_exp option a file with the same name as the hb file but
>> a .jpg extension is taken as the cover image (jpg files are blocked
>> by this plugin). BookPlug is a simplification (and extension) of the
>> HBPlug used by the Humanity Library collections. BookPlug is faster
>> as it expects the input files to be cleaner (The input to the HDL
>> collections contains lots of excess html tags around
>> &amp;lt;&amp;lt;TOC&amp;gt;&amp;gt; tags, uses
>> &amp;lt;&amp;lt;I&amp;gt;&amp;gt; tags to specify images, and simply
>> takes all text between &amp;lt;&amp;lt;TOC&amp;gt;&amp;gt; tags and
>> start of text to be Title metadata). If you're marking up documents
>> to be displayed in the same way as the HDL collections, use this
>> plugin instead of HBPlug.</Desc>
>> <Abstract>no</Abstract>
>> <Inherits>yes</Inherits>
>> <Processes>(?i).hb$</Processes>
>> <Blocks>.jpg$</Blocks>
>> <Explodes>no</Explodes>
>> </PlugInfo>
>> Can't locate loadable object for module XML::Parser::Expat in @INC
>> (@INC contains: /Library/WebServer/Documents/gsdl/perllib/dynamic/
>> lib/
>> site_perl/5.005/i686-linux /Library/WebServer/Documents/gsdl/perllib/
>> cpan /Library/WebServer/Documents/gsdl/perllib /Library/WebServer/
>> Documents/gsdl/perllib/plugins /Library/WebServer/Documents/gsdl/
>> perllib/cpan /Library/WebServer/Documents/gsdl/perllib /System/
>> Library/Perl/5.8.6/darwin-thread-multi-2level /System/Library/Perl/
>> 5.8.6 /Library/Perl/5.8.6/darwin-thread-multi-2level /Library/Perl/
>> 5.8.6 /Library/Perl /Network/Library/Perl/5.8.6/darwin-thread-
>> multi-2level /Network/Library/Perl/5.8.6 /Network/Library/Perl /
>> System/Library/Perl/Extras/5.8.6/darwin-thread-multi-2level /System/
>> Library/Perl/Extras/5.8.6 /Library/Perl/5.8.1/darwin-thread-
>> multi-2level /Library/Perl/5.8.1 . /Library/WebServer/Documents/gsdl/
>> perllib/cpan/perl-5.8) at /Library/WebServer/Documents/gsdl/perllib/
>> cpan/perl-5.8/XML/Parser.pm line 15
>> Compilation failed in require at /Library/WebServer/Documents/gsdl/
>> perllib/cpan/perl-5.8/XML/Parser.pm line 15.
>> BEGIN failed--compilation aborted at /Library/WebServer/Documents/
>> gsdl/perllib/cpan/perl-5.8/XML/Parser.pm line 27.
>> Compilation failed in require at /Library/WebServer/Documents/gsdl/
>> perllib/XMLParser.pm line 30.
>> BEGIN failed--compilation aborted at /Library/WebServer/Documents/
>> gsdl/perllib/XMLParser.pm line 30.
>> Compilation failed in require at /Library/WebServer/Documents/gsdl/
>> perllib/plugins/HTMLPlug.pm line 42.
>> BEGIN failed--compilation aborted at /Library/WebServer/Documents/
>> gsdl/perllib/plugins/HTMLPlug.pm line 42.
>> Compilation failed in require at /Library/WebServer/Documents/gsdl/
>> perllib/plugins/ConvertToPlug.pm line 37.
>> BEGIN failed--compilation aborted at /Library/WebServer/Documents/
>> gsdl/perllib/plugins/ConvertToPlug.pm line 37.
>> Compilation failed in require at /Library/WebServer/Documents/gsdl/
>> perllib/plugins/CONTENTdmPlug.pm line 27.
>> BEGIN failed--compilation aborted at /Library/WebServer/Documents/
>> gsdl/perllib/plugins/CONTENTdmPlug.pm line 27.
>> Compilation failed in require at /Library/WebServer/Documents/gsdl/
>> perllib/plugin.pm line 61.
>>
>>
>>
>>
>>
>>
>>
>> Jerome McDonough, Asst. Professor
>> Graduate School of Library & Information Science
>> University of Illinois, Urbana-Champaign
>> 501 E. Daniel Street, Room 202
>> Champaign, IL 61820
>> (217) 244-5916
>> jmcdonou@uiuc.edu
>>
>>
>>
>>
>>
>>
>> ------------------------------
>>
>> _______________________________________________
>> greenstone-users mailing list
>> greenstone-users@list.scms.waikato.ac.nz
>> https://list.scms.waikato.ac.nz/mailman/listinfo/greenstone-users
>>
>>
>> End of greenstone-users Digest, Vol 53, Issue 21
>> ************************************************
>
> Jerome McDonough, Asst. Professor
> Graduate School of Library & Information Science
> University of Illinois, Urbana-Champaign
> 501 E. Daniel Street, Room 202
> Champaign, IL 61820
> (217) 244-5916
> jmcdonou@uiuc.edu
>
>
>
>
> _______________________________________________
> greenstone-users mailing list
> greenstone-users@list.scms.waikato.ac.nz
> https://list.scms.waikato.ac.nz/mailman/listinfo/greenstone-users