[greenstone-users] Problem Building Remote Collection

From Anupama of Greenstone Team
DateWed Mar 11 12:37:09 2009
Subject [greenstone-users] Problem Building Remote Collection
In-Reply-To (49B6B7E9-9080708-hawaii-edu)
Hi again Daniel,

I'm so sorry, but I made a mistake in my last email: you are using the
remote greenstone server of course, yet, instead of directing you to
editing the appropriate client-gli.sh/client-gli.bat scripts, I asked
you to modify the gli scripts that are used with a local greenstone server.

1. Will you please try my suggestion given in the previous email again,
but this time make the edits to the client-gli script appropriate to the
operating system your running the remote client on (.sh file for Linux,
.bat on Windows).


> Let me know if there's any other data I could give you to assist in
> the troubleshooting.

Thanks. The idea was that we'd try to generate test data consisting of
about 1200 small files and attempt something similar to what you've been
going through and then debug the code. However, if (1) above does not
work out for you, and if your remote server is available to the public
to access, we can try rebuilding your collection from my machine using a
remote client here and try to debug the client code where this stack
exception is occurring. This may be a little difficult since we can't
modify the gli server code on your end with debug statements, but it's
apparently a GLI problem rather than anything to do with the server.

Fingers crossed and sorry again about the error I made,
Anupama


Daniel Ishimitsu wrote:
> Hi Anupama,
>
> Thanks for your response. I changed the 128M to 512M in gli.sh on the
> server, but the same error appears at the same point. Let me know if
> there's any other data I could give you to assist in the troubleshooting.
>
> Daniel Ishimitsu
>
> University of Hawaii
> Hamilton Library, DNS
>
>
>
> Anupama of Greenstone Team wrote:
>> Hi Daniel,
>>
>> We'll have to investigate your problem, but in the meantime, could you
>> try evading it by allocating a larger stacksize to the Java Virtual
>> Machine when running GLI.
>>
>>
>> 1. Open the file gli.bat (if you're on Windows) or gli.sh (if you're
>> on Linux) in a very basic text editor. If on Windows, I'd advise you
>> to open it in Notepad rather than Wordpad.
>>
>>
>> 2 a. If you're on Linux, then in *gli.sh*, find the line:
>>
>> basic_command="$javapath -Xmx128M -classpath
>> classes/:GLI.jar:lib/apache.jar:lib/qfslib.jar
>> org.greenstone.gatherer.GathererProg"
>>
>> b. If you're on Windows, then in *gli.bat* find the TWO occurrences of
>> the line
>> - "%JAVA_EXECUTABLE%" -Xmx128M -cp
>> classes/;GLI.jar;lib/apache.jar;lib/qfslib.jar ....
>>
>>
>> 3. Change the occurrences of "-Xmx128M" to "-Xmx256M". Try running GLI
>> again and see if the StackOverflow problem is gone.
>>
>>
>> 4. If the above did not work, repeate but by changing the value to
>> -Xmx512M.
>> This is now allocating half a Gigabyte of memory to the program,
>> instead of the original 128 Megabytes.
>>
>>
>> Tell us if the above doesn't help.
>>
>> Hopefully, it will allow you to finally get on with your work, while
>> we look into the real causes behind it.
>>
>> Regards,
>> Anupama
>>
>>
>>
>> Daniel Ishimitsu wrote:
>>> Hi Everyone,
>>>
>>> Any help would be appreciated! I have a Greenstone collection that
>>> consists of 1,200 web pages with a total of 16,000 images (gif and
>>> jpg). I'm trying to build the collection using the remote GLI (I
>>> tried both the applet and the separate client program) but I get a
>>> StackOverflowError along the way.
>>> Loading the collection is fine (it takes about 4 minutes), I click on
>>> the Create tab, then click on Complete Rebuild, then Build
>>> Collection. After about 15 minutes I get an import complete message,
>>> but the program seems stuck in a loop and the console displays the
>>> stackoverflowerror. Below is a snipped version of the logs and also
>>> the stacktrace.
>>> Other added bits of messiness:
>>> We're hosting the server on a linux machine, but using the remote
>>> client/applet from a windows machine. The collection was initially
>>> built on a windows box and moved to the linux machine (I tried
>>> creating it from scratch remotely but the Gather stage stopped
>>> responding after a few hundred images were uploaded)
>>>
>>>
>>> Greenstone Message Log:
>>> ************** Import Started **************
>>> The file is being processed by IndexPlugin.
>>> The file ttp_htms/3146.html is being processed by StructuredHTMLPlugin.
>>> The file ttp_htms/2847.html is being processed by StructuredHTMLPlugin.
>>> The file ttp_htms/2482.html is being processed by StructuredHTMLPlugin.
>>> The file ttp_htms/4038.html is being processed by StructuredHTMLPlugin.
>>> ...
>>> The file ttp_htms/3026.html is being processed by StructuredHTMLPlugin.
>>> The file ttp_htms/3101.html is being processed by StructuredHTMLPlugin.
>>> ************** Import Finished **************
>>> 1227 documents were considered for processing:
>>> 1222 documents were processed and included in the collection.
>>> 5 were rejected.
>>>
>>>
>>> Console Output:
>>> *********************************************
>>> Import complete
>>> *********************************************
>>> <ImportComplete considered='1227' processed='1222' blocked='0'
>>> ignored='0' failed='5'>
>>> * 1227 documents were considered for processing
>>> * 1222 were processed and included in the collection
>>> * 5 were rejected
>>> See /opt/gsdl/collect/trustter/etc/fail.log for a list of
>>> unrecognised and/or rejected documents
>>>
>>> at
>>> org.greenstone.gatherer.remote.RemoteGreenstoneServer.sendCommandToServer(RemoteGreenstoneServer.java:505)
>>>
>>> at
>>> org.greenstone.gatherer.remote.RemoteGreenstoneServerAction$RunScriptAction.perform(RemoteGreenstoneServerAction.java:537)
>>>
>>> at
>>> org.greenstone.gatherer.remote.ActionQueue.run(ActionQueue.java:134)
>>> Exception in thread "RemoteGreenstoneServerActionQueue"
>>> java.lang.StackOverflowError
>>> at sun.awt.AppContext.get(Unknown Source)
>>> at
>>> com.sun.java.swing.SwingUtilities3.getDelegateRepaintManager(Unknown
>>> Source)
>>> at javax.swing.RepaintManager.getDelegate(Unknown Source)
>>> at javax.swing.RepaintManager.addDirtyRegion(Unknown Source)
>>> at javax.swing.JComponent.repaint(Unknown Source)
>>> at java.awt.Component.repaint(Unknown Source)
>>> at javax.swing.JComponent.setFont(Unknown Source)
>>> at javax.swing.LookAndFeel.installColorsAndFont(Unknown Source)
>>> at javax.swing.plaf.basic.BasicLabelUI.installDefaults(Unknown
>>> Source)
>>> at javax.swing.plaf.basic.BasicLabelUI.installUI(Unknown Source)
>>> at javax.swing.JComponent.setUI(Unknown Source)
>>> at javax.swing.JLabel.setUI(Unknown Source)
>>> at javax.swing.JLabel.updateUI(Unknown Source)
>>> at javax.swing.JLabel.<init>(Unknown Source)
>>> at javax.swing.JLabel.<init>(Unknown Source)
>>> at
>>> javax.swing.plaf.basic.BasicOptionPaneUI.addMessageComponents(Unknown
>>> Source)
>>> at
>>> javax.swing.plaf.basic.BasicOptionPaneUI.addMessageComponents(Unknown
>>> Source)
>>> at
>>> javax.swing.plaf.basic.BasicOptionPaneUI.addMessageComponents(Unknown
>>> Source)
>>> at
>>> javax.swing.plaf.basic.BasicOptionPaneUI.addMessageComponents(Unknown
>>> Source)
>>> at
>>> javax.swing.plaf.basic.BasicOptionPaneUI.addMessageComponents(Unknown
>>> Source)
>>> ...(continues for about 1000 times)
>>>
>>> Thanks,
>>>
>>
>