[greenstone-users] Further information - Re: OAI corrections

From Greenstone Team
DateThu Jun 9 15:22:18 2011
Subject [greenstone-users] Further information - Re: OAI corrections
Hi again Mariana,

I have further news in addition to the email I sent in the last hour or
so (and which I have copied again below). Please read that email first.

Using the very recent corrections made to the Greenstone 2 code
regarding the OAIServer, I visited http://oval.base-search.net/oval
It passes all the tests with some 2 Warnings and several
Recommendations. The 2nd warning was regarding the metadata I had
assigned the collection's documents, which can always be altered. The
first warning could not be helped: Oval tells me that all OAI requests
were redirected to Greenstone's oaiserver.cgi page, which may not be
recommeded, but it's the way Greenstone 2 currently functions. However,
this is not an error and should therefore not affect the harvesting
process itself.

The good news is that one of the tests that it does pass is related to
the ListRecords Granularity you mentioned: the message I get now is "OK:
Incremenetal harvesting (day granularity) of ListRecords works."

Therefore, if you are keen, do try out a binary from the caveat-emptor
page (link in previous email) which contains all the fixes to GS2 to
make these things work again.
Bear in mind that these binaries are work-in-progress versions. If one
does not install properly or has issues starting up or something, then
you can try another binary from a few days before that. These are
accessible from the Dates links at the top of the Caveat-Emptor page.

Regards,
Anupama


Hi Mariana,

We recently updated the GS2.84 code to get it validated against the
official online OAI validation tool. It passed all the tests there thus
far. For instance, the updated 2.84 now passes the GetRecord test. It is
possible that other errors, such as the one you more recently
discovered, have been corrected as well as a flow-on effect.

I don't remember whether I got round to emailing you to try out the most
up-to-date binary to see whether it resolved the problems you were
encountering. However, would you be interested in trying it now, just to
see whether the OAI errors and problems you saw in the 2.84 binary have
indeed at last disappeared?


If you would like to test it out, then:

1. If GLI is running *close* any collections you may have open and then
quit GLI. Or if you had just the Greenstone server running, stop the
server.

2. go to http://www.greenstone.org/caveat-emptor/

3. scroll down to the section "Greenstone2 Binary Releases"

4. download the executable for your operating system and install it

5. copy your old 2.84's OAI-enabled collection into this new Greenstone
installation. Move the Greenstone2.84/collect/<oai-collection-name
folder into the New-GS2/collect directory.

6. Start up GLI, rebuild your ready-made collection. Note that
rebuilding is necessary in order to work out the earliestDatestamp
information required by the OAI specification. Finally try out the OAI
server in the usual manner.

7. Write back if anything failed, and if so, providing as many details
about the failure as you can will help us diagnose the problem better.
For instance, what OAI-type URL did it fail on, and if any dates values
are involved, can you also give us the output of the OAI Identify
operation, so we can generate suitable tests to simulate your situation?

However, if things worked out well instead, as I'm anticipating that
they might, could you please tell us that too, so that we know someone
making natural use of the OAI server has tried it out and things succeeded?

Thank you for bringing these problems to our notice,
Anupama

mariana@fahce.unlp.edu.ar wrote:
> Sirs,
>
> I regret to inform you that I found another problem of the OAI server.
> I'm using a new tool for validation:
> http://oval.base-search.net/oval
> and appear a new problem that works fine in 2.82 at least:
> The harvesting section report inform an ERROR, that means for them
> "Serious protocol violation that will affect the harvesting process":
>
> ERROR: No incremental harvesting (day granularity) of ListRecords:
> Harvest
> for reference date 2011-04-19 returned no records.
>
> This error don't appear in the running system validation (2.82), that
> say:
> OK: Incremental harvesting (day granularity) of ListRecords works.
>
> I test this argument with FROM optional argument for day granularity and
> the 2.84 oaiserver say:
> Error Code noRecordsMatch
>
> Will have relationship with the GetRecord problem?
> I regret to report that, this version had very good progress in other
> aspects of the OAI protocol...
> Regards
> Mariana Pichinini
>
>> On 24/05/2011 8:45 p.m., John Rose wrote:
>>> Dear Mariana,
>>>
>>> Thanks for this good news and not so good. I think that it is really
>>> the highest priority to get OAI-PMH working right, and am copying to
>>> David Bainbridge to see if he can increase the priority of the
>>> GetRecord problem and let us know when it can be resolved (would be
>>> nice if it could be in the new "Luigi version" of 2.84). Best regards,
>>> John
>>>
>>> On 23/05/2011 22:22, mariana@fahce.unlp.edu.ar wrote:
>>>> Hello gentlemen
>>>>
>>>> Thanks for asking, John!!
>>>> I'm diving deep into this 2.84 release obscurities of OAI server!!!!
>>>>
>>>> About your questions, I agree with Eduardo.
>>>> Resumption Token work fine now.
>>>> Document URL is mapped into dc.identifier with several options
>>>> Datestamp is correctly assigned from the source file.
>>>>
>>>> These corrections and implementations are nice indeed.
>>>>
>>>> Nevertheless I have a problem with this version....
>>>> The verb GetRecord don't work.
>>>> The OAI identifier (that is now composed from the repository
>>>> identifier
>>>> and the collection's name) displays normally in the other verbs,
>>>> but when
>>>> I do a GetRecord, I get this:
>>>>
>>>> Error Code idDoesNotExist
>>>>
>> Anu's just been through a big round of compliance testing against
>> against some OAI validation servers. There were several adjustments to
>> the code made at that point but -- based on her reply -- it seems that
>> it doesn't seem to have caught this particular issue. We'll start by
>> looking to reproduce the problem at our end, and then based on that
>> assess how much work we think it will take to fix. Given that it was
>> working in 2.83 but not in 2.84 I'm optimistic that this should be a
>> minor coding issue that we can get in to 2.84 Luigi.
>>
>> David.
>>
>>>> Last week I wrote an email to Anupama about this issue and he
>>>> answered:
>>>>
>>>>> Thanks for bringing this to our attention. There's a few more
>>>>> pressing
>>>>> bugs to look at first, but I've added this to the queue and also
>>>>> created
>>>>> a ticket for it.
>>>>> Regards,
>>>>> Anupama
>>>> The ticket is http://trac.greenstone.org/ticket/751
>>>>
>>>> Perhaps this verb isn't very useful, since only it retrieves only one
>>>> record (I don't know), but it seems a regression since it worked fine
>>>> in
>>>> v. 2.83....
>>>>
>>>> In other way, Eduardo had consulted about the harvesting granularity,
>>>> that
>>>> it can't be configured in GSDL.....this should be included in the OAI
>>>> server TODO.
>>>>
>>>> I think that is all.....I will continue digging, deeper and deeper....
>>>> Thanks for your attention and best regards
>>>>
>>>> Mariana Pichinini
>>>>
>>>>
>>>>
>>>>> Hello everybody
>>>>>
>>>>>
>>>>>
>>>>> Here you have the provisional OAI adress of our digital library in
>>>>> Greenstone 2.84:
>>>>>
>>>>>
>>>>>
>>>>> http://ibdigital.uib.es:8000/greenstone/cgi-bin/oaiserver.cgi
>>>>>
>>>>>
>>>>>
>>>>> Now we are moving our collections to the new version of Greenstone
>>>>> 2.84
>>>>> and
>>>>> we have not yet done any kind of OAI validation.
>>>>>
>>>>>
>>>>>
>>>>> i) Resumption token: now it seems that the resumption token is
>>>>> working
>>>>> correctly (check for example the set bellpuig).
>>>>>
>>>>> ii) Document url by default in the Identifier field: now there
>>>>> is the
>>>>> document url.
>>>>>
>>>>> iii) Datestamp inconsistency: it seems that the problem has been
>>>>> corrected.
>>>>>
>>>>>
>>>>>
>>>>> We have not yet done any test with an OAI validator.
>>>>>
>>>>>
>>>>>
>>>>> Thank you
>>>>>
>>>>>
>>>>>
>>>>> Eduardo del Valle
>>>>>
>>>>>
>>>>>
>>>>> -----Mensaje original-----
>>>>> De: John Rose [mailto:john.rose1@free.fr]
>>>>> Enviado el: s□bado, 21 de mayo de 2011 16:23
>>>>> Para: Eduardo del Valle; Mariana Pichinini
>>>>> CC: Diego Spano
>>>>> Asunto: OAI corrections
>>>>>
>>>>>
>>>>>
>>>>> Dear Eduardo, Mariana,
>>>>>
>>>>>
>>>>>
>>>>> In December 2009, we had discussions about three problems with the
>>>>>
>>>>> Greenstone OAI server: i) Resumption token inconsistency, ii) lack of
>>>>>
>>>>> document url by default in the Identifier field, and iii) datestamp
>>>>>
>>>>> inconsistency. I had the impression that Waikato had fixed all of
>>>>> these
>>>>>
>>>>> and included them in version 2.83 released in October 2010.
>>>>>
>>>>>
>>>>>
>>>>> Now I see that only the first is in the release notes of 2.83 and the
>>>>>
>>>>> second two in the release notes of 2.84. Have you both checked with
>>>>> 2.84
>>>>>
>>>>> to see that OAI server is working fine for all three aspects?
>>>>>
>>>>>
>>>>>
>>>>> Thanks and best regards, John
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> ************
>>>>>
>>>>>
>>>>>
>>>>> John B. Rose
>>>>>
>>>>> 1 Bis rue des Ch□tre-Sacs
>>>>>
>>>>> 92310 S□vres, France
>>>>>
>>>>>
>>>>>
>>>>> Email: john.rose1@free.fr
>>>>>
>>>>> Alternate email: johnrose@alumni.caltech.edu
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>
>
>
>
>