[greenstone-users] Re: OAI corrections

From Anupama of Greenstone Team
DateThu Jun 9 14:19:23 2011
Subject [greenstone-users] Re: OAI corrections
In-Reply-To (113a0efbfcdf702ee101691ef3affed8-squirrel-webmail-fahce-unlp-edu-ar)
Hi Mariana,

We recently updated the GS2.84 code to get it validated against the
official online OAI validation tool. It passed all the tests there thus
far. For instance, the updated 2.84 now passes the GetRecord test. It is
possible that other errors, such as the one you more recently
discovered, have been corrected as well as a flow-on effect.

I don't remember whether I got round to emailing you to try out the most
up-to-date binary to see whether it resolved the problems you were
encountering. However, would you be interested in trying it now, just to
see whether the OAI errors and problems you saw in the 2.84 binary have
indeed at last disappeared?


If you would like to test it out, then:

1. If GLI is running *close* any collections you may have open and then
quit GLI. Or if you had just the Greenstone server running, stop the server.

2. go to http://www.greenstone.org/caveat-emptor/

3. scroll down to the section "Greenstone2 Binary Releases"

4. download the executable for your operating system and install it

5. copy your old 2.84's OAI-enabled collection into this new Greenstone
installation. Move the Greenstone2.84/collect/<oai-collection-name
folder into the New-GS2/collect directory.

6. Start up GLI, rebuild your ready-made collection. Note that
rebuilding is necessary in order to work out the earliestDatestamp
information required by the OAI specification. Finally try out the OAI
server in the usual manner.

7. Write back if anything failed, and if so, providing as many details
about the failure as you can will help us diagnose the problem better.
For instance, what OAI-type URL did it fail on, and if any dates values
are involved, can you also give us the output of the OAI Identify
operation, so we can generate suitable tests to simulate your situation?

However, if things worked out well instead, as I'm anticipating that
they might, could you please tell us that too, so that we know someone
making natural use of the OAI server has tried it out and things succeeded?

Thank you for bringing these problems to our notice,
Anupama

mariana@fahce.unlp.edu.ar wrote:
> Sirs,
>
> I regret to inform you that I found another problem of the OAI server.
> I'm using a new tool for validation:
> http://oval.base-search.net/oval
> and appear a new problem that works fine in 2.82 at least:
> The harvesting section report inform an ERROR, that means for them
> "Serious protocol violation that will affect the harvesting process":
>
> ERROR: No incremental harvesting (day granularity) of ListRecords: Harvest
> for reference date 2011-04-19 returned no records.
>
> This error don't appear in the running system validation (2.82), that say:
> OK: Incremental harvesting (day granularity) of ListRecords works.
>
> I test this argument with FROM optional argument for day granularity and
> the 2.84 oaiserver say:
> Error CodenoRecordsMatch
>
> Will have relationship with the GetRecord problem?
> I regret to report that, this version had very good progress in other
> aspects of the OAI protocol...
> Regards
> Mariana Pichinini
>
>> On 24/05/2011 8:45 p.m., John Rose wrote:
>>> Dear Mariana,
>>>
>>> Thanks for this good news and not so good. I think that it is really
>>> the highest priority to get OAI-PMH working right, and am copying to
>>> David Bainbridge to see if he can increase the priority of the
>>> GetRecord problem and let us know when it can be resolved (would be
>>> nice if it could be in the new "Luigi version" of 2.84). Best regards,
>>> John
>>>
>>> On 23/05/2011 22:22, mariana@fahce.unlp.edu.ar wrote:
>>>> Hello gentlemen
>>>>
>>>> Thanks for asking, John!!
>>>> I'm diving deep into this 2.84 release obscurities of OAI server!!!!
>>>>
>>>> About your questions, I agree with Eduardo.
>>>> Resumption Token work fine now.
>>>> Document URL is mapped into dc.identifier with several options
>>>> Datestamp is correctly assigned from the source file.
>>>>
>>>> These corrections and implementations are nice indeed.
>>>>
>>>> Nevertheless I have a problem with this version....
>>>> The verb GetRecord don't work.
>>>> The OAI identifier (that is now composed from the repository identifier
>>>> and the collection's name) displays normally in the other verbs,
>>>> but when
>>>> I do a GetRecord, I get this:
>>>>
>>>> Error Code idDoesNotExist
>>>>
>> Anu's just been through a big round of compliance testing against
>> against some OAI validation servers. There were several adjustments to
>> the code made at that point but -- based on her reply -- it seems that
>> it doesn't seem to have caught this particular issue. We'll start by
>> looking to reproduce the problem at our end, and then based on that
>> assess how much work we think it will take to fix. Given that it was
>> working in 2.83 but not in 2.84 I'm optimistic that this should be a
>> minor coding issue that we can get in to 2.84 Luigi.
>>
>> David.
>>
>>>> Last week I wrote an email to Anupama about this issue and he answered:
>>>>
>>>>> Thanks for bringing this to our attention. There's a few more pressing
>>>>> bugs to look at first, but I've added this to the queue and also
>>>>> created
>>>>> a ticket for it.
>>>>> Regards,
>>>>> Anupama
>>>> The ticket is http://trac.greenstone.org/ticket/751
>>>>
>>>> Perhaps this verb isn't very useful, since only it retrieves only one
>>>> record (I don't know), but it seems a regression since it worked fine
>>>> in
>>>> v. 2.83....
>>>>
>>>> In other way, Eduardo had consulted about the harvesting granularity,
>>>> that
>>>> it can't be configured in GSDL.....this should be included in the OAI
>>>> server TODO.
>>>>
>>>> I think that is all.....I will continue digging, deeper and deeper....
>>>> Thanks for your attention and best regards
>>>>
>>>> Mariana Pichinini
>>>>
>>>>
>>>>
>>>>> Hello everybody
>>>>>
>>>>>
>>>>>
>>>>> Here you have the provisional OAI adress of our digital library in
>>>>> Greenstone 2.84:
>>>>>
>>>>>
>>>>>
>>>>> http://ibdigital.uib.es:8000/greenstone/cgi-bin/oaiserver.cgi
>>>>>
>>>>>
>>>>>
>>>>> Now we are moving our collections to the new version of Greenstone
>>>>> 2.84
>>>>> and
>>>>> we have not yet done any kind of OAI validation.
>>>>>
>>>>>
>>>>>
>>>>> i) Resumption token: now it seems that the resumption token is
>>>>> working
>>>>> correctly (check for example the set bellpuig).
>>>>>
>>>>> ii) Document url by default in the Identifier field: now there
>>>>> is the
>>>>> document url.
>>>>>
>>>>> iii) Datestamp inconsistency: it seems that the problem has been
>>>>> corrected.
>>>>>
>>>>>
>>>>>
>>>>> We have not yet done any test with an OAI validator.
>>>>>
>>>>>
>>>>>
>>>>> Thank you
>>>>>
>>>>>
>>>>>
>>>>> Eduardo del Valle
>>>>>
>>>>>
>>>>>
>>>>> -----Mensaje original-----
>>>>> De: John Rose [mailto:john.rose1@free.fr]
>>>>> Enviado el: s□bado, 21 de mayo de 2011 16:23
>>>>> Para: Eduardo del Valle; Mariana Pichinini
>>>>> CC: Diego Spano
>>>>> Asunto: OAI corrections
>>>>>
>>>>>
>>>>>
>>>>> Dear Eduardo, Mariana,
>>>>>
>>>>>
>>>>>
>>>>> In December 2009, we had discussions about three problems with the
>>>>>
>>>>> Greenstone OAI server: i) Resumption token inconsistency, ii) lack of
>>>>>
>>>>> document url by default in the Identifier field, and iii) datestamp
>>>>>
>>>>> inconsistency. I had the impression that Waikato had fixed all of
>>>>> these
>>>>>
>>>>> and included them in version 2.83 released in October 2010.
>>>>>
>>>>>
>>>>>
>>>>> Now I see that only the first is in the release notes of 2.83 and the
>>>>>
>>>>> second two in the release notes of 2.84. Have you both checked with
>>>>> 2.84
>>>>>
>>>>> to see that OAI server is working fine for all three aspects?
>>>>>
>>>>>
>>>>>
>>>>> Thanks and best regards, John
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> ************
>>>>>
>>>>>
>>>>>
>>>>> John B. Rose
>>>>>
>>>>> 1 Bis rue des Ch□tre-Sacs
>>>>>
>>>>> 92310 S□vres, France
>>>>>
>>>>>
>>>>>
>>>>> Email: john.rose1@free.fr
>>>>>
>>>>> Alternate email: johnrose@alumni.caltech.edu
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>
>
>
>
>