RE: [greenstone-users] dates

From Newman, Linda (newmanld)
DateWed, 17 Dec 2003 10:23:32 -0500
Subject RE: [greenstone-users] dates
Katherine -- Thank you again for your response!

I had already tried a variation with "-extract_historical_years" as an
option with the HTMLPlug, with no change in results. However, maybe the
crux of the problem here is that I am not working with documents, but with
images. For each image (jpgs) I have metadata information that includes a
date field, but there are no documents per se. The date index and the
datelist classifier are both working from the date field in the metadata
files. Is it possible to get the date range search to work from the date
field in the metadata files, rather than a date field coming from documents?
Or, can I generate documents from the metadata?

-----Original Message-----
From: Katherine Don [mailto:kjdon@cs.waikato.ac.nz]
Sent: Tuesday, December 16, 2003 6:04 PM
To: Newman, Linda (newmanld)
Cc: greenstone-users@list.scms.waikato.ac.nz
Subject: Re: [greenstone-users] dates


Hi Linda

Yes, the date search does not appear in the form search interface. This is
just
an oversight . You can search on the Coverage element in the form (which is
what the date search uses) but you can't do range searching. At some stage
I will look at putting the date range search into the form as well. This
will
probably happen early next year - if it is urgent for you, let me know and I
will do it sooner (and send you the fix).

To get the date searching working you need to put the
-extract_historical_years
option with the plugins that actually process the documents - it doesn't
work
with RecPlug (I have only just realised this otherwise I could have told you
last email).
So if your documents are in html, add the option to HTMLPlug, etc. RecPLug
doesn't actually process any documents, just goes through directories.
You will need to add the option to all plugins that match types of documents
in
your collection.

I hope this works this time :-)
Regards,
Katherine Don

"Newman, Linda (newmanld)" wrote:

> Katherine -- thanks for continuing this thread and discussion over the
> 'format queryinterface datesearch' option with mgpp collections.
>
> I have done the following:
> Upgraded to Greenstone 2.41 (BTW, the installshield uninstall also
> uninstalled my collections as earlier users reported, which I did have
> backed up, but this would be more than inconvenient with a large
production
> collection! Not, however, the point of this email.)
> Changed my configuration file per the notes you sent.
>
> I now see the date range search (with a 'Begin Search' submit type of
> button) on the 'plain' version of the search interface, although not on
the
> 'form' version. I assume that this is normal. However, the date range
> search consistently returns zero results, even though the date index
search
> and datelist classifier both work. Below is my config file, and I hope
that
> you can again tell me what I'm doing wrong.
>
> -- Linda Newman
>
> collect.cfg file:
> ************************
> creator ucdp@uc.edu
> maintainer ucdp@uc.edu
> public true
>
> searchtype plain form
> indexes allfields Title Date CardNum
> defaultindex Title
>
> plugin ZIPPlug
> plugin GAPlug
> plugin TEXTPlug
> plugin ImagePlug
> plugin HTMLPlug -block_exp (?i).(gif|jpe?g|jpe|png|css)$|(~$)
> -metadata_fields ex.Encoding
> plugin EMAILPlug
> plugin PDFPlug
> plugin RTFPlug
> plugin WordPlug
> plugin PSPlug
> plugin ArcPlug
> plugin RecPlug -use_metadata_files -extract_historical_years
>
> classify AZCompactList metadata=Title buttonname=names
> classify AZCompactList metadata=CardNum buttonname=card#s
> classify DateList bymonth=1
>
> format SearchVList
> "<td valign=top>[link][icon][/link]</td><td
valign=top>{If}{[parent(All':
> '):Title],[parent(All':
> '):Title]:}[link][Title][/link]</td><td><small>{If}{[Date],<i> [Type]:
> </i>[Date]}</td><td><small>{If}{[CardNum],<i>card number:
> </i>[CardNum]}</small></td>"
> format CL1VList
> "<td valign=top>[link][icon][/link]</td><td
>
valign=top>[highlight]{Or}{[Title],Untitled}[/highlight]<small>{If}{[Date],<
> br><i>[Type]: </i>[Date]}</small></td>"
> format CL2VList
> "<td valign=top>[link][icon][/link]</td><td
>
valign=top>[highlight]{Or}{[Title],Untitled}[/highlight]<small>{If}{[Date],<
> br><i>[Type]: </i>[Date]}{If}{[CardNum],<br><i>card number:
> </i>[CardNum]}</small></td>"
> format DateList
> "<td valign=top>[link][icon][/link]</td><td
>
valign=top>[highlight]{Or}{[Title],Untitled}[/highlight]<small>{If}{[Date],<
> br><i>[Type]: </i>[Date]}</small></td>"
>
> format DocumentText '<center><table width="537">
> <tr><td><img src="/gsdl/collect/gsarch/index/assoc/[assocfilepath]/[Image]">
> <br>Name: [Title]<br>Date: [Date]<br>Card Number: [CardNum]
> <br>Image: [Source]<br>Type: [Type]</td></tr></table></center>'
>
> format QueryInterface DateSearch
>
> collectionmeta collectionname "Hamilton County Ohio Birth & Death
> Records:1865-1910"
> collectionmeta collectionextra "This is a collection of birth & death
> certificates produced in Hamilton County Ohio during the period of
> 1865-1910."
> collectionmeta .Title "Names"
> collectionmeta .Date "Dates"
> collectionmeta .CardNum "Card Numbers"
> collectionmeta .allfields "All Fields"
>
> metadataset dc "dc.mds"
> metadataset dls "dls.mds"
> metadataset greenstone "extracted.mds"
> ************************
>
> -----Original Message-----
> From: Katherine Don [mailto:kjdon@cs.waikato.ac.nz]
> Sent: Sunday, December 14, 2003 10:21 PM
> To: Newman, Linda (newmanld)
> Cc: greenstone-users@list.scms.waikato.ac.nz
> Subject: Re: [greenstone-users] dates
>
> Hi Linda
>
> Actually, the date search only works with mgpp collections. However, even
> with
> an mg collection you should still have seen the normal search form (and
been
> able to do searching but which would search for the wrong stuff). I have
> never
> heard of the query form disappearing before.
>
> You could try with mgpp and see if the problem goes away.
> Make the following changes to your config file to use mgpp
>
> add:
> searchtype plain
> change the index lines to
> indexes allfields CardNum Date Title
> defaultindex Title
>
> collectionmeta becomes
> collectionmeta .CardNum "Card Number"
> collectionmeta .Date "Date"
> collectionmeta .allfields "all fields" etc
>
> all of these can be done through the librarian interface -go to search
types
> pane in design and switch on enable advanced searching.
> Then the indexes specification changes to use mgpp - you will need to
enter
> in
> the new index data.
>
> I would recommend downloading the latest version (2.41) of greenstone -
the
> librarian is much more stable, and there are other minor fixes to gs.
> Remember
> to back up your collections though before uninstalling.
>
> hope this helps,
> Katherine Don
>
> "Newman, Linda (newmanld)" wrote:
>
> > Katherine -- Thank you for your reply.
> >
> > I have not downloaded the version just announced (2.41), but the date of
> my
> > server.exe file is 7/1/2003. I am running in the Windows 2000
> environment.
> >
> > Below is the current version of my collect.cfg file, for what is now
just
> a
> > test database. I'm not using form searching with mgpp, although had
> wanted
> > to use that in the future. My original search box still disappears if I
> add
> > the format QueryInterface DateSearch option. If I remove that one line,
> then
> > the search box re-appears, but of course without the date range search
> > option. Additionally, I can't submit a date search, as there is no
submit
> > button.
> > -------------------------------------------------------------------
> > creator ucdp@uc.edu
> > maintainer ucdp@uc.edu
> > public true
> >
> > indexes document:CardNum document:Date document:Title
> > document:Title,Date
> > defaultindex document:Title
> >
> > plugin ZIPPlug
> > plugin GAPlug
> > plugin TEXTPlug
> > plugin ImagePlug
> > plugin HTMLPlug -block_exp (?i).(gif|jpe?g|jpe|png|css)$|(~$)
> > -metadata_fields ex.Encoding
> > plugin EMAILPlug
> > plugin PDFPlug
> > plugin RTFPlug
> > plugin WordPlug
> > plugin PSPlug
> > plugin ArcPlug
> > plugin RecPlug -use_metadata_files -extract_historical_years
> >
> > classify AZCompactList metadata=Title buttonname=names
> > classify AZCompactList metadata=CardNum buttonname=card#s
> > classify DateList bymonth=1
> >
> > format DocumentText '<center><table width="537">
> > <tr><td><img src="/gsdl/collect/gsarch/index/assoc/[assocfilepath]/[Image]">
> > <br>Name: [Title]<br>Date: [Date]<br>Card Number: [CardNum]
> > <br>Image: [Source]<br>Type: [Type]</td></tr></table></center>'
> >
> > format QueryInterface DateSearch
> >
> > collectionmeta collectionname "Hamilton County Ohio Birth & Death
> > Records:1865-1910"
> > collectionmeta collectionextra "This is a collection of birth & death
> > certificates produced in Hamilton County Ohio during the period of
> > 1865-1910."
> > collectionmeta .document:CardNum "Card Number"
> > collectionmeta .document:Date "Date"
> > collectionmeta .document:Title "Name"
> > collectionmeta .document:Title,Date "Name & Date"
> >
> > metadataset dc "dc.mds"
> > metadataset dls "dls.mds"
> > metadataset greenstone "extracted.mds"
> >
> > -----Original Message-----
> > From: Katherine Don [mailto:kjdon@cs.waikato.ac.nz]
> > Sent: Thursday, December 11, 2003 3:22 PM
> > To: Newman, Linda (newmanld)
> > Cc: greenstone-users@list.scms.waikato.ac.nz
> > Subject: Re: [greenstone-users] dates
> >
> > Hi Linda
> >
> > I have just tried adding the DateSearch format statement to my
collections
> > config files - it worked (mostly) fine. The date stuff is added to the
> query
> > form below the query box and search button. The single format line is
> enough
> > to get the date options added.
> >
> > I found one bug- it doesn't get added if you are using form searching
> (with
> > mgpp collections).
> > However, in all cases, my original search box and parameters were still
> > present.
> >
> > What version of Greenstone are you using? I am using v2.41 - perhaps an
> > older
> > version had a bug in this date stuff??
> >
> > regards,
> > Katherine Don
> >
> > "Newman, Linda (newmanld)" wrote:
> >
> > > Thank you to John McPherson for clarifying the date format of YYYYMMDD
> > > instead of YYYY-MM-DD.
> > > That solved problem 1) below.
> > >
> > > In regard to the 2nd issue, when I include the statement "format
> > > QueryInterface DateSearch" in my collect.cfg file, it results in the
> > > following two problems:
> > >
> > > *My ordinary search options deriving from my other indexes (title,
> etc.)
> > > are gone.
> > >
> > > *The date query search form doesn't have a submit button or other
way
> > for
> > > the user to actually submit the date search.
> > >
> > > My guess is that there is some additional syntax beyond "format
> > > QueryInterface DateSearch" to solve these two issues, but I can find
no
> > > mention of this format statement in any of the pdf files that I
> > downloaded.
> > > Any one know how to extend this format statement?
> > >
> > > -----Original Message-----
> > > From: John R. McPherson [mailto:jrm21@cs.waikato.ac.nz]
> > > Sent: Tuesday, December 09, 2003 3:56 PM
> > > To: Newman, Linda (newmanld)
> > > Cc: greenstone-users@list.scms.waikato.ac.nz
> > > Subject: Re: [greenstone-users] dates
> > >
> > > On Tue, Dec 09, 2003 at 03:44:58PM -0500, Newman, Linda (newmanld)
> wrote:
> > > > 1) Display problem with dates:
> > > >
> > > > I'm using dates in my metadata.xml file, and although searching and
> > > browsing
> > > > seem to work correctly, the display of the date info is strange.
> > > >
> > > > I can't find anything specifying the format in any of the greenstone
> pdf
> > > > files, so I went with what I found online for the Dublin Core:
> > > YYYY-MM-DD.
> > > >
> > > >
> > > > However, the display of a field such as this:
> > > > <Metadata name="Date" mode="accumulate">1878-01-03</Metadata>
> > > >
> > > > will be shown as:
> > > > Date: 1- _textmonth-0_ 1878
> > > >
> > > > This display shows up both in the datelist classifier and in the
> > > > documentText format.
> > >
> > > Greenstone needs dates to be in YYYYMMYY format to be able to parse
> > > them.
> > > _textmonthXX_ is a greenstone macro that turns the XXth month into
> > > a string in the appropriate language... eg 20031210 (for 10th Dec
2003).
> > > So what has happened is that the plugin got 1878 as the year, and then
> > > "-0" as the month, which isn't a valid macro for a month name.
> > >
> > > > 2) Searching Question:
> > > > Is there a way to allow the user to search (not browse) within range
> of
> > > > dates, using either the MG or MGPP search engines?
> > >
> > > According to the BasPlug (base plugin), you can add an option called
> > > "-extract_historical_years" to any plugin, and also add
> > > " format QueryInterface DateSearch"
> > > to the collect.cfg file for searching based on the year.
> > >
> > > I have no idea if this actually works or not though... but I recall
> > > that it used to...
> > >
> > > John McPherson
> > >
> > > _______________________________________________
> > > greenstone-users mailing list
> > > greenstone-users@list.scms.waikato.ac.nz
> > > https://list.scms.waikato.ac.nz/mailman/listinfo/greenstone-users