Thank you, Alistair. Sorry that I replied to your previous email, the later one
was placed in my spam folder by my email client, and I only noticed it this
morning. I include that email in the history below for completeness.
These two log extracts
> the initial ingest from all foxml files for one specific resource that shows
> the behaviour - initial–ingest-fgs.log
>
> the log from indexing that resource from pid - index–from-pid.log
both show the generated index document, and they are identical. As I said in
the previous reply, I thought that the problem was that one of them would lack
the dcterms fields. Since they are identical, which is as it should be, I think
I need your explanation again about what the problem is.
Gert
On 19/08/2013, at 10.26, Alistair Young wrote:
> the later email should contain that info Gert. Just wondering if the initial
> indexing gets hold of a cached version of the resource (before the DCTERMS
> namespace was renamed to be the same as the XSLT).
>
> thanks,
>
> Alistair
>
> --
> mov eax,1
> mov ebx,0
> int 80h
>
> From: Gert Schmeltz Pedersen <[email protected]>
> Reply-To: "Support and info exchange list for Fedora users."
> <[email protected]>
> Date: Friday, 16 August 2013 09:25
> To: "Support and info exchange list for Fedora users."
> <[email protected]>
> Subject: Re: [fcrepo-user] gsearch not indexing dcterms on upload
>
> Thank you, Alistair. Your log lines show me part of one index document
> generation, which contains the dct (dcterms) fields. However, if I understand
> you right, the problem is that you get an index document without the dct
> fields, when you call the updateIndex in one of the other ways (fromPid,
> fromFoxmlFiles, triggered by ingest/update). So, in order to compare the
> successful case and the problematic case, I need to see also the log lines
> from the problematic index document generation, and I need to see all the log
> lines of both operations, that is, from the INFO log line starting the
> operation to the INFO log line at the end of the operation.
>
> Gert
On 15/08/2013, at 10.58, Alistair Young wrote:
> I've dug deeper into the logs Gert and attached are:
>
> the initial ingest from all foxml files for one specific resource that shows
> the behaviour - initial–ingest-fgs.log
>
> the log from indexing that resource from pid - index–from-pid.log
>
> and the corresponding solr log extracts for both indexing job -
> solr–extract.log
>
> There appear to be two ingests initially for that resource into solr?
> DEBUG - 2013-08-15 09:31:47.790
> DEBUG - 2013-08-15 09:34:29.375
> they're from the same indexing job, not run separately
>
> thanks,
>
> Alistair
>
> --
>
>
> On 14/08/2013, at 17.52, Alistair Young wrote:
>
>> Hope this is of some use…
>>
>> Alistair
>>
>>
>> -----------------
>> mov eax,1
>> mov ebx,0
>> int 80
>>
>> From: Gert Schmeltz Pedersen <[email protected]>
>> Reply-To: "Support and info exchange list for Fedora users."
>> <[email protected]>
>> Date: Wednesday, 14 August 2013 16:15
>> To: "Support and info exchange list for Fedora users."
>> <[email protected]>
>> Subject: Re: [fcrepo-user] gsearch not indexing dcterms on upload
>>
>> Let me see the log lines.
>>
>> Gert
>>
>>
>> On 14/08/2013, at 17.04, Alistair Young wrote:
>>
>>> Having said that, I have to force index via pid to get it to index the
>>> updated resources.
>>>
>>> Renamed all dcterms datastreams namespaces to be same as xslt in the
>>> objectStore. Rebuilt the resourceIndex and database. Blew away the solr
>>> index. Index from scratch and it won't index dcterms on the resources that
>>> had their dcterms renamed. Have to force update on them by pid and then it
>>> indexes their dcterms.
>>>
>>> Any idea why an initial index would fail while an individual index via pid
>>> would work?
>>>
>>> Alistair
>>>
>>> -----------------
>>> mov eax,1
>>> mov ebx,0
>>> int 80
>>>
>>> From: Alistair Young <[email protected]>
>>> Reply-To: "Support and info exchange list for Fedora users."
>>> <[email protected]>
>>> Date: Wednesday, 14 August 2013 09:53
>>> To: "Support and info exchange list for Fedora users."
>>> <[email protected]>
>>> Subject: Re: [fcrepo-user] gsearch not indexing dcterms on upload
>>>
>>> Thanks for that Gert. Finally tracked it down to different representations
>>> of the same namespace in the repo. Some objects have their dcterms
>>> namespace declared as per the xslt, others don't. I feel some metadata
>>> munging coming on. Thanks for confirming the single xslt though. At the end
>>> of this hoping to produce a tutorial on fedora/gsearch/solr. This seems to
>>> be the last thorn in the bush.
>>>
>>> Cheers,
>>>
>>> Alistair
>>>
>>>
>>> From: Gert Schmeltz Pedersen <[email protected]>
>>> Reply-To: "Support and info exchange list for Fedora users."
>>> <[email protected]>
>>> Date: Tuesday, 13 August 2013 20:51
>>> To: "Support and info exchange list for Fedora users."
>>> <[email protected]>
>>> Subject: Re: [fcrepo-user] gsearch not indexing dcterms on upload
>>>
>>> No separate xslt, gsearch uses the same indexing xslt, whether triggered by
>>> an ingest/update of the Fedora object, or by an updateIndex fromPid
>>> operation, or by an updateIndex fromFoxmlFiles operation. If you get
>>> different indexing documents in these three cases, then the way to find out
>>> why, is to compare the log lines in debug mode. I may help, if you send me
>>> those log lines.
>>>
>>> Gert
>>>
>>>
>>> On 13/08/2013, at 18.47, Alistair Young wrote:
>>>
>>>> Nup , not that. Even uploading the DCTERMS first it doesn't index them. Is
>>>> there a separate xslt it uses for updating a resource? The symptoms are
>>>> identical to when the main xslt was lacking the dcterms namespace.
>>>>
>>>> Alistair
>>>>
>>>> -------------------
>>>> Alistair Young
>>>> Àrd Innleadair air Bathair-bog
>>>> UHI@Sabhal Mòr Ostaig
>>>>
>>>> From: Alistair Young <[email protected]>
>>>> Reply-To: "Support and info exchange list for Fedora users."
>>>> <[email protected]>
>>>> Date: Tuesday, 13 August 2013 16:44
>>>> To: "Support and info exchange list for Fedora users."
>>>> <[email protected]>
>>>> Subject: Re: [fcrepo-user] gsearch not indexing dcterms on upload
>>>>
>>>> I think the problem is down to the multi part commit technique being used
>>>> when uploading to Fedora:
>>>>
>>>> 1 – new object pid is generated by a POST
>>>> 2 – the DC datastream is then PUT
>>>> 3 - the DCTERMS datastream is then PUT
>>>> 4 - the content datastream is then POST
>>>>
>>>> so I suspect 3 (or 4) is not getting to solr. gsearch indexes on 2 but not
>>>> on 3 I'm assuming. All the datastreams are present in the foxml files when
>>>> doing a new index from scratch so they work.
>>>>
>>>> Is it possible to turn off auto indexing on upload and instead force it
>>>> via a REST call to gsearch once the complete resource is uploaded?
>>>>
>>>> Alistair
>>>>
>>>> --
>>>> mov eax,1
>>>> mov ebx,0
>>>> int 80h
>>>>
>>>> From: Alistair Young <[email protected]>
>>>> Reply-To: "Support and info exchange list for Fedora users."
>>>> <[email protected]>
>>>> Date: Tuesday, 13 August 2013 16:10
>>>> To: "Support and info exchange list for Fedora users."
>>>> <[email protected]>
>>>> Subject: [fcrepo-user] gsearch not indexing dcterms on upload
>>>>
>>>> I think I must be missing a file mod or something. Indexing from scratch,
>>>> using from foxml files works fine and dcterms in all resources are indexed
>>>> as the namespaces are in the xslt. When uploading a new resource to fedora
>>>> it gets indexed but only on dc, not dcterms. Does gsearch use a different
>>>> xslt when indexing from an upload rather than a clean start?
>>>>
>>>> Alistair
>>>>
>>>> --
>>>> mov eax,1
>>>> mov ebx,0
>>>> int 80h
>>>> ------------------------------------------------------------------------------
>>>> Get 100% visibility into Java/.NET code with AppDynamics Lite!
>>>> It's a free troubleshooting tool designed for production.
>>>> Get down to code-level detail for bottlenecks, with <2% overhead.
>>>> Download for free and get started troubleshooting in minutes.
>>>> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk_______________________________________________
>>>> Fedora-commons-users mailing list
>>>> [email protected]
>>>> https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
>>>
>>> ------------------------------------------------------------------------------
>>> Get 100% visibility into Java/.NET code with AppDynamics Lite!
>>> It's a free troubleshooting tool designed for production.
>>> Get down to code-level detail for bottlenecks, with <2% overhead.
>>> Download for free and get started troubleshooting in minutes.
>>> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk_______________________________________________
>>> Fedora-commons-users mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
>>
>> <fglogs.tar.gz>------------------------------------------------------------------------------
>> Get 100% visibility into Java/.NET code with AppDynamics Lite!
>> It's a free troubleshooting tool designed for production.
>> Get down to code-level detail for bottlenecks, with <2% overhead.
>> Download for free and get started troubleshooting in minutes.
>> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk_______________________________________________
>> Fedora-commons-users mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
>
> ------------------------------------------------------------------------------
> Get 100% visibility into Java/.NET code with AppDynamics Lite!
> It's a free troubleshooting tool designed for production.
> Get down to code-level detail for bottlenecks, with <2% overhead.
> Download for free and get started troubleshooting in minutes.
> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk_______________________________________________
> Fedora-commons-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Fedora-commons-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
