Re: possible status codes from solr during a (DIH) data import process
On 6/1/2012 11:51 AM, Savvas Andreas Moysidis wrote: Hello, Driven by the same requirements we also implemented the same polling mechanism (in java) and found it a bit awkward and error prone having to search through the returned response for occurrences of the terms "failure" or "Rollback" etc. It would be *really* handy if the status command returned numeric values to reflect the current state of the DIH process (similar to the HTTP status codes a server sends to a web browser). Our 2 cents.. :) On 1 June 2012 15:29, geeky2 wrote: thank you ALL for the great feedback - very much appreciated! -- View this message in context: http://lucene.472066.n3.nabble.com/possible-status-codes-from-solr-during-a-DIH-data-import-process-tp3987110p3987263.html Sent from the Solr - User mailing list archive at Nabble.com. I have filed some Jira issues on DIH status, and created a patch for one of them. https://issues.apache.org/jira/browse/SOLR-2729 https://issues.apache.org/jira/browse/SOLR-2728 I thought I had filed an issue for redoing the status response so there's a machine readable section and a human readable section, but now I can't seem to find it, so perhaps I never did. Thanks, Shawn
Re: possible status codes from solr during a (DIH) data import process
Hello, Driven by the same requirements we also implemented the same polling mechanism (in java) and found it a bit awkward and error prone having to search through the returned response for occurrences of the terms "failure" or "Rollback" etc. It would be *really* handy if the status command returned numeric values to reflect the current state of the DIH process (similar to the HTTP status codes a server sends to a web browser). Our 2 cents.. :) On 1 June 2012 15:29, geeky2 wrote: > thank you ALL for the great feedback - very much appreciated! > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/possible-status-codes-from-solr-during-a-DIH-data-import-process-tp3987110p3987263.html > Sent from the Solr - User mailing list archive at Nabble.com.
Re: possible status codes from solr during a (DIH) data import process
thank you ALL for the great feedback - very much appreciated! -- View this message in context: http://lucene.472066.n3.nabble.com/possible-status-codes-from-solr-during-a-DIH-data-import-process-tp3987110p3987263.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: possible status codes from solr during a (DIH) data import process
there is at least one scenario where no error is reported when it should be, if the host runs out of disk when optimizing, it is not reported. There is a jira issue open I think -- View this message in context: http://lucene.472066.n3.nabble.com/possible-status-codes-from-solr-during-a-DIH-data-import-process-tp3987110p3987144.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: possible status codes from solr during a (DIH) data import process
Hi,
Thats correct.
For failure, you have to check for the text *"Indexing failed. Rolled back
changes"* under the tag.
One more thing to note here is that there may be a time during the indexing
process where the indexing is complete but the index is not committed and
optimized yet.
You would need to check if the response listed below is present along with
the success message to term it as a complete success.
*2012-05-31 15:10:45
2012-05-31 15:10:45*
On Thu, May 31, 2012 at 3:42 PM, geeky2 wrote:
> hello all,
>
> i have been asked to write a small polling script (bash) to periodically
> check the status of an import on our Master. our import times are small,
> but there are business reasons why we want to know the status of an import
> after a specified amount of time.
>
> i need to perform certain actions based on the "status" of the import, and
> therefore need to quantify which tags to check and their appropriate
> states.
>
> i am using the command from the DataImportHandler HTTP API to get the
> status
> of the import:
>
> OUTPUT=$(curl -v
> http://${SERVER}:${PORT}/somecore/dataimport?command=status)
>
>
>
>
> can someone tell me if i have these rules correct?
>
> 1) during an import - the status tag will have a busy state:
>
> example:
>
> busy
>
> 2) at the completion of an import (regardless of failure or success) the
> status tag will have an "idle" state:
>
> example:
>
> idle
>
>
> 3) to determine if an import failed or succeeded - you must interrogate the
> tags underand specifically look for :
>
> success:
> Indexing completed. Added/Updated: 603378 documents. Deleted 0
> documents.
>
> failure:
> Indexing completed. Added/Updated: 603378 documents. Deleted 0
> documents.
>
> thank you,
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/possible-status-codes-from-solr-during-a-DIH-data-import-process-tp3987110.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
--
Thanks and Regards
Rahul A. Warawdekar
RE: possible status codes from solr during a (DIH) data import process
You've got it right. Here's a summary:
- "status" = "busy" means its in-process.
- "status" = "idle" means its finished (success or failure).
- You can drill down further by looking at sub-elements under "statusMessages" :
> if there is , it means the last import was cancelled
> with "command=abort"
> look at the body of .
o If it begins with "Indexing completed.", then it finished with a success.
o If it begins with "Indexing failed.", then it finished with a failure.
Just be careful to test your script whenever you change DIH versions. This
status screen isn't the best and no doubt it will change sometime in the
future. Also, keep in mind that as soon as the next import begins the old
statuses get lost so you'll need to plan your script runs around that.
Someday it'll be nice if we can come up with a better way than this to
programitically interact with DIH...
James Dyer
E-Commerce Systems
Ingram Content Group
(615) 213-4311
-Original Message-
From: geeky2 [mailto:gee...@hotmail.com]
Sent: Thursday, May 31, 2012 2:43 PM
To: solr-user@lucene.apache.org
Subject: possible status codes from solr during a (DIH) data import process
hello all,
i have been asked to write a small polling script (bash) to periodically
check the status of an import on our Master. our import times are small,
but there are business reasons why we want to know the status of an import
after a specified amount of time.
i need to perform certain actions based on the "status" of the import, and
therefore need to quantify which tags to check and their appropriate states.
i am using the command from the DataImportHandler HTTP API to get the status
of the import:
OUTPUT=$(curl -v
http://${SERVER}:${PORT}/somecore/dataimport?command=status)
can someone tell me if i have these rules correct?
1) during an import - the status tag will have a busy state:
example:
busy
2) at the completion of an import (regardless of failure or success) the
status tag will have an "idle" state:
example:
idle
3) to determine if an import failed or succeeded - you must interrogate the
tags underand specifically look for :
success:
Indexing completed. Added/Updated: 603378 documents. Deleted 0
documents.
failure:
Indexing completed. Added/Updated: 603378 documents. Deleted 0
documents.
thank you,
--
View this message in context:
http://lucene.472066.n3.nabble.com/possible-status-codes-from-solr-during-a-DIH-data-import-process-tp3987110.html
Sent from the Solr - User mailing list archive at Nabble.com.
