Re: how to index data in solr form database automatically

[Shawn Heisey]

On 2/4/2015 12:01 AM, nvmai wrote:
> I use crontab 
> * */1 * * * wget http://blog.nhipvang.com
> :8983/solr/imuzik/dataimport?command=full-import&clean=false

That will attempt to start a full-import once a minute.  Requests sent
while an import is already underway will fail, but this basically means
that your solr instance will *always* be doing a full import.

That's probably not what you want.

A time specification of "0 * * * *" would do the import once an hour.

Thanks,
Shawn

Re: how to index data in solr form database automatically

[nvmai]

I use crontab 
* */1 * * * wget http://blog.nhipvang.com
:8983/solr/imuzik/dataimport?command=full-import&clean=false



-
mẹo làm đẹp 
--
View this message in context: 
http://lucene.472066.n3.nabble.com/how-to-index-data-in-solr-form-database-automatically-tp3102893p4183792.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: how to index data in solr form database automatically

[Charith Mahawatta]

Hi Mates,

I followed your steps and it worked for me. Thank you very much.

regards,
Charith Mahawatta,

Re: how to index data in solr form database automatically

[Renato Eschini]

On 27/06/11 11:01, Romi wrote:

Can i not use it for full-import ??

yes!!!
You must only change

urlCmd='http://localhost:8080/solr/mycore/dataimport?command=delta-import&clean=false'

in

urlCmd='http://localhost:8080/solr/mycore/dataimport?command=full-import&clean=false'

or


urlCmd='http://localhost:8080/solr/mycore/dataimport?command=full-import&clean=true'


or as you desire!!

Consult DIH handler wiki for the complete list of support params..



--

Renato Eschini
Inera srl
Via Mazzini 138
56100 Pisa (PI)
Tel:(+39) (0)50 9911800
Fax:(+39) (0)50 9911830
Int:(+39) (0)50 9911819
Email:  r.esch...@inera.it
Msn:r_esch...@hotmail.com
Skype:  renato.eschini
WWW:http://www.inera.it

Rispetta l'ambiente - è veramente necessario stampare questa e-mail?
Please consider the environment - do you really need to print this e-mail?

Re: how to index data in solr form database automatically

[Romi]

Can i not use it for full-import ??

-
Thanks & Regards
Romi
--
View this message in context: 
http://lucene.472066.n3.nabble.com/how-to-index-data-in-solr-form-database-automatically-tp3102893p3113002.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: how to index data in solr form database automatically

[Renato Eschini]

Start from this point:

http://wiki.apache.org/solr/DataImportHandler

So, you can try to use this script:

#!/bin/bash
wgetProgram=wget
urlCmd='http://localhost:8080/solr/mycore/dataimport?command=delta-import&clean=false'
statusCmd='http://localhost:8080/solr/mycore/dataimport?command=status'
outputDir=.
$wgetProgram -O $outputDir/check_status_update_index.txt ${statusCmd} 
2>/dev/null

status=`fgrep idle $outputDir/check_status_update_index.txt`
if [[ ${status} == *idle* ]]
then
$wgetProgram -O $outputDir/status_update_index.txt ${urlCmd} 
2>/dev/null

fi


save it somewhere (ex. /root/script_solr) like "update-solr.sh" and 
chmod +x.



Now you can config it from root user in crontab:
linux:~# crontab -e

and insert for run every 2 minutes:

*/2 * * * * cd /root/script_solr && /root/script_solr/update-solr.sh



On 27/06/11 09:01, Romi wrote:

Thanks for answering but i don't know  how to write script for fetching data
from database and index it in to solr and how to setup this script as
cron-job to run automatically at certain interval.


Please suggest




--

Renato Eschini
Inera srl
Via Mazzini 138
56100 Pisa (PI)
Tel:(+39) (0)50 9911800
Fax:(+39) (0)50 9911830
Int:(+39) (0)50 9911819
Email:  r.esch...@inera.it
Msn:r_esch...@hotmail.com
Skype:  renato.eschini
WWW:http://www.inera.it

Rispetta l'ambiente - è veramente necessario stampare questa e-mail?
Please consider the environment - do you really need to print this e-mail?

Re: how to index data in solr form database automatically

[Romi]

Thanks for answering but i don't know  how to write script for fetching data
from database and index it in to solr and how to setup this script as
cron-job to run automatically at certain interval.


Please suggest .


-
Thanks & Regards
Romi
--
View this message in context: 
http://lucene.472066.n3.nabble.com/how-to-index-data-in-solr-form-database-automatically-tp3102893p3112772.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: how to index data in solr form database automatically

[Renato Eschini]

Why don't you use DataImportHandler?

We use DIH, we have a wget based bash script that is runned by cron 
every about 2 minutes. DIH is called in delta-query mode.

The bash script waork in this way:

1) first call a wget on DIH status

2) analyze wget DIH status "/dataimport?command=status"

2.1) if status is **busy** do nothing and exit (beacuse DIH is already 
running)


2.2) if status is **idle** do "/dataimport?command=delta-import&clean=false"

3) exit



On 24/06/11 15:20, Mohammad Shariq wrote:

First write a Script in Python ( or JAVA or PHP or anyLanguage)  which reads
the data from database and index into Solr.

Now setup this script as cron-job to run automatically at certain interval.



--

Renato Eschini
Inera srl
Via Mazzini 138
56100 Pisa (PI)
Tel:(+39) (0)50 9911800
Fax:(+39) (0)50 9911830
Int:(+39) (0)50 9911819
Email:  r.esch...@inera.it
Msn:r_esch...@hotmail.com
Skype:  renato.eschini
WWW:http://www.inera.it

Rispetta l'ambiente - è veramente necessario stampare questa e-mail?
Please consider the environment - do you really need to print this e-mail?

Re: how to index data in solr form database automatically

[Mohammad Shariq]

First write a Script in Python ( or JAVA or PHP or anyLanguage)  which reads
the data from database and index into Solr.

Now setup this script as cron-job to run automatically at certain interval.





On 24 June 2011 17:23, Romi  wrote:

> would you please tell me how can i use Cron for auto index my database
> tables
> in solr
>
> -
> Thanks & Regards
> Romi
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/how-to-index-data-in-solr-form-database-automatically-tp3102893p3103768.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>



-- 
Thanks and Regards
Mohammad Shariq

Re: how to index data in solr form database automatically

[Romi]

would you please tell me how can i use Cron for auto index my database tables
in solr

-
Thanks & Regards
Romi
--
View this message in context: 
http://lucene.472066.n3.nabble.com/how-to-index-data-in-solr-form-database-automatically-tp3102893p3103768.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: how to index data in solr form database automatically

[Pranav Prakash]

Cron is a time-based job scheduler in Unix-like computer operating systems.
en.wikipedia.org/wiki/Cron

*Pranav Prakash*

"temet nosce"

Twitter  | Blog  |
Google 


On Fri, Jun 24, 2011 at 12:26, Romi  wrote:

> Yeah i am using data-import to get data from database and indexing it. but
> what is cron can you please provide a link for it
>
> -
> Thanks & Regards
> Romi
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/how-to-index-data-in-solr-form-database-automatically-tp3102893p3103072.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: how to index data in solr form database automatically

[Romi]

Yeah i am using data-import to get data from database and indexing it. but
what is cron can you please provide a link for it

-
Thanks & Regards
Romi
--
View this message in context: 
http://lucene.472066.n3.nabble.com/how-to-index-data-in-solr-form-database-automatically-tp3102893p3103072.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: how to index data in solr form database automatically

[Anshum]

How about having a delta-import and a cron to trigger the post?

--
Anshum Gupta
http://ai-cafe.blogspot.com


On Fri, Jun 24, 2011 at 11:13 AM, Romi  wrote:

> I have MySql database for my application. i implemented solr search and
> used
> dataimporthandler(DIH)to index data from database into solr. my question
> is:
> is there any way that if database gets updated then my solr indexes
> automatically gets update for new data added in the database. . It means i
> need not to run index process manually every time data base tables
> changes.If yes then please tell me how can i achieve this.
>
> -
> Thanks & Regards
> Romi
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/how-to-index-data-in-solr-form-database-automatically-tp3102893p3102893.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>