Re: Use DIH with more than one entity at the same time

2012-05-19 Thread ketan kore 
i want to configure the solr on tomcat in eclipse ,can u plz help me
out ,how to do this thing
eagerly waiting for ur reply...

On 5/20/12, Lance Norskog  wrote:
> One DataImportHandler  entry is a single-threaded job.
> Always.
>
> You can make two  entries with the same config file and use
> parameter variables to choose which entity to run. They will not share any
> cached data like the SQL caching tool.
>
> On Fri, May 18, 2012 at 8:14 AM, Sergio Martín Cantero <
> sergio.mar...@playence.com> wrote:
>
>>  Yes, James, I have tried it myself.
>>
>> http://localhost:8080/solr/dataimportUsers?command=full-import&entity=users&entity=products
>>
>> Will run secuentially, first entity users and once it has finished,
>> entity
>> products.
>>
>> The problem with that approach is that you are not able to lauch the load
>> of a third entity while those are loading.
>>
>> Let´s say you have 2 entities:
>> - users (takes 30 minutes to finish)
>> - products (takes 1 hour to finish)
>>
>> You cuold want to launch users indexing every 1 hour and product every 15
>> minutes. With the approach of
>> command=full-import&entity=users&entity=products
>> that´s not possible, while having two repeated dataimporters makes it
>> possible.
>>
>> Again, thanks a lot for your help.
>>
>>   [image: www.playence.com] 
>>  Sergio Martín Cantero
>>  Office (ES) +34 91 733 73 97
>>   playence Spain SL sergio.mar...@playence.com
>> Calle Vicente Gaceo 19
>>
>>   28029 Madrid - España
>>
>>
>> El 18/05/12 17:04, Dyer, James escribió:
>>
>>  I don't see a drawback with the 2-handler approach you show.  I think
>> this is the best way to have parallel processing with DIH currently.
>>
>> ** **
>>
>> My understanding of the wiki (
>> http://wiki.apache.org/solr/DataImportHandler#Commands) is that it seems
>> to say that it will run "users" and "products" at the same time with a
>> single handler if you do something like this:
>>
>> ** **
>>
>>
>> http://localhost:8080/solr/dataimportUsers?command=full-import&entity=users&entity=products
>> 
>>
>> ** **
>>
>> But to me this feature seems obscure at best and like the "threads"
>> feature, I would imagine, it would be buggy.  I was wondering if anyone
>> out there is using it with success, or if perhaps the wiki is in error on
>> this point.
>>
>> ** **
>>
>> *James Dyer*
>>
>> E-Commerce Systems
>>
>> Ingram Content Group
>>
>> (615) 213-4311
>>
>> ** **
>>
>> *From:* Sergio Martín Cantero
>> [mailto:sergio.mar...@playence.com]
>>
>> *Sent:* Friday, May 18, 2012 9:33 AM
>> *To:* Dyer, James; solr-user@lucene.apache.org
>>
>> *Subject:* Re: Use DIH with more than one entity at the same time
>>
>>  ** **
>>
>> I see.
>>
>> What I need is not multiple threads for one entity but multiple entities
>> at the same time.
>>
>> What I have done is rename the DIH for each of the entities in
>> solrconfig,
>> altough the are using the same data-import-confg.xml.
>> Something like:
>>   
>>   > class="org.apache.solr.handler.dataimport.DataImportHandler">
>> 
>>   data-import-config.xml
>> 
>>   
>>   
>>   > class="org.apache.solr.handler.dataimport.DataImportHandler">
>> 
>>   data-import-config.xml
>> 
>>   
>>
>> Then I can run each entity at the same time with:
>> http://localhost:8080/solr/dataimportUsers?command=full-import&entity=users
>>
>> http://localhost:8080/solr/dataimportProducts?command=full-import&entity=products
>>
>> Being users and products entities defined in the same
>> data-import-config.xml
>>
>> This way, I don´t need to wait  to run products until users has finished.
>> This allows me to call full-import for users lets say each 15 min and for
>> products each 10 min, and don´t need to wait until one has finsihed. Both
>> can be overlaping.
>>
>> Any drawback to this approach?
>>
>> Thanks!!
>>
>> Sergio
>>
>> El 18/05/12 16:21, Dyer, James escribió: 
>>
>> "threads" lets you run a single entity with multiple threads, so tis
>> probably not what you wanted.  What we've done here is partition the
>> source data and then we have multiple handlers running at the same time,
>> each processing its own partition.  So we multi-thread the import without
>> using the "threads" parameter.  
>>
>>  
>>
>> Even if this sounds like something useful, I recommend against using it.
>> "threads"
>> has tons of bugs, although some fixes were made for Solr 3.6.  For Solr
>> 4.0 this feature is removed.
>>
>>  
>>
>> *James Dyer*
>>
>> E-Commerce Systems
>>
>> Ingram Content Group
>>
>> (615) 213-4311
>>
>>  
>>
>> *From:* Sergio Martín Cantero
>> [mailto:sergio.mar...@playence.com]
>>
>> *Sent:* Friday, May 18, 2012 6:23 AM
>> *To:* solr-user@lucene.apache.org
>> *Cc:* Dyer, James
>>
>> *Subject:* Re: Use DIH with more than one entity at the same time
>> 
>>
>>   
>>
>> What the wiki indicates actually works, altough it´s not what I wanted. I
>> have tri

Re: Indexing & Searching MySQL table with Hindi and English data

2012-05-19 Thread Lance Norskog 
Also, try saving data from a query into a file and verify that it is
UTF-8 and the characters are correct.

On Fri, May 18, 2012 at 7:54 AM, Jack Krupansky  wrote:
> Check the analyzers for the field types containing Hindi text to be sure
> that they are not using a character mapping or "folding" filter that might
> mangle the Hindi characters. Post the field type, say for the "title" field.
>
> Also, try manually (using curl or the post jar) adding a single document
> that has Hindi data and see if that works.
>
> -- Jack Krupansky
>
> -Original Message- From: KP Sanjailal
> Sent: Thursday, May 17, 2012 5:55 AM
> To: solr-user@lucene.apache.org
> Subject: Indexing & Searching MySQL table with Hindi and English data
>
>
> Hi,
>
> I tried to setup indexing of MySQL tables in Apache Solr 3.6.
>
> Everything works fine but text in Hindi script (only some 10% of total
> records) not getting indexed properly.
>
> A search with keyword in Hindi retrieve emptly result set.  Also a
> retrieved hindi record displays junk characters.
>
> The database tables contains bibliographical details of books such as
> title, author, publisher, isbn, publishing place, series etc. and out of
> the total records about 10% of records contains text in Hindi in title,
> author, publisher fields.
>
> Example:
>
> *Search Results from MySQL using PHP*
>
>  1.
> 
>  *Title:* सौर ऊर्जा Saur
> oorja
> *Author(s):* विनोद कुमार मिश्र MISHRA (VK) *Material:* Books **  **
> *Search Results from Apache Solr (searched using keyword in English)*
>
>  1.
> 
>  *Title:* सौर ऊरॠजा Saur
> oorja
> *Author(s):* विनोद कॠमार मिशॠर MISHRA (VK) *
> Material:* Books
>
>
> How do I go about solving this language problem.
>
> Thanks in advace.
>
> K. P. Sanjailal
> --
>



-- 
Lance Norskog
goks...@gmail.com

Re: Use DIH with more than one entity at the same time

2012-05-19 Thread Lance Norskog 
One DataImportHandler  entry is a single-threaded job.
Always.

You can make two  entries with the same config file and use
parameter variables to choose which entity to run. They will not share any
cached data like the SQL caching tool.

On Fri, May 18, 2012 at 8:14 AM, Sergio Martín Cantero <
sergio.mar...@playence.com> wrote:

>  Yes, James, I have tried it myself.
>
> http://localhost:8080/solr/dataimportUsers?command=full-import&entity=users&entity=products
>
> Will run secuentially, first entity users and once it has finished, entity
> products.
>
> The problem with that approach is that you are not able to lauch the load
> of a third entity while those are loading.
>
> Let´s say you have 2 entities:
> - users (takes 30 minutes to finish)
> - products (takes 1 hour to finish)
>
> You cuold want to launch users indexing every 1 hour and product every 15
> minutes. With the approach of command=full-import&entity=users&entity=products
> that´s not possible, while having two repeated dataimporters makes it
> possible.
>
> Again, thanks a lot for your help.
>
>   [image: www.playence.com] 
>  Sergio Martín Cantero
>  Office (ES) +34 91 733 73 97
>   playence Spain SL sergio.mar...@playence.com
> Calle Vicente Gaceo 19
>
>   28029 Madrid - España
>
>
> El 18/05/12 17:04, Dyer, James escribió:
>
>  I don't see a drawback with the 2-handler approach you show.  I think
> this is the best way to have parallel processing with DIH currently.
>
> ** **
>
> My understanding of the wiki (
> http://wiki.apache.org/solr/DataImportHandler#Commands) is that it seems
> to say that it will run "users" and "products" at the same time with a
> single handler if you do something like this:
>
> ** **
>
>
> http://localhost:8080/solr/dataimportUsers?command=full-import&entity=users&entity=products
> 
>
> ** **
>
> But to me this feature seems obscure at best and like the "threads"
> feature, I would imagine, it would be buggy.  I was wondering if anyone
> out there is using it with success, or if perhaps the wiki is in error on
> this point.
>
> ** **
>
> *James Dyer*
>
> E-Commerce Systems
>
> Ingram Content Group
>
> (615) 213-4311
>
> ** **
>
> *From:* Sergio Martín Cantero 
> [mailto:sergio.mar...@playence.com]
>
> *Sent:* Friday, May 18, 2012 9:33 AM
> *To:* Dyer, James; solr-user@lucene.apache.org
>
> *Subject:* Re: Use DIH with more than one entity at the same time
>
>  ** **
>
> I see.
>
> What I need is not multiple threads for one entity but multiple entities
> at the same time.
>
> What I have done is rename the DIH for each of the entities in solrconfig,
> altough the are using the same data-import-confg.xml.
> Something like:
>   
>class="org.apache.solr.handler.dataimport.DataImportHandler">
> 
>   data-import-config.xml
> 
>   
>   
>class="org.apache.solr.handler.dataimport.DataImportHandler">
> 
>   data-import-config.xml
> 
>   
>
> Then I can run each entity at the same time with:
> http://localhost:8080/solr/dataimportUsers?command=full-import&entity=users
>
> http://localhost:8080/solr/dataimportProducts?command=full-import&entity=products
>
> Being users and products entities defined in the same
> data-import-config.xml
>
> This way, I don´t need to wait  to run products until users has finished.
> This allows me to call full-import for users lets say each 15 min and for
> products each 10 min, and don´t need to wait until one has finsihed. Both
> can be overlaping.
>
> Any drawback to this approach?
>
> Thanks!!
>
> Sergio
>
> El 18/05/12 16:21, Dyer, James escribió: 
>
> "threads" lets you run a single entity with multiple threads, so tis
> probably not what you wanted.  What we've done here is partition the
> source data and then we have multiple handlers running at the same time,
> each processing its own partition.  So we multi-thread the import without
> using the "threads" parameter.  
>
>  
>
> Even if this sounds like something useful, I recommend against using it.  
> "threads"
> has tons of bugs, although some fixes were made for Solr 3.6.  For Solr
> 4.0 this feature is removed.
>
>  
>
> *James Dyer*
>
> E-Commerce Systems
>
> Ingram Content Group
>
> (615) 213-4311
>
>  
>
> *From:* Sergio Martín Cantero 
> [mailto:sergio.mar...@playence.com]
>
> *Sent:* Friday, May 18, 2012 6:23 AM
> *To:* solr-user@lucene.apache.org
> *Cc:* Dyer, James
>
> *Subject:* Re: Use DIH with more than one entity at the same time
> 
>
>   
>
> What the wiki indicates actually works, altough it´s not what I wanted. I
> have tried it and works fine.
>
>
> I have also tried Jack´s approach and also works fine (and is what I was
> looking for :-)
>
> Still, I have one more question. You wrote: " This is a 1.4.1
> installation, back when there was no "threads" option in DIH. ". I´m using
> 3.5 Solr. What would the use of threads change. How could I take advantage
> ot it, instead

Re: Editing long Solr URLs - Chrome Extension

2012-05-19 Thread Amit Nithian 
All,

I have placed a new version of the extension (suffixed _0.3) at
https://github.com/ANithian/url_edit_extension/downloads. A few of the
bugs resolved:
1) Switching to a tab in a new window and clicking on the extension
wasn't loading the right URL
2) Complex SOLR URLs (ironic as this was the purpose) weren't being
handled properly. I had to ditch the 3rd party URL parser in favor of
my own which should better handle these complex parameters.
3) Replaced the edit box of the parameter value from a single line
textbox to a multiple line textarea. This doesn't solve the "tab" to
edit the next row but it helps a bit in that problem.

Please keep submitting issues as you encounter them and I'll address
them as best as possible. I hope that this helps everyone!

Thanks!
Amit

On Tue, May 15, 2012 at 6:20 PM, Amit Nithian  wrote:
> Erick
>
> Yes thanks I did see that and am working on a solution to that already. Hope
> to post a new revision shortly and eventually migrate to the extension
> "store".
>
> Cheers
> Amit
>
> On May 15, 2012 9:20 AM, "Erick Erickson"  wrote:
>>
>> I think I put one up already, but in case I messed up github, complex
>> params like the fq here:
>>
>> http://localhost:8983/solr/select?q=:&fq={!geofilt sfield=store
>> pt=52.67,7.30 d=5}
>>
>> aren't properly handled.
>>
>> But I'm already using it occasionally
>>
>> Erick
>>
>> On Tue, May 15, 2012 at 10:02 AM, Amit Nithian  wrote:
>> > Jan
>> >
>> > Thanks for your feedback! If possible can you file these requests on the
>> > github page for the extension so I can work on them? They sound like
>> > great
>> > ideas and I'll try to incorporate all of them in future releases.
>> >
>> > Thanks
>> > Amit
>> > On May 11, 2012 9:57 AM, "Jan Høydahl"  wrote:
>> >
>> >> I've been testing
>> >>
>> >> https://chrome.google.com/webstore/detail/mbnigpeabbgkmbcbhkkbnlidcobbapff?hl=enbut
>> >> I don't think it's great.
>> >>
>> >> Great work on this one. Simple and straight forward. A few wishes:
>> >> * Sticky mode? This tool would make sense in a sidebar, to do rapid
>> >> refinements
>> >> * If you edit a value and click "TAB", it is not updated :(
>> >> * It should not be necessary to URLencode all non-ascii chars - why not
>> >> leave colon, caret (^) etc as is, for better readability?
>> >> * Some param values in Solr may be large, such as "fl", "qf" or "bf".
>> >> Would be nice if the edit box was multi-line, or perhaps adjusts to the
>> >> size of the content
>> >>
>> >> --
>> >> Jan Høydahl, search solution architect
>> >> Cominvent AS - www.facebook.com/Cominvent
>> >> Solr Training - www.solrtraining.com
>> >>
>> >> On 11. mai 2012, at 07:32, Amit Nithian wrote:
>> >>
>> >> > Hey all,
>> >> >
>> >> > I don't know about you but most of the Solr URLs I issue are fairly
>> >> > lengthy full of parameters on the query string and browser location
>> >> > bars aren't long enough/have multi-line capabilities. I tried to find
>> >> > something that does this but couldn't so I wrote a chrome extension
>> >> > to
>> >> > help.
>> >> >
>> >> > Please check out my blog post on the subject and please let me know
>> >> > if
>> >> > something doesn't work or needs improvement. Of course this can work
>> >> > for any URL with a query string but my motivation was to help edit my
>> >> > long Solr URLs.
>> >> >
>> >> >
>> >>
>> >> http://hokiesuns.blogspot.com/2012/05/manipulating-urls-with-long-query.html
>> >> >
>> >> > Thanks!
>> >> > Amit
>> >>
>> >>

Re: Meaning of avgTimePerRequest & avgRequestsPerSecond in SOLR stats page

2012-05-19 Thread shinkanze 
hi , 
am also confused abt the same when doing the maths results are different

rajat

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Meaning-of-avgTimePerRequest-avgRequestsPerSecond-in-SOLR-stats-page-tp1922692p3984848.html
Sent from the Solr - User mailing list archive at Nabble.com.