Re: SolrJ - IOException

2012-09-25 Thread Toke Eskildsen
On Tue, 2012-09-25 at 01:50 +0200, balaji.gandhi wrote: I am encountering this error randomly (under load) when posting to Solr using SolrJ. Has anyone encountered a similar error? org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at:

Re: Admin-UI: multiple facet

2012-09-25 Thread Stefan Matheis
Hey Alex Right now it's not possible .. but we already have an Issue for that: https://issues.apache.org/jira/browse/SOLR-3838 There is no real reason for that limitation .. it's just not done, yet :) Stefan On Tuesday, September 25, 2012 at 1:54 AM, Alexandre Rafalovitch wrote: Hello,

Re: /solr/dataimport not found

2012-09-25 Thread johnohod
*Michael Della Bitta:* /solr/admin/dataimport.jsp shows a gray page with the text: Select handler. *Chris Hostetter:* 1) 3.6.1 2) Yes. I've tried every alternative way of writing the url that I can think of. 3) I think so. I can see the /solr/admin-page and click around without problems. I

AutoIndexing

2012-09-25 Thread darshan
Hi All, Is there any way where I can auto-index whenever there is changes in my database. Thanks, Darshan

Antwort: Re: Solr Cell Questions

2012-09-25 Thread Johannes . Schwendinger
Thank you Erick for your respone, I've already tried what you've suggested and got some out of memory exceptions. Because of this i like the solution with solr Cell where i can send the file directly to solr via stream and don't collect them in my memory. And another question that came to my

Listening to index updates in SolrJ?

2012-09-25 Thread Mathias Hodler
Hi, I'm using Embedded Solr 4.0 with SolrJ. In solrconfig.xml you can specify a RunExecutableListener. Is there something similar in SolrJ, so I can get an event, if the index gets updated? This can be very useful if using SolrCloud, to get an event if other shards updating the index. Thanks.

Re: UIMA for lemmatization

2012-09-25 Thread Tommaso Teofili
Hi, I think you'd better ask this on u...@uima.apache.org list as this is more related to Apache UIMA itself rather than to Apache Solr. Regards, Tommaso 2012/9/25 abhayd ajdabhol...@hotmail.com hi I m new to UIMA. Solr doea not have lemmatization component, i was thinking of using UIMA

Re: AutoIndexing

2012-09-25 Thread Tom Mortimer
Hi Darshan, Can you give us some more details, e.g. what do you mean by database? A RDBMS? Which software? How are you indexing it (or intending to index it) to Solr? etc... cheers, Tom On 25 Sep 2012, at 09:55, darshan dk...@dreamsoftech.com wrote: Hi All, Is there any

Re: How can I create about 100000 independent indexes in Solr?

2012-09-25 Thread Tom Mortimer
Hi, Why do you think that the indexes should be independent? What would be the problem with using a single index and filter queries? Tom On 25 Sep 2012, at 03:21, 韦震宇 weizhe...@win-trust.com wrote: Dear all, The company I'm working in have a website to server more than 10

RE: AutoIndexing

2012-09-25 Thread darshan
My Document is Database(yes RDBMS) and software for it is postgresql, where any change in it's table should be reflected, without re-indexing. I am indexing it via DIH process Thanks, Darshan -Original Message- From: Tom Mortimer [mailto:tom.m.f...@gmail.com] Sent: Tuesday, September 25,

Re: AutoIndexing

2012-09-25 Thread Tom Mortimer
I'm afraid I don't have any DIH experience myself, but some googling suggests that using a postgresql trigger to start a delta import might be one approach: http://wiki.apache.org/solr/DataImportHandler#Using_delta-import_command and

Re: DIH problem

2012-09-25 Thread Gian Marco Tagliani
Ok, I'll try to verify if there is the same issue in the 4.0 and I'll open the issue in Jira. thanks -- Gian Marco On Sat, Sep 22, 2012 at 9:34 PM, Dyer, James james.d...@ingramcontent.comwrote: Gian, Even if you can't write a failing unit test (if you did it would be awesome), please

Re: AW: AW: auto completion search with solr using NGrams in SOLR

2012-09-25 Thread aniljayanti
Hi, Finally got Solved using TermComponents. schema.xml -- fieldType name=edgytext class=solr.TextField analyzer tokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.LowerCaseFilterFactory/ /analyzer /fieldType field name=title type=edgytext

Re: How can I create about 100000 independent indexes in Solr?

2012-09-25 Thread Toke Eskildsen
On Tue, 2012-09-25 at 12:06 +0200, Tom Mortimer wrote: Why do you think that the indexes should be independent? What would be the problem with using a single index and filter queries? Ranking will be done with regard to the full corpus instead of just the single customer. Whether that is an

Join in Solr 3.6

2012-09-25 Thread Peter Kirk
Hi. I have read there is join functionality in Solr 4 beta. Is there a join in Solr 3.6? Thanks, Peter

Re: Join in Solr 3.6

2012-09-25 Thread Rafał Kuć
Hello! If you talk about this: https://issues.apache.org/jira/browse/SOLR-2272 than it is only available since 4.0-alpha. -- Regards, Rafał Kuć Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch Hi. I have read there is join functionality in Solr 4 beta. Is there a

AW: How can I create about 100000 independent indexes in Solr?

2012-09-25 Thread Christian Bordis
Hi weizhenyu! (can you write down your name in roman latters) We also tried to provide dedicated cores for each customer in one of our early system design to get maximum config possibilities. 40k Webshops = 40k Solr Cores, which didn't work out. Problems we encountered: - Threads: Solr

Re: Copyfield query

2012-09-25 Thread Rafał Kuć
Hello! As you can is in the http://wiki.apache.org/solr/SchemaXml#Copy_Fields the actual copying is done before analysis and indexing, so it doesn't matter if you store fields you use as source for your copy fields. -- Regards, Rafał Kuć Sematext :: http://sematext.com/ :: Solr - Lucene

Re: /solr/dataimport not found

2012-09-25 Thread Mukund Chouhan
Hi John, I have created one POC with MySql database which is working fine. I have attached conf folder zip file. please create dummy database and try this. Thanks and Regards, Mukund Chouhan On Tue, Sep 25, 2012 at 1:48 PM, johnohod [via Lucene] ml-node+s472066n4010087...@n3.nabble.com

Re: Re: Solr Cell Questions

2012-09-25 Thread Erick Erickson
bq: how many documents per minute, second, what ever can i put into solr Too many variables to say. I've seen several thousand truly simple docs/sec. But since you're doing the Tika processing that's probably going to be your limiting factor. And it'll be many fewer... I don't understand your

complex Solr Sorting Expression in 4.0 alpha

2012-09-25 Thread lavesh
I want to perform complex sorting expression for which i have upgraded to solr-aplha 4.0 apache-solr-4.0.0-BETA/select?q=:sort=map(sum(SCORE,11),77,77,sum(SCORE,33),sum(SCORE,44)) desc but when i try to run this , its not working. can anyone help me for such complex queries of solr -- View

Antwort: Re: Re: Solr Cell Questions

2012-09-25 Thread Johannes . Schwendinger
The difference with solr cell is, that i'am sending every single document to solr cell and don't collect them until i have a couple of them in my memory. Using mainly the code form here: http://wiki.apache.org/solr/ExtractingRequestHandler#SolrJ Erick Erickson erickerick...@gmail.com schrieb

Re: Solr Cell Questions

2012-09-25 Thread Alexandre Rafalovitch
Are you by any chance committing after every file being indexed? That could cause the speed issues. Also, have you tried to optimize your indexer's java memory params. I use this for mine which used to run out of memory as well: java -server -Xms512m -Xmx2048m Regards, Alex. P.s. I may have

Re: Solr Cell Questions

2012-09-25 Thread Jack Krupansky
Maybe we should even contemplate direct support for Tika/SolrCell in SolrJ - call it SolrJCell. This might also make it a lot easier for apps to apply post-processing after document parser but before data is sent to Solr. And maybe even have an option for multi-process support (invoke Tika as

Re: How can I create about 100000 independent indexes in Solr?

2012-09-25 Thread Toke Eskildsen
On Tue, 2012-09-25 at 04:21 +0200, 韦震宇 wrote: The company I'm working in have a website to server more than 10 customers, and every customer should have it's own search cataegory. So I should create independent index for every customer. How many of the customers are active at any given

Re: mod function not working - Solr 4 beta

2012-09-25 Thread Jack Krupansky
It was added to 4.0 after the beta release. See: http://svn.apache.org/viewvc?view=revisionrevision=1379233 Author:hossman Date:Thu Aug 30 23:49:05 2012 UTC (3 weeks, 4 days ago) Log Message:SOLR-2768: new mod(x,y) function for computing the modulus of two value sources (merge

Re: mod function not working - Solr 4 beta

2012-09-25 Thread Jack Krupansky
Final release date? Uhhh... they're working on it! There is already one RC, but another is expected. Next week maybe? But that's just my personal guess. -- Jack Krupansky -Original Message- From: reeuv Sent: Tuesday, September 25, 2012 10:56 AM To: solr-user@lucene.apache.org

Re: How can I create about 100000 independent indexes in Solr?

2012-09-25 Thread Tanguy Moal
That is an interesting issue... I was wondering if relying on dynamic fields could be an option... Something like : * field_name: field_type * customer : string * *_field_a1 : type_a * *_field_a2 : type_a * *_field_b1 : type_b * ... And the prefix each field by the customer name, so for

Re: /solr/dataimport not found

2012-09-25 Thread Michael Della Bitta
On Tue, Sep 25, 2012 at 4:18 AM, johnohod john-o...@tyde.no wrote: *Michael Della Bitta:* /solr/admin/dataimport.jsp shows a gray page with the text: Select handler. Are there any links to click on on that page? If not, it means the handler isn't configured correctly or the DIH JARs are not

Re: complex Solr Sorting Expression in 4.0 alpha

2012-09-25 Thread Jack Krupansky
Sorry, but as the wiki for the map function states, min,max,target[, and optional default value] are constants, not full function value expressions. Good improvement to request though. See: http://wiki.apache.org/solr/FunctionQuery#map You probably need to resort to if functions. -- Jack

Re: complex Solr Sorting Expression in 4.0 alpha

2012-09-25 Thread lavesh
How to specify a range using if query ? please help me in re-framing this using if query sort=map(sum(SCORE,11),77,77,sum(SCORE,33),sum(SCORE,44)) desc -- View this message in context: http://lucene.472066.n3.nabble.com/complex-Solr-Sorting-Expression-in-4-0-alpha-tp4010160p4010188.html

Re: How to boost date field while boosting a text field?

2012-09-25 Thread srinalluri
Thanks Hoss for your reply. I already seen that Solr wiki, but I am unable to merge that solution with my query. The example in the wiki gave to search for one string. I tried this, but the date is not sorted order, why? ?q={!boost b=recip(ms(NOW,date),3.16e-11,1,1)}ipadfl=date How to merge

Re: How to more gracefully handle field format exceptions?

2012-09-25 Thread Chris Hostetter
: In my thinking (and for this situation), it would be much better to : just ignore the malformed field and keep the doc - is there any way to : configure this or enable this behavior instead? 4.0-ALPHA added some new UpdateProcessorFactories that can be configured in your solrconfig.xml when

Indexing in Solr: invalid UTF-8

2012-09-25 Thread Patrick Oliver Glauner
Hello We use Solr 3.1 and Jetty to index previously extracted fulltexts from PDFs, DOC etc. Our indexing script is written in Python 2.4 using solrpy: [...] text = remove_control_characters(text) # except \r, \t, \n utext = unicode(text, 'utf-8') SOLR_CONNECTION.add(id=recid, fulltext=utext)

RE: Indexing in Solr: invalid UTF-8

2012-09-25 Thread Markus Jelsma
Hi - you need to get rid of all non-character code points. http://unicode.org/cldr/utility/list-unicodeset.jsp?a=[:Noncharacter_Code_Point=True:] -Original message- From:Patrick Oliver Glauner patrick.oliver.glau...@cern.ch Sent: Tue 25-Sep-2012 18:47 To: solr-user@lucene.apache.org

RE: Indexing in Solr: invalid UTF-8

2012-09-25 Thread Patrick Oliver Glauner
Hi Thanks. But I see that 0xd835 is missing in this list (see my exceptions). What's the best way to get rid of all of them in Python? I am new to unicode in Python but I am sure that this use case is quite frequent. Patrick From: Markus Jelsma

Re: AutoIndexing

2012-09-25 Thread Amit Nithian
There's a couple ways to accomplish this from easy to hard depending on your database schema: 1) Use DB trigger - I don't like triggers too much b/c to me they couple your database layer with your application layer which leads to untestable and sometimes unmaintainable code - Also it gets

Making a case to migrate from GSA to Solr

2012-09-25 Thread Matthew Shapiro
Hi all, I don't know if this is the correct mailing list, so I apologize if it isn't. I wasn't sure what other list it would go to. Anyways, my company a while back (before I started) got Google envy and decided to purchase a GSA system to store our searchable data. While the GSA seems ok for

Re: Making a case to migrate from GSA to Solr

2012-09-25 Thread vikash . iitb
Solr can handle all of your pain points. You can sort with any indexed field. It returns correct count. Faceting is trivial. OR conditions are totally fine, it can handle really complex conditional statements. Sent from my iPad On 26-Sep-2012, at 12:48 AM, Matthew Shapiro m...@mshapiro.net

DIH - 'url' element in 'entity' does not work with url parameters

2012-09-25 Thread Billy Newman
Hello All. I am trying to configure the DIH to import XML data from a URL. The url attribute contains parameters in he link and I cannot get it to work. dataConfig dataSource type=URLDataSource / document entity name=test pk=link

Re: DIH - 'url' element in 'entity' does not work with url parameters

2012-09-25 Thread Jack Krupansky
In XML, needs to be written as amp;. In this case amp is the entity name. -- Jack Krupansky -Original Message- From: Billy Newman Sent: Tuesday, September 25, 2012 5:28 PM To: solr-user@lucene.apache.org Subject: DIH - 'url' element in 'entity' does not work with url parameters

Untokenized lowercase string

2012-09-25 Thread am
I am new to Solr. Just wetting my feet, trying to set it up and to migrate our in-house search to it. Is it possible to define a field type that is not tokenized, but has a lowercase filtering? I'm sure I can do it in java code, but I am looking for an XML file solution. Basically Foo Bar and foo

Re: Untokenized lowercase string

2012-09-25 Thread Alexandre Rafalovitch
Each field has a type. Each type defines what happens with the text. You can certainly select to do one thing but not another. Just look towards the bottom of the schema.xml and compare field types definition for string and text, it should be fairly obvious. You'll most probably make up a new

Re: Untokenized lowercase string

2012-09-25 Thread Jack Krupansky
Use the KeywordTokenizerFactory for your text field tokenizer to keep the text from being tokenized, and then use the LowerCaseFilterFactory token filter to do the lowercasing. Unfortunately, string (StrField) does not support analysis. -- Jack Krupansky -Original Message- From: am

Wouldn't be nice: free public example solr server

2012-09-25 Thread Alexandre Rafalovitch
I was just thinking that there is now a couple of SOLR hosting providers (e.g. https://www.dotcloud.com/, but I think there were several). Wouldn't be nice if one of them had a fully setup SOLR4 instance with the 'example' configuration loaded. Then, anybody could go experiment with it even

Re: Untokenized lowercase string

2012-09-25 Thread am
Alexandre Rafalovitch wrote Each field has a type. Each type defines what happens with the text. You can certainly select to do one thing but not another. Understood. But it seemed to me that only TextField allows adding filters to it and filters go in conjunction with tokenizers. I could not

Re: Untokenized lowercase string

2012-09-25 Thread am
That sounds right, thanks! I missed KeywordTokenizerFactory, with a name like that it did not sound like what I wanted. I expected NullTokenizerFactory or something standing out like that :) Jack Krupansky-2 wrote Use the KeywordTokenizerFactory for your text field tokenizer to keep the text

Re: Making a case to migrate from GSA to Solr

2012-09-25 Thread Otis Gospodnetic
Hi Matthew, Solr can do everything you mentioned. Re 3) This won't show anywhere in the UI, but you will get errors in the indexer during indexing. Re 5) You may want to search search-lucene.com for info about boolean queries, which are not true boolean, in case true boolean is really what you

Re: Re: Re: Solr Cell Questions

2012-09-25 Thread Erick Erickson
OK, I was thinking more along the lines of this blog: http://searchhub.org/dev/2012/02/14/indexing-with-solrj/ which uses Tika directly to process the docs on the client (wherever you run it) and only sends the results to Solr The SolrJ program you're referencing uses a different

Re: How to boost date field while boosting a text field?

2012-09-25 Thread Erick Erickson
Did you just try ?q={!boost b=recip(ms(NOW,date),3.16e-11,1,1)}ipad^2fl=date Works just fine. Appending debugQuery=on will show you why the doc ranked as it did, although the output can be cryptic... Best Erick On Tue, Sep 25, 2012 at 11:56 AM, srinalluri nallurisr...@yahoo.com wrote: Thanks

Re: Untokenized lowercase string

2012-09-25 Thread am
Just wanted to confirm that this: fieldtype name=string_lc class=solr.TextField sortMissingLast=true omitNorms=true analyzer filter class=solr.LowerCaseFilterFactory/ tokenizer class=solr.KeywordTokenizerFactory/ /analyzer /fieldtype ...works beautifully for untokenized lowercase

Re: How can I create about 100000 independent indexes in Solr?

2012-09-25 Thread 韦震宇
Hi,Tom I'm prefer performance, so if the performance of single index solution is better than mutilcore solution, I would choice the former. Best Regards! - Original Message - From: Tom Mortimer tom.m.f...@gmail.com To: solr-user@lucene.apache.org Sent: Tuesday, September 25, 2012

Re: How can I create about 100000 independent indexes in Solr?

2012-09-25 Thread Otis Gospodnetic
Hi, Have a look at ElasticSearch. While it's probably no better at handling 100K cores than Solr it, ES has document and query routing functionality that can help you route documents and queries to specific shards, thus avoiding needing to query all 100K users' data. I think Solr doesn't have

Re: How can I create about 100000 independent indexes in Solr?

2012-09-25 Thread 韦震宇
Hi, Bordis Thank you very much for your precious experience! I would choice the single index solution. I'm sorry my name is not pronounced good in english ,you would call me monton. Best Regards! Monton - Original Message - From: Christian Bordis c.bor...@epages.com To:

Re: How can I create about 100000 independent indexes in Solr?

2012-09-25 Thread 韦震宇
Hi, Eskildsen The customer is company, so the active user is tenfold more than the customers. I'm choicing the a single index and splitting index into multiple shards. Best Regards! Monton - Original Message - From: Toke Eskildsen t...@statsbiblioteket.dk To:

Re: How can I create about 100000 independent indexes in Solr?

2012-09-25 Thread 韦震宇
Hi, Tanguy I would do as your suggestion. Best Regards! Monton - Original Message - From: Tanguy Moal tanguy.m...@gmail.com To: solr-user@lucene.apache.org; t...@statsbiblioteket.dk Sent: Tuesday, September 25, 2012 11:05 PM Subject: Re: How can I create about 10 independent

Re: Indexing in Solr: invalid UTF-8

2012-09-25 Thread Robert Muir
On Tue, Sep 25, 2012 at 2:02 PM, Patrick Oliver Glauner patrick.oliver.glau...@cern.ch wrote: Hi Thanks. But I see that 0xd835 is missing in this list (see my exceptions). What's the best way to get rid of all of them in Python? I am new to unicode in Python but I am sure that this use case

Access Error after changing Solr Code

2012-09-25 Thread deniz
Hello, I had changed Solr Beta 4.0 source for learning and testing. I have added a funcitonality for this ticket which was opened after my post on this forum: http://issues.apache.org/jira/browse/SOLR-3862 Basically, now I am able to delete an entry from a multivalued queue, by using curl http