Re: While creating collection in SolrCloud can we manually select machines(nodes)

[Jack Krupansky]

One shard with one replica would be a single machine, so maybe you mean 
either two shards each with one replica or one shard with two replicas.


-- Jack Krupansky

-Original Message- 
From: vidit.asthana

Sent: Saturday, June 28, 2014 5:09 PM
To: solr-user@lucene.apache.org
Subject: While creating collection in SolrCloud can we manually select 
machines(nodes)


I have a 4 machine cluster. I want to create a collection with 1 shard and 1
replica. So I only need 2 machines. Is there a way I can explicitly define
the machines on which my new collection should be created.





--
View this message in context: 
http://lucene.472066.n3.nabble.com/While-creating-collection-in-SolrCloud-can-we-manually-select-machines-nodes-tp4144593.html
Sent from the Solr - User mailing list archive at Nabble.com. 

Re: Streaming large updates with SolrJ

[Joe Gresock]

Yeah, I think that's what I'll have to do, Mikhail.  I was just testing the
waters to see if there was a way to do it with SolrJ.


On Sat, Jun 28, 2014 at 4:11 PM, Mikhail Khludnev <
mkhlud...@griddynamics.com> wrote:

> Joe,
> if the heap is so tight, couldn't you post
> http://wiki.apache.org/solr/UpdateXmlMessages by own optimized code?
>
>
> On Sat, Jun 28, 2014 at 3:13 AM, Joe Gresock  wrote:
>
> > Is there a standard way to stream updates to Solr using SolrJ?
> >  Specifically, we have some atomic updates for large field values
> (hundreds
> > of MB) we'd like to send.  We're currently sending partial updates using
> > SolrInputDocument objects, but we'd love to be able to keep less on the
> > heap in our client code.
> >
> > Thanks,
> > Joe
> >
> > --
> > I know what it is to be in need, and I know what it is to have plenty.  I
> > have learned the secret of being content in any and every situation,
> > whether well fed or hungry, whether living in plenty or in want.  I can
> do
> > all this through him who gives me strength.*-Philippians 4:12-13*
> >
>
>
>
> --
> Sincerely yours
> Mikhail Khludnev
> Principal Engineer,
> Grid Dynamics
>
> 
>  
>



-- 
I know what it is to be in need, and I know what it is to have plenty.  I
have learned the secret of being content in any and every situation,
whether well fed or hungry, whether living in plenty or in want.  I can do
all this through him who gives me strength.*-Philippians 4:12-13*

While creating collection in SolrCloud can we manually select machines(nodes)

[vidit.asthana]

I have a 4 machine cluster. I want to create a collection with 1 shard and 1
replica. So I only need 2 machines. Is there a way I can explicitly define
the machines on which my new collection should be created.





--
View this message in context: 
http://lucene.472066.n3.nabble.com/While-creating-collection-in-SolrCloud-can-we-manually-select-machines-nodes-tp4144593.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Endeca to Solr Migration

[Mikhail Khludnev]

Hello,
Please check inlined below

On Sat, Jun 28, 2014 at 10:10 PM, mrg81  wrote:

> Hello --
>
> I wanted to get some details on Endeca to Solr Migration. I am
> interested in few topics:
>
> 1. We would like to migrate the Faceted Navigation, Boosting individual
> records and a few other items.
> 2. But the biggest question is about the UI [Experience Manager] - I have
> not found a tool that comes close to Experience Manager. I did read about
> Hue [In response to Gareth's question on Migration], but it seems that we
> will have to do a lot of customization to use that.
>
> Questions:
>
> 1. Is there a UI that we can use? Is it possible to un-hook the Experience
> Manager UI and point to Solr?
>
 AFAIK, Experience Manager is close to Adobe's one, and they are both are
clones of guess what.. http://jackrabbit.apache.org/ (check wiki for jcr or
try to visit day.com). I suppose you can employ almost any CMS system
instead, which you consider affordable and handy.

2. How long does a typical migration take? Assuming that we have to migrate
> the Faceted Navigation and Boosted records?
>
I suppose it's not a piece of cake.. I suppose that it takes few month mid
size project to launch it. The challenges are:
- Faceted Navigation, which is done via Dimensions, exposed to frontend
that's quite unnatural for Solr. To be honest, Solr doesn't navigate
taxonomies out-of-the-box, but just provides a few hints to do so. Also,
navigating nested SKUs sometimes reveals some gaps, you know...
- Endeca also has some smart text search features like phrase guessing or
so. It's need to research how much you relies on it, and leverage some
Solr's straightforwardness
note:
- whatever boosting not a problem for Solr
- Hue is a data analytic interactive UI or even "IDE", I don't think you
need to look at.


>
> Thanks
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Endeca-to-Solr-Migration-tp4144582.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>



-- 
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics


 

Re: Streaming large updates with SolrJ

[Mikhail Khludnev]

Joe,
if the heap is so tight, couldn't you post
http://wiki.apache.org/solr/UpdateXmlMessages by own optimized code?


On Sat, Jun 28, 2014 at 3:13 AM, Joe Gresock  wrote:

> Is there a standard way to stream updates to Solr using SolrJ?
>  Specifically, we have some atomic updates for large field values (hundreds
> of MB) we'd like to send.  We're currently sending partial updates using
> SolrInputDocument objects, but we'd love to be able to keep less on the
> heap in our client code.
>
> Thanks,
> Joe
>
> --
> I know what it is to be in need, and I know what it is to have plenty.  I
> have learned the secret of being content in any and every situation,
> whether well fed or hungry, whether living in plenty or in want.  I can do
> all this through him who gives me strength.*-Philippians 4:12-13*
>



-- 
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics


 

Delete single field from solr index

[vidit.asthana]

Solr Experts,

Is there a way to delete a single field from the index(without reindexing).
Lets say i have documents like below:



abab ababa 
bh vsha sa


abab ababa 
bh vsha sa



Now I want to delete field named "data_type" from *ALL *the documents. Is
this possible without reindexing?

I can use atomic updates in case I want to do this only on SOME of the
documents, by setting that field to null. But atomic updates is not
practical for setting that field to null or deleting it from ALL the
documents.




--
View this message in context: 
http://lucene.472066.n3.nabble.com/Delete-single-field-from-solr-index-tp4144589.html
Sent from the Solr - User mailing list archive at Nabble.com.

Endeca to Solr Migration

[mrg81]

Hello --

I wanted to get some details on Endeca to Solr Migration. I am
interested in few topics:

1. We would like to migrate the Faceted Navigation, Boosting individual
records and a few other items. 
2. But the biggest question is about the UI [Experience Manager] - I have
not found a tool that comes close to Experience Manager. I did read about
Hue [In response to Gareth's question on Migration], but it seems that we
will have to do a lot of customization to use that. 

Questions:

1. Is there a UI that we can use? Is it possible to un-hook the Experience
Manager UI and point to Solr?
2. How long does a typical migration take? Assuming that we have to migrate
the Faceted Navigation and Boosted records? 

Thanks



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Endeca-to-Solr-Migration-tp4144582.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Streaming large updates with SolrJ

[Ahmet Arslan]

Hi Joe,

I think it is not possible with SolrJ/SolrInputDocument.
Please vote/watch the issue : https://issues.apache.org/jira/browse/SOLR-6199

Ahmet



On Saturday, June 28, 2014 2:13 AM, Joe Gresock  wrote:
Is there a standard way to stream updates to Solr using SolrJ?
Specifically, we have some atomic updates for large field values (hundreds
of MB) we'd like to send.  We're currently sending partial updates using
SolrInputDocument objects, but we'd love to be able to keep less on the
heap in our client code.

Thanks,
Joe

-- 
I know what it is to be in need, and I know what it is to have plenty.  I
have learned the secret of being content in any and every situation,
whether well fed or hungry, whether living in plenty or in want.  I can do
all this through him who gives me strength.    *-Philippians 4:12-13*

Re: Some questions about Solrcloud

[Shawn Heisey]

On 6/28/2014 10:27 AM, spirtt0 wrote:
> Great. I hope my last questions :) Does zookeeper manage changes in other xml
> config files, except solrconfig.xml and schema.xml ? For example, if i add
> new rule for delta-import to existing dataimport.xml, use zkCli for update
> and reload collection - is it right way to got changes in dataimporthandler
> without restarting SolrCloud ? 

Zookeeper holds the entire configuration, including files other than
schema and solrconfig, like the dataimport configuration.  Uploading the
config with zkCli and restarting the collection is the way to deal with
changes in those files as well.

Thanks,
Shawn

Re: Some questions about Solrcloud

[spirtt0]

Great. I hope my last questions :) Does zookeeper manage changes in other xml
config files, except solrconfig.xml and schema.xml ? For example, if i add
new rule for delta-import to existing dataimport.xml, use zkCli for update
and reload collection - is it right way to got changes in dataimporthandler
without restarting SolrCloud ? 



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Some-questions-about-Solrcloud-tp4144463p4144580.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: How to count children's documents

[Mikhail Khludnev]

Hello,

what about searching children with some constant score query, enabling sum
score mode, and cutting off scores by range? Just an idea, didn't do that
before.


On Sat, Jun 28, 2014 at 7:01 PM, Alexandre Rafalovitch 
wrote:

> Hello,
>
> I want to find a parent document that has more than X children
> documents.  Is there a way to do so on a Solr side? I don't mind
> whether it is precomputed in URP or some sort of value math. I am just
> having troubles figuring out the correct way to do it.
>
> Regards,
>Alex.
> Personal website: http://www.outerthoughts.com/
> Current project: http://www.solr-start.com/ - Accelerating your Solr
> proficiency
>



-- 
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics


 

How to count children's documents

[Alexandre Rafalovitch]

Hello,

I want to find a parent document that has more than X children
documents.  Is there a way to do so on a Solr side? I don't mind
whether it is precomputed in URP or some sort of value math. I am just
having troubles figuring out the correct way to do it.

Regards,
   Alex.
Personal website: http://www.outerthoughts.com/
Current project: http://www.solr-start.com/ - Accelerating your Solr proficiency

Re: CollapsingQParserPlugin throws Exception when useFilterForSortedQuery=true

[Joel Bernstein]

OK, I see the problem. When you use  true
 Solr builds a docSet in a way that seems to be
incompatible with the CollapsingQParserPlugin. With 
true , Solr doesn't run the main query again when
collecting the DocSet. The getDocSetScore() method is expecting the main
query to present, because the CollapsingQParserPlugin may need the scores
generated from the main query, to select the group head.

I think trying to make  true
 compatible with CollapsingQParsePlugin is
probably not possible. So, a nice error message would be a good thing.

Joel Bernstein
Search Engineer at Heliosearch


On Tue, Jun 24, 2014 at 3:31 AM, Umesh Prasad  wrote:

> Hi ,
> Found another bug with CollapsignQParserPlugin. Not a critical one.
>
> It throws an exception when used with
>
>  true 
>
> Patch attached (against 4.8.1 but reproducible in other branches also)
>
>
> 518 T11 C0 oasc.SolrCore.execute [collection1] webapp=null path=null
> params={q=*%3A*&fq=%7B%21collapse+field%3Dgroup_s%7D&defType=edismax&bf=field%28test_ti%29}
> hits=2 status=0 QTime=99
> 4557 T11 C0 oasc.SolrCore.execute [collection1] webapp=null path=null
> params={q=*%3A*&fq=%7B%21collapse+field%3Dgroup_s+nullPolicy%3Dexpand+min%3Dtest_tf%7D&defType=edismax&bf=field%28test_ti%29&sort=}
> hits=4 status=0 QTime=15
> 4587 T11 C0 oasc.SolrException.log ERROR
> java.lang.UnsupportedOperationException: Query  does not implement
> createWeight
> at org.apache.lucene.search.Query.createWeight(Query.java:80)
> at
> org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:684)
> at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:297)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocSetScore(SolrIndexSearcher.java:879)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocSet(SolrIndexSearcher.java:902)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1381)
> at
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:478)
> at
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:461)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:218)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1952)
> at org.apache.solr.util.TestHarness.query(TestHarness.java:295)
> at org.apache.solr.util.TestHarness.query(TestHarness.java:278)
> at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:676)
> at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:669)
> at
> org.apache.solr.search.TestCollapseQParserPlugin.testCollapseQueries(TestCollapseQParserPlugin.java:106)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
> at
> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
> at
> com.carrotsearch.rand

External File Field eating memory

[Kamal Kishore Aggarwal]

Hi Team,

I have recently implemented EFF in solr. There are about 1.5 lacs(unsorted)
values in the external file. After this implementation, the server has
become slow. The solr query time has also increased.

Can anybody confirm me if these issues are because of this implementation.
Is that memory does EFF eats up?

Regards
Kamal Kishore