Re: solr4.6.0 with Glassfish 3.1.2.2

[Varun Krishna Parthasarathy]

Shawn
Thank you so much.



With Regards
Varun Krishna. P


On Thu, Dec 12, 2013 at 11:47 AM, Shawn Heisey  wrote:

> On 12/11/2013 8:54 PM, Varun Krishna Parthasarathy wrote:
> > I am trying to install solr 4.6.0 under glassfish 3.1.2.2, and have
> > followed the instructions given at
> > http://wiki.apache.org/solr/SolrGlassfish and
> > http://solrwithglassfish.blogspot.in/
> > But either of the two does not seem to work for me. I am getting the http
> > status 404 the requested resource is not available.
> >
> > Can you please help me configuring solr4.6.0 with glassfish 3.1.2.2.
>
> There are two main hurdles with any servlet container other than the
> included Jetty.  One is getting it to extract and start the .war file,
> the other (if using Solr 4.3 or later) is the logging jars and the
> logging config (log4j.properties), which are no longer included in the
> .war file.  For best results, the logging jars must be made available to
> the servlet container before Solr is loaded.  Solr will not start if the
> SLF4J jars cannot be properly loaded.
>
> http://wiki.apache.org/solr/SolrLogging#Solr_4.3_and_above
>
> One option, if you can't figure out how to get external logging jars in
> the proper location, is to add the logging jars to the war file, under
> WEB-INF/lib, which is how Solr was packaged before version 4.3.
>
> I know nothing about Glassfish.  The Jetty included in the example is
> the only container that's actually tested, and it's strongly recommended
> that you use that instead of any other container choice.  It is a
> production quality container.  It would not be included if it were only
> good enough for proof of concept installs.  Also, a number of
> large-scale public-facing systems use Jetty.
>
> http://www.infoq.com/news/2009/08/google-chose-jetty
>
> Beyond what I've said here, you may need help from Glassfish support
> resources.
>
> Thanks,
> Shawn
>
>

Load existing HDFS files into solr?

[cynosure]

Folks,
Our current data is stored in hive tables. Is there a way to specify solr
to index the existing hdfs files directly? or I have to import each hive
table to solr?
Can any one point to me some reference?
Thank you very much!
Chen

Re: Importing/Indexing the DB2 XML FieldType in SOLR

[ravi1984]

Thanks again Shawn. The CLOB transformer helped.

Thanks
Ravi



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Importing-Indexing-the-DB2-XML-FieldType-in-SOLR-tp4105079p4106307.html
Sent from the Solr - User mailing list archive at Nabble.com.

unable to facet range query

[Nutan]

My schema has :



this is my field which i want to facet:



I replaced above fieldtype with this:


But now this shows error in elevate component.My elevate.xml is









integer
elevate.xml


*Logs:*
Caused by: org.apache.solr.common.SolrException: Error initializing
QueryElevationComponent.
at
org.apache.solr.handler.component.QueryElevationComponent.inform(QueryElevationComponent.java:218)
at
org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:592)
at org.apache.solr.core.SolrCore.(SolrCore.java:801)
... 13 more
Caused by: org.apache.solr.common.SolrException: Invalid Number: promotions
at
org.apache.solr.analysis.TrieTokenizer.reset(TrieTokenizerFactory.java:122)

i read that range queries are for numeric fields,isn't IntField a numeric
one?
What are the other datatypes that support range queries?



--
View this message in context: 
http://lucene.472066.n3.nabble.com/unable-to-facet-range-query-tp4106305.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: solr4.6.0 with Glassfish 3.1.2.2

[Shawn Heisey]

On 12/11/2013 8:54 PM, Varun Krishna Parthasarathy wrote:
> I am trying to install solr 4.6.0 under glassfish 3.1.2.2, and have
> followed the instructions given at
> http://wiki.apache.org/solr/SolrGlassfish and
> http://solrwithglassfish.blogspot.in/
> But either of the two does not seem to work for me. I am getting the http
> status 404 the requested resource is not available.
> 
> Can you please help me configuring solr4.6.0 with glassfish 3.1.2.2.

There are two main hurdles with any servlet container other than the
included Jetty.  One is getting it to extract and start the .war file,
the other (if using Solr 4.3 or later) is the logging jars and the
logging config (log4j.properties), which are no longer included in the
.war file.  For best results, the logging jars must be made available to
the servlet container before Solr is loaded.  Solr will not start if the
SLF4J jars cannot be properly loaded.

http://wiki.apache.org/solr/SolrLogging#Solr_4.3_and_above

One option, if you can't figure out how to get external logging jars in
the proper location, is to add the logging jars to the war file, under
WEB-INF/lib, which is how Solr was packaged before version 4.3.

I know nothing about Glassfish.  The Jetty included in the example is
the only container that's actually tested, and it's strongly recommended
that you use that instead of any other container choice.  It is a
production quality container.  It would not be included if it were only
good enough for proof of concept installs.  Also, a number of
large-scale public-facing systems use Jetty.

http://www.infoq.com/news/2009/08/google-chose-jetty

Beyond what I've said here, you may need help from Glassfish support
resources.

Thanks,
Shawn

Re: Solr Profiler

[Otis Gospodnetic]

Hi,

Are you looking for a Java profiler?  Or a Solr monitoring tool?
For a profiler I'd recommend YourKit -- http://www.yourkit.com/
For Solr monitoring I'd recommend our SPM --
http://sematext.com/spm/solr-performance-monitoring/index.html

Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/


On Wed, Dec 11, 2013 at 3:46 PM, Monica Skidmore <
monica.skidm...@careerbuilder.com> wrote:

> We're trying to improve the speed of some custom Solr code we've written,
> and we'd like to use a profiler to help us focus our efforts.  However,
> we've tried both JProfiler and NewRelic, and we've found it challenging to
> configure them correctly to be able to tell where our bottlenecks really
> are.
>
> What profilers/configurations have people successfully used for Solr?
>
> Monica Skidmore
> Engineering Lead, Core Search
> CareerBuilder.com
>
>

Re: Solr hardware memory question

[Otis Gospodnetic]

Hi Gil,

I'd look at the number and type of fields you sort and facet on (this stuff
likes memory).
I'd keep in mind heaps over 32 GB use bigger pointers, so maybe more
smaller heaps are better than one big one.
You didn't mention the # of CPU cores, but keep that in mind when sharding.
 When a query comes in, you want to put all your CPU cores to work.
...

Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/


On Tue, Dec 10, 2013 at 11:51 AM, Hoggarth, Gil  wrote:

> We're probably going to be building a Solr service to handle a dataset
> of ~60TB, which for our data and schema typically gives a Solr index
> size of 1/10th - i.e., 6TB. Given there's a general rule about the
> amount of hardware memory required should exceed the size of the Solr
> index (exceed to also allow for the operating system etc.), how have
> people handled this situation? Do I really need, for example, 12 servers
> with 512GB RAM, or are there other techniques to handling this?
>
>
>
> Many thanks in advance for any general/conceptual/specific
> ideas/comments/answers!
>
> Gil
>
>
>
>
>
> Gil Hoggarth
>
> Web Archiving Technical Services Engineer
>
> The British Library, Boston Spa, West Yorkshire, LS23 7BQ
>
>

Re: CollapsingQParserPlugin scores incorrectly in Solr 4.6.0 when multiple sort criteria are used

[Umesh Prasad]

Issue occurs in Single Segment index also ..

sort: "score desc,floSalesRank asc"
response: {

   - numFound: 21461,
   - start: 0,
   - maxScore: 4.4415073,
   - docs: [
  - {
 - floSalesRank: 0,
 - score: 0.123750895,
 - [docid]: 9208
 -




On Thu, Dec 12, 2013 at 9:50 AM, Umesh Prasad  wrote:

> Hi All,
> I am using new CollapsingQParserPlugin for Grouping and found that it
> works incorrectly when I use multiple sort criteria.
>
>
>
> http://localhost:8080/solr/toys/select/?q=car%20and%20toys&version=2.2&start=0&rows=10&indent=on&sort=score%20desc,floSalesRank%20asc&facet=on&facet.field=store_path&facet.mincount=1&bq=store_path:%22mgl/ksc/gcv%22
> ^10&wt=json&fl=score,floSalesRank,[docid]&bq=id:STFDCHZM3552AHXE^1000&fq={!collapse%20field=item_id}
>
>
>- sort: "score desc,floSalesRank asc",
>- fl: "score,floSalesRank,[docid]",
>- start: "0",
>- q: "car and toys",
>- facet.field: "store_path",
>- fq: "{!collapse field=item_id}"
>
>
> response:
>
> {
>
>- numFound: 21461,
>- start: 0,
>- maxScore: 4.447499,
>- docs: [
>   - {
>  - floSalesRank: 0,
>  - score: 0.12396862,
>  - [docid]: 9703
>  },
>   - {
>   -
>
>
> I found a bug opened for same
> https://issues.apache.org/jira/browse/SOLR-5408 ..
>
>
> The bug is closed but I am not really sure that it works specially for
> Multiple segment parts ..
>
> I am using Solr 4.6.0 and my index contains 4 segments ..
>
> Have anyone else faced the same issue ?
>
> ---
> Thanks & Regards
> Umesh Prasad
>



-- 
---
Thanks & Regards
Umesh Prasad

CollapsingQParserPlugin scores incorrectly in Solr 4.6.0 when multiple sort criteria are used

[Umesh Prasad]

Hi All,
I am using new CollapsingQParserPlugin for Grouping and found that it
works incorrectly when I use multiple sort criteria.


http://localhost:8080/solr/toys/select/?q=car%20and%20toys&version=2.2&start=0&rows=10&indent=on&sort=score%20desc,floSalesRank%20asc&facet=on&facet.field=store_path&facet.mincount=1&bq=store_path:%22mgl/ksc/gcv%22
^10&wt=json&fl=score,floSalesRank,[docid]&bq=id:STFDCHZM3552AHXE^1000&fq={!collapse%20field=item_id}


   - sort: "score desc,floSalesRank asc",
   - fl: "score,floSalesRank,[docid]",
   - start: "0",
   - q: "car and toys",
   - facet.field: "store_path",
   - fq: "{!collapse field=item_id}"


response:

{

   - numFound: 21461,
   - start: 0,
   - maxScore: 4.447499,
   - docs: [
  - {
 - floSalesRank: 0,
 - score: 0.12396862,
 - [docid]: 9703
 },
  - {
  -


I found a bug opened for same
https://issues.apache.org/jira/browse/SOLR-5408 ..


The bug is closed but I am not really sure that it works specially for
Multiple segment parts ..

I am using Solr 4.6.0 and my index contains 4 segments ..

Have anyone else faced the same issue ?

---
Thanks & Regards
Umesh Prasad

solr4.6.0 with Glassfish 3.1.2.2

[Varun Krishna Parthasarathy]

Hello
I am trying to install solr 4.6.0 under glassfish 3.1.2.2, and have
followed the instructions given at
http://wiki.apache.org/solr/SolrGlassfish and
http://solrwithglassfish.blogspot.in/
But either of the two does not seem to work for me. I am getting the http
status 404 the requested resource is not available.

Can you please help me configuring solr4.6.0 with glassfish 3.1.2.2.




Thanking You
With Regards
Varun Krishna. P

solrcloud split problem

[xuke...@gmail.com]

hello, i use solr 4.5.1. i use solrcloud and split a collection shard to two 
shard,then i unload the original shard. when i select the collection,  i get a 
error.

{
  "responseHeader": {
"status": 500,
"QTime": 15,
"params": {
  "indent": "true",
  "q": "*:*",
  "_": "1386812476045",
  "wt": "json"
}
  },
  "error": {
"trace": "java.lang.NullPointerException\n\tat 
org.apache.solr.handler.component.QueryComponent.createRetrieveDocs(QueryComponent.java:934)\n\t
at 
org.apache.solr.handler.component.QueryComponent.regularDistributedProcess(QueryComponent.java:617)\n\t
at 
org.apache.solr.handler.component.QueryComponent.distributedProcess(QueryComponent.java:565)\n\t
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:244)\n\t
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)\n\t
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)\n\t
at 
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703)\n\t
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406)\n\t
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)\n\t
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)\n\t
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)\n\t
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)\n\t
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)\n\t
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)\n\t
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)\n\t
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)\n\t
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)\n\t
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)\n\t
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)\n\t
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)\n\t
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)\n\t
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)\n\t
at org.eclipse.jetty.server.Server.handle(Server.java:368)\n\t
at 
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)\n\t
at 
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)\n\t
at 
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)\n\t
at 
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)\n\t
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)\n\t
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)\n\t
at 
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)\n\t
at 
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)\n\t
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)\n\t
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)\n\t
at java.lang.Thread.run(Thread.java:662)\n",
"code": 500
  }
}




xukexu

Re: Solr hardware memory question

[Joel Bernstein]

Curious how many documents per shard you were planning? The number of
documents per shard and field type will drive the amount of a RAM needed to
sort and facet.


On Wed, Dec 11, 2013 at 7:02 AM, Toke Eskildsen wrote:

> On Tue, 2013-12-10 at 17:51 +0100, Hoggarth, Gil wrote:
> > We're probably going to be building a Solr service to handle a dataset
> > of ~60TB, which for our data and schema typically gives a Solr index
> > size of 1/10th - i.e., 6TB. Given there's a general rule about the
> > amount of hardware memory required should exceed the size of the Solr
> > index (exceed to also allow for the operating system etc.), how have
> > people handled this situation?
>
> By acknowledging that it is cheaper to buy SSDs instead of trying to
> compensate for slow spinning drives with excessive amounts of RAM.
>
> Our plans for an estimated 20TB of indexes out of 372TB of raw web data
> is to use SSDs controlled by a single machine with 512GB of RAM (or was
> it 256GB? I'll have to ask the hardware guys):
> https://sbdevel.wordpress.com/2013/12/06/danish-webscale/
>
> As always YMMW and the numbers you quite elsewhere indicates that your
> queries are quite complex. You might want to be a bit of profiling to
> see if they are heavy enough to make the CPU the bottleneck.
>
> Regards,
> Toke Eskildsen, State and University Library, Denmark
>
>
>


-- 
Joel Bernstein
Search Engineer at Heliosearch

Re: Solr Cloud error with shard update

[Shawn Heisey]

On 12/11/2013 5:01 PM, dboychuck wrote:
> I am running a data import and it is throwing all kinds of errors. I am
> upgrading to 4.6 from 4.5.1 with the exact schema and solrconfig and dih
> configs.
> 
> Here is the error I am getting:
> 
> org.apache.solr.common.SolrException: ERROR: [doc=k-690kohler!670614] Error
> adding field 'weight'='java.math.BigDecimal:9.28' msg=For input string:
> "java.math.BigDecimal:9.28"
>   at
> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:167)
>   at
> org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:77)
>   at
> org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:215)
>   at
> org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69)
>   at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
>   at
> org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:569)
>   at
> org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:705)
>   at
> org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:435)
>   at org.apache.solr.update.PeerSync.handleUpdates(PeerSync.java:482)
>   at org.apache.solr.update.PeerSync.handleResponse(PeerSync.java:330)
>   at org.apache.solr.update.PeerSync.sync(PeerSync.java:260)
>   at
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:387)
>   at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:244)
> Caused by: java.lang.NumberFormatException: For input string:
> "java.math.BigDecimal:9.28"
>   at 
> sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1241)
>   at java.lang.Float.parseFloat(Float.java:452)
>   at org.apache.solr.schema.TrieField.createField(TrieField.java:593)
>   at org.apache.solr.schema.TrieField.createFields(TrieField.java:650)
>   at 
> org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:47)
>   at
> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:118)
>   ... 12 more

I have been discussing this with David on IRC.  Everything works
perfectly for him on 4.5.1 with identical configs.  On 4.6.0, the update
succeeds on the server where DIH is running, but fails when the update
is forwarded to SolrCloud replicas.  The source database is MSSQL, with
a decimal DB column and a TrieFloatField in Solr.

I am not sure yet what the problem is.  I know that there's been a lot
of code tightening going on in order to fix SolrCloud indexing deadlocks.

Thanks,
Shawn

Re: Solr Cloud error with shard update

[dboychuck]

Also I have tried setting my the schema defintion to float for the offending
fields in the schema.xml as well as tried to cast my columns to strings in
the query. Both still give the same result.

My java version is:

java version "1.7.0_10"
Java(TM) SE Runtime Environment (build 1.7.0_10-b18)
Java HotSpot(TM) 64-Bit Server VM (build 23.6-b04, mixed mode)



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-Cloud-error-with-shard-update-tp4106260p4106271.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr Cloud error with shard update

[dboychuck]

And here are the logs of one of the replicas:


2286617 [Thread-146] WARN  org.apache.solr.cloud.RecoveryStrategy  –
Stopping recovery for zkNodeName=core_node2core=productindex
2286627 [Thread-147] WARN  org.apache.solr.cloud.RecoveryStrategy  –
Stopping recovery for zkNodeName=core_node2core=productindex
2286638 [Thread-159] WARN  org.apache.solr.cloud.RecoveryStrategy  –
Stopping recovery for zkNodeName=core_node2core=productindex
2286641 [Thread-160] WARN  org.apache.solr.cloud.RecoveryStrategy  –
Stopping recovery for zkNodeName=core_node2core=productindex
2286643 [Thread-158] WARN  org.apache.solr.cloud.RecoveryStrategy  –
Stopping recovery for zkNodeName=core_node2core=productindex
2286645 [Thread-154] WARN  org.apache.solr.cloud.RecoveryStrategy  –
Stopping recovery for zkNodeName=core_node2core=productindex
2286647 [Thread-155] WARN  org.apache.solr.cloud.RecoveryStrategy  –
Stopping recovery for zkNodeName=core_node2core=productindex
2286648 [Thread-157] WARN  org.apache.solr.cloud.RecoveryStrategy  –
Stopping recovery for zkNodeName=core_node2core=productindex
2286650 [Thread-152] WARN  org.apache.solr.cloud.RecoveryStrategy  –
Stopping recovery for zkNodeName=core_node2core=productindex
2286652 [Thread-150] WARN  org.apache.solr.cloud.RecoveryStrategy  –
Stopping recovery for zkNodeName=core_node2core=productindex
2286654 [Thread-151] WARN  org.apache.solr.cloud.RecoveryStrategy  –
Stopping recovery for zkNodeName=core_node2core=productindex
2286655 [Thread-148] WARN  org.apache.solr.cloud.RecoveryStrategy  –
Stopping recovery for zkNodeName=core_node2core=productindex
2286657 [Thread-156] WARN  org.apache.solr.cloud.RecoveryStrategy  –
Stopping recovery for zkNodeName=core_node2core=productindex
2290696 [RecoveryThread] ERROR org.apache.solr.update.PeerSync  – PeerSync:
core=productindex url=http://solr-shard-5.sys.id.build.com:8080/solr Error
applying updates from [Ljava.lang.String;@6c8cbe52 ,update=[1,
1454173058518482944, SolrInputDocument(fields: [productAddDate=Fri Aug 17
12:03:00 PDT 2007, keywords_84_txtws=[K-690,  Vinnata],
InstallationType_string=Deck Mounted, InstallationAvailable_boolean=true,
popularity_82_i=98960, pricebook_26_fs=[700.88, 483.64, 630.30],
keywords_15_txtws=[K-690,  Vinnata], keywords_33_txtws=[K-690,  Vinnata],
keywords_5_txtws=[K-690,  Vinnata], uniquePricebook_28_f=700.88,
PulloutSpray_boolean=true, pricebook_28_fs=[700.88, 483.64, 630.30],
uniquePricebook_27_f=700.88, type_string=faucet, popularity_84_i=97124,
uniquePricebook_29_f=700.88, popularity_85_i=3940, productid=k-690,
masterFinish=[Chromes, Nickel Tones, Chromes, Bronze Tones, Nickel Tones,
Bronze Tones], description=**Note: This faucet is extremely large
please see specification sheet below for dimensions. If you are looking for
this same faucet but a little smaller.  Please check out the
K-691.
Product Features:

Fully covered under Kohler's limited lifetime warranty
All-metal faucet body and handle construction
Superior finishing process - faucet finish covered under lifetime
warranty
Includes cover plate (escutcheon) for sinks with 3-holes (8"
centers)
Insulated pullout spray faucet head and retractable hose
Vinnata kitchen faucets not only boast good looks, but superior
functionality as well
Featuring a MasterClean spray face that resists hard water build-up
Spout swivels 360 degrees to allow for unobstructed sink access
High-arch gooseneck spout further allows for unobstructed sink
access

Product Specifications:

Overall Height: 16-5/8" (measured from counter top to highest point of
faucet)
Spout Height: 10" (measured from counter top to faucet outlet)
Spout Reach: 9-1/2" (measured from center of faucet base to center of
faucet outlet)
Number of installation holes required: 1
2.2 gallons-per-minute flow rate
Maximum Deck Thickness: 1-3/8"
1 handle included
ADA compliant
Low lead compliant - complies with federal and state regulations for
lead content
Designed to easily connect to standard U.S. plumbing supply bibs
Secure mounting assembly
All necessary mounting hardware included

Variations:

K-690: This model
K-691: Bar (smaller) version of this model

About Kohler:
Gracious living is characterized by qualities of charm, good taste,
generosity of spirit and the enhancement of nature. It is Kohler’s
mission that you can improve your sense of gracious living with every
experience you have with a Kohler product or service.
With everything that Kohler does it lives on the leading edge of design
and technology, while maintaining a constant level of quality regardless of
price. Kohler sets the standard of excellence in its field, and yet is
constantly striving to improve.
From sustainable products in the kitchen and bath, to engines for lawn
tractors and generators, to power systems that protect data centers and
urban landscapes all over the world; this is what the Kohler Company is all
about., siteid=[4, 82, 84], NumberOfHandles_numeric=1,
groupid=k-690kohler, Filtering_boolean=false, popularity_4_

Re: Solr Cloud error with shard update

[dboychuck]

Here is the output from the logs of the server running the import:

598413 [updateExecutor-1-thread-62] ERROR
org.apache.solr.update.StreamingSolrServers  – error
org.apache.solr.common.SolrException: Bad Request



request:
http://solr-shard-5.sys.id.build.com:8080/solr/productindex/update?update.distrib=FROMLEADER&distrib.from=http%3A%2F%2Fsolr-shard-4.sys.id.build.com%3A8080%2Fsolr%2Fproductindex%2F&wt=javabin&version=2
at
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner.run(ConcurrentUpdateSolrServer.java:240)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2598413 [updateExecutor-1-thread-63] ERROR
org.apache.solr.update.StreamingSolrServers  – error
org.apache.solr.common.SolrException: Bad Request



request:
http://solr-shard-6.sys.id.build.com:8080/solr/productindex/update?update.distrib=FROMLEADER&distrib.from=http%3A%2F%2Fsolr-shard-4.sys.id.build.com%3A8080%2Fsolr%2Fproductindex%2F&wt=javabin&version=2
at
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner.run(ConcurrentUpdateSolrServer.java:240)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2598449 [updateExecutor-1-thread-63] ERROR
org.apache.solr.update.StreamingSolrServers  – error
org.apache.solr.common.SolrException: Bad Request



request:
http://solr-shard-5.sys.id.build.com:8080/solr/productindex/update?update.distrib=FROMLEADER&distrib.from=http%3A%2F%2Fsolr-shard-4.sys.id.build.com%3A8080%2Fsolr%2Fproductindex%2F&wt=javabin&version=2
at
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner.run(ConcurrentUpdateSolrServer.java:240)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2598449 [updateExecutor-1-thread-62] ERROR
org.apache.solr.update.StreamingSolrServers  – error
org.apache.solr.common.SolrException: Bad Request



request:
http://solr-shard-6.sys.id.build.com:8080/solr/productindex/update?update.distrib=FROMLEADER&distrib.from=http%3A%2F%2Fsolr-shard-4.sys.id.build.com%3A8080%2Fsolr%2Fproductindex%2F&wt=javabin&version=2
at
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner.run(ConcurrentUpdateSolrServer.java:240)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2598493 [updateExecutor-1-thread-63] ERROR
org.apache.solr.update.StreamingSolrServers  – error
org.apache.solr.common.SolrException: Bad Request



request:
http://solr-shard-6.sys.id.build.com:8080/solr/productindex/update?update.distrib=FROMLEADER&distrib.from=http%3A%2F%2Fsolr-shard-4.sys.id.build.com%3A8080%2Fsolr%2Fproductindex%2F&wt=javabin&version=2
at
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner.run(ConcurrentUpdateSolrServer.java:240)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2598494 [updateExecutor-1-thread-62] ERROR
org.apache.solr.update.StreamingSolrServers  – error
org.apache.solr.common.SolrException: Bad Request



request:
http://solr-shard-5.sys.id.build.com:8080/solr/productindex/update?update.distrib=FROMLEADER&distrib.from=http%3A%2F%2Fsolr-shard-4.sys.id.build.com%3A8080%2Fsolr%2Fproductindex%2F&wt=javabin&version=2
at
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner.run(ConcurrentUpdateSolrServer.java:240)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2598538 [updateExecutor-1-thread-63] ERROR
org.apache.solr.update.StreamingSolrServers  – error
org.apache.solr.common.SolrException: Bad Request



request:
http://solr-shard-6.sys.id.build.com:8080/solr/productindex/update?update.distrib=FROMLEADER&distrib.from=http%3A%2F%2Fsolr-shard-4.sys.id.build.com%3A8080%2Fsolr%2Fproductindex%2F&wt=javabin&version=2
at
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner.run(ConcurrentUpdateSolrServer.java:240)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)

Re: SolrCloud with large synonym files

[gf80]

Tx a lot Jack tomorrow I split synonyms file. Anyway, do you know how can I set 
jute.maxbuffer? 
--giuseppe 
Inviato da iPhone

Il giorno 11/dic/2013, alle ore 22:40, "Jack Krupansky-2 [via Lucene]" 
 ha scritto:

> Note that the synonym filter accepts a comma-separated list of synonym 
> files, so you can split your huge synonym file into two or more smaller 
> files. 
> 
> --Jack Krupansky 
> 
> -Original Message- 
> From: gf80 
> Sent: Wednesday, December 11, 2013 5:21 PM 
> To: [hidden email] 
> Subject: RE: SolrCloud with large synonym files 
> 
> Hi all, 
> 
> I have the same exception 
> 
> org.apache.zookeeper.KeeperException$ConnectionLossException: 
> KeeperErrorCode = 
> ConnectionLoss for /configs/myconf/synonyms.txt 
> 
> synonyms.txt is greater than 1,5Mb 
> 
> I have tried to set jute.maxbuffer specifying -Djute.maxbuffer=1024M at the 
> solrcloud startup but I'm not able to figure out the problem. 
> Please let me know your suggestions/solutions. 
> Thanks a lot in advance, 
> Giuseppe 
> 
> 
> 
> -- 
> View this message in context: 
> http://lucene.472066.n3.nabble.com/SolrCloud-with-large-synonym-files-tp3473568p4106239.html
> Sent from the Solr - User mailing list archive at Nabble.com. 
> 
> 
> 
> If you reply to this email, your message will be added to the discussion 
> below:
> http://lucene.472066.n3.nabble.com/SolrCloud-with-large-synonym-files-tp3473568p4106246.html
> To unsubscribe from SolrCloud with large synonym files, click here.
> NAML




--
View this message in context: 
http://lucene.472066.n3.nabble.com/SolrCloud-with-large-synonym-files-tp3473568p4106268.html
Sent from the Solr - User mailing list archive at Nabble.com.

Solr Cloud error with shard update

[dboychuck]

I am running a data import and it is throwing all kinds of errors. I am
upgrading to 4.6 from 4.5.1 with the exact schema and solrconfig and dih
configs.

Here is the error I am getting:

org.apache.solr.common.SolrException: ERROR: [doc=k-690kohler!670614] Error
adding field 'weight'='java.math.BigDecimal:9.28' msg=For input string:
"java.math.BigDecimal:9.28"
at
org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:167)
at
org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:77)
at
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:215)
at
org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69)
at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:569)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:705)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:435)
at org.apache.solr.update.PeerSync.handleUpdates(PeerSync.java:482)
at org.apache.solr.update.PeerSync.handleResponse(PeerSync.java:330)
at org.apache.solr.update.PeerSync.sync(PeerSync.java:260)
at
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:387)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:244)
Caused by: java.lang.NumberFormatException: For input string:
"java.math.BigDecimal:9.28"
at 
sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1241)
at java.lang.Float.parseFloat(Float.java:452)
at org.apache.solr.schema.TrieField.createField(TrieField.java:593)
at org.apache.solr.schema.TrieField.createFields(TrieField.java:650)
at 
org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:47)
at
org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:118)
... 12 more

It looks like there is a problem with the readJavaFormat() function on a
tried field?



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-Cloud-error-with-shard-update-tp4106260.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrCloud and MoreLikeThis: SOLR-788

[gf80]

Hi guys,

could you kindly help me to apply patch for MoreLikeThis on solrcloud. 
I'm using Solr 4.6 and I'm using solrcloud with 10 shards. 
The problem is described here
https://issues.apache.org/jira/browse/SOLR-4414 
but I think that it was solved but not already delivered in Solr4.6.

Thanks a lot in advance,
Giuseppe

P.S. Rakudten: Did you figure out the problem applying patch? Tx



--
View this message in context: 
http://lucene.472066.n3.nabble.com/SolrCloud-and-MoreLikeThis-SOLR-788-tp4022581p4106258.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Error when creating collection in Solr 4.6

[Yago Riveiro]

Adding genericCoreNodeNames to solr.xml work for me.


-- 
Yago Riveiro
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)


On Wednesday, December 11, 2013 at 10:47 PM, dboychuck wrote:

> Where did you add that directive? I am having the same problem.
> 
> 
> 
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Error-when-creating-collection-in-Solr-4-6-tp4103536p4106248.html
> Sent from the Solr - User mailing list archive at Nabble.com 
> (http://Nabble.com).
> 
> 

Re: Edismax and prefixlen for fuzzy queries

[Furkan KAMACI]

Hi;

I've read the issue. When I have time I will apply a patch for it.

Thanks;
Furkan KAMACI


11 Aralık 2013 Çarşamba tarihinde Ahmet Arslan  adlı
kullanıcı şöyle yazdı:
> Somehow relevant jira : https://issues.apache.org/jira/browse/SOLR-218
>
>
>
> On Wednesday, December 11, 2013 2:15 AM, Furkan KAMACI <
furkankam...@gmail.com> wrote:
> Hi;
>
> I've debugged edismax of Solr 4.5.1 and I see that line:
>
> *case FUZZY: return super.getFuzzyQuery(field, val, flt);*
>
> it calls that one:
>
> * // called from parser*
> *  protected Query getFuzzyQuery(String field, String termStr, float
> minSimilarity) throws SyntaxError {*
> *termStr = analyzeIfMultitermTermText(field, termStr,
> schema.getFieldType(field));*
> *Term t = new Term(field, termStr);*
> *return newFuzzyQuery(t, minSimilarity, getFuzzyPrefixLength());*
> *  }*
>
> and getFuzzyPrefixLength() returns that:
>
> *int fuzzyPrefixLength = FuzzyQuery.defaultPrefixLength;*
>
> FuzzyQuery.defaultPrefixLength is 0 and *fuzzyPrefixLength *variable's
setter
> is not called anywhere else and that variable's value is not changed..
Here
> is a similar question:
> http://grokbase.com/t/lucene/solr-user/11a7a1bkg1/fuzzy-prefix-length that
> has no answer. If you can not find an answer I can fire a Jira and apply a
> patch for it.
>
> Thanks;
> Furkan KAMACI
>
>
>
> 2013/12/11 Mhd Wrk 
>
>> Does edismax query parser support prefixlen (length of common (non-fuzzy)
>> prefix)?
>>
>> Thanks
>>
>
>

Re: Replacing Deprecated CachedSqlEntityProcessor with SqlEntityProcessor with a cacheImpl parameter

[Furkan KAMACI]

Hi;

Good to hear that I could help you.

Thanks;
Furkan KAMACI


11 Aralık 2013 Çarşamba tarihinde O. Olson  adlı
kullanıcı şöyle yazdı:
> Thank you very much Furkan for the detailed explanation. I simply set the
> processor to SqlEntityProcessor and cacheImpl to SortedMapBackedCache, and
> it worked for me i.e. from my original post I had/did:
>
>  query="SELECT DocID, Title FROM solr.DOCS_TABLE">
> 
> 
>  query="SELECT CategoryName, DocID FROM
solr.CAT_DOCS_MAP
> WHERE CategoryLevel=1"
> cacheKey="DocID" cacheLookup="Doc.DocID"
> processor="SqlEntityProcessor" cacheImpl="SortedMapBackedCache">
> 
> 
> 
>
> Thanks again,
> O. O.
>
>
>
>
> --
> View this message in context:
http://lucene.472066.n3.nabble.com/Replacing-Deprecated-CachedSqlEntityProcessor-with-SqlEntityProcessor-with-a-cacheImpl-parameter-tp4106018p4106227.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: Error when creating collection in Solr 4.6

[dboychuck]

Where did you add that directive? I am having the same problem.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Error-when-creating-collection-in-Solr-4-6-tp4103536p4106248.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrCloud with large synonym files

[Jack Krupansky]

Note that the synonym filter accepts a comma-separated list of synonym 
files, so you can split your huge synonym file into two or more smaller 
files.


-- Jack Krupansky

-Original Message- 
From: gf80

Sent: Wednesday, December 11, 2013 5:21 PM
To: solr-user@lucene.apache.org
Subject: RE: SolrCloud with large synonym files

Hi all,

I have the same exception

org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode =
ConnectionLoss for /configs/myconf/synonyms.txt

synonyms.txt is greater than 1,5Mb

I have tried to set jute.maxbuffer specifying -Djute.maxbuffer=1024M at the
solrcloud startup but I'm not able to figure out the problem.
Please let me know your suggestions/solutions.
Thanks a lot in advance,
Giuseppe



--
View this message in context: 
http://lucene.472066.n3.nabble.com/SolrCloud-with-large-synonym-files-tp3473568p4106239.html
Sent from the Solr - User mailing list archive at Nabble.com. 

RE: SolrCloud with large synonym files

[gf80]

Hi all,

I have the same exception 

org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode =
ConnectionLoss for /configs/myconf/synonyms.txt

synonyms.txt is greater than 1,5Mb

I have tried to set jute.maxbuffer specifying -Djute.maxbuffer=1024M at the
solrcloud startup but I'm not able to figure out the problem.
Please let me know your suggestions/solutions.
Thanks a lot in advance,
Giuseppe



--
View this message in context: 
http://lucene.472066.n3.nabble.com/SolrCloud-with-large-synonym-files-tp3473568p4106239.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Configurable collectors for custom ranking

[Peter Keegan]

Thanks very much for the guidance. I'd be happy to donate a working
solution.

Peter


On Wed, Dec 11, 2013 at 3:53 PM, Joel Bernstein  wrote:

> SOLR-5020 has the commit info, it's mainly changes to SolrIndexSearcher I
> believe. They might apply to 4.3.
> I think as long you have the finish method that's all you'll need. If you
> can get this working it would be excellent if you could donate back the
> Scale PostFilter.
>
>
> On Wed, Dec 11, 2013 at 3:36 PM, Peter Keegan  >wrote:
>
> > This is what I was looking for, but the DelegatingCollector 'finish'
> method
> > doesn't exist in 4.3.0 :(   Can this be patched in and are there any
> other
> > PostFilter dependencies on 4.5?
> >
> > Thanks,
> > Peter
> >
> >
> > On Wed, Dec 11, 2013 at 3:16 PM, Joel Bernstein 
> > wrote:
> >
> > > Here is one approach to use in a postfilter
> > >
> > > 1) In the collect() method call score for each doc. Use the scores to
> > > create your scaleInfo.
> > > 2) Keep a bitset of the hits and a priorityQueue of your top X
> ScoreDocs.
> > > 3) Don't delegate any documents to lower collectors in the collect()
> > > method.
> > > 4) In the finish method create a score mapping (use the hppc
> > > IntFloatOpenHashMap) with your top X docIds pointing to their score,
> > using
> > > the priorityQueue created in step 2. Then iterate the bitset (also
> > created
> > > in step 2) sending down each doc to the lower collectors, retrieving
> and
> > > scaling the score from the score map. If the document is not in the
> score
> > > map then send down 0.
> > >
> > > You'll have setup a dummy scorer to feed to lower collectors. The
> > > CollapsingQParserPlugin has an example of how to do this.
> > >
> > >
> > >
> > >
> > > On Wed, Dec 11, 2013 at 2:05 PM, Peter Keegan  > > >wrote:
> > >
> > > > Hi Joel,
> > > >
> > > > I thought about using a PostFilter, but the problem is that the
> 'scale'
> > > > function must be done after all matching docs have been scored but
> > before
> > > > adding them to the PriorityQueue that sorts just the rows to be
> > returned.
> > > > Doing the 'scale' function wrapped in a 'query' is proving to be too
> > slow
> > > > when it visits every document in the index.
> > > >
> > > > In the Collector, I can see how to get the field values like this:
> > > >
> > > >
> > >
> >
> indexSearcher.getSchema().getField("field(myfield").getType().getValueSource(SchemaField,
> > > > QParser).getValues()
> > > >
> > > > But, 'getValueSource' needs a QParser, which isn't available.
> > > > And I can't create a QParser without a SolrQueryRequest, which isn't
> > > > available.
> > > >
> > > > Thanks,
> > > > Peter
> > > >
> > > >
> > > > On Wed, Dec 11, 2013 at 1:48 PM, Joel Bernstein 
> > > > wrote:
> > > >
> > > > > Peter,
> > > > >
> > > > > It sounds like you could achieve what you want to do in a
> PostFilter
> > > > rather
> > > > > then extending the TopDocsCollector. Is there a reason why a
> > PostFilter
> > > > > won't work for you?
> > > > >
> > > > > Joel
> > > > >
> > > > >
> > > > > On Tue, Dec 10, 2013 at 3:24 PM, Peter Keegan <
> > peterlkee...@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > Quick question:
> > > > > > In the context of a custom collector, how does one get the values
> > of
> > > a
> > > > > > field of type 'ExternalFileField'?
> > > > > >
> > > > > > Thanks,
> > > > > > Peter
> > > > > >
> > > > > >
> > > > > > On Tue, Dec 10, 2013 at 1:18 PM, Peter Keegan <
> > > peterlkee...@gmail.com
> > > > > > >wrote:
> > > > > >
> > > > > > > Hi Joel,
> > > > > > >
> > > > > > > This is related to another thread on function query matching (
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://lucene.472066.n3.nabble.com/Function-query-matching-td4099807.html#a4105513
> > > > > > ).
> > > > > > > The patch in SOLR-4465 will allow me to extend TopDocsCollector
> > and
> > > > > > perform
> > > > > > > the 'scale' function on only the documents matching the main
> > dismax
> > > > > > query.
> > > > > > > As you mention, it is a slightly intrusive design and requires
> > > that I
> > > > > > > manage my own PriorityQueue (and a local duplicate of
> HitQueue),
> > > but
> > > > > > should
> > > > > > > work. I think a better design would hide the PQ from the
> plugin.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Peter
> > > > > > >
> > > > > > >
> > > > > > > On Sun, Dec 8, 2013 at 5:32 PM, Joel Bernstein <
> > joels...@gmail.com
> > > >
> > > > > > wrote:
> > > > > > >
> > > > > > >> Hi Peter,
> > > > > > >>
> > > > > > >> I've been meaning to revisit configurable ranking collectors,
> > but
> > > I
> > > > > > >> haven't
> > > > > > >> yet had a chance. It's on the shortlist of things I'd like to
> > > tackle
> > > > > > >> though.
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> On Fri, Dec 6, 2013 at 4:17 PM, Peter Keegan <
> > > > peterlkee...@gmail.com>
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >> > I looked at SOLR-4465 and SOLR-5045, where it appears that
> > t

Re: Configurable collectors for custom ranking

[Joel Bernstein]

SOLR-5020 has the commit info, it's mainly changes to SolrIndexSearcher I
believe. They might apply to 4.3.
I think as long you have the finish method that's all you'll need. If you
can get this working it would be excellent if you could donate back the
Scale PostFilter.


On Wed, Dec 11, 2013 at 3:36 PM, Peter Keegan wrote:

> This is what I was looking for, but the DelegatingCollector 'finish' method
> doesn't exist in 4.3.0 :(   Can this be patched in and are there any other
> PostFilter dependencies on 4.5?
>
> Thanks,
> Peter
>
>
> On Wed, Dec 11, 2013 at 3:16 PM, Joel Bernstein 
> wrote:
>
> > Here is one approach to use in a postfilter
> >
> > 1) In the collect() method call score for each doc. Use the scores to
> > create your scaleInfo.
> > 2) Keep a bitset of the hits and a priorityQueue of your top X ScoreDocs.
> > 3) Don't delegate any documents to lower collectors in the collect()
> > method.
> > 4) In the finish method create a score mapping (use the hppc
> > IntFloatOpenHashMap) with your top X docIds pointing to their score,
> using
> > the priorityQueue created in step 2. Then iterate the bitset (also
> created
> > in step 2) sending down each doc to the lower collectors, retrieving and
> > scaling the score from the score map. If the document is not in the score
> > map then send down 0.
> >
> > You'll have setup a dummy scorer to feed to lower collectors. The
> > CollapsingQParserPlugin has an example of how to do this.
> >
> >
> >
> >
> > On Wed, Dec 11, 2013 at 2:05 PM, Peter Keegan  > >wrote:
> >
> > > Hi Joel,
> > >
> > > I thought about using a PostFilter, but the problem is that the 'scale'
> > > function must be done after all matching docs have been scored but
> before
> > > adding them to the PriorityQueue that sorts just the rows to be
> returned.
> > > Doing the 'scale' function wrapped in a 'query' is proving to be too
> slow
> > > when it visits every document in the index.
> > >
> > > In the Collector, I can see how to get the field values like this:
> > >
> > >
> >
> indexSearcher.getSchema().getField("field(myfield").getType().getValueSource(SchemaField,
> > > QParser).getValues()
> > >
> > > But, 'getValueSource' needs a QParser, which isn't available.
> > > And I can't create a QParser without a SolrQueryRequest, which isn't
> > > available.
> > >
> > > Thanks,
> > > Peter
> > >
> > >
> > > On Wed, Dec 11, 2013 at 1:48 PM, Joel Bernstein 
> > > wrote:
> > >
> > > > Peter,
> > > >
> > > > It sounds like you could achieve what you want to do in a PostFilter
> > > rather
> > > > then extending the TopDocsCollector. Is there a reason why a
> PostFilter
> > > > won't work for you?
> > > >
> > > > Joel
> > > >
> > > >
> > > > On Tue, Dec 10, 2013 at 3:24 PM, Peter Keegan <
> peterlkee...@gmail.com
> > > > >wrote:
> > > >
> > > > > Quick question:
> > > > > In the context of a custom collector, how does one get the values
> of
> > a
> > > > > field of type 'ExternalFileField'?
> > > > >
> > > > > Thanks,
> > > > > Peter
> > > > >
> > > > >
> > > > > On Tue, Dec 10, 2013 at 1:18 PM, Peter Keegan <
> > peterlkee...@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > Hi Joel,
> > > > > >
> > > > > > This is related to another thread on function query matching (
> > > > > >
> > > > >
> > > >
> > >
> >
> http://lucene.472066.n3.nabble.com/Function-query-matching-td4099807.html#a4105513
> > > > > ).
> > > > > > The patch in SOLR-4465 will allow me to extend TopDocsCollector
> and
> > > > > perform
> > > > > > the 'scale' function on only the documents matching the main
> dismax
> > > > > query.
> > > > > > As you mention, it is a slightly intrusive design and requires
> > that I
> > > > > > manage my own PriorityQueue (and a local duplicate of HitQueue),
> > but
> > > > > should
> > > > > > work. I think a better design would hide the PQ from the plugin.
> > > > > >
> > > > > > Thanks,
> > > > > > Peter
> > > > > >
> > > > > >
> > > > > > On Sun, Dec 8, 2013 at 5:32 PM, Joel Bernstein <
> joels...@gmail.com
> > >
> > > > > wrote:
> > > > > >
> > > > > >> Hi Peter,
> > > > > >>
> > > > > >> I've been meaning to revisit configurable ranking collectors,
> but
> > I
> > > > > >> haven't
> > > > > >> yet had a chance. It's on the shortlist of things I'd like to
> > tackle
> > > > > >> though.
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> On Fri, Dec 6, 2013 at 4:17 PM, Peter Keegan <
> > > peterlkee...@gmail.com>
> > > > > >> wrote:
> > > > > >>
> > > > > >> > I looked at SOLR-4465 and SOLR-5045, where it appears that
> there
> > > is
> > > > a
> > > > > >> goal
> > > > > >> > to be able to do custom sorting and ranking in a PostFilter.
> So
> > > far,
> > > > > it
> > > > > >> > looks like only custom aggregation can be implemented in
> > > PostFilter
> > > > > >> (5045).
> > > > > >> > Custom sorting/ranking can be done in a pluggable collector
> > > (4465),
> > > > > but
> > > > > >> > this patch is no longer in dev.
> > > > > >> >
> > > > > >> > Is there any other dev. being done

Re: Change Velocity Template Directory in Solr 4.6

[O. Olson]

Thank you iorixxx. Yes, when I run: 

> java -Dsolr.allow.unsafe.resourceloading=true -jar start.jar

And I then load the root of my site, I get: 

ERROR - 2013-12-11 14:36:03.434; org.apache.solr.common.SolrException;
null:java.io.IOException: Unable to find resource 'browse.vm'
at
org.apache.solr.response.VelocityResponseWriter.getTemplate(VelocityResponseWriter.java:174)
at
org.apache.solr.response.VelocityResponseWriter.write(VelocityResponseWriter.java:50)




In the above case, in the solrconfig.xml I have set: 

MyVMTemplates 

And my velocity templates are in /corename/conf/MyVMTemplates . If you look
at the VelocityResponseWriter at
http://svn.apache.org/viewvc/lucene/dev/branches/lucene_solr_4_6/solr/contrib/velocity/src/java/org/apache/solr/response/VelocityResponseWriter.java?revision=1541081&view=markup
 
nowhere does it use v.base_dir. So it seems that you need to name the
velocity template directory as "velocity". (I tried to set it to
"/corename/conf/velocity" and it works without any errors.) 

Thank you,
O. O.




--
View this message in context: 
http://lucene.472066.n3.nabble.com/Change-Velocity-Template-Directory-in-Solr-4-6-tp4105381p4106232.html
Sent from the Solr - User mailing list archive at Nabble.com.

Solr Profiler

[Monica Skidmore]

We're trying to improve the speed of some custom Solr code we've written, and 
we'd like to use a profiler to help us focus our efforts.  However, we've tried 
both JProfiler and NewRelic, and we've found it challenging to configure them 
correctly to be able to tell where our bottlenecks really are.

What profilers/configurations have people successfully used for Solr?

Monica Skidmore
Engineering Lead, Core Search
CareerBuilder.com

Re: Configurable collectors for custom ranking

[Peter Keegan]

This is what I was looking for, but the DelegatingCollector 'finish' method
doesn't exist in 4.3.0 :(   Can this be patched in and are there any other
PostFilter dependencies on 4.5?

Thanks,
Peter


On Wed, Dec 11, 2013 at 3:16 PM, Joel Bernstein  wrote:

> Here is one approach to use in a postfilter
>
> 1) In the collect() method call score for each doc. Use the scores to
> create your scaleInfo.
> 2) Keep a bitset of the hits and a priorityQueue of your top X ScoreDocs.
> 3) Don't delegate any documents to lower collectors in the collect()
> method.
> 4) In the finish method create a score mapping (use the hppc
> IntFloatOpenHashMap) with your top X docIds pointing to their score, using
> the priorityQueue created in step 2. Then iterate the bitset (also created
> in step 2) sending down each doc to the lower collectors, retrieving and
> scaling the score from the score map. If the document is not in the score
> map then send down 0.
>
> You'll have setup a dummy scorer to feed to lower collectors. The
> CollapsingQParserPlugin has an example of how to do this.
>
>
>
>
> On Wed, Dec 11, 2013 at 2:05 PM, Peter Keegan  >wrote:
>
> > Hi Joel,
> >
> > I thought about using a PostFilter, but the problem is that the 'scale'
> > function must be done after all matching docs have been scored but before
> > adding them to the PriorityQueue that sorts just the rows to be returned.
> > Doing the 'scale' function wrapped in a 'query' is proving to be too slow
> > when it visits every document in the index.
> >
> > In the Collector, I can see how to get the field values like this:
> >
> >
> indexSearcher.getSchema().getField("field(myfield").getType().getValueSource(SchemaField,
> > QParser).getValues()
> >
> > But, 'getValueSource' needs a QParser, which isn't available.
> > And I can't create a QParser without a SolrQueryRequest, which isn't
> > available.
> >
> > Thanks,
> > Peter
> >
> >
> > On Wed, Dec 11, 2013 at 1:48 PM, Joel Bernstein 
> > wrote:
> >
> > > Peter,
> > >
> > > It sounds like you could achieve what you want to do in a PostFilter
> > rather
> > > then extending the TopDocsCollector. Is there a reason why a PostFilter
> > > won't work for you?
> > >
> > > Joel
> > >
> > >
> > > On Tue, Dec 10, 2013 at 3:24 PM, Peter Keegan  > > >wrote:
> > >
> > > > Quick question:
> > > > In the context of a custom collector, how does one get the values of
> a
> > > > field of type 'ExternalFileField'?
> > > >
> > > > Thanks,
> > > > Peter
> > > >
> > > >
> > > > On Tue, Dec 10, 2013 at 1:18 PM, Peter Keegan <
> peterlkee...@gmail.com
> > > > >wrote:
> > > >
> > > > > Hi Joel,
> > > > >
> > > > > This is related to another thread on function query matching (
> > > > >
> > > >
> > >
> >
> http://lucene.472066.n3.nabble.com/Function-query-matching-td4099807.html#a4105513
> > > > ).
> > > > > The patch in SOLR-4465 will allow me to extend TopDocsCollector and
> > > > perform
> > > > > the 'scale' function on only the documents matching the main dismax
> > > > query.
> > > > > As you mention, it is a slightly intrusive design and requires
> that I
> > > > > manage my own PriorityQueue (and a local duplicate of HitQueue),
> but
> > > > should
> > > > > work. I think a better design would hide the PQ from the plugin.
> > > > >
> > > > > Thanks,
> > > > > Peter
> > > > >
> > > > >
> > > > > On Sun, Dec 8, 2013 at 5:32 PM, Joel Bernstein  >
> > > > wrote:
> > > > >
> > > > >> Hi Peter,
> > > > >>
> > > > >> I've been meaning to revisit configurable ranking collectors, but
> I
> > > > >> haven't
> > > > >> yet had a chance. It's on the shortlist of things I'd like to
> tackle
> > > > >> though.
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Fri, Dec 6, 2013 at 4:17 PM, Peter Keegan <
> > peterlkee...@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >> > I looked at SOLR-4465 and SOLR-5045, where it appears that there
> > is
> > > a
> > > > >> goal
> > > > >> > to be able to do custom sorting and ranking in a PostFilter. So
> > far,
> > > > it
> > > > >> > looks like only custom aggregation can be implemented in
> > PostFilter
> > > > >> (5045).
> > > > >> > Custom sorting/ranking can be done in a pluggable collector
> > (4465),
> > > > but
> > > > >> > this patch is no longer in dev.
> > > > >> >
> > > > >> > Is there any other dev. being done on adding custom sorting
> (after
> > > > >> > collection) via a plugin?
> > > > >> >
> > > > >> > Thanks,
> > > > >> > Peter
> > > > >> >
> > > > >>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> Joel Bernstein
> > > > >> Search Engineer at Heliosearch
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Joel Bernstein
> > > Search Engineer at Heliosearch
> > >
> >
>
>
>
> --
> Joel Bernstein
> Search Engineer at Heliosearch
>

Re: Replacing Deprecated CachedSqlEntityProcessor with SqlEntityProcessor with a cacheImpl parameter

[O. Olson]

Thank you very much Furkan for the detailed explanation. I simply set the
processor to SqlEntityProcessor and cacheImpl to SortedMapBackedCache, and
it worked for me i.e. from my original post I had/did: 

 
 
 
 
 
 


Thanks again,
O. O.




--
View this message in context: 
http://lucene.472066.n3.nabble.com/Replacing-Deprecated-CachedSqlEntityProcessor-with-SqlEntityProcessor-with-a-cacheImpl-parameter-tp4106018p4106227.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Configurable collectors for custom ranking

[Joel Bernstein]

Here is one approach to use in a postfilter

1) In the collect() method call score for each doc. Use the scores to
create your scaleInfo.
2) Keep a bitset of the hits and a priorityQueue of your top X ScoreDocs.
3) Don't delegate any documents to lower collectors in the collect() method.
4) In the finish method create a score mapping (use the hppc
IntFloatOpenHashMap) with your top X docIds pointing to their score, using
the priorityQueue created in step 2. Then iterate the bitset (also created
in step 2) sending down each doc to the lower collectors, retrieving and
scaling the score from the score map. If the document is not in the score
map then send down 0.

You'll have setup a dummy scorer to feed to lower collectors. The
CollapsingQParserPlugin has an example of how to do this.




On Wed, Dec 11, 2013 at 2:05 PM, Peter Keegan wrote:

> Hi Joel,
>
> I thought about using a PostFilter, but the problem is that the 'scale'
> function must be done after all matching docs have been scored but before
> adding them to the PriorityQueue that sorts just the rows to be returned.
> Doing the 'scale' function wrapped in a 'query' is proving to be too slow
> when it visits every document in the index.
>
> In the Collector, I can see how to get the field values like this:
>
> indexSearcher.getSchema().getField("field(myfield").getType().getValueSource(SchemaField,
> QParser).getValues()
>
> But, 'getValueSource' needs a QParser, which isn't available.
> And I can't create a QParser without a SolrQueryRequest, which isn't
> available.
>
> Thanks,
> Peter
>
>
> On Wed, Dec 11, 2013 at 1:48 PM, Joel Bernstein 
> wrote:
>
> > Peter,
> >
> > It sounds like you could achieve what you want to do in a PostFilter
> rather
> > then extending the TopDocsCollector. Is there a reason why a PostFilter
> > won't work for you?
> >
> > Joel
> >
> >
> > On Tue, Dec 10, 2013 at 3:24 PM, Peter Keegan  > >wrote:
> >
> > > Quick question:
> > > In the context of a custom collector, how does one get the values of a
> > > field of type 'ExternalFileField'?
> > >
> > > Thanks,
> > > Peter
> > >
> > >
> > > On Tue, Dec 10, 2013 at 1:18 PM, Peter Keegan  > > >wrote:
> > >
> > > > Hi Joel,
> > > >
> > > > This is related to another thread on function query matching (
> > > >
> > >
> >
> http://lucene.472066.n3.nabble.com/Function-query-matching-td4099807.html#a4105513
> > > ).
> > > > The patch in SOLR-4465 will allow me to extend TopDocsCollector and
> > > perform
> > > > the 'scale' function on only the documents matching the main dismax
> > > query.
> > > > As you mention, it is a slightly intrusive design and requires that I
> > > > manage my own PriorityQueue (and a local duplicate of HitQueue), but
> > > should
> > > > work. I think a better design would hide the PQ from the plugin.
> > > >
> > > > Thanks,
> > > > Peter
> > > >
> > > >
> > > > On Sun, Dec 8, 2013 at 5:32 PM, Joel Bernstein 
> > > wrote:
> > > >
> > > >> Hi Peter,
> > > >>
> > > >> I've been meaning to revisit configurable ranking collectors, but I
> > > >> haven't
> > > >> yet had a chance. It's on the shortlist of things I'd like to tackle
> > > >> though.
> > > >>
> > > >>
> > > >>
> > > >> On Fri, Dec 6, 2013 at 4:17 PM, Peter Keegan <
> peterlkee...@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > I looked at SOLR-4465 and SOLR-5045, where it appears that there
> is
> > a
> > > >> goal
> > > >> > to be able to do custom sorting and ranking in a PostFilter. So
> far,
> > > it
> > > >> > looks like only custom aggregation can be implemented in
> PostFilter
> > > >> (5045).
> > > >> > Custom sorting/ranking can be done in a pluggable collector
> (4465),
> > > but
> > > >> > this patch is no longer in dev.
> > > >> >
> > > >> > Is there any other dev. being done on adding custom sorting (after
> > > >> > collection) via a plugin?
> > > >> >
> > > >> > Thanks,
> > > >> > Peter
> > > >> >
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Joel Bernstein
> > > >> Search Engineer at Heliosearch
> > > >>
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > Joel Bernstein
> > Search Engineer at Heliosearch
> >
>



-- 
Joel Bernstein
Search Engineer at Heliosearch

Re: Configurable collectors for custom ranking

[Peter Keegan]

>From the Collector context, I suppose I can access the FileFloatSource
directly like this, although it's not generic:

SchemaField field = indexSearcher.getSchema().getField(fieldName);
dataDir = indexSearcher.getSchema().getResourceLoader().getDataDir();
ExternalFileField eff = (ExternalFileField)field.getType();
fieldValues = eff.getFileFloatSource(field, dataDir);

And then read the values in 'setNextReader'

Peter


On Wed, Dec 11, 2013 at 2:05 PM, Peter Keegan wrote:

> Hi Joel,
>
> I thought about using a PostFilter, but the problem is that the 'scale'
> function must be done after all matching docs have been scored but before
> adding them to the PriorityQueue that sorts just the rows to be returned.
> Doing the 'scale' function wrapped in a 'query' is proving to be too slow
> when it visits every document in the index.
>
> In the Collector, I can see how to get the field values like this:
> indexSearcher.getSchema().getField("field(myfield").getType().getValueSource(SchemaField,
> QParser).getValues()
>
> But, 'getValueSource' needs a QParser, which isn't available.
> And I can't create a QParser without a SolrQueryRequest, which isn't
> available.
>
> Thanks,
> Peter
>
>
> On Wed, Dec 11, 2013 at 1:48 PM, Joel Bernstein wrote:
>
>> Peter,
>>
>> It sounds like you could achieve what you want to do in a PostFilter
>> rather
>> then extending the TopDocsCollector. Is there a reason why a PostFilter
>> won't work for you?
>>
>> Joel
>>
>>
>> On Tue, Dec 10, 2013 at 3:24 PM, Peter Keegan > >wrote:
>>
>> > Quick question:
>> > In the context of a custom collector, how does one get the values of a
>> > field of type 'ExternalFileField'?
>> >
>> > Thanks,
>> > Peter
>> >
>> >
>> > On Tue, Dec 10, 2013 at 1:18 PM, Peter Keegan > > >wrote:
>> >
>> > > Hi Joel,
>> > >
>> > > This is related to another thread on function query matching (
>> > >
>> >
>> http://lucene.472066.n3.nabble.com/Function-query-matching-td4099807.html#a4105513
>> > ).
>> > > The patch in SOLR-4465 will allow me to extend TopDocsCollector and
>> > perform
>> > > the 'scale' function on only the documents matching the main dismax
>> > query.
>> > > As you mention, it is a slightly intrusive design and requires that I
>> > > manage my own PriorityQueue (and a local duplicate of HitQueue), but
>> > should
>> > > work. I think a better design would hide the PQ from the plugin.
>> > >
>> > > Thanks,
>> > > Peter
>> > >
>> > >
>> > > On Sun, Dec 8, 2013 at 5:32 PM, Joel Bernstein 
>> > wrote:
>> > >
>> > >> Hi Peter,
>> > >>
>> > >> I've been meaning to revisit configurable ranking collectors, but I
>> > >> haven't
>> > >> yet had a chance. It's on the shortlist of things I'd like to tackle
>> > >> though.
>> > >>
>> > >>
>> > >>
>> > >> On Fri, Dec 6, 2013 at 4:17 PM, Peter Keegan > >
>> > >> wrote:
>> > >>
>> > >> > I looked at SOLR-4465 and SOLR-5045, where it appears that there
>> is a
>> > >> goal
>> > >> > to be able to do custom sorting and ranking in a PostFilter. So
>> far,
>> > it
>> > >> > looks like only custom aggregation can be implemented in PostFilter
>> > >> (5045).
>> > >> > Custom sorting/ranking can be done in a pluggable collector (4465),
>> > but
>> > >> > this patch is no longer in dev.
>> > >> >
>> > >> > Is there any other dev. being done on adding custom sorting (after
>> > >> > collection) via a plugin?
>> > >> >
>> > >> > Thanks,
>> > >> > Peter
>> > >> >
>> > >>
>> > >>
>> > >>
>> > >> --
>> > >> Joel Bernstein
>> > >> Search Engineer at Heliosearch
>> > >>
>> > >
>> > >
>> >
>>
>>
>>
>> --
>> Joel Bernstein
>> Search Engineer at Heliosearch
>>
>
>

Re: Getting Solr Document Attributes from a Custom Function

[Kydryavtsev Andrey]

As I know (not 100% sure actually), function queries don't work with 
multivalued fields. Why do you need multivalued fields here? Your "price" and 
"numberOfCities" don't look like multivalued. At least you can try to use, you 
know, some tricky format like 
"50;40;20" to index multivalued field as single-valued and then parse this into 
values list in function.

11.12.2013, 11:13, "Mukundaraman valakumaresan" :
> Hi Kydryavtsev
>
> Thanks a lot it works,  but how do i pass a multivalued field values to a
> function query?
>
> Can it be passed as a String array?
>
> Thanks & Regards
> Mukund
>
> On Tue, Dec 10, 2013 at 12:05 PM, Kydryavtsev Andrey 
> wrote:
>
>>  You can implement it in this way:
>>  Index number of cities as new int field (like >  name="numberOfCities">2) and implement user function like
>>
>>  "customFunction(price, numberOfCities, 1, 2000, 5)"
>>
>>  Custom parser should parse this into value sources list. From first two
>>  field sources we can get per doc value for this particular fields, another
>>  three will be ConstValueSource instances - just constants, so we can access
>>  all 5 values and implement custom formula per doc id. Find examples in
>>  ValueSourceParser and solr functions like DefFunction or MinFloatFunction
>>
>>  10.12.2013, 09:31, "Mukundaraman valakumaresan" :
>>>  Hi Hoss,
>>>
>>>  Thanks a lot for your response. The actual problem is,
>>>
>>>  For every record that I query, I have to execute a formula and sort the
>>>  records based on the value of the formula.
>>>  The formula has elements from the record.
>>>
>>>  For eg. for the following document ,I need to apply the formula
>>  (maxprice -
>>>  solrprice)/ (maxprice - minprice)  +  count(cities)/totalcities.
>>>  where maxprice, maxprice and total cities will be available at run time.
>>>
>>>  So for the following record, it has to execute as  (1 -
>>>  *5000*)/(1-2000)
>>>  + *2*/5   (where 5000 and 2, which are in bold are from the document)
>>>
>>>  
>>>  apartment_1
>>>  Casa Grande
>>>  chennai
>>>  bangalore
>>>  5000
>>>  
>>>
>>>  Thanks & Regards
>>>  Mukund
>>>
>>>  On Tue, Dec 10, 2013 at 12:22 AM, Chris Hostetter
>>>  wrote:
   Smells like an XY problem ...

   Can you please describe what your end goal is in writing a custom
   function, and what you would do with things like the "name" field
>>  inside
   your funciton?

   In general, accessing stored field values for indexed documents ca be
   prohibitively expensive, it rather defeats the entire point of the
   inverted index data structure.  If you help us understand what your
>>  goal
   is, people may be able to offer performant suggestions.

   https://people.apache.org/~hossman/#xyproblem
   XY Problem

   Your question appears to be an "XY Problem" ... that is: you are
>>  dealing
   with "X", you are assuming "Y" will help you, and you are asking about
>>  "Y"
   without giving more details about the "X" so that we can understand the
   full issue.  Perhaps the best solution doesn't involve "Y" at all?
   See Also: http://www.perlmonks.org/index.pl?node_id=542341

   : Date: Mon, 9 Dec 2013 20:24:15 +0530
   : From: Mukundaraman valakumaresan 
   : Reply-To: solr-user@lucene.apache.org
   : To: solr-user@lucene.apache.org
   : Subject: Getting Solr Document Attributes from a Custom Function
   :
   : Hi All,
   :
   : I have a written a custom solr function and I would like to read a
   property
   : of the document inside my custom function. Is it possible to get that
   using
   : Solr?
   :
   : For eg. inside the floatVal method, I would like to get the value of
>>  the
   : attribute "name"
   :
   : public class CustomValueSource extends ValueSource {
   :
   : @Override
   : public FunctionValues getValues(Map context,
   : AtomicReaderContext readerContext) throws IOException {
   :  return new FloatDocValues(this) { @Override public float
>>  floatVal(int
   doc)
   : {
   : /***
   :  getDocument(doc).getAttribute("name")
   :
   : / }}}
   :
   : Thanks & Regards
   : Mukund
   :

   -Hoss
   http://www.lucidworks.com/

Re: Configurable collectors for custom ranking

[Peter Keegan]

Hi Joel,

I thought about using a PostFilter, but the problem is that the 'scale'
function must be done after all matching docs have been scored but before
adding them to the PriorityQueue that sorts just the rows to be returned.
Doing the 'scale' function wrapped in a 'query' is proving to be too slow
when it visits every document in the index.

In the Collector, I can see how to get the field values like this:
indexSearcher.getSchema().getField("field(myfield").getType().getValueSource(SchemaField,
QParser).getValues()

But, 'getValueSource' needs a QParser, which isn't available.
And I can't create a QParser without a SolrQueryRequest, which isn't
available.

Thanks,
Peter


On Wed, Dec 11, 2013 at 1:48 PM, Joel Bernstein  wrote:

> Peter,
>
> It sounds like you could achieve what you want to do in a PostFilter rather
> then extending the TopDocsCollector. Is there a reason why a PostFilter
> won't work for you?
>
> Joel
>
>
> On Tue, Dec 10, 2013 at 3:24 PM, Peter Keegan  >wrote:
>
> > Quick question:
> > In the context of a custom collector, how does one get the values of a
> > field of type 'ExternalFileField'?
> >
> > Thanks,
> > Peter
> >
> >
> > On Tue, Dec 10, 2013 at 1:18 PM, Peter Keegan  > >wrote:
> >
> > > Hi Joel,
> > >
> > > This is related to another thread on function query matching (
> > >
> >
> http://lucene.472066.n3.nabble.com/Function-query-matching-td4099807.html#a4105513
> > ).
> > > The patch in SOLR-4465 will allow me to extend TopDocsCollector and
> > perform
> > > the 'scale' function on only the documents matching the main dismax
> > query.
> > > As you mention, it is a slightly intrusive design and requires that I
> > > manage my own PriorityQueue (and a local duplicate of HitQueue), but
> > should
> > > work. I think a better design would hide the PQ from the plugin.
> > >
> > > Thanks,
> > > Peter
> > >
> > >
> > > On Sun, Dec 8, 2013 at 5:32 PM, Joel Bernstein 
> > wrote:
> > >
> > >> Hi Peter,
> > >>
> > >> I've been meaning to revisit configurable ranking collectors, but I
> > >> haven't
> > >> yet had a chance. It's on the shortlist of things I'd like to tackle
> > >> though.
> > >>
> > >>
> > >>
> > >> On Fri, Dec 6, 2013 at 4:17 PM, Peter Keegan 
> > >> wrote:
> > >>
> > >> > I looked at SOLR-4465 and SOLR-5045, where it appears that there is
> a
> > >> goal
> > >> > to be able to do custom sorting and ranking in a PostFilter. So far,
> > it
> > >> > looks like only custom aggregation can be implemented in PostFilter
> > >> (5045).
> > >> > Custom sorting/ranking can be done in a pluggable collector (4465),
> > but
> > >> > this patch is no longer in dev.
> > >> >
> > >> > Is there any other dev. being done on adding custom sorting (after
> > >> > collection) via a plugin?
> > >> >
> > >> > Thanks,
> > >> > Peter
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> Joel Bernstein
> > >> Search Engineer at Heliosearch
> > >>
> > >
> > >
> >
>
>
>
> --
> Joel Bernstein
> Search Engineer at Heliosearch
>

Re: Configurable collectors for custom ranking

[Joel Bernstein]

Peter,

It sounds like you could achieve what you want to do in a PostFilter rather
then extending the TopDocsCollector. Is there a reason why a PostFilter
won't work for you?

Joel


On Tue, Dec 10, 2013 at 3:24 PM, Peter Keegan wrote:

> Quick question:
> In the context of a custom collector, how does one get the values of a
> field of type 'ExternalFileField'?
>
> Thanks,
> Peter
>
>
> On Tue, Dec 10, 2013 at 1:18 PM, Peter Keegan  >wrote:
>
> > Hi Joel,
> >
> > This is related to another thread on function query matching (
> >
> http://lucene.472066.n3.nabble.com/Function-query-matching-td4099807.html#a4105513
> ).
> > The patch in SOLR-4465 will allow me to extend TopDocsCollector and
> perform
> > the 'scale' function on only the documents matching the main dismax
> query.
> > As you mention, it is a slightly intrusive design and requires that I
> > manage my own PriorityQueue (and a local duplicate of HitQueue), but
> should
> > work. I think a better design would hide the PQ from the plugin.
> >
> > Thanks,
> > Peter
> >
> >
> > On Sun, Dec 8, 2013 at 5:32 PM, Joel Bernstein 
> wrote:
> >
> >> Hi Peter,
> >>
> >> I've been meaning to revisit configurable ranking collectors, but I
> >> haven't
> >> yet had a chance. It's on the shortlist of things I'd like to tackle
> >> though.
> >>
> >>
> >>
> >> On Fri, Dec 6, 2013 at 4:17 PM, Peter Keegan 
> >> wrote:
> >>
> >> > I looked at SOLR-4465 and SOLR-5045, where it appears that there is a
> >> goal
> >> > to be able to do custom sorting and ranking in a PostFilter. So far,
> it
> >> > looks like only custom aggregation can be implemented in PostFilter
> >> (5045).
> >> > Custom sorting/ranking can be done in a pluggable collector (4465),
> but
> >> > this patch is no longer in dev.
> >> >
> >> > Is there any other dev. being done on adding custom sorting (after
> >> > collection) via a plugin?
> >> >
> >> > Thanks,
> >> > Peter
> >> >
> >>
> >>
> >>
> >> --
> >> Joel Bernstein
> >> Search Engineer at Heliosearch
> >>
> >
> >
>



-- 
Joel Bernstein
Search Engineer at Heliosearch

Ever increasing open searchers

[Manish Kumar]

Hi,

I am actually struggling with one of my custom search plugins is causing a
SolrCore to keep many open searchers. This is causing my slave boxes to run
out of disk space in couple of days.

Following is the problemm description :

My solr set-up comprises a master, a repeater and two slaves. Slaves poll
their respective masters every 10 secs.Searches are handled only by slave
nodes while updates are done only to the master node. These are deployed on
Jboss application servers on Linux boxes. The sorl version is 4.0.

The problem we are facing is that the slave boxes run out of hard-disk
space in couple of days which we are currently solving by restarting the
slave nodes once every week. The reason for running out of hard-disk space
is the Jboss holding reference to hundreds of deleted index files.

Deleted file reference are alive because I see SolrCore having many open
searchers. There should be maximum 2 open searchers at any time as per
solrconfig.xml. [image: enter image description here]

Another thing I have been able to establish that SolrCore holds the open
searcher only when a query has been running and a replication cycle changes
the registered searcher for the core while the last query hasn't finished
yet. The searcher when the query began is held in memory by the SolrCore
forever.

This problem comes only when the query uses a SearchPlugin that we have
written. In the search plugin we are closing the *SolrRequest* and
*SolrCore* object. But still it doesn't seem to close the associated
searcher.

SolrCore portfolioIndexCore = container.getCore(portfolioCoreName);
SolrIndexSearcher portfolioIndexSearcher =
portfolioIndexCore.getSearcher().get();

When search is complete :

finally {

  if (null != portfolioSolrReq) portfolioSolrReq.close();

  if (null != portfolioIndexCore) {
portfolioIndexCore.close();
  }
}

I tried changing the above finally block to :

finally {

  if (null != portfolioSolrReq) portfolioSolrReq.close();

  if (null != portfolioIndexCore) {
RefCounted searcher =
portfolioIndexCore.getNewestSearcher(false);
if (searcher.get() != portfolioIndexSearcher) {
  log.warn("Current Searcher for the Core " + portfolioIndexCore
  + " has changed. Old Searcher=[" +
portfolioIndexSearcher + "], new Searcher=["
  + searcher.get() + "]");
  portfolioIndexSearcher.close();
  portfolioIndexSearcher = null;
}
searcher.decref();
portfolioIndexCore.close();
  }
}

But this also doesn't seem to help.

-- 
Regards,
Manish

Re: /select with 'q' parameter does not work

[Nutan]

solrconfig is a xml file,it is not possible to run xml files,you open it in
browser and its code gets displayed.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/select-with-q-parameter-does-not-work-tp4106099p4106176.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Importing/Indexing the DB2 XML FieldType in SOLR

[Shawn Heisey]

On 12/11/2013 3:09 AM, ravi1984 wrote:
> Thanks much for the reply! We have this working through Java layer and
> facing the issues when integrated with SOLR layer. 
> 
> We tried to tweak the Query itself to return the String - following are the
> options we tried, and still there was no luck. Do we need to tweak anything
> in the Schema.xml, with regard to the field: cust_data? 
> 
> select XMLSERIALIZE(DTL_XML as CLOB(1M)) as DTL_XML from testSch.SUMMRY_DTL
> where DTL_XML IS NOT NULL FETCH FIRST 50 ROWS ONLY 
> 
> Select XMLCAST(DTL_XML AS CLOB(1M)) as DTL_XML from testSch.SUMMRY_DTL where
> DTL_XML IS NOT NULL FETCH FIRST 50 ROWS ONLY 

Upgrading the JDBC driver and talking to IBM are your best options to
try first.  In a newer driver version, IBM might have implemented a
toString() method on the XML object type.  I have no idea whether this
is the case or not -- I'm completely unfamiliar with DB2.

You could try returning the XML column as a CLOB and then using the
ClobTransformer.  Reading what I can find on XMLCAST, it seems as though
this probably is returning as CLOB, which Solr is unable to deal with
natively -- requiring the transformer.

http://wiki.apache.org/solr/DataImportHandler#ClobTransformer

If those don't yield something useful, you might need to write some
custom code.  That might be a straight modification of DIH to use the
IBM data types directly, or you might write a custom transformer or a
custom update handler.

Thanks,
Shawn

RE: Cloud graph gone after manually editing clusterstate.json

[michael.boom]

I had a look, but all looks fine there too:

[Wed Dec 11 2013 17:04:41 GMT+0100 (CET)] runRoute get #/~cloud
GET tpl/cloud.html?_=1386777881244
200 OK
57ms
GET /solr/zookeeper?wt=json&_=1386777881308
200 OK
509ms   
GET /solr/zookeeper?wt=json&path=%2Flive_nodes&_=1386777881822
200 OK
62ms
GET
/solr/zookeeper?wt=json&detail=true&path=%2Fclusterstate.json&_=1386777881886
200 OK
84ms




-
Thanks,
Michael
--
View this message in context: 
http://lucene.472066.n3.nabble.com/Cloud-graph-gone-after-manually-editing-clusterstate-json-tp4106142p4106172.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Java heap space:out of memory

[Michael Della Bitta]

Just for future reference, I think Tomcat uses CATALINA_OPTS for the server
runtime options.

Michael Della Bitta

Applications Developer

o: +1 646 532 3062  | c: +1 917 477 7906

appinions inc.

“The Science of Influence Marketing”

18 East 41st Street

New York, NY 10017

t: @appinions  | g+:
plus.google.com/appinions
w: appinions.com 


On Wed, Dec 11, 2013 at 2:14 AM, sweety  wrote:

> You were right the changes made in JAVA_OPTs didn't show increase in the
> heap
> size, I made changes in the UI of tomcat
> Initial pool memory : 512 MB
> Maximum pool memory : 1024 MB
>
> Now the heap size has increased.
> Thanks you all for your suggestions,it really saved my time.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Java-heap-space-out-of-memory-tp4105903p4106082.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: /select with 'q' parameter does not work

[Nutan]

No exception,it works normal.
other queries like :
http://localhost:8080/solr/document/select?q=author:dgfb
works.
Only for 'contents' field it does not work.
all other functionalities like analysis,indexing etc work fine.

But for spell check :
http://localhost:8080/solr/document/spell?q=programmer
it shows this output:
 

This is wrong because 'programmer' is the right word.It is in many docs,as
below
 
(this is output for q=*:*)






--
View this message in context: 
http://lucene.472066.n3.nabble.com/select-with-q-parameter-does-not-work-tp4106099p4106168.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Cloud graph gone after manually editing clusterstate.json

[Tim Potter]

I'm not sure at this point as what you're describing seems fine to me ... I'm 
not too familiar with Solr's UI implementation, but I suspect the cloud graph 
stuff may be client side, so are you seeing any JavaScript errors in the dev 
console in your browser?

Timothy Potter
Sr. Software Engineer, LucidWorks
www.lucidworks.com


From: michael.boom 
Sent: Wednesday, December 11, 2013 8:21 AM
To: solr-user@lucene.apache.org
Subject: RE: Cloud graph gone after manually editing clusterstate.json

Thanks for the reply Tim,

Yes, that was just a typo, i used "cat" not "cate".
As for the checks everything looks fine, my edits were:
1. updating the shard range
2. removed the header which looked log information, as below:
* removed header start here*
Connecting to solr3:9983
2013-12-11 16:15:05,372 [myid:] - INFO  [main:Environment@100] - Client
environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
2013-12-11 16:15:05,376 [myid:] - INFO  [main:Environment@100] - Client
environment:host.name=solr3.internal
2013-12-11 16:15:05,377 [myid:] - INFO  [main:Environment@100] - Client
environment:java.version=1.7.0_25
2013-12-11 16:15:05,377 [myid:] - INFO  [main:Environment@100] - Client
environment:java.vendor=Oracle Corporation
2013-12-11 16:15:05,378 [myid:] - INFO  [main:Environment@100] - Client
environment:java.home=/usr/lib/jvm/java-7-openjdk-amd64/jre
2013-12-11 16:15:05,378 [myid:] - INFO  [main:Environment@100] - Client
environment:java.class.path=/opt/zookeeper/bin/../build/classes:/opt/zookeeper/bin/../build/lib/*.jar:/opt/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/opt/z$
2013-12-11 16:15:05,378 [myid:] - INFO  [main:Environment@100] - Client
environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/jni:/lib:/usr/lib
2013-12-11 16:15:05,379 [myid:] - INFO  [main:Environment@100] - Client
environment:java.io.tmpdir=/tmp
2013-12-11 16:15:05,379 [myid:] - INFO  [main:Environment@100] - Client
environment:java.compiler=
2013-12-11 16:15:05,380 [myid:] - INFO  [main:Environment@100] - Client
environment:os.name=Linux
2013-12-11 16:15:05,380 [myid:] - INFO  [main:Environment@100] - Client
environment:os.arch=amd64
2013-12-11 16:15:05,381 [myid:] - INFO  [main:Environment@100] - Client
environment:os.version=3.2.0-4-amd64
2013-12-11 16:15:05,381 [myid:] - INFO  [main:Environment@100] - Client
environment:user.name=solr
2013-12-11 16:15:05,382 [myid:] - INFO  [main:Environment@100] - Client
environment:user.home=/home/solr
2013-12-11 16:15:05,382 [myid:] - INFO  [main:Environment@100] - Client
environment:user.dir=/opt/zookeeper
2013-12-11 16:15:05,384 [myid:] - INFO  [main:ZooKeeper@438] - Initiating
client connection, connectString=solr3:9983 sessionTimeout=3
watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@58a5f543
2013-12-11 16:15:05,412 [myid:] - INFO
[main-SendThread(solr3.productdb.internal:9983):ClientCnxn$SendThread@966] -
Opening socket connection to server solr3.internal/10.33.182.78:9983. Will
not attempt to authenticate $
2013-12-11 16:15:05,419 [myid:] - INFO
[main-SendThread(solr3.productdb.internal:9983):ClientCnxn$SendThread@849] -
Socket connection established to solr3.internal/10.33.182.78:9983,
initiating session
2013-12-11 16:15:05,427 [myid:] - INFO
[main-SendThread(solr3.productdb.internal:9983):ClientCnxn$SendThread@1207]
- Session establishment complete on server solr3.internal/10.33.182.78:9983,
sessionid = 0x142e187355000$

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
*<< i removed the above until here*
{
  "offers_collection_GB":{
"shards":{
  "shard1":{
"range":"8000-bfff",
"state":"active",
"replicas":{
.. and so on


Could this be the problem?



-
Thanks,
Michael
--
View this message in context: 
http://lucene.472066.n3.nabble.com/Cloud-graph-gone-after-manually-editing-clusterstate-json-tp4106142p4106161.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Cloud graph gone after manually editing clusterstate.json

[michael.boom]

Thanks for the reply Tim,

Yes, that was just a typo, i used "cat" not "cate".
As for the checks everything looks fine, my edits were:
1. updating the shard range
2. removed the header which looked log information, as below:
* removed header start here* 
Connecting to solr3:9983   
2013-12-11 16:15:05,372 [myid:] - INFO  [main:Environment@100] - Client
environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
2013-12-11 16:15:05,376 [myid:] - INFO  [main:Environment@100] - Client
environment:host.name=solr3.internal
2013-12-11 16:15:05,377 [myid:] - INFO  [main:Environment@100] - Client
environment:java.version=1.7.0_25
2013-12-11 16:15:05,377 [myid:] - INFO  [main:Environment@100] - Client
environment:java.vendor=Oracle Corporation
2013-12-11 16:15:05,378 [myid:] - INFO  [main:Environment@100] - Client
environment:java.home=/usr/lib/jvm/java-7-openjdk-amd64/jre
2013-12-11 16:15:05,378 [myid:] - INFO  [main:Environment@100] - Client
environment:java.class.path=/opt/zookeeper/bin/../build/classes:/opt/zookeeper/bin/../build/lib/*.jar:/opt/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/opt/z$
2013-12-11 16:15:05,378 [myid:] - INFO  [main:Environment@100] - Client
environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/jni:/lib:/usr/lib
2013-12-11 16:15:05,379 [myid:] - INFO  [main:Environment@100] - Client
environment:java.io.tmpdir=/tmp
2013-12-11 16:15:05,379 [myid:] - INFO  [main:Environment@100] - Client
environment:java.compiler=
2013-12-11 16:15:05,380 [myid:] - INFO  [main:Environment@100] - Client
environment:os.name=Linux
2013-12-11 16:15:05,380 [myid:] - INFO  [main:Environment@100] - Client
environment:os.arch=amd64
2013-12-11 16:15:05,381 [myid:] - INFO  [main:Environment@100] - Client
environment:os.version=3.2.0-4-amd64
2013-12-11 16:15:05,381 [myid:] - INFO  [main:Environment@100] - Client
environment:user.name=solr
2013-12-11 16:15:05,382 [myid:] - INFO  [main:Environment@100] - Client
environment:user.home=/home/solr
2013-12-11 16:15:05,382 [myid:] - INFO  [main:Environment@100] - Client
environment:user.dir=/opt/zookeeper
2013-12-11 16:15:05,384 [myid:] - INFO  [main:ZooKeeper@438] - Initiating
client connection, connectString=solr3:9983 sessionTimeout=3
watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@58a5f543
2013-12-11 16:15:05,412 [myid:] - INFO 
[main-SendThread(solr3.productdb.internal:9983):ClientCnxn$SendThread@966] -
Opening socket connection to server solr3.internal/10.33.182.78:9983. Will
not attempt to authenticate $
2013-12-11 16:15:05,419 [myid:] - INFO 
[main-SendThread(solr3.productdb.internal:9983):ClientCnxn$SendThread@849] -
Socket connection established to solr3.internal/10.33.182.78:9983,
initiating session
2013-12-11 16:15:05,427 [myid:] - INFO 
[main-SendThread(solr3.productdb.internal:9983):ClientCnxn$SendThread@1207]
- Session establishment complete on server solr3.internal/10.33.182.78:9983,
sessionid = 0x142e187355000$

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
*<< i removed the above until here*
{
  "offers_collection_GB":{
"shards":{   
  "shard1":{
"range":"8000-bfff",
"state":"active",
"replicas":{
.. and so on


Could this be the problem?



-
Thanks,
Michael
--
View this message in context: 
http://lucene.472066.n3.nabble.com/Cloud-graph-gone-after-manually-editing-clusterstate-json-tp4106142p4106161.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Cloud graph gone after manually editing clusterstate.json

[Tim Potter]

Hi Michael,

Can you /get clusterstate.json again to see the contents? Also, maybe just a 
typo but you have `cate clusterstate.json` vs. `cat ..`

Timothy Potter
Sr. Software Engineer, LucidWorks
www.lucidworks.com


From: michael.boom 
Sent: Wednesday, December 11, 2013 6:37 AM
To: solr-user@lucene.apache.org
Subject: Cloud graph gone after manually editing clusterstate.json

HI,

Today I changed my ZK config, removing one instance in the quorum and then
restarted both all ZKs and all Solr instances.
After this operation i noticed that one of the shards in one collection was
missing the range ("range":null). Router for that collection was
compositeId.

So, I proceeded adding the missing range manually by editing
clusterstate.json
$ zkCli.sh -server zk1:9983 get /clusterstate.json > clusterstate.json
i did my edits, and then:
$ zkCli.sh -server zk1:9983 set /clusterstate.json "`cate
clusterstate.json`"

Everything fine, I check in the Admin - the clusterstate.json was updated,
but now when i try to see the graph view or radial graph i can't see
anything. Just white space.

Any idea why?
Thanks!





-
Thanks,
Michael
--
View this message in context: 
http://lucene.472066.n3.nabble.com/Cloud-graph-gone-after-manually-editing-clusterstate-json-tp4106142.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Equivalent of SQL JOIN in SOLR across multiple cores

[Ahmet Arslan]

Hi,

There are some join support :

1) Query Time Join (works across cores)

https://cwiki.apache.org/confluence/display/solr/Other+Parsers#OtherParsers-JoinQueryParser

http://wiki.apache.org/solr/Join


2) Block Joins

https://cwiki.apache.org/confluence/display/solr/Other+Parsers#OtherParsers-BlockJoinQueryParsers


An open jira : https://issues.apache.org/jira/browse/SOLR-4787



On Wednesday, December 11, 2013 4:28 PM, bijalcm  wrote:

1, Are the cores join-able?
2. Could you give me an example about how to write a multiple core join? 

3. Can we do equivalent of JOIN in SOLR across multiple cores

    Select T1.*,T2.* 
    FROM Table1 T1,Table2 T2
    WHERE T1.id = T2.id





--
View this message in context: 
http://lucene.472066.n3.nabble.com/Equivalent-of-SQL-JOIN-in-SOLR-across-multiple-cores-tp4106152.html
Sent from the Solr - User mailing list archive at Nabble.com.

Equivalent of SQL JOIN in SOLR across multiple cores

[bijalcm]

1, Are the cores join-able?
2. Could you give me an example about how to write a multiple core join? 

3. Can we do equivalent of JOIN in SOLR across multiple cores

Select T1.*,T2.* 
FROM Table1 T1,Table2 T2
WHERE T1.id = T2.id





--
View this message in context: 
http://lucene.472066.n3.nabble.com/Equivalent-of-SQL-JOIN-in-SOLR-across-multiple-cores-tp4106152.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Edismax and prefixlen for fuzzy queries

[Ahmet Arslan]

Somehow relevant jira : https://issues.apache.org/jira/browse/SOLR-218



On Wednesday, December 11, 2013 2:15 AM, Furkan KAMACI  
wrote:
Hi;

I've debugged edismax of Solr 4.5.1 and I see that line:

*case FUZZY: return super.getFuzzyQuery(field, val, flt);*

it calls that one:

* // called from parser*
*  protected Query getFuzzyQuery(String field, String termStr, float
minSimilarity) throws SyntaxError {*
*    termStr = analyzeIfMultitermTermText(field, termStr,
schema.getFieldType(field));*
*    Term t = new Term(field, termStr);*
*    return newFuzzyQuery(t, minSimilarity, getFuzzyPrefixLength());*
*  }*

and getFuzzyPrefixLength() returns that:

*int fuzzyPrefixLength = FuzzyQuery.defaultPrefixLength;*

FuzzyQuery.defaultPrefixLength is 0 and *fuzzyPrefixLength *variable's setter
is not called anywhere else and that variable's value is not changed.. Here
is a similar question:
http://grokbase.com/t/lucene/solr-user/11a7a1bkg1/fuzzy-prefix-length that
has no answer. If you can not find an answer I can fire a Jira and apply a
patch for it.

Thanks;
Furkan KAMACI



2013/12/11 Mhd Wrk 

> Does edismax query parser support prefixlen (length of common (non-fuzzy)
> prefix)?
>
> Thanks
>

Cloud graph gone after manually editing clusterstate.json

[michael.boom]

HI,

Today I changed my ZK config, removing one instance in the quorum and then
restarted both all ZKs and all Solr instances.
After this operation i noticed that one of the shards in one collection was
missing the range ("range":null). Router for that collection was
compositeId.

So, I proceeded adding the missing range manually by editing
clusterstate.json
$ zkCli.sh -server zk1:9983 get /clusterstate.json > clusterstate.json
i did my edits, and then:
$ zkCli.sh -server zk1:9983 set /clusterstate.json "`cate
clusterstate.json`"

Everything fine, I check in the Admin - the clusterstate.json was updated,
but now when i try to see the graph view or radial graph i can't see
anything. Just white space.

Any idea why?
Thanks!





-
Thanks,
Michael
--
View this message in context: 
http://lucene.472066.n3.nabble.com/Cloud-graph-gone-after-manually-editing-clusterstate-json-tp4106142.html
Sent from the Solr - User mailing list archive at Nabble.com.

Updating shard range in Zookeeper

[michael.boom]

Hi, 

Somehow my Zookeeper clusterstate has gotten messed up and after a restart
of both Zookeeper instances and my Solr instances in one of my collections ,
for one shard, the "range" is now null.
Everything else it's fine, but I can't index documents now because i get an
error : No active slice servicing hash code 2c7ade4d in DocCollection.

The router of my collection is compositIed.

If i look to the other collections ranges i can guess that the missing range
should be "0-3fff".
Any idea how can i update it ? (tools, procedures)

Thanks!





-
Thanks,
Michael
--
View this message in context: 
http://lucene.472066.n3.nabble.com/Updating-shard-range-in-Zookeeper-tp4106138.html
Sent from the Solr - User mailing list archive at Nabble.com.

Fwd: FuzzyLookupFactory fwfsta.bin

[Harun Reşit Zafer]

With the configration below:


   
  suggest
  name="classname">org.apache.solr.spelling.suggest.Suggester
  name="lookupImpl">org.apache.solr.spelling.suggest.fst.FuzzyLookupFactory

  fuzzy_suggest_analyzing
  true

  text_tr

  suggestions.txt

  
  true
  false

   

   lowercase


*I got th**e error:*

...\solr-4.6.0\example\solr\collection1\data\fuzzy_suggest_analyzing\fwfsta.bin 
(The system cannot find the file specified)



--
Harun Reşit Zafer
TÜBİTAK BİLGEM BTE
Metin Madenciliği ve Kaynaştırma Sistemleri Bölümü
T +90 262 675 3268
Whttp://www.hrzafer.com

Re: Constantly increasing time of full data import

[michallos]

I took a few thread dumps and here and the results:
- service which are indexing stuck on this stack trace:

"cmdDistribExecutor-3-thread-17669" prio=10 tid=0x7f1aae4a6800
nid=0x44a9 runnable [0x7f1a6c0f6000]
   java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:152)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:166)
at
org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:90)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:281)
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:92)
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
at
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
at
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
at
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
at
org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
at
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
at
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:712)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:517)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at
org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:401)
at
org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:375)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)

   Locked ownable synchronizers:
- <0x7f1db2911048> (a 
java.util.concurrent.ThreadPoolExecutor$Worker)


Other servers stuck on:

"http-solr03-app-pci.dc2%2F10.210.20.219-8080-177" daemon prio=10
tid=0x7f24cc117800 nid=0x422c waiting on condition [0x7f098d383000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x7f2719890408> (a
java.util.concurrent.FutureTask)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:425)
at java.util.concurrent.FutureTask.get(FutureTask.java:187)
at
org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:621)
at
org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95)
at
org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1240)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1219)
at
org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:69)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1904)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:659)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:362)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:158)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.Appl

Re: /select with 'q' parameter does not work

[Amit Aggarwal]

When you start solr , do you find any error or exception
 Java -jar ./start.jar ... Then see if there is any problem ...
Otherwise take solr solrconfig.xml and try to run .. it should run
On 11-Dec-2013 5:41 PM, "Nutan"  wrote:

>  default="true">
>
>  
>explicit
>20
>*
>id
>2.1
>  
>   
>
>
>   
>   
> ${solr.document.data.dir:}
>   
>   
>
>
>   
>
>   class="solr.FieldAnalysisRequestHandler" />
>  
>  
>
> 
>  
>explicit
>10
>contents
>  
> 
>
> i made changes n this new solrconfig.xml ,but still the query does not
> work.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/select-with-q-parameter-does-not-work-tp4106099p4106133.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: /select with 'q' parameter does not work

[Nutan]

 
   explicit 
   20
   *
   id
   2.1
 
  
  
   
  
  
${solr.document.data.dir:}

  
  
   
  
  
   
 
 
 

 
   explicit
   10
   contents
 


i made changes n this new solrconfig.xml ,but still the query does not work.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/select-with-q-parameter-does-not-work-tp4106099p4106133.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr hardware memory question

[Toke Eskildsen]

On Tue, 2013-12-10 at 17:51 +0100, Hoggarth, Gil wrote:
> We're probably going to be building a Solr service to handle a dataset
> of ~60TB, which for our data and schema typically gives a Solr index
> size of 1/10th - i.e., 6TB. Given there's a general rule about the
> amount of hardware memory required should exceed the size of the Solr
> index (exceed to also allow for the operating system etc.), how have
> people handled this situation?

By acknowledging that it is cheaper to buy SSDs instead of trying to
compensate for slow spinning drives with excessive amounts of RAM. 

Our plans for an estimated 20TB of indexes out of 372TB of raw web data
is to use SSDs controlled by a single machine with 512GB of RAM (or was
it 256GB? I'll have to ask the hardware guys):
https://sbdevel.wordpress.com/2013/12/06/danish-webscale/

As always YMMW and the numbers you quite elsewhere indicates that your
queries are quite complex. You might want to be a bit of profiling to
see if they are heavy enough to make the CPU the bottleneck.

Regards,
Toke Eskildsen, State and University Library, Denmark

Re: /select with 'q' parameter does not work

[Nutan]

But below that i have also defined this,

 
   explicit
   10
   contents
 


Initially it worked,now all of a sudden why no results??



--
View this message in context: 
http://lucene.472066.n3.nabble.com/select-with-q-parameter-does-not-work-tp4106099p4106128.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Importing/Indexing the DB2 XML FieldType in SOLR

[ravi1984]

Hi Shawn, 
Thanks much for the reply! We have this working through Java layer and
facing the issues when integrated with SOLR layer. 

We tried to tweak the Query itself to return the String - following are the
options we tried, and still there was no luck. Do we need to tweak anything
in the Schema.xml, with regard to the field: cust_data? 

select XMLSERIALIZE(DTL_XML as CLOB(1M)) as DTL_XML from testSch.SUMMRY_DTL
where DTL_XML IS NOT NULL FETCH FIRST 50 ROWS ONLY 

Select XMLCAST(DTL_XML AS CLOB(1M)) as DTL_XML from testSch.SUMMRY_DTL where
DTL_XML IS NOT NULL FETCH FIRST 50 ROWS ONLY 

Thanks 
Ravi



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Importing-Indexing-the-DB2-XML-FieldType-in-SOLR-tp4105079p4106111.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Importing/Indexing the DB2 XML FieldType in SOLR

[ravi1984]

Hi Shawn,
Thanks much for the reply! We have this working through Java layer and
facing the issues when integrated with SOLR layer.

We tried to tweak the Query itself to return the String - following are the
options we tried, and still there was no luck. Do we need to tweak anything
in the Schema.xml, with regard to the field: cust_data?

select XMLSERIALIZE(ITNRY_DTL_TX as CLOB(1M)) as ITNRY_DTL_TX from
TITIN00.ITNRY_DTL where ITNRY_DTL_TX IS NOT NULL FETCH FIRST 50 ROWS ONLY

Select XMLCAST(ITNRY_DTL_TX AS CLOB(1M)) as ITNRY_DTL_TX from
TITIN00.ITNRY_DTL where ITNRY_DTL_TX IS NOT NULL FETCH FIRST 50 ROWS ONLY

Thanks
Ravi



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Importing-Indexing-the-DB2-XML-FieldType-in-SOLR-tp4105079p4106110.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: /select with 'q' parameter does not work

[Amit Aggarwal]

Because in your solrconfig ... Against /select ... DirectUpdateHandler is
mentioned . It should be solr.searchhanlder ..
On 11-Dec-2013 3:11 PM, "Nutan"  wrote:

> I have indexed 9 docs.
> this my* schema.xml*
>
> 
> 
>
>  multiValued="false"/>
>  required="true"
> multiValued="false"/>
>  multiValued="false"/>
>  multiValued="true"/>
> 
>  multiValued="false"/>
> 
>  multiValued="true"/>
>
>  stored="false" />
> 
> 
>
> 
>
>  positionIncrementGap="100" >
> 
> 
> 
> 
> 
> 
>
>
> 
>  class="solr.StrField" />
>  positionIncrementGap="0"/>
> 
> 
> 
> 
> 
> 
>
> 
> 
>
>
> 
> 
> 
> 
>  splitOnCaseChange="1" generateNumberParts="1" splitOnNumerics="1" />
>  dictionary="my_stemmer.txt" />
> 
>  ignoreCase="true" expand="false" />
> 
> 
> 
> 
> 
> 
>  splitOnCaseChange="1" generateNumberParts="1" splitOnNumerics="1" />
>  dictionary="my_stemmer.txt" />
> 
> 
> 
> 
> 
> contents
> id
> 
>
> *solrconfig.xml* is:
>
> 
>
> 
>
>   LUCENE_42
>
>   ${solr.document.data.dir:}
>
>   
>multipartUploadLimitInKB="8500" />
> 
>
>
>
>
>default="true">
>
>  
>explicit
>20
>*
>id
>2.1
>  
>   
>
>   
>   
> ${solr.document.data.dir:}
>   
>   
>
>   class="solr.FieldAnalysisRequestHandler" />
>  
>  
> 
>  
>explicit
>10
>contents
>  
> 
> 
> (i have also added extract,analysis,elevator,promotion,spell,suggester
> components in solrconfig but i guess that wont select query)
> When i run this:
> http://localhost:8080/solr/document/select?q=*:*   --> all the 9 docs are
> replaced
>
> but when i run this:
> http://localhost:8080/solr/document/select?q=programmer or anything in
> place
> of programmer --> output shows numfound=0 evenif there are about 34 times
> programmer has appeared in docs.
>
> Initially it worked fine,but not now.
> Why is it so?
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/select-with-q-parameter-does-not-work-tp4106099.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Possible regression for Solr 4.6.0 - commitWithin does not work with replicas

[Elodie Sannier]

Hello,

I am using SolrCloud 4.6.0 with two shards, two replicas by shard and with two 
collections.

collection fr_blue:
- shard1 -> server-01 (replica1), server-01 (replica2)
- shard2 -> server-02 (replica1), server-02 (replica2)

collection fr_green:
- shard1 -> server-01 (replica1), server-01 (replica2)
- shard2 -> server-02 (replica1), server-02 (replica2)

I add documents using solrj CloudSolrServer and using commitWithin feature :
int commitWithinMs = 3;
SolrServer server = new CloudSolrServer(zkHost);
server.add(doc, commitWithinMs);

When I query an instance,  for 5 indexed documents, the numFound value changes 
for each call, randomly 0,1,4 or 5.
When I query the instances with distrib=false, I have:
- leader shard1: numFound=1
- leader shard2: numFound=4
- replica shard1: numFound=0
- replica shard1: numFound=0

The documents are not commited in the replicas, even after waiting more than 30 
seconds.

If I force a commit usinghttp://server-01:8080/solr/update/?commit=true, the 
documents are commited in the replicas and numFound=5.
I suppose that the leader forwards the documents to the replica, but they are 
not commited.

Is it a new bug with commitWithin feature for distributed mode ?

This problem does not occur with the version 4.5.1.

Elodie Sannier


Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 8, rue du Sentier 75002 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention 
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce 
message, merci de le détruire et d'en avertir l'expéditeur.

/select with 'q' parameter does not work

[Nutan]

I have indexed 9 docs.
this my* schema.xml*


 












 













 


  













 



 





 






contents
id


*solrconfig.xml* is:




  
  LUCENE_42
  
  ${solr.document.data.dir:}
  
  
  



   
   
  

 
   explicit 
   20
   *
   id
   2.1
 

  
  
  
${solr.document.data.dir:}

  
  
   
 
 

 
   explicit
   10
   contents
 


(i have also added extract,analysis,elevator,promotion,spell,suggester
components in solrconfig but i guess that wont select query)
When i run this:
http://localhost:8080/solr/document/select?q=*:*   --> all the 9 docs are
replaced

but when i run this:
http://localhost:8080/solr/document/select?q=programmer or anything in place
of programmer --> output shows numfound=0 evenif there are about 34 times
programmer has appeared in docs.

Initially it worked fine,but not now.
Why is it so?



--
View this message in context: 
http://lucene.472066.n3.nabble.com/select-with-q-parameter-does-not-work-tp4106099.html
Sent from the Solr - User mailing list archive at Nabble.com.