RE: Lucene tests killed one other SSD - Policeman Jenkins
Hi, the broken SSD was replaced yesterday evening. All seems fine. I moved the Jenkins workspace and (snapshots of the) virtual machine disks to the new SSD and game is going on! Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Uwe Schindler [mailto:u...@thetaphi.de] > Sent: Monday, August 19, 2013 5:04 PM > To: dev@lucene.apache.org > Subject: Lucene tests killed one other SSD - Policeman Jenkins > > Hi, > > there were some problems with Policeman Jenkins the last days. The server > died 6 times the last month, recently 2 times in 24 hours. After I moved away > the swap file from the SSD, the failures were no longer fatal for the server > but fatal for some Jenkins runs :-) > > Finally the SSD device got unresponsible and only after a power cycle it was > responsible again. The error messages in dmesg look similar to other dying > OCX Vertex 2 drives. > > Now the statistics: During the whole lifetime of this SSD (2.5 years; which is > the lifetime of the server), it was mostly unused (it was just a "addon", > provided by the hosting provider, thanks to Serverloft / Plusserver). 1.5 > years > ago, Robert Muir and also Mike McCandless decided to use the server of my > own company SD DataSolutions to do more than idling most of the time: We > installed Jenkins and 2 additional virtualbox machines on this server after > the > 2012 Lucene Revolution conference and the "spare" SSD was given as base > for swap file, Jenkins Workspace and virtual disks for the Windows and > Haskintosh machines. > > During this time (1 year, 3 months) the SSD did hard work, according to > SMART: > ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED > WHEN_FAILED RAW_VALUE > 1 Raw_Read_Error_Rate 0x000f 112 112 050Pre-fail Always > - > 0/61244435 > 5 Retired_Block_Count 0x0033 100 100 003Pre-fail Always > - 0 > 9 Power_On_Hours_and_Msec 0x0032 100 100 000Old_age Always > - 21904h+48m+22.180s > 12 Power_Cycle_Count 0x0032 100 100 000Old_age Always > - > 19 > 171 Program_Fail_Count 0x0032 000 000 000Old_age Always > - 0 > 172 Erase_Fail_Count0x0032 000 000 000Old_age Always > - 0 > 174 Unexpect_Power_Loss_Ct 0x0030 000 000 000Old_age Offline > - > 6 > 177 Wear_Range_Delta0x 000 000 000Old_age Offline > - 2 > 181 Program_Fail_Count 0x0032 000 000 000Old_age Always > - 0 > 182 Erase_Fail_Count0x0032 000 000 000Old_age Always > - 0 > 187 Reported_Uncorrect 0x0032 100 100 000Old_age Always > - 0 > 194 Temperature_Celsius 0x0022 001 129 000Old_age Always > - > 1 (0 127 0 129) > 195 ECC_Uncorr_Error_Count 0x001c 112 112 000Old_age Offline > - > 0/61244435 > 196 Reallocated_Event_Count 0x0033 100 100 000Pre-fail Always > - > 0 > 231 SSD_Life_Left 0x0013 096 096 010Pre-fail Always > - 0 > 233 SandForce_Internal 0x 000 000 000Old_age Offline > - > 18752 > 234 SandForce_Internal 0x0032 000 000 000Old_age Always > - > 53376 > 241 Lifetime_Writes_GiB 0x0032 000 000 000Old_age Always > - > 53376 > 242 Lifetime_Reads_GiB 0x0032 000 000 000Old_age Always > - > 22784 > > Last 2 lines are interesting: > 53 Terabytes written to it and 22 Terabytes read from it. Ignore swap (mostly > unused as swappiness is low), so our tests are reading and writing a lot! > > And unfortunately after that it died (or almost died) this morning. Cause is > unclear, it could also be broken SATA cable, but from the web the given error > messages in "dmesg" seem to also be caused by drive failure (especially as it > is a timeout, not DMA error)! See https://paste.apache.org/bjAH > > So just to conclude: Lucene kills SSDs :-) Mike still has one Vertex 3 running > (his Intel one died before). > > Of course as this is a rented server, the hosting provider will replace the > SSD > (I was able to copy the data off, but the Jenkins workspace is not really > important data, more the virtual machines). After that one more year with a > new SSD, or should it survive longer? Let's see what type I will get as > replacement. I have no idea when it is replaced, so excuse any jenkins > downtime and after that maybe broken builds until all is settled again. At the > moment Jenkins is running much slower from the RAID 1 harddisks (with lots > of IOWAITS!). > > Uwe > > - > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > > > > -
[JENKINS] Lucene-Solr-SmokeRelease-4.x - Build # 101 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-4.x/101/
No tests ran.
Build Log:
[...truncated 34381 lines...]
prepare-release-no-sign:
[mkdir] Created dir:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeRelease
[copy] Copying 416 files to
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeRelease/lucene
[copy] Copying 194 files to
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeRelease/solr
[exec] JAVA6_HOME is /home/hudson/tools/java/latest1.6
[exec] JAVA7_HOME is /home/hudson/tools/java/latest1.7
[exec] NOTE: output encoding is US-ASCII
[exec]
[exec] Load release URL
"file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeRelease/"...
[exec]
[exec] Test Lucene...
[exec] test basics...
[exec] get KEYS
[exec] 0.1 MB in 0.02 sec (5.8 MB/sec)
[exec] check changes HTML...
[exec] download lucene-4.5.0-src.tgz...
[exec] 27.1 MB in 0.04 sec (658.1 MB/sec)
[exec] verify md5/sha1 digests
[exec] download lucene-4.5.0.tgz...
[exec] 49.0 MB in 0.07 sec (662.4 MB/sec)
[exec] verify md5/sha1 digests
[exec] download lucene-4.5.0.zip...
[exec] 58.8 MB in 0.10 sec (595.2 MB/sec)
[exec] verify md5/sha1 digests
[exec] unpack lucene-4.5.0.tgz...
[exec] verify JAR/WAR metadata...
[exec] test demo with 1.6...
[exec] got 5713 hits for query "lucene"
[exec] test demo with 1.7...
[exec] got 5713 hits for query "lucene"
[exec] check Lucene's javadoc JAR
[exec]
[exec]
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeReleaseTmp/unpack/lucene-4.5.0/docs/core/org/apache/lucene/util/AttributeSource.html
[exec] broken details HTML: Method Detail: addAttributeImpl: closing
"" does not match opening ""
[exec] broken details HTML: Method Detail: getAttribute: closing
"" does not match opening ""
[exec] Traceback (most recent call last):
[exec] File
"/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py",
line 1450, in
[exec] main()
[exec] File
"/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py",
line 1394, in main
[exec] smokeTest(baseURL, svnRevision, version, tmpDir, isSigned,
testArgs)
[exec] File
"/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py",
line 1431, in smokeTest
[exec] unpackAndVerify('lucene', tmpDir, artifact, svnRevision,
version, testArgs)
[exec] File
"/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py",
line 607, in unpackAndVerify
[exec] verifyUnpacked(project, artifact, unpackPath, svnRevision,
version, testArgs)
[exec] File
"/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py",
line 786, in verifyUnpacked
[exec] checkJavadocpath('%s/docs' % unpackPath)
[exec] File
"/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py",
line 904, in checkJavadocpath
[exec] raise RuntimeError('missing javadocs package summaries!')
[exec] RuntimeError: missing javadocs package summaries!
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/build.xml:314:
exec returned: 1
Total time: 19 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Email was triggered for: Failure
Sending email for trigger: Failure
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting
[
https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13745938#comment-13745938
]
William Harris commented on SOLR-2894:
--
shards.tolerant=true did indeed yield a more descriptive error:
{code}
ERROR - 2013-08-21 12:54:17.392; org.apache.solr.common.SolrException;
null:java.lang.NullPointerException
at
org.apache.solr.handler.component.FacetComponent.refinePivotFacets(FacetComponent.java:882)
at
org.apache.solr.handler.component.FacetComponent.handleResponses(FacetComponent.java:411)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:311)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1850)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:724
{code}
I also reindexed everything replacing the values of all string fields with
their corresponding hashes in order to see if the error could be caused by some
odd strings, but the same error occurs.
> Implement distributed pivot faceting
>
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
>Reporter: Erik Hatcher
> Fix For: 4.5
>
> Attachments: SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
> SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
> SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
> SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894-reworked.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports
> undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-5178) Admin UI - Memory Graph on Dashboard shows NaN for unused Swap
Stefan Matheis (steffkes) created SOLR-5178:
---
Summary: Admin UI - Memory Graph on Dashboard shows NaN for unused
Swap
Key: SOLR-5178
URL: https://issues.apache.org/jira/browse/SOLR-5178
Project: Solr
Issue Type: Bug
Components: web gui
Affects Versions: 4.4, 4.3
Reporter: Stefan Matheis (steffkes)
Assignee: Stefan Matheis (steffkes)
Priority: Minor
Fix For: 4.5, 5.0
If the System doesn't use Swap, the displayed memory graph on the dashboard
shows {{NaN}} (not a number) because it tries to divide by zero.
{code}"system":{
"name":"Linux",
"version":"3.2.0-39-virtual",
"arch":"amd64",
"systemLoadAverage":3.38,
"committedVirtualMemorySize":32454287360,
"freePhysicalMemorySize":912945152,
"freeSwapSpaceSize":0,
"processCpuTime":5627465000,
"totalPhysicalMemorySize":71881908224,
"totalSwapSpaceSize":0,
"openFileDescriptorCount":350,
"maxFileDescriptorCount":4096,
"uname": "Linux ip-xxx-xxx-xxx-xxx 3.2.0-39-virtual #62-Ubuntu SMP Thu
Feb 28 00:48:27 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux\n",
"uptime":" 11:24:39 up 4 days, 23:03, 1 user, load average: 3.38, 3.10,
2.95\n"
}{code}
We should add an additional check for that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5178) Admin UI - Memory Graph on Dashboard shows NaN for unused Swap
[
https://issues.apache.org/jira/browse/SOLR-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Matheis (steffkes) updated SOLR-5178:
Attachment: screenshot-vladimir.jpeg
> Admin UI - Memory Graph on Dashboard shows NaN for unused Swap
> --
>
> Key: SOLR-5178
> URL: https://issues.apache.org/jira/browse/SOLR-5178
> Project: Solr
> Issue Type: Bug
> Components: web gui
>Affects Versions: 4.3, 4.4
>Reporter: Stefan Matheis (steffkes)
>Assignee: Stefan Matheis (steffkes)
>Priority: Minor
> Fix For: 4.5, 5.0
>
> Attachments: screenshot-vladimir.jpeg
>
>
> If the System doesn't use Swap, the displayed memory graph on the dashboard
> shows {{NaN}} (not a number) because it tries to divide by zero.
> {code}"system":{
> "name":"Linux",
> "version":"3.2.0-39-virtual",
> "arch":"amd64",
> "systemLoadAverage":3.38,
> "committedVirtualMemorySize":32454287360,
> "freePhysicalMemorySize":912945152,
> "freeSwapSpaceSize":0,
> "processCpuTime":5627465000,
> "totalPhysicalMemorySize":71881908224,
> "totalSwapSpaceSize":0,
> "openFileDescriptorCount":350,
> "maxFileDescriptorCount":4096,
> "uname": "Linux ip-xxx-xxx-xxx-xxx 3.2.0-39-virtual #62-Ubuntu SMP Thu
> Feb 28 00:48:27 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux\n",
> "uptime":" 11:24:39 up 4 days, 23:03, 1 user, load average: 3.38, 3.10,
> 2.95\n"
> }{code}
> We should add an additional check for that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-4449) Enable backup requests for the internal solr load balancer
[ https://issues.apache.org/jira/browse/SOLR-4449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] philip hoy updated SOLR-4449: - Attachment: solr-back-request-lb-plugin.jar Now SOLR-4448 is committed the backup request load balancer is pluggable, attached is a java7 plugin built against lucene solr 4.4.0. > Enable backup requests for the internal solr load balancer > -- > > Key: SOLR-4449 > URL: https://issues.apache.org/jira/browse/SOLR-4449 > Project: Solr > Issue Type: New Feature > Components: SolrCloud >Reporter: philip hoy >Priority: Minor > Attachments: patch-4449.txt, SOLR-4449.patch, SOLR-4449.patch, > SOLR-4449.patch, solr-back-request-lb-plugin.jar > > > Add the ability to configure the built-in solr load balancer such that it > submits a backup request to the next server in the list if the initial > request takes too long. Employing such an algorithm could improve the latency > of the 9xth percentile albeit at the expense of increasing overall load due > to additional requests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-SmokeRelease-4.x - Build # 101 - Still Failing
I'm changing the wildcard from A to T. Consistently java has proven
they can't get this right!
On Wed, Aug 21, 2013 at 4:18 AM, Apache Jenkins Server
wrote:
> Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-4.x/101/
>
> No tests ran.
>
> Build Log:
> [...truncated 34381 lines...]
> prepare-release-no-sign:
> [mkdir] Created dir:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeRelease
> [copy] Copying 416 files to
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeRelease/lucene
> [copy] Copying 194 files to
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeRelease/solr
> [exec] JAVA6_HOME is /home/hudson/tools/java/latest1.6
> [exec] JAVA7_HOME is /home/hudson/tools/java/latest1.7
> [exec] NOTE: output encoding is US-ASCII
> [exec]
> [exec] Load release URL
> "file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeRelease/"...
> [exec]
> [exec] Test Lucene...
> [exec] test basics...
> [exec] get KEYS
> [exec] 0.1 MB in 0.02 sec (5.8 MB/sec)
> [exec] check changes HTML...
> [exec] download lucene-4.5.0-src.tgz...
> [exec] 27.1 MB in 0.04 sec (658.1 MB/sec)
> [exec] verify md5/sha1 digests
> [exec] download lucene-4.5.0.tgz...
> [exec] 49.0 MB in 0.07 sec (662.4 MB/sec)
> [exec] verify md5/sha1 digests
> [exec] download lucene-4.5.0.zip...
> [exec] 58.8 MB in 0.10 sec (595.2 MB/sec)
> [exec] verify md5/sha1 digests
> [exec] unpack lucene-4.5.0.tgz...
> [exec] verify JAR/WAR metadata...
> [exec] test demo with 1.6...
> [exec] got 5713 hits for query "lucene"
> [exec] test demo with 1.7...
> [exec] got 5713 hits for query "lucene"
> [exec] check Lucene's javadoc JAR
> [exec]
> [exec]
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeReleaseTmp/unpack/lucene-4.5.0/docs/core/org/apache/lucene/util/AttributeSource.html
> [exec] broken details HTML: Method Detail: addAttributeImpl: closing
> "" does not match opening ""
> [exec] broken details HTML: Method Detail: getAttribute: closing
> "" does not match opening ""
> [exec] Traceback (most recent call last):
> [exec] File
> "/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py",
> line 1450, in
> [exec] main()
> [exec] File
> "/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py",
> line 1394, in main
> [exec] smokeTest(baseURL, svnRevision, version, tmpDir, isSigned,
> testArgs)
> [exec] File
> "/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py",
> line 1431, in smokeTest
> [exec] unpackAndVerify('lucene', tmpDir, artifact, svnRevision,
> version, testArgs)
> [exec] File
> "/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py",
> line 607, in unpackAndVerify
> [exec] verifyUnpacked(project, artifact, unpackPath, svnRevision,
> version, testArgs)
> [exec] File
> "/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py",
> line 786, in verifyUnpacked
> [exec] checkJavadocpath('%s/docs' % unpackPath)
> [exec] File
> "/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py",
> line 904, in checkJavadocpath
> [exec] raise RuntimeError('missing javadocs package summaries!')
> [exec] RuntimeError: missing javadocs package summaries!
>
> BUILD FAILED
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.x/build.xml:314:
> exec returned: 1
>
> Total time: 19 minutes 32 seconds
> Build step 'Invoke Ant' marked build as failure
> Email was triggered for: Failure
> Sending email for trigger: Failure
>
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Using PatternAnalyzer
Hi, According to the changelog in 4.0.0alpha [1] the Pattern Analyzer was deprecated from the org.apache.lucene.analysis.miscellaneous package. From where do i use the Analyzer now? [1] http://lucene.apache.org/core/4_3_0/changes/Changes.html#4.0.0-alpha.api_changes -- Thanks and regards, Ajay Bhat
RE: [JENKINS] Lucene-Solr-SmokeRelease-4.x - Build # 101 - Still Failing
Hi Robert,
Thanks. But this is not a new issue, the warning was always there. The issue
that failed the build is a missing package-summary:
[exec] RuntimeError: missing javadocs package summaries!
But I don’t understand the output so I have no idea which ones causes this. I
assume it is Solr's new Join package?
Uwe
-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de
> -Original Message-
> From: Robert Muir [mailto:rcm...@gmail.com]
> Sent: Wednesday, August 21, 2013 3:23 PM
> To: dev@lucene.apache.org
> Subject: Re: [JENKINS] Lucene-Solr-SmokeRelease-4.x - Build # 101 - Still
> Failing
>
> I'm changing the wildcard from A to T. Consistently java has proven they can't
> get this right!
>
> On Wed, Aug 21, 2013 at 4:18 AM, Apache Jenkins Server
> wrote:
> > Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-4.x/101/
> >
> > No tests ran.
> >
> > Build Log:
> > [...truncated 34381 lines...]
> > prepare-release-no-sign:
> > [mkdir] Created dir: /usr/home/hudson/hudson-
> slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeRelease
> > [copy] Copying 416 files to /usr/home/hudson/hudson-
> slave/workspace/Lucene-Solr-SmokeRelease-
> 4.x/lucene/build/fakeRelease/lucene
> > [copy] Copying 194 files to /usr/home/hudson/hudson-
> slave/workspace/Lucene-Solr-SmokeRelease-
> 4.x/lucene/build/fakeRelease/solr
> > [exec] JAVA6_HOME is /home/hudson/tools/java/latest1.6
> > [exec] JAVA7_HOME is /home/hudson/tools/java/latest1.7
> > [exec] NOTE: output encoding is US-ASCII
> > [exec]
> > [exec] Load release URL "file:/usr/home/hudson/hudson-
> slave/workspace/Lucene-Solr-SmokeRelease-
> 4.x/lucene/build/fakeRelease/"...
> > [exec]
> > [exec] Test Lucene...
> > [exec] test basics...
> > [exec] get KEYS
> > [exec] 0.1 MB in 0.02 sec (5.8 MB/sec)
> > [exec] check changes HTML...
> > [exec] download lucene-4.5.0-src.tgz...
> > [exec] 27.1 MB in 0.04 sec (658.1 MB/sec)
> > [exec] verify md5/sha1 digests
> > [exec] download lucene-4.5.0.tgz...
> > [exec] 49.0 MB in 0.07 sec (662.4 MB/sec)
> > [exec] verify md5/sha1 digests
> > [exec] download lucene-4.5.0.zip...
> > [exec] 58.8 MB in 0.10 sec (595.2 MB/sec)
> > [exec] verify md5/sha1 digests
> > [exec] unpack lucene-4.5.0.tgz...
> > [exec] verify JAR/WAR metadata...
> > [exec] test demo with 1.6...
> > [exec] got 5713 hits for query "lucene"
> > [exec] test demo with 1.7...
> > [exec] got 5713 hits for query "lucene"
> > [exec] check Lucene's javadoc JAR
> > [exec]
> > [exec] /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-
> SmokeRelease-4.x/lucene/build/fakeReleaseTmp/unpack/lucene-
> 4.5.0/docs/core/org/apache/lucene/util/AttributeSource.html
> > [exec] broken details HTML: Method Detail: addAttributeImpl: closing
> "" does not match opening ""
> > [exec] broken details HTML: Method Detail: getAttribute: closing
> "" does not match opening ""
> > [exec] Traceback (most recent call last):
> > [exec] File "/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-
> SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py", line 1450, in
>
> > [exec] main()
> > [exec] File "/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-
> SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py", line 1394, in
> main
> > [exec] smokeTest(baseURL, svnRevision, version, tmpDir, isSigned,
> testArgs)
> > [exec] File "/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-
> SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py", line 1431, in
> smokeTest
> > [exec] unpackAndVerify('lucene', tmpDir, artifact, svnRevision,
> version, testArgs)
> > [exec] File "/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-
> SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py", line 607, in
> unpackAndVerify
> > [exec] verifyUnpacked(project, artifact, unpackPath, svnRevision,
> version, testArgs)
> > [exec] File "/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-
> SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py", line 786, in
> verifyUnpacked
> > [exec] checkJavadocpath('%s/docs' % unpackPath)
> > [exec] File "/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-
> SmokeRelease-4.x/dev-tools/scripts/smokeTestRelease.py", line 904, in
> checkJavadocpath
> > [exec] raise RuntimeError('missing javadocs package summaries!')
> > [exec] RuntimeError: missing javadocs package summaries!
> >
> > BUILD FAILED
> > /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-
> 4.x/b
> > uild.xml:314: exec returned: 1
> >
> > Total time: 19 minutes 32 seconds
> > Build step 'Invoke Ant' marked build as failure Email was triggered
> > for: Failure Sending email for trigger
Re: Using PatternAnalyzer
See: org.apache.lucene.analysis.pattern There's a bunch of stuff there that I think is what you want. Best, Erick On Wed, Aug 21, 2013 at 9:55 AM, Ajay Bhat wrote: > Hi, > > According to the changelog in 4.0.0alpha [1] the Pattern Analyzer was > deprecated from the org.apache.lucene.analysis.miscellaneous package. From > where do i use the Analyzer now? > > [1] > http://lucene.apache.org/core/4_3_0/changes/Changes.html#4.0.0-alpha.api_changes > -- > Thanks and regards, > Ajay Bhat >
Re: Using PatternAnalyzer
Ajay, In the future, please ask questions about *using* Lucene on the java-user mailing list - the dev list is for Lucene/Solr development discussion. PatternAnalyzer is deprecated, but will be available until Lucene 5.0. Alternatively, you can create your own equivalent analyzer using PatternTokenizer, LowerCaseFilter and StopFilter. Steve On Aug 21, 2013, at 9:55 AM, Ajay Bhat wrote: > Hi, > > According to the changelog in 4.0.0alpha [1] the Pattern Analyzer was > deprecated from the org.apache.lucene.analysis.miscellaneous package. From > where do i use the Analyzer now? > > [1] > http://lucene.apache.org/core/4_3_0/changes/Changes.html#4.0.0-alpha.api_changes > -- > Thanks and regards, > Ajay Bhat - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4787) Join Contrib
[
https://issues.apache.org/jira/browse/SOLR-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13746126#comment-13746126
]
Steven Bower commented on SOLR-4787:
When using the pJoin is relevance still applied to the results on the left side
of the join?
> Join Contrib
>
>
> Key: SOLR-4787
> URL: https://issues.apache.org/jira/browse/SOLR-4787
> Project: Solr
> Issue Type: New Feature
> Components: search
>Affects Versions: 4.2.1
>Reporter: Joel Bernstein
>Priority: Minor
> Fix For: 4.5, 5.0
>
> Attachments: SOLR-4787-deadlock-fix.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787-pjoin-long-keys.patch
>
>
> This contrib provides a place where different join implementations can be
> contributed to Solr. This contrib currently includes 2 join implementations.
> The initial patch was generated from the Solr 4.3 tag. Because of changes in
> the FieldCache API this patch will only build with Solr 4.2 or above.
> *PostFilterJoinQParserPlugin aka "pjoin"*
> The pjoin provides a join implementation that filters results in one core
> based on the results of a search in another core. This is similar in
> functionality to the JoinQParserPlugin but the implementation differs in a
> couple of important ways.
> The first way is that the pjoin is designed to work with integer join keys
> only. So, in order to use pjoin, integer join keys must be included in both
> the to and from core.
> The second difference is that the pjoin builds memory structures that are
> used to quickly connect the join keys. It also uses a custom SolrCache named
> "join" to hold intermediate DocSets which are needed to build the join memory
> structures. So, the pjoin will need more memory then the JoinQParserPlugin to
> perform the join.
> The main advantage of the pjoin is that it can scale to join millions of keys
> between cores.
> Because it's a PostFilter, it only needs to join records that match the main
> query.
> The syntax of the pjoin is the same as the JoinQParserPlugin except that the
> plugin is referenced by the string "pjoin" rather then "join".
> fq=\{!pjoin fromCore=collection2 from=id_i to=id_i\}user:customer1
> The example filter query above will search the fromCore (collection2) for
> "user:customer1". This query will generate a list of values from the "from"
> field that will be used to filter the main query. Only records from the main
> query, where the "to" field is present in the "from" list will be included in
> the results.
> The solrconfig.xml in the main query core must contain the reference to the
> pjoin.
> class="org.apache.solr.joins.PostFilterJoinQParserPlugin"/>
> And the join contrib jars must be registed in the solrconfig.xml.
>
> The solrconfig.xml in the fromcore must have the "join" SolrCache configured.
> class="solr.LRUCache"
> size="4096"
> initialSize="1024"
> />
> *ValueSourceJoinParserPlugin aka vjoin*
> The second implementation is the ValueSourceJoinParserPlugin aka "vjoin".
> This implements a ValueSource function query that can return a value from a
> second core based on join keys and limiting query. The limiting query can be
> used to select a specific subset of data from the join core. This allows
> customer specific relevance data to be stored in a separate core and then
> joined in the main query.
> The vjoin is called using the "vjoin" function query. For example:
> bf=vjoin(fromCore, fromKey, fromVal, toKey, query)
> This example shows "vjoin" being called by the edismax boost function
> parameter. This example will return the "fromVal" from the "fromCore". The
> "fromKey" and "toKey" are used to link the records from the main query to the
> records in the "fromCore". The "query" is used to select a specific set of
> records to join with in fromCore.
> Currently the fromKey and toKey must be longs but this will change in future
> versions. Like the pjoin, the "join" SolrCache is used to hold the join
> memory structures.
> To configure the vjoin you must register the ValueSource plugin in the
> solrconfig.xml as follows:
> class="org.apache.solr.joins.ValueSourceJoinParserPlugin" />
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5084) new field type - EnumField
[ https://issues.apache.org/jira/browse/SOLR-5084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13746521#comment-13746521 ] Erick Erickson commented on SOLR-5084: -- I have a few cycles to devote to this. [~elrand] What's the state of the most recent patch? You were going to attach a new patch that had some unit tests, is that forthcoming or did I miss it being attached? > new field type - EnumField > -- > > Key: SOLR-5084 > URL: https://issues.apache.org/jira/browse/SOLR-5084 > Project: Solr > Issue Type: New Feature >Reporter: Elran Dvir > Attachments: enumsConfig.xml, schema_example.xml, Solr-5084.patch, > Solr-5084.patch, Solr-5084.patch > > > We have encountered a use case in our system where we have a few fields > (Severity. Risk etc) with a closed set of values, where the sort order for > these values is pre-determined but not lexicographic (Critical is higher than > High). Generically this is very close to how enums work. > To implement, I have prototyped a new type of field: EnumField where the > inputs are a closed predefined set of strings in a special configuration > file (similar to currency.xml). > The code is based on 4.2.1. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4479) TermVectorComponent NPE when running Solr Cloud
[
https://issues.apache.org/jira/browse/SOLR-4479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13746552#comment-13746552
]
Yakov commented on SOLR-4479:
-
Hi,
problem is still exists on 4.4. Reindexing, as Dimitris suggested is not fixing
the problem in my case. On fresh index it produces:
{noformat}
"trace":"java.lang.NullPointerException\n\tat
org.apache.solr.handler.component.TermVectorComponent.finishStage(TermVectorComponent.java:437)\n\tat
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:317)\n\tat
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)\n\tat
org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:241)\n\tat
org.apache.solr.core.SolrCore.execute(SolrCore.java:1904)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:659)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:362)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:158)\n\tat
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)\n\tat
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)\n\tat
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)\n\tat
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)\n\tat
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)\n\tat
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)\n\tat
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)\n\tat
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)\n\tat
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)\n\tat
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)\n\tat
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)\n\tat
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)\n\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\n\tat
java.lang.Thread.run(Thread.java:722)\n",
"code":500
{noformat}
Actually, this problem is really important for me, how could I help to fix it
faster?
> TermVectorComponent NPE when running Solr Cloud
> ---
>
> Key: SOLR-4479
> URL: https://issues.apache.org/jira/browse/SOLR-4479
> Project: Solr
> Issue Type: Bug
>Affects Versions: 4.1
>Reporter: Vitali Kviatkouski
>
> When running Solr Cloud (just simply 2 shards - as described in wiki), got NPE
> java.lang.NullPointerException
> at
> org.apache.solr.handler.component.TermVectorComponent.finishStage(TermVectorComponent.java:437)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:317)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at
> org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:242)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1816)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:448)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:269)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:560)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1072)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
> . Skipped
> To reproduce, follow the guide in wiki
> (http://wiki.apache.org/solr/SolrCloud), add some documents and then request
> http://localhost:8983/solr/collection1/tvrh?q=*
[jira] [Created] (SOLR-5179) Add Support for other operator other than AND/ OR in solr query (eg:SAME )
narendra created SOLR-5179: -- Summary: Add Support for other operator other than AND/ OR in solr query (eg:SAME ) Key: SOLR-5179 URL: https://issues.apache.org/jira/browse/SOLR-5179 Project: Solr Issue Type: Improvement Components: query parsers Affects Versions: 4.4 Reporter: narendra Fix For: 5.0 Add Support for other operator other than AND/ OR in solr query (eg:SAME ) Or please let me know who we can do this and which class and method is responsible for this so i can override the behaviors -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5179) Add Support for other operator other than AND/ OR in solr query (eg:SAME )
[ https://issues.apache.org/jira/browse/SOLR-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13746643#comment-13746643 ] Uwe Schindler commented on SOLR-5179: - What does "SAME" mean for you? There is no such query in Lucene, so Solr cannot handle this. > Add Support for other operator other than AND/ OR in solr query (eg:SAME ) > -- > > Key: SOLR-5179 > URL: https://issues.apache.org/jira/browse/SOLR-5179 > Project: Solr > Issue Type: Improvement > Components: query parsers >Affects Versions: 4.4 >Reporter: narendra > Fix For: 5.0 > > > Add Support for other operator other than AND/ OR in solr query (eg:SAME ) > Or please let me know who we can do this and which class and method is > responsible for this so i can override the behaviors -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4787) Join Contrib
[
https://issues.apache.org/jira/browse/SOLR-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13746647#comment-13746647
]
Joel Bernstein commented on SOLR-4787:
--
Yes, the main query still has relevance applied. The filter join query does not
impact relevance though.
There is going to be a large set of code changes to this ticket, probably next
week. The implementation of the pjoin is changing and another more scalable
join will be added as well.
The vjoin is going to remain the same for the time being.
> Join Contrib
>
>
> Key: SOLR-4787
> URL: https://issues.apache.org/jira/browse/SOLR-4787
> Project: Solr
> Issue Type: New Feature
> Components: search
>Affects Versions: 4.2.1
>Reporter: Joel Bernstein
>Priority: Minor
> Fix For: 4.5, 5.0
>
> Attachments: SOLR-4787-deadlock-fix.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787-pjoin-long-keys.patch
>
>
> This contrib provides a place where different join implementations can be
> contributed to Solr. This contrib currently includes 2 join implementations.
> The initial patch was generated from the Solr 4.3 tag. Because of changes in
> the FieldCache API this patch will only build with Solr 4.2 or above.
> *PostFilterJoinQParserPlugin aka "pjoin"*
> The pjoin provides a join implementation that filters results in one core
> based on the results of a search in another core. This is similar in
> functionality to the JoinQParserPlugin but the implementation differs in a
> couple of important ways.
> The first way is that the pjoin is designed to work with integer join keys
> only. So, in order to use pjoin, integer join keys must be included in both
> the to and from core.
> The second difference is that the pjoin builds memory structures that are
> used to quickly connect the join keys. It also uses a custom SolrCache named
> "join" to hold intermediate DocSets which are needed to build the join memory
> structures. So, the pjoin will need more memory then the JoinQParserPlugin to
> perform the join.
> The main advantage of the pjoin is that it can scale to join millions of keys
> between cores.
> Because it's a PostFilter, it only needs to join records that match the main
> query.
> The syntax of the pjoin is the same as the JoinQParserPlugin except that the
> plugin is referenced by the string "pjoin" rather then "join".
> fq=\{!pjoin fromCore=collection2 from=id_i to=id_i\}user:customer1
> The example filter query above will search the fromCore (collection2) for
> "user:customer1". This query will generate a list of values from the "from"
> field that will be used to filter the main query. Only records from the main
> query, where the "to" field is present in the "from" list will be included in
> the results.
> The solrconfig.xml in the main query core must contain the reference to the
> pjoin.
> class="org.apache.solr.joins.PostFilterJoinQParserPlugin"/>
> And the join contrib jars must be registed in the solrconfig.xml.
>
> The solrconfig.xml in the fromcore must have the "join" SolrCache configured.
> class="solr.LRUCache"
> size="4096"
> initialSize="1024"
> />
> *ValueSourceJoinParserPlugin aka vjoin*
> The second implementation is the ValueSourceJoinParserPlugin aka "vjoin".
> This implements a ValueSource function query that can return a value from a
> second core based on join keys and limiting query. The limiting query can be
> used to select a specific subset of data from the join core. This allows
> customer specific relevance data to be stored in a separate core and then
> joined in the main query.
> The vjoin is called using the "vjoin" function query. For example:
> bf=vjoin(fromCore, fromKey, fromVal, toKey, query)
> This example shows "vjoin" being called by the edismax boost function
> parameter. This example will return the "fromVal" from the "fromCore". The
> "fromKey" and "toKey" are used to link the records from the main query to the
> records in the "fromCore". The "query" is used to select a specific set of
> records to join with in fromCore.
> Currently the fromKey and toKey must be longs but this will change in future
> versions. Like the pjoin, the "join" SolrCache is used to hold the join
> memory structures.
> To configure the vjoin you must register the ValueSource plugin in the
> solrconfig.xml as follows:
> class="org.apache.solr.joins.ValueSourceJoinParserPlugin" />
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/softw
[jira] [Updated] (SOLR-5173) Solr-core's Maven configuration includes test-only Hadoop dependencies as indirect compile-time dependencies
[ https://issues.apache.org/jira/browse/SOLR-5173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe updated SOLR-5173: - Attachment: SOLR-5173.patch This patch no longer removes hadoop-auth, hadoop-hdfs, and hdfs-annotations as Ivy and Maven dependencies. Instead, the solr-core Maven dependencies are trimmed down to a minimum, and no longer include indirect jetty 6 dependencies. Committing shortly. > Solr-core's Maven configuration includes test-only Hadoop dependencies as > indirect compile-time dependencies > > > Key: SOLR-5173 > URL: https://issues.apache.org/jira/browse/SOLR-5173 > Project: Solr > Issue Type: Bug > Components: Build >Affects Versions: 4.4 >Reporter: Steve Rowe >Assignee: Steve Rowe >Priority: Blocker > Fix For: 4.5 > > Attachments: SOLR-5173.patch, SOLR-5173.patch > > > Chris Collins [reported on > solr-user|http://markmail.org/message/evhpcougs5ppafjk] that solr-core 4.4 > has dependencies on hadoop, and indirectly on jetty 6. > As a workaround for Maven dependencies, the indirect jetty 6 dependency/ies > can be excluded. > The Maven configuration should exclude any compile-time (and also run-time) > dependencies that are not Ant/Ivy compile- and run-time dependencies, > including jetty 6. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5173) Solr-core's Maven configuration includes test-only Hadoop dependencies as indirect compile-time dependencies
[ https://issues.apache.org/jira/browse/SOLR-5173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13746736#comment-13746736 ] ASF subversion and git services commented on SOLR-5173: --- Commit 1516264 from [~steve_rowe] in branch 'dev/trunk' [ https://svn.apache.org/r1516264 ] SOLR-5173: Solr-core's Maven configuration includes test-only Hadoop dependencies as indirect compile-time dependencies > Solr-core's Maven configuration includes test-only Hadoop dependencies as > indirect compile-time dependencies > > > Key: SOLR-5173 > URL: https://issues.apache.org/jira/browse/SOLR-5173 > Project: Solr > Issue Type: Bug > Components: Build >Affects Versions: 4.4 >Reporter: Steve Rowe >Assignee: Steve Rowe >Priority: Blocker > Fix For: 4.5 > > Attachments: SOLR-5173.patch, SOLR-5173.patch > > > Chris Collins [reported on > solr-user|http://markmail.org/message/evhpcougs5ppafjk] that solr-core 4.4 > has dependencies on hadoop, and indirectly on jetty 6. > As a workaround for Maven dependencies, the indirect jetty 6 dependency/ies > can be excluded. > The Maven configuration should exclude any compile-time (and also run-time) > dependencies that are not Ant/Ivy compile- and run-time dependencies, > including jetty 6. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5173) Solr-core's Maven configuration includes test-only Hadoop dependencies as indirect compile-time dependencies
[ https://issues.apache.org/jira/browse/SOLR-5173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13746768#comment-13746768 ] ASF subversion and git services commented on SOLR-5173: --- Commit 1516273 from [~steve_rowe] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1516273 ] SOLR-5173: Solr-core's Maven configuration includes test-only Hadoop dependencies as indirect compile-time dependencies (merged trunk r1516264) > Solr-core's Maven configuration includes test-only Hadoop dependencies as > indirect compile-time dependencies > > > Key: SOLR-5173 > URL: https://issues.apache.org/jira/browse/SOLR-5173 > Project: Solr > Issue Type: Bug > Components: Build >Affects Versions: 4.4 >Reporter: Steve Rowe >Assignee: Steve Rowe >Priority: Blocker > Fix For: 4.5 > > Attachments: SOLR-5173.patch, SOLR-5173.patch > > > Chris Collins [reported on > solr-user|http://markmail.org/message/evhpcougs5ppafjk] that solr-core 4.4 > has dependencies on hadoop, and indirectly on jetty 6. > As a workaround for Maven dependencies, the indirect jetty 6 dependency/ies > can be excluded. > The Maven configuration should exclude any compile-time (and also run-time) > dependencies that are not Ant/Ivy compile- and run-time dependencies, > including jetty 6. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3192) NettySolrServer (supported by netty/protobuf) such as CommonsHttpSolrServer
[
https://issues.apache.org/jira/browse/SOLR-3192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13746776#comment-13746776
]
Kranti Parisa commented on SOLR-3192:
-
Anyone working on this (integration of solr + netty) ?
> NettySolrServer (supported by netty/protobuf) such as CommonsHttpSolrServer
> ---
>
> Key: SOLR-3192
> URL: https://issues.apache.org/jira/browse/SOLR-3192
> Project: Solr
> Issue Type: New Feature
>Reporter: Linbin Chen
> Fix For: 4.5, 5.0
>
> Attachments: solr.proto
>
>
> solr support netty tcp, netty/tcp can handle asynchronous,efficient,keepalive
> ...
> it's used on solr cloud or solrj
> solr.proto maybe
> {code:java}
> package org.apache.solr.common.netty;
> option java_package = "org.apache.solr.common.netty";
> option java_outer_classname = "SolrProtocol";
> option optimize_for = SPEED;
> message SolrRequest {
> //request id
> required int64 rid = 1;
> //for string, json format, like http params
> required string params = 2;
> //(xml, json, solr_javabin) fix by params rt (request type)
> optional int32 streamsFormat = 3;
> repeated bytes streams = 4;
> }
> message SolrResponse {
> //response request id,
> required int64 rid = 1;
> //response format (xml, json, solr_javabin, csv ...)
> optional int32 responseFormat = 2;
> optional bytes response = 3;
> optional int32 errorCode = 4;
> optional string errorStr = 5;
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5057) queryResultCache should not related with the order of fq's list
[ https://issues.apache.org/jira/browse/SOLR-5057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13746805#comment-13746805 ] ASF subversion and git services commented on SOLR-5057: --- Commit 1516299 from [~erickoerickson] in branch 'dev/trunk' [ https://svn.apache.org/r1516299 ] SOLR-5057 - queryResultCache should match out-of-order fq clauses > queryResultCache should not related with the order of fq's list > --- > > Key: SOLR-5057 > URL: https://issues.apache.org/jira/browse/SOLR-5057 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 4.0, 4.1, 4.2, 4.3 >Reporter: Feihong Huang >Assignee: Erick Erickson >Priority: Minor > Attachments: SOLR-5057.patch, SOLR-5057.patch, SOLR-5057.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > There are two case query with the same meaning below. But the case2 can't use > the queryResultCache when case1 is executed. > case1: q=*:*&fq=field1:value1&fq=field2:value2 > case2: q=*:*&fq=field2:value2&fq=field1:value1 > I think queryResultCache should not be related with the order of fq's list. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-5173) Solr-core's Maven configuration includes test-only Hadoop dependencies as indirect compile-time dependencies
[ https://issues.apache.org/jira/browse/SOLR-5173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe resolved SOLR-5173. -- Resolution: Fixed Fix Version/s: 5.0 Committed to trunk and branch_4x. Thanks Chris! > Solr-core's Maven configuration includes test-only Hadoop dependencies as > indirect compile-time dependencies > > > Key: SOLR-5173 > URL: https://issues.apache.org/jira/browse/SOLR-5173 > Project: Solr > Issue Type: Bug > Components: Build >Affects Versions: 4.4 >Reporter: Steve Rowe >Assignee: Steve Rowe >Priority: Blocker > Fix For: 4.5, 5.0 > > Attachments: SOLR-5173.patch, SOLR-5173.patch > > > Chris Collins [reported on > solr-user|http://markmail.org/message/evhpcougs5ppafjk] that solr-core 4.4 > has dependencies on hadoop, and indirectly on jetty 6. > As a workaround for Maven dependencies, the indirect jetty 6 dependency/ies > can be excluded. > The Maven configuration should exclude any compile-time (and also run-time) > dependencies that are not Ant/Ivy compile- and run-time dependencies, > including jetty 6. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5057) queryResultCache should not related with the order of fq's list
[ https://issues.apache.org/jira/browse/SOLR-5057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson updated SOLR-5057: - Attachment: SOLR-5057.patch Applies cleanly, I suspect there is a charset problem. > queryResultCache should not related with the order of fq's list > --- > > Key: SOLR-5057 > URL: https://issues.apache.org/jira/browse/SOLR-5057 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 4.0, 4.1, 4.2, 4.3 >Reporter: Feihong Huang >Assignee: Erick Erickson >Priority: Minor > Attachments: SOLR-5057.patch, SOLR-5057.patch, SOLR-5057.patch, > SOLR-5057.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > There are two case query with the same meaning below. But the case2 can't use > the queryResultCache when case1 is executed. > case1: q=*:*&fq=field1:value1&fq=field2:value2 > case2: q=*:*&fq=field2:value2&fq=field1:value1 > I think queryResultCache should not be related with the order of fq's list. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-5057) queryResultCache should not related with the order of fq's list
[ https://issues.apache.org/jira/browse/SOLR-5057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson resolved SOLR-5057. -- Resolution: Fixed Fix Version/s: 5.0 4.5 Thanks to Feihong Huang! > queryResultCache should not related with the order of fq's list > --- > > Key: SOLR-5057 > URL: https://issues.apache.org/jira/browse/SOLR-5057 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 4.0, 4.1, 4.2, 4.3 >Reporter: Feihong Huang >Assignee: Erick Erickson >Priority: Minor > Fix For: 4.5, 5.0 > > Attachments: SOLR-5057.patch, SOLR-5057.patch, SOLR-5057.patch, > SOLR-5057.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > There are two case query with the same meaning below. But the case2 can't use > the queryResultCache when case1 is executed. > case1: q=*:*&fq=field1:value1&fq=field2:value2 > case2: q=*:*&fq=field2:value2&fq=field1:value1 > I think queryResultCache should not be related with the order of fq's list. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5057) queryResultCache should not related with the order of fq's list
[ https://issues.apache.org/jira/browse/SOLR-5057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13746858#comment-13746858 ] ASF subversion and git services commented on SOLR-5057: --- Commit 1516307 from [~erickoerickson] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1516307 ] SOLR-5057 - queryResultCache should match out-of-order fq clauses > queryResultCache should not related with the order of fq's list > --- > > Key: SOLR-5057 > URL: https://issues.apache.org/jira/browse/SOLR-5057 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 4.0, 4.1, 4.2, 4.3 >Reporter: Feihong Huang >Assignee: Erick Erickson >Priority: Minor > Attachments: SOLR-5057.patch, SOLR-5057.patch, SOLR-5057.patch, > SOLR-5057.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > There are two case query with the same meaning below. But the case2 can't use > the queryResultCache when case1 is executed. > case1: q=*:*&fq=field1:value1&fq=field2:value2 > case2: q=*:*&fq=field2:value2&fq=field1:value1 > I think queryResultCache should not be related with the order of fq's list. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-5185) licenses/*.jar.sha1 don't belong in Lucene and Solr binary distributions
Steve Rowe created LUCENE-5185:
--
Summary: licenses/*.jar.sha1 don't belong in Lucene and Solr
binary distributions
Key: LUCENE-5185
URL: https://issues.apache.org/jira/browse/LUCENE-5185
Project: Lucene - Core
Issue Type: Improvement
Components: general/build
Reporter: Steve Rowe
Assignee: Steve Rowe
Priority: Minor
Fix For: 5.0, 4.5
On LUCENE-3945, where external dependency checksum verification was put in
place, [~hossman_luc...@fucit.org] wrote:
bq. So i propose that we include checksum files in svn and in our source
releases that can be used by users to verify that the jars they get from ivy
match the jars we tested against.
That is, checksum files in *binary* distributions was not part of the proposal.
And [in his comment associated with the final
patch|https://issues.apache.org/jira/browse/LUCENE-3945?focusedCommentId=13246476&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13246476]:
bq. 2) fixes the binary releases to exlcude the sha1 files
Somewhere between then and now, {{\*.jar.sha1}} files snuck back into the
Lucene and Solr binary releases, under the {{licenses/}} directory. They
should not be there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5185) licenses/*.jar.sha1 don't belong in Lucene and Solr binary distributions
[
https://issues.apache.org/jira/browse/LUCENE-5185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13746892#comment-13746892
]
Robert Muir commented on LUCENE-5185:
-
{quote}
Somewhere between then and now, *.jar.sha1 files snuck back into the Lucene and
Solr binary releases, under the licenses/ directory.
{quote}
There was no sneakiness about it, it was totally intentional: LUCENE-4267
{quote}
They should not be there.
{quote}
Thats your personal opinion: I disagree (please see the issue for discussion).
> licenses/*.jar.sha1 don't belong in Lucene and Solr binary distributions
>
>
> Key: LUCENE-5185
> URL: https://issues.apache.org/jira/browse/LUCENE-5185
> Project: Lucene - Core
> Issue Type: Improvement
> Components: general/build
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Minor
> Fix For: 5.0, 4.5
>
>
> On LUCENE-3945, where external dependency checksum verification was put in
> place, [~hossman_luc...@fucit.org] wrote:
> bq. So i propose that we include checksum files in svn and in our source
> releases that can be used by users to verify that the jars they get from ivy
> match the jars we tested against.
> That is, checksum files in *binary* distributions was not part of the
> proposal.
> And [in his comment associated with the final
> patch|https://issues.apache.org/jira/browse/LUCENE-3945?focusedCommentId=13246476&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13246476]:
> bq. 2) fixes the binary releases to exlcude the sha1 files
> Somewhere between then and now, {{\*.jar.sha1}} files snuck back into the
> Lucene and Solr binary releases, under the {{licenses/}} directory. They
> should not be there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-5180) Core admin reload for an invalid core name returns 500 rather than 400 status code
Jack Krupansky created SOLR-5180:
Summary: Core admin reload for an invalid core name returns 500
rather than 400 status code
Key: SOLR-5180
URL: https://issues.apache.org/jira/browse/SOLR-5180
Project: Solr
Issue Type: Bug
Components: multicore
Affects Versions: 4.4
Reporter: Jack Krupansky
A core admin request to reload a nonexistent core name returns a 500 Server
Error rather than a 400 Invalid Request status code.
The request:
{code}
curl
"http://localhost:8983/solr/admin/cores?action=reload&core=bogus&indent=true";
{code}
The response:
{code}
500
5
Error handling 'reload' action
org.apache.solr.common.SolrException: Error handling
'reload' action
at
org.apache.solr.handler.admin.CoreAdminHandler.handleReloadAction(CoreAdminHandler.java:673)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:172)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at
org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:655)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:246)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:368)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Unknown Source)
Caused by: org.apache.solr.common.SolrException: Unable to reload core: bogus
at
org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:930)
at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:685)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleReloadAction(CoreAdminHandler.java:671)
... 30 more
Caused by: org.apache.solr.common.SolrException: No such core: bogus
at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:636)
... 31 more
500
{code}
The code at CoreContainer.reload(CoreContainer.java:636) correctly throws a
Solr Bad Request Exception:
{Code}
if (core == null)
throw new SolrException( SolrException.ErrorCode.BAD_REQUEST, "No such
core: " + name );
{code}
But then the code at
CoreAdminHandler.handleReloadAction(CoreAdminHandler.java:673) catches it and
throws the Server Error exception:
{code}
try {
coreContainer.reload(cname);
} catch (Exception ex) {
throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error
handling 'reload' action", ex);
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrator
[jira] [Commented] (SOLR-4787) Join Contrib
[
https://issues.apache.org/jira/browse/SOLR-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13746912#comment-13746912
]
Kranti Parisa commented on SOLR-4787:
-
Joel:
That's cool, thanks. Did you ever thought of supporting "fq" parameter for the
Join, like how we support for "v" for query. the idea is to use filter cache on
the child core while executing the join query.
> Join Contrib
>
>
> Key: SOLR-4787
> URL: https://issues.apache.org/jira/browse/SOLR-4787
> Project: Solr
> Issue Type: New Feature
> Components: search
>Affects Versions: 4.2.1
>Reporter: Joel Bernstein
>Priority: Minor
> Fix For: 4.5, 5.0
>
> Attachments: SOLR-4787-deadlock-fix.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787-pjoin-long-keys.patch
>
>
> This contrib provides a place where different join implementations can be
> contributed to Solr. This contrib currently includes 2 join implementations.
> The initial patch was generated from the Solr 4.3 tag. Because of changes in
> the FieldCache API this patch will only build with Solr 4.2 or above.
> *PostFilterJoinQParserPlugin aka "pjoin"*
> The pjoin provides a join implementation that filters results in one core
> based on the results of a search in another core. This is similar in
> functionality to the JoinQParserPlugin but the implementation differs in a
> couple of important ways.
> The first way is that the pjoin is designed to work with integer join keys
> only. So, in order to use pjoin, integer join keys must be included in both
> the to and from core.
> The second difference is that the pjoin builds memory structures that are
> used to quickly connect the join keys. It also uses a custom SolrCache named
> "join" to hold intermediate DocSets which are needed to build the join memory
> structures. So, the pjoin will need more memory then the JoinQParserPlugin to
> perform the join.
> The main advantage of the pjoin is that it can scale to join millions of keys
> between cores.
> Because it's a PostFilter, it only needs to join records that match the main
> query.
> The syntax of the pjoin is the same as the JoinQParserPlugin except that the
> plugin is referenced by the string "pjoin" rather then "join".
> fq=\{!pjoin fromCore=collection2 from=id_i to=id_i\}user:customer1
> The example filter query above will search the fromCore (collection2) for
> "user:customer1". This query will generate a list of values from the "from"
> field that will be used to filter the main query. Only records from the main
> query, where the "to" field is present in the "from" list will be included in
> the results.
> The solrconfig.xml in the main query core must contain the reference to the
> pjoin.
> class="org.apache.solr.joins.PostFilterJoinQParserPlugin"/>
> And the join contrib jars must be registed in the solrconfig.xml.
>
> The solrconfig.xml in the fromcore must have the "join" SolrCache configured.
> class="solr.LRUCache"
> size="4096"
> initialSize="1024"
> />
> *ValueSourceJoinParserPlugin aka vjoin*
> The second implementation is the ValueSourceJoinParserPlugin aka "vjoin".
> This implements a ValueSource function query that can return a value from a
> second core based on join keys and limiting query. The limiting query can be
> used to select a specific subset of data from the join core. This allows
> customer specific relevance data to be stored in a separate core and then
> joined in the main query.
> The vjoin is called using the "vjoin" function query. For example:
> bf=vjoin(fromCore, fromKey, fromVal, toKey, query)
> This example shows "vjoin" being called by the edismax boost function
> parameter. This example will return the "fromVal" from the "fromCore". The
> "fromKey" and "toKey" are used to link the records from the main query to the
> records in the "fromCore". The "query" is used to select a specific set of
> records to join with in fromCore.
> Currently the fromKey and toKey must be longs but this will change in future
> versions. Like the pjoin, the "join" SolrCache is used to hold the join
> memory structures.
> To configure the vjoin you must register the ValueSource plugin in the
> solrconfig.xml as follows:
> class="org.apache.solr.joins.ValueSourceJoinParserPlugin" />
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.o
[jira] [Commented] (LUCENE-5185) licenses/*.jar.sha1 don't belong in Lucene and Solr binary distributions
[
https://issues.apache.org/jira/browse/LUCENE-5185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13746924#comment-13746924
]
Steve Rowe commented on LUCENE-5185:
bq. it was totally intentional: LUCENE-4267
Thanks for the pointer, I guess I didn't realize at the time that checksums
were included.
{quote}
bq. They should not be there.
Thats your personal opinion: I disagree (please see the issue for discussion).
{quote}
AFAICT, the only discussion that happened there that's applicable to the
checksums was your rationale in the issue description:
bq. additional verification for consumers.
Is there some other discussion I'm missing?
I don't disagree that consumers could theoretically use the checksums to verify
3rd party .jar integrity, but:
# We don't provide checksums for our own .jars, or for anything else in the
binary distributions - why are 3rd party jars special in this regard?
# Jenkins regularly validates the checksums against the files Ivy downloads,
and the smoke checker runs 'ant validate' against the source distribution,
which indirectly validates the integrity of the 3rd party jars included in the
binary distribution.
Maybe the smoke tester could be modified to directly test the checksums against
the binary artifacts' 3rd party jars?
My thinking here is that we should keep our distributions lean, including only
the things that must be there, and I don't think these checksum files qualify.
> licenses/*.jar.sha1 don't belong in Lucene and Solr binary distributions
>
>
> Key: LUCENE-5185
> URL: https://issues.apache.org/jira/browse/LUCENE-5185
> Project: Lucene - Core
> Issue Type: Improvement
> Components: general/build
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Minor
> Fix For: 5.0, 4.5
>
>
> On LUCENE-3945, where external dependency checksum verification was put in
> place, [~hossman_luc...@fucit.org] wrote:
> bq. So i propose that we include checksum files in svn and in our source
> releases that can be used by users to verify that the jars they get from ivy
> match the jars we tested against.
> That is, checksum files in *binary* distributions was not part of the
> proposal.
> And [in his comment associated with the final
> patch|https://issues.apache.org/jira/browse/LUCENE-3945?focusedCommentId=13246476&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13246476]:
> bq. 2) fixes the binary releases to exlcude the sha1 files
> Somewhere between then and now, {{\*.jar.sha1}} files snuck back into the
> Lucene and Solr binary releases, under the {{licenses/}} directory. They
> should not be there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5184) CountFacetRequest does not seem to sum the totals of the subResult values.
[ https://issues.apache.org/jira/browse/LUCENE-5184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-5184: --- Attachment: LUCENE-5184.patch I modified the patch into a test case (attached). This is happening because of LUCENE-4715, where we changed the default OrdinalPolicy to ALL_BUT_DIMENSION; if you change the test to set the OrdinalPolicy to either NO_PARENTS or ALL_PARENTS then you should see count 1 for "book". > CountFacetRequest does not seem to sum the totals of the subResult values. > -- > > Key: LUCENE-5184 > URL: https://issues.apache.org/jira/browse/LUCENE-5184 > Project: Lucene - Core > Issue Type: Bug > Components: modules/facet >Affects Versions: 4.4 > Environment: Windows 7, Java 1.6 (64 bit), Eclipse >Reporter: Karl Nicholas > Labels: CountFacetRequest, facet > Attachments: FacetTest.java, LUCENE-5184.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > CountFacetRequest does not seem to sum the totals of the subResult values > when the query searches in a facet hierarchy. Seemed to be better behaved in > version 4.0, and changed when I updated to version 4.4, though I did have to > change code as well. I am using facets to create a hierarchy. Will attempt to > upload sample code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-5181) Core admin CREATE with missing solrconfig file in 4.5 gives unhelpful too-terse message compared to 4.4
Jack Krupansky created SOLR-5181:
Summary: Core admin CREATE with missing solrconfig file in 4.5
gives unhelpful too-terse message compared to 4.4
Key: SOLR-5181
URL: https://issues.apache.org/jira/browse/SOLR-5181
Project: Solr
Issue Type: Bug
Components: multicore
Affects Versions: 4.5
Reporter: Jack Krupansky
Priority: Minor
In branch_4x, a core admin CREATE request with a missing solrconfig.xml file
gives a too-short message that does not indicate what the real problem was. In
4.4, the message included the cause.
The request:
{code}
curl
"http://localhost:8983/solr/admin/cores?action=CREATE&name=collection2&indent=true";
{code}
The response using a recent nightly build of 4.5:
{code}
400
35
Error CREATEing SolrCore 'collection2': Unable to create
core: collection2
400
{code}
But that doesn't indicate any reason for why the core could not be created.
The response using 4.4:
{code}
400
16
Error CREATEing SolrCore 'collection2': Unable to create
core: collection2 Caused by: Can't find resource 'solrconfig.xml' in classpath
or 'solr\collection2\conf/',
cwd=C:\cygwin\home\projects\solr-4.4.0\solr-4.4.0\example-test
400
{code}
That tells me exactly why.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5185) licenses/*.jar.sha1 don't belong in Lucene and Solr binary distributions
[
https://issues.apache.org/jira/browse/LUCENE-5185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13747006#comment-13747006
]
Robert Muir commented on LUCENE-5185:
-
{quote}
Maybe the smoke tester could be modified to directly test the checksums against
the binary artifacts' 3rd party jars?
{quote}
+1
{quote}
My thinking here is that we should keep our distributions lean, including only
the things that must be there, and I don't think these checksum files qualify.
{quote}
Under this line of thinking: then we shouldnt do a binary release at all. But
if we are going to do one, one thats filled with over 100 third party
dependencies, then I think some "paperwork" to go along with those is totally
appropriate.
> licenses/*.jar.sha1 don't belong in Lucene and Solr binary distributions
>
>
> Key: LUCENE-5185
> URL: https://issues.apache.org/jira/browse/LUCENE-5185
> Project: Lucene - Core
> Issue Type: Improvement
> Components: general/build
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Minor
> Fix For: 5.0, 4.5
>
>
> On LUCENE-3945, where external dependency checksum verification was put in
> place, [~hossman_luc...@fucit.org] wrote:
> bq. So i propose that we include checksum files in svn and in our source
> releases that can be used by users to verify that the jars they get from ivy
> match the jars we tested against.
> That is, checksum files in *binary* distributions was not part of the
> proposal.
> And [in his comment associated with the final
> patch|https://issues.apache.org/jira/browse/LUCENE-3945?focusedCommentId=13246476&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13246476]:
> bq. 2) fixes the binary releases to exlcude the sha1 files
> Somewhere between then and now, {{\*.jar.sha1}} files snuck back into the
> Lucene and Solr binary releases, under the {{licenses/}} directory. They
> should not be there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (LUCENE-5123) invert the codec postings API
[ https://issues.apache.org/jira/browse/LUCENE-5123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless reassigned LUCENE-5123: -- Assignee: Michael McCandless > invert the codec postings API > - > > Key: LUCENE-5123 > URL: https://issues.apache.org/jira/browse/LUCENE-5123 > Project: Lucene - Core > Issue Type: Wish >Reporter: Robert Muir >Assignee: Michael McCandless > > Currently FieldsConsumer/PostingsConsumer/etc is a "push" oriented api, e.g. > FreqProxTermsWriter streams the postings at flush, and the default merge() > takes the incoming codec api and filters out deleted docs and "pushes" via > same api (but that can be overridden). > It could be cleaner if we allowed for a "pull" model instead (like > DocValues). For example, maybe FreqProxTermsWriter could expose a Terms of > itself and just passed this to the codec consumer. > This would give the codec more flexibility to e.g. do multiple passes if it > wanted to do things like encode high-frequency terms more efficiently with a > bitset-like encoding or other things... > A codec can try to do things like this to some extent today, but its very > difficult (look at buffering in Pulsing). We made this change with DV and it > made a lot of interesting optimizations easy to implement... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5123) invert the codec postings API
[ https://issues.apache.org/jira/browse/LUCENE-5123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-5123: --- Attachment: LUCENE-5123.patch Here's a starting patch with lots of nocommits, but tests are passing ... This patch adds a postings (Fields) impl during flush (FreqProxFields), reading the postings from FreqProxTermWriter's RAM buffers, and another for merging (MappedMultiFields). It also adds a new PostingsFormat.write to write the postings ... the idea is this will eventually replace PostingsFormat.fieldsConsumer, but for now, so we can cutover existing postings formats iteratively, I made a default impl for write that takes a Fields and steps through Fields/Terms/Postings for the FieldsConsumer API. > invert the codec postings API > - > > Key: LUCENE-5123 > URL: https://issues.apache.org/jira/browse/LUCENE-5123 > Project: Lucene - Core > Issue Type: Wish >Reporter: Robert Muir >Assignee: Michael McCandless > Attachments: LUCENE-5123.patch > > > Currently FieldsConsumer/PostingsConsumer/etc is a "push" oriented api, e.g. > FreqProxTermsWriter streams the postings at flush, and the default merge() > takes the incoming codec api and filters out deleted docs and "pushes" via > same api (but that can be overridden). > It could be cleaner if we allowed for a "pull" model instead (like > DocValues). For example, maybe FreqProxTermsWriter could expose a Terms of > itself and just passed this to the codec consumer. > This would give the codec more flexibility to e.g. do multiple passes if it > wanted to do things like encode high-frequency terms more efficiently with a > bitset-like encoding or other things... > A codec can try to do things like this to some extent today, but its very > difficult (look at buffering in Pulsing). We made this change with DV and it > made a lot of interesting optimizations easy to implement... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-5186) Add CachingWrapperFilter.getFilter()
Trejkaz created LUCENE-5186: --- Summary: Add CachingWrapperFilter.getFilter() Key: LUCENE-5186 URL: https://issues.apache.org/jira/browse/LUCENE-5186 Project: Lucene - Core Issue Type: Improvement Components: core/search Reporter: Trejkaz Priority: Minor There are a couple of use cases I can think of where being able to get the underlying filter out of CachingWrapperFilter would be useful: 1. You might want to introspect the filter to figure out what's in it (the use case we hit.) 2. You might want to serialise the filter since Lucene no longer supports that itself. We currently work around this by subclassing, keeping another copy of the underlying filter reference and implementing a trivial getter, which is an easy workaround, but the trap is that a junior developer could unknowingly create a CachingWrapperFilter without knowing that the BetterCachingWrapperFilter exists, introducing a filter which cannot be introspected. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5123) invert the codec postings API
[ https://issues.apache.org/jira/browse/LUCENE-5123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13747034#comment-13747034 ] Robert Muir commented on LUCENE-5123: - This is exciting! One idea is to just keep the old API (at least for now)? Then, we dont have to cutover tons of code at once and we just have a new low level api (and back compat by accident). I think it would be good if we wrote or converted a 'demo' codec (simpletext is ok for example, or a new simple one) to the new api first, just to see if we are happy with it. Like maybe its just fine that if you are implementing the new API you have to compute the stats in your codec yourself, maybe its simple, or maybe we just plan on keeping the higher level API and not deprecating it. > invert the codec postings API > - > > Key: LUCENE-5123 > URL: https://issues.apache.org/jira/browse/LUCENE-5123 > Project: Lucene - Core > Issue Type: Wish >Reporter: Robert Muir >Assignee: Michael McCandless > Attachments: LUCENE-5123.patch > > > Currently FieldsConsumer/PostingsConsumer/etc is a "push" oriented api, e.g. > FreqProxTermsWriter streams the postings at flush, and the default merge() > takes the incoming codec api and filters out deleted docs and "pushes" via > same api (but that can be overridden). > It could be cleaner if we allowed for a "pull" model instead (like > DocValues). For example, maybe FreqProxTermsWriter could expose a Terms of > itself and just passed this to the codec consumer. > This would give the codec more flexibility to e.g. do multiple passes if it > wanted to do things like encode high-frequency terms more efficiently with a > bitset-like encoding or other things... > A codec can try to do things like this to some extent today, but its very > difficult (look at buffering in Pulsing). We made this change with DV and it > made a lot of interesting optimizations easy to implement... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.8.0-ea-b102) - Build # 3176 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/3176/ Java: 64bit/jdk1.8.0-ea-b102 -XX:+UseCompressedOops -XX:+UseG1GC All tests passed Build Log: [...truncated 10161 lines...] BUILD FAILED C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:389: The following error occurred while executing this line: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:369: The following error occurred while executing this line: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:39: The following error occurred while executing this line: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build.xml:551: The following error occurred while executing this line: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1885: The following error occurred while executing this line: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\analysis\build.xml:99: The following error occurred while executing this line: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\analysis\build.xml:38: The following error occurred while executing this line: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:357: impossible to resolve dependencies: resolve failed - see output for details Total time: 11 minutes 35 seconds Build step 'Invoke Ant' marked build as failure Description set: Java: 64bit/jdk1.8.0-ea-b102 -XX:+UseCompressedOops -XX:+UseG1GC Archiving artifacts Recording test results Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5184) CountFacetRequest does not seem to sum the totals of the subResult values.
[ https://issues.apache.org/jira/browse/LUCENE-5184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13747057#comment-13747057 ] Karl Nicholas commented on LUCENE-5184: --- Well, this manages to solve the issue I was experiencing. This may be difficult to figure out without clear documentation, but of course that's a different problem. Thanks. Karl. > CountFacetRequest does not seem to sum the totals of the subResult values. > -- > > Key: LUCENE-5184 > URL: https://issues.apache.org/jira/browse/LUCENE-5184 > Project: Lucene - Core > Issue Type: Bug > Components: modules/facet >Affects Versions: 4.4 > Environment: Windows 7, Java 1.6 (64 bit), Eclipse >Reporter: Karl Nicholas > Labels: CountFacetRequest, facet > Attachments: FacetTest.java, LUCENE-5184.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > CountFacetRequest does not seem to sum the totals of the subResult values > when the query searches in a facet hierarchy. Seemed to be better behaved in > version 4.0, and changed when I updated to version 4.4, though I did have to > change code as well. I am using facets to create a hierarchy. Will attempt to > upload sample code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4787) Join Contrib
[
https://issues.apache.org/jira/browse/SOLR-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13747147#comment-13747147
]
David Smiley commented on SOLR-4787:
Excellent suggestion [~kranti2006] on supporting filter queries on the
from-side of the join. I implemented a custom join query and it has a
from-side filter query feature. It can help a great deal with performance. It
wasn't that hard to add, either.
> Join Contrib
>
>
> Key: SOLR-4787
> URL: https://issues.apache.org/jira/browse/SOLR-4787
> Project: Solr
> Issue Type: New Feature
> Components: search
>Affects Versions: 4.2.1
>Reporter: Joel Bernstein
>Priority: Minor
> Fix For: 4.5, 5.0
>
> Attachments: SOLR-4787-deadlock-fix.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787-pjoin-long-keys.patch
>
>
> This contrib provides a place where different join implementations can be
> contributed to Solr. This contrib currently includes 2 join implementations.
> The initial patch was generated from the Solr 4.3 tag. Because of changes in
> the FieldCache API this patch will only build with Solr 4.2 or above.
> *PostFilterJoinQParserPlugin aka "pjoin"*
> The pjoin provides a join implementation that filters results in one core
> based on the results of a search in another core. This is similar in
> functionality to the JoinQParserPlugin but the implementation differs in a
> couple of important ways.
> The first way is that the pjoin is designed to work with integer join keys
> only. So, in order to use pjoin, integer join keys must be included in both
> the to and from core.
> The second difference is that the pjoin builds memory structures that are
> used to quickly connect the join keys. It also uses a custom SolrCache named
> "join" to hold intermediate DocSets which are needed to build the join memory
> structures. So, the pjoin will need more memory then the JoinQParserPlugin to
> perform the join.
> The main advantage of the pjoin is that it can scale to join millions of keys
> between cores.
> Because it's a PostFilter, it only needs to join records that match the main
> query.
> The syntax of the pjoin is the same as the JoinQParserPlugin except that the
> plugin is referenced by the string "pjoin" rather then "join".
> fq=\{!pjoin fromCore=collection2 from=id_i to=id_i\}user:customer1
> The example filter query above will search the fromCore (collection2) for
> "user:customer1". This query will generate a list of values from the "from"
> field that will be used to filter the main query. Only records from the main
> query, where the "to" field is present in the "from" list will be included in
> the results.
> The solrconfig.xml in the main query core must contain the reference to the
> pjoin.
> class="org.apache.solr.joins.PostFilterJoinQParserPlugin"/>
> And the join contrib jars must be registed in the solrconfig.xml.
>
> The solrconfig.xml in the fromcore must have the "join" SolrCache configured.
> class="solr.LRUCache"
> size="4096"
> initialSize="1024"
> />
> *ValueSourceJoinParserPlugin aka vjoin*
> The second implementation is the ValueSourceJoinParserPlugin aka "vjoin".
> This implements a ValueSource function query that can return a value from a
> second core based on join keys and limiting query. The limiting query can be
> used to select a specific subset of data from the join core. This allows
> customer specific relevance data to be stored in a separate core and then
> joined in the main query.
> The vjoin is called using the "vjoin" function query. For example:
> bf=vjoin(fromCore, fromKey, fromVal, toKey, query)
> This example shows "vjoin" being called by the edismax boost function
> parameter. This example will return the "fromVal" from the "fromCore". The
> "fromKey" and "toKey" are used to link the records from the main query to the
> records in the "fromCore". The "query" is used to select a specific set of
> records to join with in fromCore.
> Currently the fromKey and toKey must be longs but this will change in future
> versions. Like the pjoin, the "join" SolrCache is used to hold the join
> memory structures.
> To configure the vjoin you must register the ValueSource plugin in the
> solrconfig.xml as follows:
> class="org.apache.solr.joins.ValueSourceJoinParserPlugin" />
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e
[jira] [Comment Edited] (SOLR-4787) Join Contrib
[
https://issues.apache.org/jira/browse/SOLR-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13747147#comment-13747147
]
David Smiley edited comment on SOLR-4787 at 8/22/13 1:53 AM:
-
Excellent suggestion [~krantiparisa] on supporting filter queries on the
from-side of the join. I implemented a custom join query and it has a
from-side filter query feature. It can help a great deal with performance. It
wasn't that hard to add, either.
was (Author: dsmiley):
Excellent suggestion [~kranti2006] on supporting filter queries on the
from-side of the join. I implemented a custom join query and it has a
from-side filter query feature. It can help a great deal with performance. It
wasn't that hard to add, either.
> Join Contrib
>
>
> Key: SOLR-4787
> URL: https://issues.apache.org/jira/browse/SOLR-4787
> Project: Solr
> Issue Type: New Feature
> Components: search
>Affects Versions: 4.2.1
>Reporter: Joel Bernstein
>Priority: Minor
> Fix For: 4.5, 5.0
>
> Attachments: SOLR-4787-deadlock-fix.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787-pjoin-long-keys.patch
>
>
> This contrib provides a place where different join implementations can be
> contributed to Solr. This contrib currently includes 2 join implementations.
> The initial patch was generated from the Solr 4.3 tag. Because of changes in
> the FieldCache API this patch will only build with Solr 4.2 or above.
> *PostFilterJoinQParserPlugin aka "pjoin"*
> The pjoin provides a join implementation that filters results in one core
> based on the results of a search in another core. This is similar in
> functionality to the JoinQParserPlugin but the implementation differs in a
> couple of important ways.
> The first way is that the pjoin is designed to work with integer join keys
> only. So, in order to use pjoin, integer join keys must be included in both
> the to and from core.
> The second difference is that the pjoin builds memory structures that are
> used to quickly connect the join keys. It also uses a custom SolrCache named
> "join" to hold intermediate DocSets which are needed to build the join memory
> structures. So, the pjoin will need more memory then the JoinQParserPlugin to
> perform the join.
> The main advantage of the pjoin is that it can scale to join millions of keys
> between cores.
> Because it's a PostFilter, it only needs to join records that match the main
> query.
> The syntax of the pjoin is the same as the JoinQParserPlugin except that the
> plugin is referenced by the string "pjoin" rather then "join".
> fq=\{!pjoin fromCore=collection2 from=id_i to=id_i\}user:customer1
> The example filter query above will search the fromCore (collection2) for
> "user:customer1". This query will generate a list of values from the "from"
> field that will be used to filter the main query. Only records from the main
> query, where the "to" field is present in the "from" list will be included in
> the results.
> The solrconfig.xml in the main query core must contain the reference to the
> pjoin.
> class="org.apache.solr.joins.PostFilterJoinQParserPlugin"/>
> And the join contrib jars must be registed in the solrconfig.xml.
>
> The solrconfig.xml in the fromcore must have the "join" SolrCache configured.
> class="solr.LRUCache"
> size="4096"
> initialSize="1024"
> />
> *ValueSourceJoinParserPlugin aka vjoin*
> The second implementation is the ValueSourceJoinParserPlugin aka "vjoin".
> This implements a ValueSource function query that can return a value from a
> second core based on join keys and limiting query. The limiting query can be
> used to select a specific subset of data from the join core. This allows
> customer specific relevance data to be stored in a separate core and then
> joined in the main query.
> The vjoin is called using the "vjoin" function query. For example:
> bf=vjoin(fromCore, fromKey, fromVal, toKey, query)
> This example shows "vjoin" being called by the edismax boost function
> parameter. This example will return the "fromVal" from the "fromCore". The
> "fromKey" and "toKey" are used to link the records from the main query to the
> records in the "fromCore". The "query" is used to select a specific set of
> records to join with in fromCore.
> Currently the fromKey and toKey must be longs but this will change in future
> versions. Like the pjoin, the "join" SolrCache is used to hold the join
> memory structures.
> To configure the vjoin you must register the ValueSource plugin in the
> solrconfig.xml as follo
[jira] [Commented] (SOLR-4787) Join Contrib
[
https://issues.apache.org/jira/browse/SOLR-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13747155#comment-13747155
]
Kranti Parisa commented on SOLR-4787:
-
That's awesome! Yes, for sure it would be a big deal for performance especially
it will allow us to cache the majority of the join query which could be common
for all the cases. Once it's filter cached, we can run additional clases thru
normal queries super fast!
I was thinking to add a new local-param to support "fq". If you already have
something, do you want to share?
> Join Contrib
>
>
> Key: SOLR-4787
> URL: https://issues.apache.org/jira/browse/SOLR-4787
> Project: Solr
> Issue Type: New Feature
> Components: search
>Affects Versions: 4.2.1
>Reporter: Joel Bernstein
>Priority: Minor
> Fix For: 4.5, 5.0
>
> Attachments: SOLR-4787-deadlock-fix.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787-pjoin-long-keys.patch
>
>
> This contrib provides a place where different join implementations can be
> contributed to Solr. This contrib currently includes 2 join implementations.
> The initial patch was generated from the Solr 4.3 tag. Because of changes in
> the FieldCache API this patch will only build with Solr 4.2 or above.
> *PostFilterJoinQParserPlugin aka "pjoin"*
> The pjoin provides a join implementation that filters results in one core
> based on the results of a search in another core. This is similar in
> functionality to the JoinQParserPlugin but the implementation differs in a
> couple of important ways.
> The first way is that the pjoin is designed to work with integer join keys
> only. So, in order to use pjoin, integer join keys must be included in both
> the to and from core.
> The second difference is that the pjoin builds memory structures that are
> used to quickly connect the join keys. It also uses a custom SolrCache named
> "join" to hold intermediate DocSets which are needed to build the join memory
> structures. So, the pjoin will need more memory then the JoinQParserPlugin to
> perform the join.
> The main advantage of the pjoin is that it can scale to join millions of keys
> between cores.
> Because it's a PostFilter, it only needs to join records that match the main
> query.
> The syntax of the pjoin is the same as the JoinQParserPlugin except that the
> plugin is referenced by the string "pjoin" rather then "join".
> fq=\{!pjoin fromCore=collection2 from=id_i to=id_i\}user:customer1
> The example filter query above will search the fromCore (collection2) for
> "user:customer1". This query will generate a list of values from the "from"
> field that will be used to filter the main query. Only records from the main
> query, where the "to" field is present in the "from" list will be included in
> the results.
> The solrconfig.xml in the main query core must contain the reference to the
> pjoin.
> class="org.apache.solr.joins.PostFilterJoinQParserPlugin"/>
> And the join contrib jars must be registed in the solrconfig.xml.
>
> The solrconfig.xml in the fromcore must have the "join" SolrCache configured.
> class="solr.LRUCache"
> size="4096"
> initialSize="1024"
> />
> *ValueSourceJoinParserPlugin aka vjoin*
> The second implementation is the ValueSourceJoinParserPlugin aka "vjoin".
> This implements a ValueSource function query that can return a value from a
> second core based on join keys and limiting query. The limiting query can be
> used to select a specific subset of data from the join core. This allows
> customer specific relevance data to be stored in a separate core and then
> joined in the main query.
> The vjoin is called using the "vjoin" function query. For example:
> bf=vjoin(fromCore, fromKey, fromVal, toKey, query)
> This example shows "vjoin" being called by the edismax boost function
> parameter. This example will return the "fromVal" from the "fromCore". The
> "fromKey" and "toKey" are used to link the records from the main query to the
> records in the "fromCore". The "query" is used to select a specific set of
> records to join with in fromCore.
> Currently the fromKey and toKey must be longs but this will change in future
> versions. Like the pjoin, the "join" SolrCache is used to hold the join
> memory structures.
> To configure the vjoin you must register the ValueSource plugin in the
> solrconfig.xml as follows:
> class="org.apache.solr.joins.ValueSourceJoinParserPlugin" />
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: htt
[jira] [Commented] (SOLR-4787) Join Contrib
[
https://issues.apache.org/jira/browse/SOLR-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13747166#comment-13747166
]
David Smiley commented on SOLR-4787:
I don't have the rights to it but I'll share the most pertinent line of code:
{code}
SolrIndexSearcher.ProcessedFilter processedFilter =
searcher.getProcessedFilter(null, filters);
{code}
See, Solr does most of the work with that one line call; everything else, such
as parsing the queries is easy/common stuff.
> Join Contrib
>
>
> Key: SOLR-4787
> URL: https://issues.apache.org/jira/browse/SOLR-4787
> Project: Solr
> Issue Type: New Feature
> Components: search
>Affects Versions: 4.2.1
>Reporter: Joel Bernstein
>Priority: Minor
> Fix For: 4.5, 5.0
>
> Attachments: SOLR-4787-deadlock-fix.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787-pjoin-long-keys.patch
>
>
> This contrib provides a place where different join implementations can be
> contributed to Solr. This contrib currently includes 2 join implementations.
> The initial patch was generated from the Solr 4.3 tag. Because of changes in
> the FieldCache API this patch will only build with Solr 4.2 or above.
> *PostFilterJoinQParserPlugin aka "pjoin"*
> The pjoin provides a join implementation that filters results in one core
> based on the results of a search in another core. This is similar in
> functionality to the JoinQParserPlugin but the implementation differs in a
> couple of important ways.
> The first way is that the pjoin is designed to work with integer join keys
> only. So, in order to use pjoin, integer join keys must be included in both
> the to and from core.
> The second difference is that the pjoin builds memory structures that are
> used to quickly connect the join keys. It also uses a custom SolrCache named
> "join" to hold intermediate DocSets which are needed to build the join memory
> structures. So, the pjoin will need more memory then the JoinQParserPlugin to
> perform the join.
> The main advantage of the pjoin is that it can scale to join millions of keys
> between cores.
> Because it's a PostFilter, it only needs to join records that match the main
> query.
> The syntax of the pjoin is the same as the JoinQParserPlugin except that the
> plugin is referenced by the string "pjoin" rather then "join".
> fq=\{!pjoin fromCore=collection2 from=id_i to=id_i\}user:customer1
> The example filter query above will search the fromCore (collection2) for
> "user:customer1". This query will generate a list of values from the "from"
> field that will be used to filter the main query. Only records from the main
> query, where the "to" field is present in the "from" list will be included in
> the results.
> The solrconfig.xml in the main query core must contain the reference to the
> pjoin.
> class="org.apache.solr.joins.PostFilterJoinQParserPlugin"/>
> And the join contrib jars must be registed in the solrconfig.xml.
>
> The solrconfig.xml in the fromcore must have the "join" SolrCache configured.
> class="solr.LRUCache"
> size="4096"
> initialSize="1024"
> />
> *ValueSourceJoinParserPlugin aka vjoin*
> The second implementation is the ValueSourceJoinParserPlugin aka "vjoin".
> This implements a ValueSource function query that can return a value from a
> second core based on join keys and limiting query. The limiting query can be
> used to select a specific subset of data from the join core. This allows
> customer specific relevance data to be stored in a separate core and then
> joined in the main query.
> The vjoin is called using the "vjoin" function query. For example:
> bf=vjoin(fromCore, fromKey, fromVal, toKey, query)
> This example shows "vjoin" being called by the edismax boost function
> parameter. This example will return the "fromVal" from the "fromCore". The
> "fromKey" and "toKey" are used to link the records from the main query to the
> records in the "fromCore". The "query" is used to select a specific set of
> records to join with in fromCore.
> Currently the fromKey and toKey must be longs but this will change in future
> versions. Like the pjoin, the "join" SolrCache is used to hold the join
> memory structures.
> To configure the vjoin you must register the ValueSource plugin in the
> solrconfig.xml as follows:
> class="org.apache.solr.joins.ValueSourceJoinParserPlugin" />
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---
[jira] [Created] (SOLR-5182) add regenerator for blockjoin cache
Robert Muir created SOLR-5182: - Summary: add regenerator for blockjoin cache Key: SOLR-5182 URL: https://issues.apache.org/jira/browse/SOLR-5182 Project: Solr Issue Type: Bug Reporter: Robert Muir The BlockJoin parsers cache by default with CachingWrapperFilter, but unless the user writes some code, the parent filter will be totally discarded every commit (losing all cached segments and behaving as if it were top-level). This defeats the point... we should provide a regenerator that just copies elements over for things like this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4787) Join Contrib
[
https://issues.apache.org/jira/browse/SOLR-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13747168#comment-13747168
]
Joel Bernstein commented on SOLR-4787:
--
I'll put up what I've been working on tomorrow. One of the joins I'll be adding
supports the "fq" on the from side of join. These joins also support both
PostFilter and traditional filter query joins.
> Join Contrib
>
>
> Key: SOLR-4787
> URL: https://issues.apache.org/jira/browse/SOLR-4787
> Project: Solr
> Issue Type: New Feature
> Components: search
>Affects Versions: 4.2.1
>Reporter: Joel Bernstein
>Priority: Minor
> Fix For: 4.5, 5.0
>
> Attachments: SOLR-4787-deadlock-fix.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787-pjoin-long-keys.patch
>
>
> This contrib provides a place where different join implementations can be
> contributed to Solr. This contrib currently includes 2 join implementations.
> The initial patch was generated from the Solr 4.3 tag. Because of changes in
> the FieldCache API this patch will only build with Solr 4.2 or above.
> *PostFilterJoinQParserPlugin aka "pjoin"*
> The pjoin provides a join implementation that filters results in one core
> based on the results of a search in another core. This is similar in
> functionality to the JoinQParserPlugin but the implementation differs in a
> couple of important ways.
> The first way is that the pjoin is designed to work with integer join keys
> only. So, in order to use pjoin, integer join keys must be included in both
> the to and from core.
> The second difference is that the pjoin builds memory structures that are
> used to quickly connect the join keys. It also uses a custom SolrCache named
> "join" to hold intermediate DocSets which are needed to build the join memory
> structures. So, the pjoin will need more memory then the JoinQParserPlugin to
> perform the join.
> The main advantage of the pjoin is that it can scale to join millions of keys
> between cores.
> Because it's a PostFilter, it only needs to join records that match the main
> query.
> The syntax of the pjoin is the same as the JoinQParserPlugin except that the
> plugin is referenced by the string "pjoin" rather then "join".
> fq=\{!pjoin fromCore=collection2 from=id_i to=id_i\}user:customer1
> The example filter query above will search the fromCore (collection2) for
> "user:customer1". This query will generate a list of values from the "from"
> field that will be used to filter the main query. Only records from the main
> query, where the "to" field is present in the "from" list will be included in
> the results.
> The solrconfig.xml in the main query core must contain the reference to the
> pjoin.
> class="org.apache.solr.joins.PostFilterJoinQParserPlugin"/>
> And the join contrib jars must be registed in the solrconfig.xml.
>
> The solrconfig.xml in the fromcore must have the "join" SolrCache configured.
> class="solr.LRUCache"
> size="4096"
> initialSize="1024"
> />
> *ValueSourceJoinParserPlugin aka vjoin*
> The second implementation is the ValueSourceJoinParserPlugin aka "vjoin".
> This implements a ValueSource function query that can return a value from a
> second core based on join keys and limiting query. The limiting query can be
> used to select a specific subset of data from the join core. This allows
> customer specific relevance data to be stored in a separate core and then
> joined in the main query.
> The vjoin is called using the "vjoin" function query. For example:
> bf=vjoin(fromCore, fromKey, fromVal, toKey, query)
> This example shows "vjoin" being called by the edismax boost function
> parameter. This example will return the "fromVal" from the "fromCore". The
> "fromKey" and "toKey" are used to link the records from the main query to the
> records in the "fromCore". The "query" is used to select a specific set of
> records to join with in fromCore.
> Currently the fromKey and toKey must be longs but this will change in future
> versions. Like the pjoin, the "join" SolrCache is used to hold the join
> memory structures.
> To configure the vjoin you must register the ValueSource plugin in the
> solrconfig.xml as follows:
> class="org.apache.solr.joins.ValueSourceJoinParserPlugin" />
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For addit
[jira] [Updated] (SOLR-5182) add regenerator for blockjoin cache
[ https://issues.apache.org/jira/browse/SOLR-5182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir updated SOLR-5182: -- Attachment: SOLR-5182.patch patch > add regenerator for blockjoin cache > --- > > Key: SOLR-5182 > URL: https://issues.apache.org/jira/browse/SOLR-5182 > Project: Solr > Issue Type: Bug >Reporter: Robert Muir > Attachments: SOLR-5182.patch > > > The BlockJoin parsers cache by default with CachingWrapperFilter, but unless > the user writes some code, the parent filter will be totally discarded every > commit (losing all cached segments and behaving as if it were top-level). > This defeats the point... we should provide a regenerator that just copies > elements over for things like this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4787) Join Contrib
[
https://issues.apache.org/jira/browse/SOLR-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13747170#comment-13747170
]
Joel Bernstein commented on SOLR-4787:
--
Yeah, that's the exact approach I took David.
> Join Contrib
>
>
> Key: SOLR-4787
> URL: https://issues.apache.org/jira/browse/SOLR-4787
> Project: Solr
> Issue Type: New Feature
> Components: search
>Affects Versions: 4.2.1
>Reporter: Joel Bernstein
>Priority: Minor
> Fix For: 4.5, 5.0
>
> Attachments: SOLR-4787-deadlock-fix.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787-pjoin-long-keys.patch
>
>
> This contrib provides a place where different join implementations can be
> contributed to Solr. This contrib currently includes 2 join implementations.
> The initial patch was generated from the Solr 4.3 tag. Because of changes in
> the FieldCache API this patch will only build with Solr 4.2 or above.
> *PostFilterJoinQParserPlugin aka "pjoin"*
> The pjoin provides a join implementation that filters results in one core
> based on the results of a search in another core. This is similar in
> functionality to the JoinQParserPlugin but the implementation differs in a
> couple of important ways.
> The first way is that the pjoin is designed to work with integer join keys
> only. So, in order to use pjoin, integer join keys must be included in both
> the to and from core.
> The second difference is that the pjoin builds memory structures that are
> used to quickly connect the join keys. It also uses a custom SolrCache named
> "join" to hold intermediate DocSets which are needed to build the join memory
> structures. So, the pjoin will need more memory then the JoinQParserPlugin to
> perform the join.
> The main advantage of the pjoin is that it can scale to join millions of keys
> between cores.
> Because it's a PostFilter, it only needs to join records that match the main
> query.
> The syntax of the pjoin is the same as the JoinQParserPlugin except that the
> plugin is referenced by the string "pjoin" rather then "join".
> fq=\{!pjoin fromCore=collection2 from=id_i to=id_i\}user:customer1
> The example filter query above will search the fromCore (collection2) for
> "user:customer1". This query will generate a list of values from the "from"
> field that will be used to filter the main query. Only records from the main
> query, where the "to" field is present in the "from" list will be included in
> the results.
> The solrconfig.xml in the main query core must contain the reference to the
> pjoin.
> class="org.apache.solr.joins.PostFilterJoinQParserPlugin"/>
> And the join contrib jars must be registed in the solrconfig.xml.
>
> The solrconfig.xml in the fromcore must have the "join" SolrCache configured.
> class="solr.LRUCache"
> size="4096"
> initialSize="1024"
> />
> *ValueSourceJoinParserPlugin aka vjoin*
> The second implementation is the ValueSourceJoinParserPlugin aka "vjoin".
> This implements a ValueSource function query that can return a value from a
> second core based on join keys and limiting query. The limiting query can be
> used to select a specific subset of data from the join core. This allows
> customer specific relevance data to be stored in a separate core and then
> joined in the main query.
> The vjoin is called using the "vjoin" function query. For example:
> bf=vjoin(fromCore, fromKey, fromVal, toKey, query)
> This example shows "vjoin" being called by the edismax boost function
> parameter. This example will return the "fromVal" from the "fromCore". The
> "fromKey" and "toKey" are used to link the records from the main query to the
> records in the "fromCore". The "query" is used to select a specific set of
> records to join with in fromCore.
> Currently the fromKey and toKey must be longs but this will change in future
> versions. Like the pjoin, the "join" SolrCache is used to hold the join
> memory structures.
> To configure the vjoin you must register the ValueSource plugin in the
> solrconfig.xml as follows:
> class="org.apache.solr.joins.ValueSourceJoinParserPlugin" />
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-4787) Join Contrib
[
https://issues.apache.org/jira/browse/SOLR-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13747170#comment-13747170
]
Joel Bernstein edited comment on SOLR-4787 at 8/22/13 2:33 AM:
---
Yeah, that's the exact approach I took David.
This was added to support nested joins but I see how the caching could really
speed up the whole join.
was (Author: joel.bernstein):
Yeah, that's the exact approach I took David.
> Join Contrib
>
>
> Key: SOLR-4787
> URL: https://issues.apache.org/jira/browse/SOLR-4787
> Project: Solr
> Issue Type: New Feature
> Components: search
>Affects Versions: 4.2.1
>Reporter: Joel Bernstein
>Priority: Minor
> Fix For: 4.5, 5.0
>
> Attachments: SOLR-4787-deadlock-fix.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787-pjoin-long-keys.patch
>
>
> This contrib provides a place where different join implementations can be
> contributed to Solr. This contrib currently includes 2 join implementations.
> The initial patch was generated from the Solr 4.3 tag. Because of changes in
> the FieldCache API this patch will only build with Solr 4.2 or above.
> *PostFilterJoinQParserPlugin aka "pjoin"*
> The pjoin provides a join implementation that filters results in one core
> based on the results of a search in another core. This is similar in
> functionality to the JoinQParserPlugin but the implementation differs in a
> couple of important ways.
> The first way is that the pjoin is designed to work with integer join keys
> only. So, in order to use pjoin, integer join keys must be included in both
> the to and from core.
> The second difference is that the pjoin builds memory structures that are
> used to quickly connect the join keys. It also uses a custom SolrCache named
> "join" to hold intermediate DocSets which are needed to build the join memory
> structures. So, the pjoin will need more memory then the JoinQParserPlugin to
> perform the join.
> The main advantage of the pjoin is that it can scale to join millions of keys
> between cores.
> Because it's a PostFilter, it only needs to join records that match the main
> query.
> The syntax of the pjoin is the same as the JoinQParserPlugin except that the
> plugin is referenced by the string "pjoin" rather then "join".
> fq=\{!pjoin fromCore=collection2 from=id_i to=id_i\}user:customer1
> The example filter query above will search the fromCore (collection2) for
> "user:customer1". This query will generate a list of values from the "from"
> field that will be used to filter the main query. Only records from the main
> query, where the "to" field is present in the "from" list will be included in
> the results.
> The solrconfig.xml in the main query core must contain the reference to the
> pjoin.
> class="org.apache.solr.joins.PostFilterJoinQParserPlugin"/>
> And the join contrib jars must be registed in the solrconfig.xml.
>
> The solrconfig.xml in the fromcore must have the "join" SolrCache configured.
> class="solr.LRUCache"
> size="4096"
> initialSize="1024"
> />
> *ValueSourceJoinParserPlugin aka vjoin*
> The second implementation is the ValueSourceJoinParserPlugin aka "vjoin".
> This implements a ValueSource function query that can return a value from a
> second core based on join keys and limiting query. The limiting query can be
> used to select a specific subset of data from the join core. This allows
> customer specific relevance data to be stored in a separate core and then
> joined in the main query.
> The vjoin is called using the "vjoin" function query. For example:
> bf=vjoin(fromCore, fromKey, fromVal, toKey, query)
> This example shows "vjoin" being called by the edismax boost function
> parameter. This example will return the "fromVal" from the "fromCore". The
> "fromKey" and "toKey" are used to link the records from the main query to the
> records in the "fromCore". The "query" is used to select a specific set of
> records to join with in fromCore.
> Currently the fromKey and toKey must be longs but this will change in future
> versions. Like the pjoin, the "join" SolrCache is used to hold the join
> memory structures.
> To configure the vjoin you must register the ValueSource plugin in the
> solrconfig.xml as follows:
> class="org.apache.solr.joins.ValueSourceJoinParserPlugin" />
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SOLR-4787) Join Contrib
[
https://issues.apache.org/jira/browse/SOLR-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13747172#comment-13747172
]
Kranti Parisa commented on SOLR-4787:
-
Great, thanks!
> Join Contrib
>
>
> Key: SOLR-4787
> URL: https://issues.apache.org/jira/browse/SOLR-4787
> Project: Solr
> Issue Type: New Feature
> Components: search
>Affects Versions: 4.2.1
>Reporter: Joel Bernstein
>Priority: Minor
> Fix For: 4.5, 5.0
>
> Attachments: SOLR-4787-deadlock-fix.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787-pjoin-long-keys.patch
>
>
> This contrib provides a place where different join implementations can be
> contributed to Solr. This contrib currently includes 2 join implementations.
> The initial patch was generated from the Solr 4.3 tag. Because of changes in
> the FieldCache API this patch will only build with Solr 4.2 or above.
> *PostFilterJoinQParserPlugin aka "pjoin"*
> The pjoin provides a join implementation that filters results in one core
> based on the results of a search in another core. This is similar in
> functionality to the JoinQParserPlugin but the implementation differs in a
> couple of important ways.
> The first way is that the pjoin is designed to work with integer join keys
> only. So, in order to use pjoin, integer join keys must be included in both
> the to and from core.
> The second difference is that the pjoin builds memory structures that are
> used to quickly connect the join keys. It also uses a custom SolrCache named
> "join" to hold intermediate DocSets which are needed to build the join memory
> structures. So, the pjoin will need more memory then the JoinQParserPlugin to
> perform the join.
> The main advantage of the pjoin is that it can scale to join millions of keys
> between cores.
> Because it's a PostFilter, it only needs to join records that match the main
> query.
> The syntax of the pjoin is the same as the JoinQParserPlugin except that the
> plugin is referenced by the string "pjoin" rather then "join".
> fq=\{!pjoin fromCore=collection2 from=id_i to=id_i\}user:customer1
> The example filter query above will search the fromCore (collection2) for
> "user:customer1". This query will generate a list of values from the "from"
> field that will be used to filter the main query. Only records from the main
> query, where the "to" field is present in the "from" list will be included in
> the results.
> The solrconfig.xml in the main query core must contain the reference to the
> pjoin.
> class="org.apache.solr.joins.PostFilterJoinQParserPlugin"/>
> And the join contrib jars must be registed in the solrconfig.xml.
>
> The solrconfig.xml in the fromcore must have the "join" SolrCache configured.
> class="solr.LRUCache"
> size="4096"
> initialSize="1024"
> />
> *ValueSourceJoinParserPlugin aka vjoin*
> The second implementation is the ValueSourceJoinParserPlugin aka "vjoin".
> This implements a ValueSource function query that can return a value from a
> second core based on join keys and limiting query. The limiting query can be
> used to select a specific subset of data from the join core. This allows
> customer specific relevance data to be stored in a separate core and then
> joined in the main query.
> The vjoin is called using the "vjoin" function query. For example:
> bf=vjoin(fromCore, fromKey, fromVal, toKey, query)
> This example shows "vjoin" being called by the edismax boost function
> parameter. This example will return the "fromVal" from the "fromCore". The
> "fromKey" and "toKey" are used to link the records from the main query to the
> records in the "fromCore". The "query" is used to select a specific set of
> records to join with in fromCore.
> Currently the fromKey and toKey must be longs but this will change in future
> versions. Like the pjoin, the "join" SolrCache is used to hold the join
> memory structures.
> To configure the vjoin you must register the ValueSource plugin in the
> solrconfig.xml as follows:
> class="org.apache.solr.joins.ValueSourceJoinParserPlugin" />
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4787) Join Contrib
[
https://issues.apache.org/jira/browse/SOLR-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13747175#comment-13747175
]
Kranti Parisa commented on SOLR-4787:
-
Nested Joins! That's exactly what I am trying :) and thought about adding fq to
the solr joins.
Using local-param in the first join: "v=$joinQ"&joinQ=(field:123 AND
_query_={another join}). So here "another join" could be passed as a FQ and it
should get results faster!!
> Join Contrib
>
>
> Key: SOLR-4787
> URL: https://issues.apache.org/jira/browse/SOLR-4787
> Project: Solr
> Issue Type: New Feature
> Components: search
>Affects Versions: 4.2.1
>Reporter: Joel Bernstein
>Priority: Minor
> Fix For: 4.5, 5.0
>
> Attachments: SOLR-4787-deadlock-fix.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787-pjoin-long-keys.patch
>
>
> This contrib provides a place where different join implementations can be
> contributed to Solr. This contrib currently includes 2 join implementations.
> The initial patch was generated from the Solr 4.3 tag. Because of changes in
> the FieldCache API this patch will only build with Solr 4.2 or above.
> *PostFilterJoinQParserPlugin aka "pjoin"*
> The pjoin provides a join implementation that filters results in one core
> based on the results of a search in another core. This is similar in
> functionality to the JoinQParserPlugin but the implementation differs in a
> couple of important ways.
> The first way is that the pjoin is designed to work with integer join keys
> only. So, in order to use pjoin, integer join keys must be included in both
> the to and from core.
> The second difference is that the pjoin builds memory structures that are
> used to quickly connect the join keys. It also uses a custom SolrCache named
> "join" to hold intermediate DocSets which are needed to build the join memory
> structures. So, the pjoin will need more memory then the JoinQParserPlugin to
> perform the join.
> The main advantage of the pjoin is that it can scale to join millions of keys
> between cores.
> Because it's a PostFilter, it only needs to join records that match the main
> query.
> The syntax of the pjoin is the same as the JoinQParserPlugin except that the
> plugin is referenced by the string "pjoin" rather then "join".
> fq=\{!pjoin fromCore=collection2 from=id_i to=id_i\}user:customer1
> The example filter query above will search the fromCore (collection2) for
> "user:customer1". This query will generate a list of values from the "from"
> field that will be used to filter the main query. Only records from the main
> query, where the "to" field is present in the "from" list will be included in
> the results.
> The solrconfig.xml in the main query core must contain the reference to the
> pjoin.
> class="org.apache.solr.joins.PostFilterJoinQParserPlugin"/>
> And the join contrib jars must be registed in the solrconfig.xml.
>
> The solrconfig.xml in the fromcore must have the "join" SolrCache configured.
> class="solr.LRUCache"
> size="4096"
> initialSize="1024"
> />
> *ValueSourceJoinParserPlugin aka vjoin*
> The second implementation is the ValueSourceJoinParserPlugin aka "vjoin".
> This implements a ValueSource function query that can return a value from a
> second core based on join keys and limiting query. The limiting query can be
> used to select a specific subset of data from the join core. This allows
> customer specific relevance data to be stored in a separate core and then
> joined in the main query.
> The vjoin is called using the "vjoin" function query. For example:
> bf=vjoin(fromCore, fromKey, fromVal, toKey, query)
> This example shows "vjoin" being called by the edismax boost function
> parameter. This example will return the "fromVal" from the "fromCore". The
> "fromKey" and "toKey" are used to link the records from the main query to the
> records in the "fromCore". The "query" is used to select a specific set of
> records to join with in fromCore.
> Currently the fromKey and toKey must be longs but this will change in future
> versions. Like the pjoin, the "join" SolrCache is used to hold the join
> memory structures.
> To configure the vjoin you must register the ValueSource plugin in the
> solrconfig.xml as follows:
> class="org.apache.solr.joins.ValueSourceJoinParserPlugin" />
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---
[jira] [Commented] (LUCENE-3069) Lucene should have an entirely memory resident term dictionary
[ https://issues.apache.org/jira/browse/LUCENE-3069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13747180#comment-13747180 ] ASF subversion and git services commented on LUCENE-3069: - Commit 1516365 from [~billy] in branch 'dev/branches/lucene3069' [ https://svn.apache.org/r1516365 ] LUCENE-3069: API refactoring on Sep/IntBlock PF > Lucene should have an entirely memory resident term dictionary > -- > > Key: LUCENE-3069 > URL: https://issues.apache.org/jira/browse/LUCENE-3069 > Project: Lucene - Core > Issue Type: Improvement > Components: core/index, core/search >Affects Versions: 4.0-ALPHA >Reporter: Simon Willnauer >Assignee: Han Jiang > Labels: gsoc2013 > Fix For: 5.0, 4.5 > > Attachments: df-ttf-estimate.txt, example.png, LUCENE-3069.patch, > LUCENE-3069.patch, LUCENE-3069.patch, LUCENE-3069.patch, LUCENE-3069.patch, > LUCENE-3069.patch, LUCENE-3069.patch, LUCENE-3069.patch, LUCENE-3069.patch, > LUCENE-3069.patch > > > FST based TermDictionary has been a great improvement yet it still uses a > delta codec file for scanning to terms. Some environments have enough memory > available to keep the entire FST based term dict in memory. We should add a > TermDictionary implementation that encodes all needed information for each > term into the FST (custom fst.Output) and builds a FST from the entire term > not just the delta. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5084) new field type - EnumField
[ https://issues.apache.org/jira/browse/SOLR-5084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13747249#comment-13747249 ] Elran Dvir commented on SOLR-5084: -- In the latest patch I removed all redundant format changes. There were no changes in logic. I hope to attach the patch with unit tests in the next following days. Thanks again for the attention. > new field type - EnumField > -- > > Key: SOLR-5084 > URL: https://issues.apache.org/jira/browse/SOLR-5084 > Project: Solr > Issue Type: New Feature >Reporter: Elran Dvir > Attachments: enumsConfig.xml, schema_example.xml, Solr-5084.patch, > Solr-5084.patch, Solr-5084.patch > > > We have encountered a use case in our system where we have a few fields > (Severity. Risk etc) with a closed set of values, where the sort order for > these values is pre-determined but not lexicographic (Critical is higher than > High). Generically this is very close to how enums work. > To implement, I have prototyped a new type of field: EnumField where the > inputs are a closed predefined set of strings in a special configuration > file (similar to currency.xml). > The code is based on 4.2.1. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5186) Add CachingWrapperFilter.getFilter()
[ https://issues.apache.org/jira/browse/LUCENE-5186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13747308#comment-13747308 ] Adrien Grand commented on LUCENE-5186: -- This sounds good to me, would you like to write a patch? > Add CachingWrapperFilter.getFilter() > > > Key: LUCENE-5186 > URL: https://issues.apache.org/jira/browse/LUCENE-5186 > Project: Lucene - Core > Issue Type: Improvement > Components: core/search >Reporter: Trejkaz >Priority: Minor > > There are a couple of use cases I can think of where being able to get the > underlying filter out of CachingWrapperFilter would be useful: > 1. You might want to introspect the filter to figure out what's in it (the > use case we hit.) > 2. You might want to serialise the filter since Lucene no longer supports > that itself. > We currently work around this by subclassing, keeping another copy of the > underlying filter reference and implementing a trivial getter, which is an > easy workaround, but the trap is that a junior developer could unknowingly > create a CachingWrapperFilter without knowing that the > BetterCachingWrapperFilter exists, introducing a filter which cannot be > introspected. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5186) Add CachingWrapperFilter.getFilter()
[ https://issues.apache.org/jira/browse/LUCENE-5186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrien Grand updated LUCENE-5186: - Assignee: Adrien Grand > Add CachingWrapperFilter.getFilter() > > > Key: LUCENE-5186 > URL: https://issues.apache.org/jira/browse/LUCENE-5186 > Project: Lucene - Core > Issue Type: Improvement > Components: core/search >Reporter: Trejkaz >Assignee: Adrien Grand >Priority: Minor > > There are a couple of use cases I can think of where being able to get the > underlying filter out of CachingWrapperFilter would be useful: > 1. You might want to introspect the filter to figure out what's in it (the > use case we hit.) > 2. You might want to serialise the filter since Lucene no longer supports > that itself. > We currently work around this by subclassing, keeping another copy of the > underlying filter reference and implementing a trivial getter, which is an > easy workaround, but the trap is that a junior developer could unknowingly > create a CachingWrapperFilter without knowing that the > BetterCachingWrapperFilter exists, introducing a filter which cannot be > introspected. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
