Re: Multiple shards for one collection on the same Solr server

[Otis Gospodnetic]

Per - keep contributing.  Your contributions so far have not gone unnoticed
- you've been spotted.  At least from here in New York. :)

Otis
--
SOLR Performance Monitoring - http://sematext.com/spm/index.html
Search Analytics - http://sematext.com/search-analytics/index.html




On Mon, Nov 26, 2012 at 2:43 PM, Per Steffensen  wrote:

> **
> Ok thanks. Guess you are not ready to nominate me for now. But who are the
> nominators? Are you a nominator, Mark? And how to get nominated - do I have
> to ask (as I did today) or do I just have to wait until a nominator gets
> the idea about me as a committer by himself?
>
>
> Regards, Per Steffensen
>
> Mark Miller skrev:
>
> On Nov 26, 2012, at 11:51 AM, Per Steffensen  
>  wrote:
>
>
>
>  I am a very experinced programmer with a long record as architect, designer, 
> mentor etc. Is there a chance that I could become committer? You will end up 
> happy - and so will I! :-)
>
>
>  You have to build up enough merit in the project to get someone to nominate 
> you. Getting someone to nominate you can come about in a few different ways, 
> but generally it requires seeing a good number of JIRA issues through, being 
> around long enough to learn and integrate with the current community vibe, 
> and sometimes other things like helping out on the mailing list, helping 
> others get JIRA issues in, etc.
>
> You have to build the merit in the community - merit doesn't transfer from 
> outside of the project.
>
> You can imagine why we are careful about this - we want to make sure the 
> community grows in a healthy manner and that the others we bring into the 
> community are a nice fit with the rest of us. We want to make sure some of 
> the Apache way has seeped into your mind and certainly that a lot of the 
> current community vibe has as well. IMO, the best committers tend to act like 
> committers before they are voted in. Once you earn the trust of another 
> committer or two, and contribute in digestible pieces, you can generally 
> build a relationship that gets your patches committed quite quickly. Once a 
> committer comes to trust your code, he tends to have an easier time 
> reviewing. Then he starts committing more of your stuff. Then he thinks, man, 
> this would just be easier if this guy was a committer.
>
> In any event, simply continuing to contribute and getting issues into Solr 
> over a decent length of time is generally enough to be nominated at some 
> point. Though it really all depends on the nominators in the end. And then of 
> course you must be voted in by the PMC.
>
> - Mark
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>
>
>

[jira] [Commented] (SOLR-3960) Searching: PostFilter plugins ignored by solr cloud for queries with grouping

[Commit Tag Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504342#comment-13504342
 ] 

Commit Tag Bot commented on SOLR-3960:
--

[branch_4x commit] Chris M. Hostetter
http://svn.apache.org/viewvc?view=revision&revision=1413983

SOLR-3960: Fixed a bug where Distributed Grouping ignored PostFilters (merge 
r1413975)



> Searching: PostFilter plugins ignored by solr cloud for queries with grouping
> -
>
> Key: SOLR-3960
> URL: https://issues.apache.org/jira/browse/SOLR-3960
> Project: Solr
>  Issue Type: Bug
>  Components: search, SolrCloud
>Affects Versions: 4.0-BETA
> Environment: Linux / jre 1.6
>Reporter: Nathan Visagan
>Assignee: Hoss Man
>  Labels: patch
> Fix For: 4.1, 5.0
>
> Attachments: SOLR-3960.patch, SOLR-3960.patch
>
>
> When specifying the cloud options (-DzkRun -DnumShards=n) in solr startup 
> parameters, solr search does not consider any PostFilters.
> For example, I have implemented a PostFilter and wrapped that in my 
> QParserPlugin sub class. Then in the solr query I am using fq to apply post 
> filtering by specifying fq={!pf1} where pf1 is the name of my QParserPlugin. 
> This all works well in solr and not in solr cloud. 
> I debugged and figured out that org.apache.solr.search.Grouping.execute() 
> method is called by the non cloud instance. This considers pf.PostFilter 
> which is the collection of all PostFilters whereas 
> org.apache.solr.search.grouping.CommandHandler.execute() which is called by 
> the solr cloud instance ignores the pf.PostFilter.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-3960) Searching: PostFilter plugins ignored by solr cloud for queries with grouping

[Hoss Man (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man resolved SOLR-3960.


   Resolution: Fixed
Fix Version/s: 5.0
   4.1

Thanks for hte review yonik, and thanks Nathan for the bug report.

Committed revision 1413975. - trunk
Committed revision 1413983. - 4x


> Searching: PostFilter plugins ignored by solr cloud for queries with grouping
> -
>
> Key: SOLR-3960
> URL: https://issues.apache.org/jira/browse/SOLR-3960
> Project: Solr
>  Issue Type: Bug
>  Components: search, SolrCloud
>Affects Versions: 4.0-BETA
> Environment: Linux / jre 1.6
>Reporter: Nathan Visagan
>Assignee: Hoss Man
>  Labels: patch
> Fix For: 4.1, 5.0
>
> Attachments: SOLR-3960.patch, SOLR-3960.patch
>
>
> When specifying the cloud options (-DzkRun -DnumShards=n) in solr startup 
> parameters, solr search does not consider any PostFilters.
> For example, I have implemented a PostFilter and wrapped that in my 
> QParserPlugin sub class. Then in the solr query I am using fq to apply post 
> filtering by specifying fq={!pf1} where pf1 is the name of my QParserPlugin. 
> This all works well in solr and not in solr cloud. 
> I debugged and figured out that org.apache.solr.search.Grouping.execute() 
> method is called by the non cloud instance. This considers pf.PostFilter 
> which is the collection of all PostFilters whereas 
> org.apache.solr.search.grouping.CommandHandler.execute() which is called by 
> the solr cloud instance ignores the pf.PostFilter.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3960) Searching: PostFilter plugins ignored by solr cloud for queries with grouping

[Commit Tag Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504325#comment-13504325
 ] 

Commit Tag Bot commented on SOLR-3960:
--

[trunk commit] Chris M. Hostetter
http://svn.apache.org/viewvc?view=revision&revision=1413975

SOLR-3960: Fixed a bug where Distributed Grouping ignored PostFilters



> Searching: PostFilter plugins ignored by solr cloud for queries with grouping
> -
>
> Key: SOLR-3960
> URL: https://issues.apache.org/jira/browse/SOLR-3960
> Project: Solr
>  Issue Type: Bug
>  Components: search, SolrCloud
>Affects Versions: 4.0-BETA
> Environment: Linux / jre 1.6
>Reporter: Nathan Visagan
>Assignee: Hoss Man
>  Labels: patch
> Attachments: SOLR-3960.patch, SOLR-3960.patch
>
>
> When specifying the cloud options (-DzkRun -DnumShards=n) in solr startup 
> parameters, solr search does not consider any PostFilters.
> For example, I have implemented a PostFilter and wrapped that in my 
> QParserPlugin sub class. Then in the solr query I am using fq to apply post 
> filtering by specifying fq={!pf1} where pf1 is the name of my QParserPlugin. 
> This all works well in solr and not in solr cloud. 
> I debugged and figured out that org.apache.solr.search.Grouping.execute() 
> method is called by the non cloud instance. This considers pf.PostFilter 
> which is the collection of all PostFilters whereas 
> org.apache.solr.search.grouping.CommandHandler.execute() which is called by 
> the solr cloud instance ignores the pf.PostFilter.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3960) Searching: PostFilter plugins ignored by solr cloud for queries with grouping

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504311#comment-13504311
 ] 

Yonik Seeley commented on SOLR-3960:


Looks good Hoss!

> Searching: PostFilter plugins ignored by solr cloud for queries with grouping
> -
>
> Key: SOLR-3960
> URL: https://issues.apache.org/jira/browse/SOLR-3960
> Project: Solr
>  Issue Type: Bug
>  Components: search, SolrCloud
>Affects Versions: 4.0-BETA
> Environment: Linux / jre 1.6
>Reporter: Nathan Visagan
>Assignee: Hoss Man
>  Labels: patch
> Attachments: SOLR-3960.patch, SOLR-3960.patch
>
>
> When specifying the cloud options (-DzkRun -DnumShards=n) in solr startup 
> parameters, solr search does not consider any PostFilters.
> For example, I have implemented a PostFilter and wrapped that in my 
> QParserPlugin sub class. Then in the solr query I am using fq to apply post 
> filtering by specifying fq={!pf1} where pf1 is the name of my QParserPlugin. 
> This all works well in solr and not in solr cloud. 
> I debugged and figured out that org.apache.solr.search.Grouping.execute() 
> method is called by the non cloud instance. This considers pf.PostFilter 
> which is the collection of all PostFilters whereas 
> org.apache.solr.search.grouping.CommandHandler.execute() which is called by 
> the solr cloud instance ignores the pf.PostFilter.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2592) Custom Hashing

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504305#comment-13504305
 ] 

Yonik Seeley commented on SOLR-2592:


OK, fixed the bugs: 
https://github.com/yonik/lucene-solr/commit/3c0de9db3f737ee24a8f47ab3db9c573a173ce7d
branch is back to working.

> Custom Hashing
> --
>
> Key: SOLR-2592
> URL: https://issues.apache.org/jira/browse/SOLR-2592
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Affects Versions: 4.0-ALPHA
>Reporter: Noble Paul
> Attachments: dbq_fix.patch, pluggable_sharding.patch, 
> pluggable_sharding_V2.patch, SOLR-2592.patch, SOLR-2592_r1373086.patch, 
> SOLR-2592_r1384367.patch, SOLR-2592_rev_2.patch, 
> SOLR_2592_solr_4_0_0_BETA_ShardPartitioner.patch
>
>
> If the data in a cloud can be partitioned on some criteria (say range, hash, 
> attribute value etc) It will be easy to narrow down the search to a smaller 
> subset of shards and in effect can achieve more efficient search.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3926) solrj should support better way of finding active sorts

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504303#comment-13504303
 ] 

Yonik Seeley commented on SOLR-3926:


bq. Lastly: I'm really not a fan of having "setSorts" and "getSorts" use 
"Map" in their APIs ... (I know, it was yonik's idea on the 
mailing list ... i cringed when i read that).

I was just running with what seemed to be the implied requirement (which 
appeared to be random access-by-name to the sort criteria).  Thinking about it 
a second longer though, I can't see any obvious use cases for that.  
List does seem more natural.

> solrj should support better way of finding active sorts
> ---
>
> Key: SOLR-3926
> URL: https://issues.apache.org/jira/browse/SOLR-3926
> Project: Solr
>  Issue Type: Improvement
>  Components: clients - java
>Affects Versions: 4.0
>Reporter: Eirik Lygre
>Priority: Minor
> Fix For: 4.1
>
> Attachments: SOLR-3926.patch, SOLR-3926.patch, SOLR-3926.patch
>
>
> The Solrj api uses ortogonal concepts for setting/removing and getting sort 
> information. Setting/removing uses a combination of (name,order), while 
> getters return a String "name order":
> {code}
> public SolrQuery setSortField(String field, ORDER order);
> public SolrQuery addSortField(String field, ORDER order);
> public SolrQuery removeSortField(String field, ORDER order);
> public String[] getSortFields();
> public String getSortField();
> {code}
> If you want to use the current sort information to present a list of active 
> sorts, with the possibility to remove then, you need to manually parse the 
> string(s) returned from getSortFields, to recreate the information required 
> by removeSortField(). Not difficult, but not convenient either :-)
> Therefore this suggestion: Add a new method {{public Map 
> getSortFieldMap();}} which returns an ordered map of active sort fields. This 
> will make introspection of the current sort setup much easier.
> {code}
>   public Map getSortFieldMap() {
> String[] actualSortFields = getSortFields();
> if (actualSortFields == null || actualSortFields.length == 0)
>   return Collections.emptyMap();
> Map sortFieldMap = new LinkedHashMap();
> for (String sortField : actualSortFields) {
>   String[] fieldSpec = sortField.trim().split(" ");
>   sortFieldMap.put(fieldSpec[0], ORDER.valueOf(fieldSpec[1]));
> }
> return Collections.unmodifiableMap(sortFieldMap);
>   }
> {code}
> For what it's worth, this is possible client code:
> {code}
> System.out.println("Active sorts");
> Map fieldMap = getSortFieldMap(query);
> for (String field : fieldMap.keySet()) {
>System.out.println("- " + field + "; dir=" + fieldMap.get(field));
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-4113) Fullpath for db-config file causes error on SolrCloud

[Deniz Durmus (JIRA)]

Deniz Durmus created SOLR-4113:
--

 Summary: Fullpath for db-config file causes error on SolrCloud
 Key: SOLR-4113
 URL: https://issues.apache.org/jira/browse/SOLR-4113
 Project: Solr
  Issue Type: Bug
Affects Versions: 5.0
Reporter: Deniz Durmus


When I put a full path for a db config file to my solrconfig.xml, dataimport 
attempt shows an error message like this:

SEVERE: null:java.io.IOException: Error opening 
/configs/mycore//Users/deniz/SolrCloudTesting/solr/mycore/conf/db-config.xml
at 
org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:82)
at 
org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:132)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:144)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1830)
at 
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:455)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:276)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
at org.eclipse.jetty.server.Server.handle(Server.java:351)
at 
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
at 
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
at 
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:890)
at 
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:944)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:634)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230)
at 
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)
at 
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.IllegalArgumentException: Invalid path string 
"/configs/mycore//Users/deniz/SolrCloudTesting/solr/mycore/conf/db-config.xml" 
caused by empty node name specified @19
at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:99)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:794)
at 
org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:193)
at 
org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:190)
at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:63)
at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:190)
at org.apache.solr.cloud.ZkController.pathExists(ZkController.java:519)
at 
org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:77)
... 30 more


and here is the part that i set my db-configs location in solrconfig.xml:



/Users/deniz/SolrCloudTesting/solr/mycore/conf/db-config.xml








--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails

[Deniz Durmus (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504279#comment-13504279
 ] 

Deniz Durmus commented on SOLR-4112:


As a feedback, would like to say that the patch that James Dyer prepared seems 
to solve the exceptions 

> Dataimporting with SolrCloud Fails
> --
>
> Key: SOLR-4112
> URL: https://issues.apache.org/jira/browse/SOLR-4112
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 5.0
>Reporter: Deniz Durmus
> Attachments: SOLR-4112.patch
>
>
> While trying to import data from db on cloud, it shows this in logs:
> SEVERE: Full Import 
> failed:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable 
> to PropertyWriter implementation:ZKPropertiesWriter 
> at 
> org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:336)
>  
> at 
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:418)
>  
> at 
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) 
> at 
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) 
> Caused by: org.apache.solr.common.cloud.ZooKeeperException: 
> ZkSolrResourceLoader does not support getConfigDir() - likely, what you are 
> trying to do is not supported in ZooKeeper mode 
> at 
> org.apache.solr.cloud.ZkSolrResourceLoader.getConfigDir(ZkSolrResourceLoader.java:100)
>  
> at 
> org.apache.solr.handler.dataimport.SimplePropertiesWriter.init(SimplePropertiesWriter.java:91)
>  
> at 
> org.apache.solr.handler.dataimport.ZKPropertiesWriter.init(ZKPropertiesWriter.java:45)
>  
> at 
> org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:334)
>  
> ... 3 more 
> Exception in thread "Thread-306" java.lang.NullPointerException 
> at 
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:427)
>  
> at 
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) 
> at 
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3926) solrj should support better way of finding active sorts

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504276#comment-13504276
 ] 

Hoss Man commented on SOLR-3926:


Eirik:

Thanks for bring this up and working on improving things -- I think the 
direction your patch is is taking looks really good, but i have a few comments 
that i think we should try to address before committing...

1) the javadocs for the new methods should clarify that when they refer to 
"field" that can actually be any sortable value (ie: field names, function, 
"score", etc...)

2) we should add javadocs to the deprecated methods explaining why they have 
been deprecated (ie: what limitations they have) with \{@link\} tags pointing 
out the corresponding new methods

3) I don't actually see any advantage in deprecating/removing the "public 
String getSortField()" since it's read only ... we should just document that it 
returns the serialized value of the "sort" param and that for programatic 
access the new methods are available

Lastly: I'm really not a fan of having "setSorts" and "getSorts" use 
"Map" in their APIs ... (I know, it was yonik's idea on the 
mailing list ... i cringed when i read that).  Even if we're using 
LinkedHashMap unde the covers it seems like it would be far to easy for a naive 
user to let a HashMap make it's way to setSorts and then not understand why the 
final order isn't what they want.

I think it would make a _lot_ more sense to introduce a new tiny little 
immutable "SortClause" class that just models the String+ORDER pair, and have 
all of these methods take/return List.  (It would also help 
simplify the javadocs for all these methods, because only the SortClause class 
would need to explain what the legal values are for the String, w/o cut/pasting 
on each of SolrQuery methods).

What do you think?

> solrj should support better way of finding active sorts
> ---
>
> Key: SOLR-3926
> URL: https://issues.apache.org/jira/browse/SOLR-3926
> Project: Solr
>  Issue Type: Improvement
>  Components: clients - java
>Affects Versions: 4.0
>Reporter: Eirik Lygre
>Priority: Minor
> Fix For: 4.1
>
> Attachments: SOLR-3926.patch, SOLR-3926.patch, SOLR-3926.patch
>
>
> The Solrj api uses ortogonal concepts for setting/removing and getting sort 
> information. Setting/removing uses a combination of (name,order), while 
> getters return a String "name order":
> {code}
> public SolrQuery setSortField(String field, ORDER order);
> public SolrQuery addSortField(String field, ORDER order);
> public SolrQuery removeSortField(String field, ORDER order);
> public String[] getSortFields();
> public String getSortField();
> {code}
> If you want to use the current sort information to present a list of active 
> sorts, with the possibility to remove then, you need to manually parse the 
> string(s) returned from getSortFields, to recreate the information required 
> by removeSortField(). Not difficult, but not convenient either :-)
> Therefore this suggestion: Add a new method {{public Map 
> getSortFieldMap();}} which returns an ordered map of active sort fields. This 
> will make introspection of the current sort setup much easier.
> {code}
>   public Map getSortFieldMap() {
> String[] actualSortFields = getSortFields();
> if (actualSortFields == null || actualSortFields.length == 0)
>   return Collections.emptyMap();
> Map sortFieldMap = new LinkedHashMap();
> for (String sortField : actualSortFields) {
>   String[] fieldSpec = sortField.trim().split(" ");
>   sortFieldMap.put(fieldSpec[0], ORDER.valueOf(fieldSpec[1]));
> }
> return Collections.unmodifiableMap(sortFieldMap);
>   }
> {code}
> For what it's worth, this is possible client code:
> {code}
> System.out.println("Active sorts");
> Map fieldMap = getSortFieldMap(query);
> for (String field : fieldMap.keySet()) {
>System.out.println("- " + field + "; dir=" + fieldMap.get(field));
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Trying out the commit bot tagger at a larger scale

[Mark Miller]

By the way David, I saw your last commit did not trigger a tag - looks because 
you used LUCENE-XXX without a colon after it. Currently it keys on that. I can 
look at making it a little more forgiving.

- Mark

On Nov 26, 2012, at 4:51 PM, Mark Miller  wrote:

> I took a look at the local repo with a git client and it seemed to show local 
> changes in an HTMLStripCharFilter…odd…
> 
> Anyway, I discarded those changes and let the bot run again and it caught up 
> with the missed tags.
> 
> Not sure if it will happen again or not (this stuff is pretty isolated and 
> untouched) - please let me know if anyone notices the tags are not being sent 
> out.
> 
> - Mark
> 
> On Nov 26, 2012, at 4:32 PM, Mark Miller  wrote:
> 
>> Thanks for the note - it actually has not been firing lately - I just
>> took a look and for some reason it has having trouble doing an update
>> (I'm using jgit under the covers). It's claiming there is a conflict
>> when I am trying to check out a branch.
>> 
>> I'll solve this and get it kicking again.
>> 
>> It's currently cron'd to run every 2 minutes.
>> 
>> - Mark
>> 
>> On Mon, Nov 26, 2012 at 4:04 PM, David Smiley (@MITRE.org)
>>  wrote:
>>> Mark,
>>> Do I need to do anything for the bot to make its comment, aside from the
>>> commit?  I just made a commit to both branches.  How much delay is there /
>>> i.e. what's its schedule?
>>> ~ David
>>> 
>>> 
>>> 
>>> -
>>> Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book
>>> --
>>> View this message in context: 
>>> http://lucene.472066.n3.nabble.com/Trying-out-the-commit-bot-tagger-at-a-larger-scale-tp4021178p4022451.html
>>> Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
>>> 
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>> 
>> 
>> 
>> 
>> -- 
>> - Mark
> 


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-3960) Searching: PostFilter plugins ignored by solr cloud for queries with grouping

[Hoss Man (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man reassigned SOLR-3960:
--

Assignee: Hoss Man

> Searching: PostFilter plugins ignored by solr cloud for queries with grouping
> -
>
> Key: SOLR-3960
> URL: https://issues.apache.org/jira/browse/SOLR-3960
> Project: Solr
>  Issue Type: Bug
>  Components: search, SolrCloud
>Affects Versions: 4.0-BETA
> Environment: Linux / jre 1.6
>Reporter: Nathan Visagan
>Assignee: Hoss Man
>  Labels: patch
> Attachments: SOLR-3960.patch, SOLR-3960.patch
>
>
> When specifying the cloud options (-DzkRun -DnumShards=n) in solr startup 
> parameters, solr search does not consider any PostFilters.
> For example, I have implemented a PostFilter and wrapped that in my 
> QParserPlugin sub class. Then in the solr query I am using fq to apply post 
> filtering by specifying fq={!pf1} where pf1 is the name of my QParserPlugin. 
> This all works well in solr and not in solr cloud. 
> I debugged and figured out that org.apache.solr.search.Grouping.execute() 
> method is called by the non cloud instance. This considers pf.PostFilter 
> which is the collection of all PostFilters whereas 
> org.apache.solr.search.grouping.CommandHandler.execute() which is called by 
> the solr cloud instance ignores the pf.PostFilter.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3960) Searching: PostFilter plugins ignored by solr cloud for queries with grouping

[Hoss Man (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man updated SOLR-3960:
---

Attachment: SOLR-3960.patch

slight test improvement to exercise distributed grouping both with a regular 
filter and with a post-filter

> Searching: PostFilter plugins ignored by solr cloud for queries with grouping
> -
>
> Key: SOLR-3960
> URL: https://issues.apache.org/jira/browse/SOLR-3960
> Project: Solr
>  Issue Type: Bug
>  Components: search, SolrCloud
>Affects Versions: 4.0-BETA
> Environment: Linux / jre 1.6
>Reporter: Nathan Visagan
>  Labels: patch
> Attachments: SOLR-3960.patch, SOLR-3960.patch
>
>
> When specifying the cloud options (-DzkRun -DnumShards=n) in solr startup 
> parameters, solr search does not consider any PostFilters.
> For example, I have implemented a PostFilter and wrapped that in my 
> QParserPlugin sub class. Then in the solr query I am using fq to apply post 
> filtering by specifying fq={!pf1} where pf1 is the name of my QParserPlugin. 
> This all works well in solr and not in solr cloud. 
> I debugged and figured out that org.apache.solr.search.Grouping.execute() 
> method is called by the non cloud instance. This considers pf.PostFilter 
> which is the collection of all PostFilters whereas 
> org.apache.solr.search.grouping.CommandHandler.execute() which is called by 
> the solr cloud instance ignores the pf.PostFilter.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3960) Searching: PostFilter plugins ignored by solr cloud for queries with grouping

[Hoss Man (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man updated SOLR-3960:
---

Attachment: SOLR-3960.patch

grouping and postfilters are two areas of internals that i'm not really 
familiar with, but based on Nathan's problem description and great job pointing 
out the methods he noticed not playing nicely, i was able to create a test case 
demonstrating the problem, and then take a blind stab at a fix.

would definitely appreciate more eyeballs on this

> Searching: PostFilter plugins ignored by solr cloud for queries with grouping
> -
>
> Key: SOLR-3960
> URL: https://issues.apache.org/jira/browse/SOLR-3960
> Project: Solr
>  Issue Type: Bug
>  Components: search, SolrCloud
>Affects Versions: 4.0-BETA
> Environment: Linux / jre 1.6
>Reporter: Nathan Visagan
>  Labels: patch
> Attachments: SOLR-3960.patch
>
>
> When specifying the cloud options (-DzkRun -DnumShards=n) in solr startup 
> parameters, solr search does not consider any PostFilters.
> For example, I have implemented a PostFilter and wrapped that in my 
> QParserPlugin sub class. Then in the solr query I am using fq to apply post 
> filtering by specifying fq={!pf1} where pf1 is the name of my QParserPlugin. 
> This all works well in solr and not in solr cloud. 
> I debugged and figured out that org.apache.solr.search.Grouping.execute() 
> method is called by the non cloud instance. This considers pf.PostFilter 
> which is the collection of all PostFilters whereas 
> org.apache.solr.search.grouping.CommandHandler.execute() which is called by 
> the solr cloud instance ignores the pf.PostFilter.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-2141) NullPointerException when using escapeSql function

[James Dyer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Dyer updated SOLR-2141:
-

Attachment: SOLR-2141.patch

this 3rd patch version contains a test that won't begin to fail on Jan 1 :)

> NullPointerException when using escapeSql function
> --
>
> Key: SOLR-2141
> URL: https://issues.apache.org/jira/browse/SOLR-2141
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - DataImportHandler
>Affects Versions: 1.4.1, 4.0
> Environment: openjdk 1.6.0 b12
>Reporter: Edward Rudd
>Assignee: James Dyer
> Fix For: 4.1, 5.0
>
> Attachments: dih-config.xml, dih-file.xml, SOLR-2141.b341f5b.patch, 
> SOLR-2141.patch, SOLR-2141.patch, SOLR-2141.patch, SOLR-2141.patch, 
> SOLR-2141.patch, SOLR-2141-sample.patch, SOLR-2141-test.patch
>
>
> I have two entities defined, nested in each other..
> 
>  
>
> 
> Now, when I run that it bombs on any article where subcategory = '' (it's a 
> NOT NULL column so empty string is there)  If i do where subcategory!='' in 
> the article query it works fine (aside from not pulling in all of the 
> articles).
> org.apache.solr.handler.dataimport.DataImportHandlerException: 
> java.lang.NullPointerException
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:424)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:383)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:242)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:180)
> at 
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:331)
> at 
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:389)
> at 
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:370)
> Caused by: java.lang.NullPointerException
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$1.evaluate(EvaluatorBag.java:75)
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$5.get(EvaluatorBag.java:216)
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$5.get(EvaluatorBag.java:204)
> at 
> org.apache.solr.handler.dataimport.VariableResolverImpl.resolve(VariableResolverImpl.java:107)
> at 
> org.apache.solr.handler.dataimport.TemplateString.fillTokens(TemplateString.java:81)
> at 
> org.apache.solr.handler.dataimport.TemplateString.replaceTokens(TemplateString.java:75)
> at 
> org.apache.solr.handler.dataimport.VariableResolverImpl.replaceTokens(VariableResolverImpl.java:87)
> at 
> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:71)
> at 
> org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:237)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:357)
> ... 6 more

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-2141) NullPointerException when using escapeSql function

[James Dyer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Dyer updated SOLR-2141:
-

Attachment: SOLR-2141.patch

better patch.  fixes problem for SOLR-3842 also.

> NullPointerException when using escapeSql function
> --
>
> Key: SOLR-2141
> URL: https://issues.apache.org/jira/browse/SOLR-2141
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - DataImportHandler
>Affects Versions: 1.4.1, 4.0
> Environment: openjdk 1.6.0 b12
>Reporter: Edward Rudd
>Assignee: James Dyer
> Fix For: 4.1, 5.0
>
> Attachments: dih-config.xml, dih-file.xml, SOLR-2141.b341f5b.patch, 
> SOLR-2141.patch, SOLR-2141.patch, SOLR-2141.patch, SOLR-2141.patch, 
> SOLR-2141-sample.patch, SOLR-2141-test.patch
>
>
> I have two entities defined, nested in each other..
> 
>  
>
> 
> Now, when I run that it bombs on any article where subcategory = '' (it's a 
> NOT NULL column so empty string is there)  If i do where subcategory!='' in 
> the article query it works fine (aside from not pulling in all of the 
> articles).
> org.apache.solr.handler.dataimport.DataImportHandlerException: 
> java.lang.NullPointerException
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:424)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:383)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:242)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:180)
> at 
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:331)
> at 
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:389)
> at 
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:370)
> Caused by: java.lang.NullPointerException
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$1.evaluate(EvaluatorBag.java:75)
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$5.get(EvaluatorBag.java:216)
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$5.get(EvaluatorBag.java:204)
> at 
> org.apache.solr.handler.dataimport.VariableResolverImpl.resolve(VariableResolverImpl.java:107)
> at 
> org.apache.solr.handler.dataimport.TemplateString.fillTokens(TemplateString.java:81)
> at 
> org.apache.solr.handler.dataimport.TemplateString.replaceTokens(TemplateString.java:75)
> at 
> org.apache.solr.handler.dataimport.VariableResolverImpl.replaceTokens(VariableResolverImpl.java:87)
> at 
> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:71)
> at 
> org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:237)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:357)
> ... 6 more

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3948) Calculate/display deleted documents in admin interface

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504181#comment-13504181
 ] 

Shawn Heisey commented on SOLR-3948:


I tried this patch against a trunk checkout today and it applied OK, though it 
did need the standard gnu fuzz of 1.

ncindex@bigindy5 /index/src/trunk $ patch -p0 < ../SOLR-3948.patch
patching file solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java
Hunk #1 succeeded at 1983 (offset -3 lines).
patching file 
solr/core/src/java/org/apache/solr/handler/admin/LukeRequestHandler.java
patching file solr/webapp/web/tpl/dashboard.html
patching file solr/webapp/web/js/scripts/dashboard.js


> Calculate/display deleted documents in admin interface
> --
>
> Key: SOLR-3948
> URL: https://issues.apache.org/jira/browse/SOLR-3948
> Project: Solr
>  Issue Type: Improvement
>  Components: web gui
>Affects Versions: 4.0
>Reporter: Shawn Heisey
>Priority: Minor
> Fix For: 4.1
>
> Attachments: SOLR-3948.patch
>
>
> The admin interface shows you two totals that let you infer how many deleted 
> documents exist in the index by subtracting Num Docs from Max Doc.  It would 
> make things much easier for novice users and for automated statistics 
> gathering if the number of deleted documents were calculated for you and 
> displayed.
> Last Modified: 3 minutes ago
> Num Docs: 12924551
> Max Doc: 13011778
> Version: 862
> Segment Count: 23

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4571) speedup disjunction with minShouldMatch

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504180#comment-13504180
 ] 

Robert Muir commented on LUCENE-4571:
-

I like the ideas Stefan!

{quote}
Using BooleanScorer will probably only give improvement by a factor (in some 
cases), but also this scorers would still generate all these useless candidates.
{quote}

It might be hard to beat this one (which already supports minShouldMatch as a 
side effect of its coord computation anyway i think) in cases with
lots of terms because I feel like inevitably supporting advance() on the 
subscorers will result in more heap operations/cpu (at least when i messed 
around on paper it seemed this way).

In fact in some situations I think even this one should be used when there are 
mandatory clauses: in any case booleanweight should have better heuristics to 
decide, limited by what the collector can deal with.

Collectors that support out of order scoring today I think are likely a lot 
faster with these minShouldMatch types of queries than BooleanScorer2. Thats 
why i brought it up: it could be an easy win for e.g. solr users.

But when the number of terms is smallish and one is super-common, its silly we 
don't try to improve the BS2 impl to at least try to handle the worst case :) 

At least for the sake of simplicity (not necessarily performance) I still think 
it would be good to factor minshouldmatch behavior out of disjunctionsumscorer 
if we try to make it use advance() on subscorers.


> speedup disjunction with minShouldMatch 
> 
>
> Key: LUCENE-4571
> URL: https://issues.apache.org/jira/browse/LUCENE-4571
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/search
>Affects Versions: 4.1
>Reporter: Mikhail Khludnev
>
> even minShouldMatch is supplied to DisjunctionSumScorer it enumerates whole 
> disjunction, and verifies minShouldMatch condition [on every 
> doc|https://github.com/apache/lucene-solr/blob/trunk/lucene/core/src/java/org/apache/lucene/search/DisjunctionSumScorer.java#L70]:
> {code}
>   public int nextDoc() throws IOException {
> assert doc != NO_MORE_DOCS;
> while(true) {
>   while (subScorers[0].docID() == doc) {
> if (subScorers[0].nextDoc() != NO_MORE_DOCS) {
>   heapAdjust(0);
> } else {
>   heapRemoveRoot();
>   if (numScorers < minimumNrMatchers) {
> return doc = NO_MORE_DOCS;
>   }
> }
>   }
>   afterNext();
>   if (nrMatchers >= minimumNrMatchers) {
> break;
>   }
> }
> 
> return doc;
>   }
> {code}
> [~spo] proposes (as well as I get it) to pop nrMatchers-1 scorers from the 
> heap first, and then push them back advancing behind that top doc. For me the 
> question no.1 is there a performance test for minShouldMatch constrained 
> disjunction. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4048) Add a "getRecursive" method to NamedList

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-4048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504170#comment-13504170
 ] 

Shawn Heisey commented on SOLR-4048:


The javadoc for this new method probably should say "One or more" instead of 
"An array of" for the args parameter.

I am curious about whether this can be considered for inclusion in Solr.


> Add a "getRecursive" method to NamedList
> 
>
> Key: SOLR-4048
> URL: https://issues.apache.org/jira/browse/SOLR-4048
> Project: Solr
>  Issue Type: New Feature
>Affects Versions: 4.0
>Reporter: Shawn Heisey
>Priority: Minor
> Fix For: 4.1
>
> Attachments: SOLR-4048.patch, SOLR-4048.patch, SOLR-4048.patch
>
>
> Most of the time when accessing data from a NamedList, what you'll be doing 
> is using get() to retrieve another NamedList, and doing so over and over 
> until you reach the final level, where you'll actually retrieve the value you 
> want.
> I propose adding a method to NamedList which would do all that heavy lifting 
> for you.  I created the following method for my own code.  It could be 
> adapted fairly easily for inclusion into NamedList itself.  The only reason I 
> did not include it as a patch is because I figure you'll want to ensure it 
> meets all your particular coding guidelines, and that the JavaDoc is much 
> better than I have done here:
> {code}
>   /**
>* Recursively parse a NamedList and return the value at the last level,
>* assuming that the object found at each level is also a NamedList. For
>* example, if "response" is the NamedList response from the Solr4 mbean
>* handler, the following code makes sense:
>* 
>* String coreName = (String) getRecursiveFromResponse(response, new
>* String[] { "solr-mbeans", "CORE", "core", "stats", "coreName" });
>* 
>* 
>* @param namedList the NamedList to parse
>* @param args A list of values to recursively request
>* @return the object at the last level.
>* @throws SolrServerException
>*/
>   @SuppressWarnings("unchecked")
>   private final Object getRecursiveFromResponse(
>   NamedList namedList, String[] args)
>   throws SolrServerException
>   {
>   NamedList list = null;
>   Object value = null;
>   try
>   {
>   for (String key : args)
>   {
>   if (list == null)
>   {
>   list = namedList;
>   }
>   else
>   {
>   list = (NamedList) value;
>   }
>   value = list.get(key);
>   }
>   return value;
>   }
>   catch (Exception e)
>   {
>   throw new SolrServerException(
>   "Failed to recursively parse 
> NamedList", e);
>   }
>   }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-4573) Improve SpatialExample.java to show distance value retrieval

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley resolved LUCENE-4573.
--

   Resolution: Fixed
Fix Version/s: 5.0
 Assignee: David Smiley

> Improve SpatialExample.java to show distance value retrieval
> 
>
> Key: LUCENE-4573
> URL: https://issues.apache.org/jira/browse/LUCENE-4573
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Minor
> Fix For: 4.1, 5.0
>
> Attachments: LUCENE-4573.patch
>
>
> I want to have SpatialExample.java depict how to get the distance.  
> I also observed that SpatialExample.java isn't tested, despite it being a 
> Junit test.  The reason is its name doesn't begin or end in "Test".  As a 
> small hack, a test method can be added elsewhere to call it.
> Patch to follow...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Trying out the commit bot tagger at a larger scale

[Mark Miller]

I took a look at the local repo with a git client and it seemed to show local 
changes in an HTMLStripCharFilter…odd…

Anyway, I discarded those changes and let the bot run again and it caught up 
with the missed tags.

Not sure if it will happen again or not (this stuff is pretty isolated and 
untouched) - please let me know if anyone notices the tags are not being sent 
out.

- Mark

On Nov 26, 2012, at 4:32 PM, Mark Miller  wrote:

> Thanks for the note - it actually has not been firing lately - I just
> took a look and for some reason it has having trouble doing an update
> (I'm using jgit under the covers). It's claiming there is a conflict
> when I am trying to check out a branch.
> 
> I'll solve this and get it kicking again.
> 
> It's currently cron'd to run every 2 minutes.
> 
> - Mark
> 
> On Mon, Nov 26, 2012 at 4:04 PM, David Smiley (@MITRE.org)
>  wrote:
>> Mark,
>> Do I need to do anything for the bot to make its comment, aside from the
>> commit?  I just made a commit to both branches.  How much delay is there /
>> i.e. what's its schedule?
>> ~ David
>> 
>> 
>> 
>> -
>> Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book
>> --
>> View this message in context: 
>> http://lucene.472066.n3.nabble.com/Trying-out-the-commit-bot-tagger-at-a-larger-scale-tp4021178p4022451.html
>> Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>> 
> 
> 
> 
> -- 
> - Mark


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4571) speedup disjunction with minShouldMatch

[Stefan Pohl (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504158#comment-13504158
 ] 

Stefan Pohl commented on LUCENE-4571:
-

Mikhail, Robert, thank you for following up on this and I'm happy to provide 
more details on ideas of how to use the minimumMatch constraint to speed up the 
execution of disjunctive queries.

Currently, one heap is used in disjunctive scorers to keep track of the next 
docid (next candidate), and only then these candidates are ruled out if below 
minimum number of terms / percentage of terms. If there is only one stop word 
or otherwise high docfreq term in the query, this will produce a huge number of 
candidates that only occur in very few query terms and only afterwards will be 
ruled out by the constraint again. As Robert pointed out, this leads to long 
processing times to attain the next valid candidate. Using BooleanScorer will 
probably only give improvement by a factor (in some cases), but also this 
scorers would still generate all these useless candidates.
It would be much more efficient to actively use the constraint as part of query 
processing and employ inverted list skipping (leap-frogging), which makes 
conjunctive queries so efficient, also for these kind of queries.

An efficient implementation could look like the following:
Assume at least 5 out of 20 query terms have to match and imagine the 
terms/inverted lists to be sorted by the next docid. Then, the first potential 
candidate that satisfies the constraint would be the docid at position 5 and 
all of the inverted list at position 0-4 can be safely advanced and have to 
match to that docid in order to generate a real candidate worth scoring. The 
order, in which to try advancing terms 0-4 should probably be guided by the 
sparsity of the lists (guided by either docfreq, if available, or a heuristic 
such as the distance to the next docid), that is, inverted list 4 should first 
be advanced to the docid, then list 3,2,1, if previous advances are successful. 
Otherwise, the algorithm can start all over and try the next docid at 5th 
position. This leads to effective leap-frogging on the most sparse lists within 
the constraint which rules out matches not satisfying the constraint, most of 
the time without even touching the very high-frequency terms. Improvements all 
depend on the type of queries and number of docs per index.

There are different implementations possible to achieve such a behavior:
1) Using the current heap and pull out the first 5 lists to get to the next 
candidate docid to which the other 4 then have to be advanced. This probably 
leads to more heap removal and addition operations than necessary with the next 
approaches.
2) Always keep the first 5 lists fully sorted and use the heap to keep track of 
the next docid within lists 6-20. This invariant should simplify implementation 
and be competitive for small minimumMatch.
3) Use two heaps: A max-heap to keep track of the largest docid within the 
first 5 inverted lists and a min-heap to keep track of the smallest docid 
within lists 6-20. This approach looks most efficient.

These implementations look like generalizations to pure disjunctive queries, 
but if there should be any impact they could be implemented as specializations 
used only for queries with a minimum match constraint.

> speedup disjunction with minShouldMatch 
> 
>
> Key: LUCENE-4571
> URL: https://issues.apache.org/jira/browse/LUCENE-4571
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/search
>Affects Versions: 4.1
>Reporter: Mikhail Khludnev
>
> even minShouldMatch is supplied to DisjunctionSumScorer it enumerates whole 
> disjunction, and verifies minShouldMatch condition [on every 
> doc|https://github.com/apache/lucene-solr/blob/trunk/lucene/core/src/java/org/apache/lucene/search/DisjunctionSumScorer.java#L70]:
> {code}
>   public int nextDoc() throws IOException {
> assert doc != NO_MORE_DOCS;
> while(true) {
>   while (subScorers[0].docID() == doc) {
> if (subScorers[0].nextDoc() != NO_MORE_DOCS) {
>   heapAdjust(0);
> } else {
>   heapRemoveRoot();
>   if (numScorers < minimumNrMatchers) {
> return doc = NO_MORE_DOCS;
>   }
> }
>   }
>   afterNext();
>   if (nrMatchers >= minimumNrMatchers) {
> break;
>   }
> }
> 
> return doc;
>   }
> {code}
> [~spo] proposes (as well as I get it) to pop nrMatchers-1 scorers from the 
> heap first, and then push them back advancing behind that top doc. For me the 
> question no.1 is there a performance test for minShouldMatch constrained 
> disjunction. 

--
This message is automatically generated by JIRA.
If you think it was sent incorr

[jira] [Commented] (LUCENE-3441) Add NRT support to facets

[Commit Tag Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504155#comment-13504155
 ] 

Commit Tag Bot commented on LUCENE-3441:


[trunk commit] Shai Erera
http://svn.apache.org/viewvc?view=revision&revision=1412235

LUCENE-3441: fix bug in ParentArray.initFromReader



> Add NRT support to facets
> -
>
> Key: LUCENE-3441
> URL: https://issues.apache.org/jira/browse/LUCENE-3441
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: modules/facet
>Reporter: Shai Erera
>Assignee: Shai Erera
>Priority: Minor
> Fix For: 4.1, 5.0
>
> Attachments: LUCENE-3441.patch, LUCENE-3441.patch
>
>
> Currently LuceneTaxonomyReader does not support NRT - i.e., on changes to 
> LuceneTaxonomyWriter, you cannot have the reader updated, like 
> IndexReader/Writer. In order to do that we need to do the following:
> # Add ctor to LuceneTaxonomyReader to allow you to instantiate it with 
> LuceneTaxonomyWriter.
> # Add API to LuceneTaxonomyWriter to expose its internal IndexReader
> # Change LTR.refresh() to return an LTR, rather than void. This is actually 
> not strictly related to that issue, but since we'll need to modify refresh() 
> impl, I think it'll be good to change its API as well. Since all of facet API 
> is @lucene.experimental, no backwards issues here (and the sooner we do it, 
> the better).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3441) Add NRT support to facets

[Commit Tag Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504154#comment-13504154
 ] 

Commit Tag Bot commented on LUCENE-3441:


[trunk commit] Shai Erera
http://svn.apache.org/viewvc?view=revision&revision=1412249

LUCENE-3441: add another test that reproduced a bug on a wrong assert



> Add NRT support to facets
> -
>
> Key: LUCENE-3441
> URL: https://issues.apache.org/jira/browse/LUCENE-3441
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: modules/facet
>Reporter: Shai Erera
>Assignee: Shai Erera
>Priority: Minor
> Fix For: 4.1, 5.0
>
> Attachments: LUCENE-3441.patch, LUCENE-3441.patch
>
>
> Currently LuceneTaxonomyReader does not support NRT - i.e., on changes to 
> LuceneTaxonomyWriter, you cannot have the reader updated, like 
> IndexReader/Writer. In order to do that we need to do the following:
> # Add ctor to LuceneTaxonomyReader to allow you to instantiate it with 
> LuceneTaxonomyWriter.
> # Add API to LuceneTaxonomyWriter to expose its internal IndexReader
> # Change LTR.refresh() to return an LTR, rather than void. This is actually 
> not strictly related to that issue, but since we'll need to modify refresh() 
> impl, I think it'll be good to change its API as well. Since all of facet API 
> is @lucene.experimental, no backwards issues here (and the sooner we do it, 
> the better).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2045) DIH doesn't release jdbc connections in conjunction with DB2

[Commit Tag Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504153#comment-13504153
 ] 

Commit Tag Bot commented on SOLR-2045:
--

[trunk commit] James Dyer
http://svn.apache.org/viewvc?view=revision&revision=1412262

SOLR-2045: workaround for Locales not supported by Derby



> DIH doesn't release jdbc connections in conjunction with DB2 
> -
>
> Key: SOLR-2045
> URL: https://issues.apache.org/jira/browse/SOLR-2045
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - DataImportHandler
>Affects Versions: 1.4.1, 3.6, 4.0
> Environment: DB2 SQLLIB 9.5, 9.7 jdbc Driver
>Reporter: Fenlor Sebastia
>Assignee: James Dyer
> Fix For: 4.1, 5.0
>
> Attachments: SOLR-2045.patch, SOLR-2045.patch
>
>
> Using the JDBCDatasource in conjunction with the DB2 JDBC Drivers results in 
> the following error when the DIH tries to close the the connection due to 
> active transactions. As a consequence each delta im port or full import opens 
> a new connection without closing it. So the maximum amount of connections 
> will be reached soon. Setting the connection to readOnly or changing the 
> transaction isolation level doesn't help neither.
> The JDBC Driver I used: "com.ibm.db2.jcc.DB2Driver" relieing in db2jcc4.jar 
> shipped with DB2 Express 9.7 for example
> Here is the stack trace...
> 14.08.2010 01:49:51 org.apache.solr.handler.dataimport.JdbcDataSource 
> closeConnection
> FATAL: Ignoring Error when closing connection
> com.ibm.db2.jcc.am.SqlException: [jcc][10251][10308][4.8.87] 
> java.sql.Connection.close() requested while a transaction is in progress on 
> the connection.The transaction remains active, and the connection cannot be 
> closed. ERRORCODE=-4471, SQLSTATE=null
>   at com.ibm.db2.jcc.am.gd.a(gd.java:660)
>   at com.ibm.db2.jcc.am.gd.a(gd.java:60)
>   at com.ibm.db2.jcc.am.gd.a(gd.java:120)
>   at com.ibm.db2.jcc.am.lb.u(lb.java:1202)
>   at com.ibm.db2.jcc.am.lb.x(lb.java:1225)
>   at com.ibm.db2.jcc.am.lb.v(lb.java:1211)
>   at com.ibm.db2.jcc.am.lb.close(lb.java:1195)
>   at com.ibm.db2.jcc.uw.UWConnection.close(UWConnection.java:838)
>   at 
> org.apache.solr.handler.dataimport.JdbcDataSource.closeConnection(JdbcDataSource.java:399)
>   at 
> org.apache.solr.handler.dataimport.JdbcDataSource.close(JdbcDataSource.java:390)
>   at 
> org.apache.solr.handler.dataimport.DataConfig$Entity.clearCache(DataConfig.java:173)
>   at 
> org.apache.solr.handler.dataimport.DataConfig.clearCaches(DataConfig.java:331)
>   at 
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:339)
>   at 
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:389)
>   at 
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:370)
> Well the issue can be solved by invoking a commit or rollback directly before 
> the connection.close() statement. Here is the code snipped of changes I made 
> in JdbcDatasource.java
>   private void closeConnection()  {
> try {
>   if (conn != null) {
>   if (conn.isReadOnly())
>   {
>   LOG.info("connection is readonly, therefore rollback");
>   conn.rollback();
>   } else
>   {
>   LOG.info("connection is not readonly, therefore 
> commit");
>   conn.commit();
>   }
> 
> conn.close();
>   }
> } catch (Exception e) {
>   LOG.error("Ignoring Error when closing connection", e);
> }
>   }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4570) release policeman tools?

[Commit Tag Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504152#comment-13504152
 ] 

Commit Tag Bot commented on LUCENE-4570:


[trunk commit] Uwe Schindler
http://svn.apache.org/viewvc?view=revision&revision=1412598

LUCENE-4570: Add missing license headers



> release policeman tools?
> 
>
> Key: LUCENE-4570
> URL: https://issues.apache.org/jira/browse/LUCENE-4570
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Robert Muir
>
> Currently there is source code in lucene/tools/src (e.g. Forbidden APIs 
> checker ant task).
> It would be convenient if you could download this thing in your ant build 
> from ivy (especially if maybe it included our definitions .txt files as 
> resources).
> In general checking for locale/charset violations in this way is a pretty 
> general useful thing for a server-side app.
> Can we either release lucene-tools.jar as an artifact, or maybe alternatively 
> move this somewhere else as a standalone project and suck it in ourselves?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4101) Provide a flag to store positions and offsets on fields defined in the schema

[Commit Tag Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504151#comment-13504151
 ] 

Commit Tag Bot commented on SOLR-4101:
--

[trunk commit] Alan Woodward
http://svn.apache.org/viewvc?view=revision&revision=1412803

SOLR-4101: Add storeOffsetsWithPositions flag to SchemaField



> Provide a flag to store positions and offsets on fields defined in the schema
> -
>
> Key: SOLR-4101
> URL: https://issues.apache.org/jira/browse/SOLR-4101
> Project: Solr
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Minor
> Fix For: 4.1, 5.0
>
> Attachments: SOLR-4101.patch
>
>
> This will be useful for highlighters (particularly ones based on interval 
> iterators, see LUCENE-2878)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-1916) investigate DIH use of default locale

[Commit Tag Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504148#comment-13504148
 ] 

Commit Tag Bot commented on SOLR-1916:
--

[trunk commit] James Dyer
http://svn.apache.org/viewvc?view=revision&revision=1413684

SOLR-1916: reinstate @Ignore'ed test



> investigate DIH use of default locale
> -
>
> Key: SOLR-1916
> URL: https://issues.apache.org/jira/browse/SOLR-1916
> Project: Solr
>  Issue Type: Task
>  Components: contrib - DataImportHandler
>Affects Versions: 3.1, 4.0-ALPHA
>Reporter: Robert Muir
>Assignee: James Dyer
> Fix For: 4.1, 5.0
>
> Attachments: SOLR-1916.patch, SOLR-1916.patch
>
>
> This is a spinoff from LUCENE-2466.
> In this issue I changed my locale to various locales and found some problems 
> in Lucene/Solr triggered by use of the default Locale.
> I noticed some use of the default-locale for Date operations in DIH 
> (TimeZone.getDefault/Locale.getDefault) and, while no tests fail, I think it 
> might be better to support a locale parameter for this.
> The wiki documents that numeric parsing can support localized numerics 
> formats: http://wiki.apache.org/solr/DataImportHandler#NumberFormatTransformer
> In both cases, I don't think we should ever use the default Locale. If no 
> Locale is provided, I find that new Locale("") <-- Unicode Root Locale, is a 
> better default for a server situation in a lot of cases, as it won't change 
> depending on the computer, or perhaps we just make Locale params mandatory 
> for this.
> Finally, in both cases, if localized numbers/dates are explicitly supported, 
> I think we should come up with a test strategy to ensure everything is 
> working. One idea is to do something similar to or make use of Lucene's 
> LocalizedTestCase.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4111) Context-Sensitive SpellCheck Collation is not really being tested on IndexBasedSpellChecker

[Commit Tag Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-4111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504150#comment-13504150
 ] 

Commit Tag Bot commented on SOLR-4111:
--

[trunk commit] James Dyer
http://svn.apache.org/viewvc?view=revision&revision=1413036

SOLR-4111: test fix for SpellCheckCollatorTest#testContextSensitiveCollate



> Context-Sensitive SpellCheck Collation is not really being tested on 
> IndexBasedSpellChecker
> ---
>
> Key: SOLR-4111
> URL: https://issues.apache.org/jira/browse/SOLR-4111
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 4.0
>Reporter: Tomás Fernández Löbbe
>Assignee: James Dyer
>Priority: Trivial
> Fix For: 4.1, 5.0
>
> Attachments: SOLR-4111.patch
>
>
> Looks like a typo in the code. 
> SpellCheckCollatorTest.testContextSensitiveCollate() has:
> {code:java}
> // DirectSolrSpellChecker   IndexBasedSpellChecker
> String[] dictionary = {"direct","default_teststop" };
> for(int i=0 ; i<1 ; i++) {
> ... test stuff using dictionary[i]...
> }
> {code}
> where it should be
> {code:java}
> // DirectSolrSpellChecker   IndexBasedSpellChecker
> String[] dictionary = {"direct","default_teststop" };
> for(int i=0 ; i<=1 ; i++) {
> ... test stuff using dictionary[i]...
> }
> {code}
> due to this, the tests are not being run with a IndexBasedSpellChecker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4093) localParams syntax for standard query parser

[Commit Tag Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-4093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504149#comment-13504149
 ] 

Commit Tag Bot commented on SOLR-4093:
--

[trunk commit] Yonik Seeley
http://svn.apache.org/viewvc?view=revision&revision=1413042

SOLR-4093: solr specific parser + localParams syntax



> localParams syntax for standard query parser
> 
>
> Key: SOLR-4093
> URL: https://issues.apache.org/jira/browse/SOLR-4093
> Project: Solr
>  Issue Type: New Feature
>Reporter: Yonik Seeley
> Fix For: 4.1
>
> Attachments: SOLR-4093.patch
>
>
> Instead of
> {code}
> _query_:"{!geodist d=10 p=20.5,30.2}"
> {code}
> one could directly use
> {code}
> {!geodist d=10 p=20.5,30.2}
>  {code}
> references: http://wiki.apache.org/solr/LocalParams
> First raised as LUCENE-4271, but moved to this Solr issue due to concerns 
> about Solr specific features in Lucene.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4086) Refactor DIH - VariableResolver & Evaluator

[Commit Tag Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-4086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504147#comment-13504147
 ] 

Commit Tag Bot commented on SOLR-4086:
--

[trunk commit] James Dyer
http://svn.apache.org/viewvc?view=revision&revision=1413729

SOLR-4086: fix trivial test mistake



> Refactor DIH - VariableResolver & Evaluator
> ---
>
> Key: SOLR-4086
> URL: https://issues.apache.org/jira/browse/SOLR-4086
> Project: Solr
>  Issue Type: Improvement
>  Components: contrib - DataImportHandler
>Affects Versions: 4.0
>Reporter: James Dyer
>Assignee: James Dyer
>Priority: Minor
> Fix For: 4.1, 5.0
>
> Attachments: SOLR-4086.patch
>
>
> This simplifies VariableResolver and moves each built-in Evaluator into its 
> own class.  Compiler warnings / missing generics are fixed.  Also, the Locale 
> bug with DateFormatEvaluator is solved.  Instead of using the machine 
> default, the Root Locale is used by default.  An optional 3rd parameter 
> allows users to specify whatever locale they want.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3441) Add NRT support to facets

[Commit Tag Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504146#comment-13504146
 ] 

Commit Tag Bot commented on LUCENE-3441:


[branch_4x commit] Shai Erera
http://svn.apache.org/viewvc?view=revision&revision=1412232

LUCENE-3441: facets NRT support



> Add NRT support to facets
> -
>
> Key: LUCENE-3441
> URL: https://issues.apache.org/jira/browse/LUCENE-3441
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: modules/facet
>Reporter: Shai Erera
>Assignee: Shai Erera
>Priority: Minor
> Fix For: 4.1, 5.0
>
> Attachments: LUCENE-3441.patch, LUCENE-3441.patch
>
>
> Currently LuceneTaxonomyReader does not support NRT - i.e., on changes to 
> LuceneTaxonomyWriter, you cannot have the reader updated, like 
> IndexReader/Writer. In order to do that we need to do the following:
> # Add ctor to LuceneTaxonomyReader to allow you to instantiate it with 
> LuceneTaxonomyWriter.
> # Add API to LuceneTaxonomyWriter to expose its internal IndexReader
> # Change LTR.refresh() to return an LTR, rather than void. This is actually 
> not strictly related to that issue, but since we'll need to modify refresh() 
> impl, I think it'll be good to change its API as well. Since all of facet API 
> is @lucene.experimental, no backwards issues here (and the sooner we do it, 
> the better).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3441) Add NRT support to facets

[Commit Tag Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504145#comment-13504145
 ] 

Commit Tag Bot commented on LUCENE-3441:


[branch_4x commit] Shai Erera
http://svn.apache.org/viewvc?view=revision&revision=1412252

LUCENE-3441: add another test that reproduced a bug on a wrong assert



> Add NRT support to facets
> -
>
> Key: LUCENE-3441
> URL: https://issues.apache.org/jira/browse/LUCENE-3441
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: modules/facet
>Reporter: Shai Erera
>Assignee: Shai Erera
>Priority: Minor
> Fix For: 4.1, 5.0
>
> Attachments: LUCENE-3441.patch, LUCENE-3441.patch
>
>
> Currently LuceneTaxonomyReader does not support NRT - i.e., on changes to 
> LuceneTaxonomyWriter, you cannot have the reader updated, like 
> IndexReader/Writer. In order to do that we need to do the following:
> # Add ctor to LuceneTaxonomyReader to allow you to instantiate it with 
> LuceneTaxonomyWriter.
> # Add API to LuceneTaxonomyWriter to expose its internal IndexReader
> # Change LTR.refresh() to return an LTR, rather than void. This is actually 
> not strictly related to that issue, but since we'll need to modify refresh() 
> impl, I think it'll be good to change its API as well. Since all of facet API 
> is @lucene.experimental, no backwards issues here (and the sooner we do it, 
> the better).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2045) DIH doesn't release jdbc connections in conjunction with DB2

[Commit Tag Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504144#comment-13504144
 ] 

Commit Tag Bot commented on SOLR-2045:
--

[branch_4x commit] James Dyer
http://svn.apache.org/viewvc?view=revision&revision=1412266

SOLR-2045: workaround for Locales not supported by Derby



> DIH doesn't release jdbc connections in conjunction with DB2 
> -
>
> Key: SOLR-2045
> URL: https://issues.apache.org/jira/browse/SOLR-2045
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - DataImportHandler
>Affects Versions: 1.4.1, 3.6, 4.0
> Environment: DB2 SQLLIB 9.5, 9.7 jdbc Driver
>Reporter: Fenlor Sebastia
>Assignee: James Dyer
> Fix For: 4.1, 5.0
>
> Attachments: SOLR-2045.patch, SOLR-2045.patch
>
>
> Using the JDBCDatasource in conjunction with the DB2 JDBC Drivers results in 
> the following error when the DIH tries to close the the connection due to 
> active transactions. As a consequence each delta im port or full import opens 
> a new connection without closing it. So the maximum amount of connections 
> will be reached soon. Setting the connection to readOnly or changing the 
> transaction isolation level doesn't help neither.
> The JDBC Driver I used: "com.ibm.db2.jcc.DB2Driver" relieing in db2jcc4.jar 
> shipped with DB2 Express 9.7 for example
> Here is the stack trace...
> 14.08.2010 01:49:51 org.apache.solr.handler.dataimport.JdbcDataSource 
> closeConnection
> FATAL: Ignoring Error when closing connection
> com.ibm.db2.jcc.am.SqlException: [jcc][10251][10308][4.8.87] 
> java.sql.Connection.close() requested while a transaction is in progress on 
> the connection.The transaction remains active, and the connection cannot be 
> closed. ERRORCODE=-4471, SQLSTATE=null
>   at com.ibm.db2.jcc.am.gd.a(gd.java:660)
>   at com.ibm.db2.jcc.am.gd.a(gd.java:60)
>   at com.ibm.db2.jcc.am.gd.a(gd.java:120)
>   at com.ibm.db2.jcc.am.lb.u(lb.java:1202)
>   at com.ibm.db2.jcc.am.lb.x(lb.java:1225)
>   at com.ibm.db2.jcc.am.lb.v(lb.java:1211)
>   at com.ibm.db2.jcc.am.lb.close(lb.java:1195)
>   at com.ibm.db2.jcc.uw.UWConnection.close(UWConnection.java:838)
>   at 
> org.apache.solr.handler.dataimport.JdbcDataSource.closeConnection(JdbcDataSource.java:399)
>   at 
> org.apache.solr.handler.dataimport.JdbcDataSource.close(JdbcDataSource.java:390)
>   at 
> org.apache.solr.handler.dataimport.DataConfig$Entity.clearCache(DataConfig.java:173)
>   at 
> org.apache.solr.handler.dataimport.DataConfig.clearCaches(DataConfig.java:331)
>   at 
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:339)
>   at 
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:389)
>   at 
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:370)
> Well the issue can be solved by invoking a commit or rollback directly before 
> the connection.close() statement. Here is the code snipped of changes I made 
> in JdbcDatasource.java
>   private void closeConnection()  {
> try {
>   if (conn != null) {
>   if (conn.isReadOnly())
>   {
>   LOG.info("connection is readonly, therefore rollback");
>   conn.rollback();
>   } else
>   {
>   LOG.info("connection is not readonly, therefore 
> commit");
>   conn.commit();
>   }
> 
> conn.close();
>   }
> } catch (Exception e) {
>   LOG.error("Ignoring Error when closing connection", e);
> }
>   }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4570) release policeman tools?

[Commit Tag Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504143#comment-13504143
 ] 

Commit Tag Bot commented on LUCENE-4570:


[branch_4x commit] Uwe Schindler
http://svn.apache.org/viewvc?view=revision&revision=1412599

LUCENE-4570: Add missing license headers



> release policeman tools?
> 
>
> Key: LUCENE-4570
> URL: https://issues.apache.org/jira/browse/LUCENE-4570
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Robert Muir
>
> Currently there is source code in lucene/tools/src (e.g. Forbidden APIs 
> checker ant task).
> It would be convenient if you could download this thing in your ant build 
> from ivy (especially if maybe it included our definitions .txt files as 
> resources).
> In general checking for locale/charset violations in this way is a pretty 
> general useful thing for a server-side app.
> Can we either release lucene-tools.jar as an artifact, or maybe alternatively 
> move this somewhere else as a standalone project and suck it in ourselves?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4101) Provide a flag to store positions and offsets on fields defined in the schema

[Commit Tag Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504142#comment-13504142
 ] 

Commit Tag Bot commented on SOLR-4101:
--

[branch_4x commit] Alan Woodward
http://svn.apache.org/viewvc?view=revision&revision=1412805

SOLR-4101: Add storeOffsetsWithPositions flag to SchemaField



> Provide a flag to store positions and offsets on fields defined in the schema
> -
>
> Key: SOLR-4101
> URL: https://issues.apache.org/jira/browse/SOLR-4101
> Project: Solr
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Minor
> Fix For: 4.1, 5.0
>
> Attachments: SOLR-4101.patch
>
>
> This will be useful for highlighters (particularly ones based on interval 
> iterators, see LUCENE-2878)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4111) Context-Sensitive SpellCheck Collation is not really being tested on IndexBasedSpellChecker

[Commit Tag Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-4111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504141#comment-13504141
 ] 

Commit Tag Bot commented on SOLR-4111:
--

[branch_4x commit] James Dyer
http://svn.apache.org/viewvc?view=revision&revision=1413038

SOLR-4111: test fix for SpellCheckCollatorTest#testContextSensitiveCollate



> Context-Sensitive SpellCheck Collation is not really being tested on 
> IndexBasedSpellChecker
> ---
>
> Key: SOLR-4111
> URL: https://issues.apache.org/jira/browse/SOLR-4111
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 4.0
>Reporter: Tomás Fernández Löbbe
>Assignee: James Dyer
>Priority: Trivial
> Fix For: 4.1, 5.0
>
> Attachments: SOLR-4111.patch
>
>
> Looks like a typo in the code. 
> SpellCheckCollatorTest.testContextSensitiveCollate() has:
> {code:java}
> // DirectSolrSpellChecker   IndexBasedSpellChecker
> String[] dictionary = {"direct","default_teststop" };
> for(int i=0 ; i<1 ; i++) {
> ... test stuff using dictionary[i]...
> }
> {code}
> where it should be
> {code:java}
> // DirectSolrSpellChecker   IndexBasedSpellChecker
> String[] dictionary = {"direct","default_teststop" };
> for(int i=0 ; i<=1 ; i++) {
> ... test stuff using dictionary[i]...
> }
> {code}
> due to this, the tests are not being run with a IndexBasedSpellChecker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4093) localParams syntax for standard query parser

[Commit Tag Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-4093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504140#comment-13504140
 ] 

Commit Tag Bot commented on SOLR-4093:
--

[branch_4x commit] Yonik Seeley
http://svn.apache.org/viewvc?view=revision&revision=1413047

SOLR-4093: solr specific parser + localParams syntax



> localParams syntax for standard query parser
> 
>
> Key: SOLR-4093
> URL: https://issues.apache.org/jira/browse/SOLR-4093
> Project: Solr
>  Issue Type: New Feature
>Reporter: Yonik Seeley
> Fix For: 4.1
>
> Attachments: SOLR-4093.patch
>
>
> Instead of
> {code}
> _query_:"{!geodist d=10 p=20.5,30.2}"
> {code}
> one could directly use
> {code}
> {!geodist d=10 p=20.5,30.2}
>  {code}
> references: http://wiki.apache.org/solr/LocalParams
> First raised as LUCENE-4271, but moved to this Solr issue due to concerns 
> about Solr specific features in Lucene.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-1916) investigate DIH use of default locale

[Commit Tag Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504139#comment-13504139
 ] 

Commit Tag Bot commented on SOLR-1916:
--

[branch_4x commit] James Dyer
http://svn.apache.org/viewvc?view=revision&revision=1413692

SOLR-1916: reinstate @Ignore'ed test



> investigate DIH use of default locale
> -
>
> Key: SOLR-1916
> URL: https://issues.apache.org/jira/browse/SOLR-1916
> Project: Solr
>  Issue Type: Task
>  Components: contrib - DataImportHandler
>Affects Versions: 3.1, 4.0-ALPHA
>Reporter: Robert Muir
>Assignee: James Dyer
> Fix For: 4.1, 5.0
>
> Attachments: SOLR-1916.patch, SOLR-1916.patch
>
>
> This is a spinoff from LUCENE-2466.
> In this issue I changed my locale to various locales and found some problems 
> in Lucene/Solr triggered by use of the default Locale.
> I noticed some use of the default-locale for Date operations in DIH 
> (TimeZone.getDefault/Locale.getDefault) and, while no tests fail, I think it 
> might be better to support a locale parameter for this.
> The wiki documents that numeric parsing can support localized numerics 
> formats: http://wiki.apache.org/solr/DataImportHandler#NumberFormatTransformer
> In both cases, I don't think we should ever use the default Locale. If no 
> Locale is provided, I find that new Locale("") <-- Unicode Root Locale, is a 
> better default for a server situation in a lot of cases, as it won't change 
> depending on the computer, or perhaps we just make Locale params mandatory 
> for this.
> Finally, in both cases, if localized numbers/dates are explicitly supported, 
> I think we should come up with a test strategy to ensure everything is 
> working. One idea is to do something similar to or make use of Lucene's 
> LocalizedTestCase.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile

[Commit Tag Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504138#comment-13504138
 ] 

Commit Tag Bot commented on SOLR-1972:
--

[branch_4x commit] Alan Woodward
http://svn.apache.org/viewvc?view=revision&revision=1413797

SOLR-1972: Add extra query stats to RequestHandler



> Need additional query stats in admin interface - median, 95th and 99th 
> percentile
> -
>
> Key: SOLR-1972
> URL: https://issues.apache.org/jira/browse/SOLR-1972
> Project: Solr
>  Issue Type: Improvement
>  Components: web gui
>Affects Versions: 1.4
>Reporter: Shawn Heisey
>Assignee: Alan Woodward
>Priority: Minor
> Fix For: 4.1
>
> Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch, 
> elyograg-1972-trunk.patch, elyograg-1972-trunk.patch, 
> SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch, 
> SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, 
> SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, 
> SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, 
> solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch, 
> SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch
>
>
> I would like to see more detailed query statistics from the admin GUI.  This 
> is what you can get now:
> requests : 809
> errors : 0
> timeouts : 0
> totalTime : 70053
> avgTimePerRequest : 86.59209
> avgRequestsPerSecond : 0.8148785 
> I'd like to see more data on the time per request - median, 95th percentile, 
> 99th percentile, and any other statistical function that makes sense to 
> include.  In my environment, the first bunch of queries after startup tend to 
> take several seconds each.  I find that the average value tends to be useless 
> until it has several thousand queries under its belt and the caches are 
> thoroughly warmed.  The statistical functions I have mentioned would quickly 
> eliminate the influence of those initial slow queries.
> The system will have to store individual data about each query.  I don't know 
> if this is something Solr does already.  It would be nice to have a 
> configurable count of how many of the most recent data points are kept, to 
> control the amount of memory the feature uses.  The default value could be 
> something like 1024 or 4096.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Trying out the commit bot tagger at a larger scale

[Mark Miller]

Thanks for the note - it actually has not been firing lately - I just
took a look and for some reason it has having trouble doing an update
(I'm using jgit under the covers). It's claiming there is a conflict
when I am trying to check out a branch.

I'll solve this and get it kicking again.

It's currently cron'd to run every 2 minutes.

- Mark

On Mon, Nov 26, 2012 at 4:04 PM, David Smiley (@MITRE.org)
 wrote:
> Mark,
> Do I need to do anything for the bot to make its comment, aside from the
> commit?  I just made a commit to both branches.  How much delay is there /
> i.e. what's its schedule?
> ~ David
>
>
>
> -
>  Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Trying-out-the-commit-bot-tagger-at-a-larger-scale-tp4021178p4022451.html
> Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>



-- 
- Mark

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: svn commit: r1413832 - in /lucene/dev/trunk/lucene/spatial/src/test/org/apache/lucene/spatial: SpatialExample.java StrategyTestCase.java TestTestFramework.java

[Dawid Weiss]

> I wish the test infrastructure simply scanned all test classes in test paths
> for @Test or JUnit Testcase subclass.  After all, the vast majority of
> classes in /test/** are going to be tests.  No big deal.

This isn't entirely correct since Lucene tests may not be annotated
and may not subclass Testcase... they're also discovered by name
(testXXX); as long as they're subclassing LuceneTestCase this is also
picked up.

You can try to change that fileset and it should work. I think it will
add overhead though. Besides -- having a logical naming scheme here
helps I think (Test*, *Test); everyone knows what's inside without
even looking.

Dawid

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3842) DataImportHandler: When attempting to use column values as field names, multivalued fields only retain the first result

[James Dyer (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504105#comment-13504105
 ] 

James Dyer commented on SOLR-3842:
--

>From Stackoverflow:

I'm trying to perform a full-import with document configuration similar to the 
following:











Let's say the field/value results from the child_table for parent.ID=1 look 
like this:

FIELD_COLUMN   VALUE_COLUMN
fieldA value1
fieldB value2
fieldB value3

And the schema configuration for fieldA and fieldB (note that fieldB is 
multivalued):


 

After running full-import, the document for parent 1 looks like this:

{"id":1,
"fieldA":"value1",
"fieldB":["value2"]}

But, I need (and was expecting) it to look like this instead (fieldB should 
have multiple values):

{"id":1,
"fieldA":"value1",
"fieldB":["value2","value3"]}


Any help would be greatly appreciated!! Thanks!

Solr version: 4.0.0.2012.08.06.22.50.47


> DataImportHandler: When attempting to use column values as field names, 
> multivalued fields only retain the first result
> ---
>
> Key: SOLR-3842
> URL: https://issues.apache.org/jira/browse/SOLR-3842
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 4.0-BETA
>Reporter: Eric Kingston
>
> Can you please verify if this issue is simply due to improper implementation, 
> or if it's a bug in Solr?
> http://stackoverflow.com/questions/12412040/solr-dataimporthandler-when-attempting-to-use-column-values-as-field-names-m
> Also, I should note that a static reference to fieldB does handle multiple 
> values correctly.
> Example: 
> It only fails when I try to set the field names dynamically.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-2141) NullPointerException when using escapeSql function

[James Dyer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Dyer updated SOLR-2141:
-

Attachment: SOLR-2141.patch

I've attached a patch which fixes the functions namespace issue.  More 
importantly, it tests for the problem described here, in SOLR-4047 & SOLR-3842. 
 

My plan is to commit this tomorrow and if you do not have any more problems, I 
will close this.

> NullPointerException when using escapeSql function
> --
>
> Key: SOLR-2141
> URL: https://issues.apache.org/jira/browse/SOLR-2141
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - DataImportHandler
>Affects Versions: 1.4.1, 4.0
> Environment: openjdk 1.6.0 b12
>Reporter: Edward Rudd
>Assignee: James Dyer
> Fix For: 4.1, 5.0
>
> Attachments: dih-config.xml, dih-file.xml, SOLR-2141.b341f5b.patch, 
> SOLR-2141.patch, SOLR-2141.patch, SOLR-2141.patch, SOLR-2141-sample.patch, 
> SOLR-2141-test.patch
>
>
> I have two entities defined, nested in each other..
> 
>  
>
> 
> Now, when I run that it bombs on any article where subcategory = '' (it's a 
> NOT NULL column so empty string is there)  If i do where subcategory!='' in 
> the article query it works fine (aside from not pulling in all of the 
> articles).
> org.apache.solr.handler.dataimport.DataImportHandlerException: 
> java.lang.NullPointerException
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:424)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:383)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:242)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:180)
> at 
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:331)
> at 
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:389)
> at 
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:370)
> Caused by: java.lang.NullPointerException
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$1.evaluate(EvaluatorBag.java:75)
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$5.get(EvaluatorBag.java:216)
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$5.get(EvaluatorBag.java:204)
> at 
> org.apache.solr.handler.dataimport.VariableResolverImpl.resolve(VariableResolverImpl.java:107)
> at 
> org.apache.solr.handler.dataimport.TemplateString.fillTokens(TemplateString.java:81)
> at 
> org.apache.solr.handler.dataimport.TemplateString.replaceTokens(TemplateString.java:75)
> at 
> org.apache.solr.handler.dataimport.VariableResolverImpl.replaceTokens(VariableResolverImpl.java:87)
> at 
> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:71)
> at 
> org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:237)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:357)
> ... 6 more

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: svn commit: r1413832 - in /lucene/dev/trunk/lucene/spatial/src/test/org/apache/lucene/spatial: SpatialExample.java StrategyTestCase.java TestTestFramework.java

[David Smiley (@MITRE.org)]

Hi Dawid.
There's an ant build file that looks for Test*, *Test, and EXCLUDES '$'
hence no inner class.
I wish the test infrastructure simply scanned all test classes in test paths
for @Test or JUnit Testcase subclass.  After all, the vast majority of
classes in /test/** are going to be tests.  No big deal.

~ David



-
 Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book
--
View this message in context: 
http://lucene.472066.n3.nabble.com/Re-svn-commit-r1413832-in-lucene-dev-trunk-lucene-spatial-src-test-org-apache-lucene-spatial-Spatiala-tp4022453p4022455.html
Sent from the Lucene - Java Developer mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: svn commit: r1413832 - in /lucene/dev/trunk/lucene/spatial/src/test/org/apache/lucene/spatial: SpatialExample.java StrategyTestCase.java TestTestFramework.java

[Dawid Weiss]

> +  @Test
> +  public void spatialExample() throws IOException {
> +//kind of a hack so that SpatialExample is tested despite
> +// it not starting or ending with "Test".
> +SpatialExample.main(null);
> +  }
> +
>  }

This is actually not the test framework's deal but an explicit fileset
that is used to collect and provide suite classes. I think (would have
to check) it would work the same even if you provided ALL classes as
the input, the difference being it'd be slower (more parsing, more
discarded classes).

Alternatively, you could probably try to embed a static public
RunAsTest class inside SpatialExample... this should also be picked up
because its name would be SpatialExample$RunAsTest (unless that
fileset explicitly discards nested classes).

Dawid

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Trying out the commit bot tagger at a larger scale

[David Smiley (@MITRE.org)]

Mark,
Do I need to do anything for the bot to make its comment, aside from the
commit?  I just made a commit to both branches.  How much delay is there /
i.e. what's its schedule?
~ David



-
 Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book
--
View this message in context: 
http://lucene.472066.n3.nabble.com/Trying-out-the-commit-bot-tagger-at-a-larger-scale-tp4021178p4022451.html
Sent from the Lucene - Java Developer mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-2141) NullPointerException when using escapeSql function

[James Dyer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Dyer updated SOLR-2141:
-

Affects Version/s: 4.0
Fix Version/s: 5.0
   4.1
 Assignee: James Dyer  (was: Koji Sekiguchi)

> NullPointerException when using escapeSql function
> --
>
> Key: SOLR-2141
> URL: https://issues.apache.org/jira/browse/SOLR-2141
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - DataImportHandler
>Affects Versions: 1.4.1, 4.0
> Environment: openjdk 1.6.0 b12
>Reporter: Edward Rudd
>Assignee: James Dyer
> Fix For: 4.1, 5.0
>
> Attachments: dih-config.xml, dih-file.xml, SOLR-2141.b341f5b.patch, 
> SOLR-2141.patch, SOLR-2141.patch, SOLR-2141-sample.patch, SOLR-2141-test.patch
>
>
> I have two entities defined, nested in each other..
> 
>  
>
> 
> Now, when I run that it bombs on any article where subcategory = '' (it's a 
> NOT NULL column so empty string is there)  If i do where subcategory!='' in 
> the article query it works fine (aside from not pulling in all of the 
> articles).
> org.apache.solr.handler.dataimport.DataImportHandlerException: 
> java.lang.NullPointerException
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:424)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:383)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:242)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:180)
> at 
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:331)
> at 
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:389)
> at 
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:370)
> Caused by: java.lang.NullPointerException
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$1.evaluate(EvaluatorBag.java:75)
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$5.get(EvaluatorBag.java:216)
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$5.get(EvaluatorBag.java:204)
> at 
> org.apache.solr.handler.dataimport.VariableResolverImpl.resolve(VariableResolverImpl.java:107)
> at 
> org.apache.solr.handler.dataimport.TemplateString.fillTokens(TemplateString.java:81)
> at 
> org.apache.solr.handler.dataimport.TemplateString.replaceTokens(TemplateString.java:75)
> at 
> org.apache.solr.handler.dataimport.VariableResolverImpl.replaceTokens(VariableResolverImpl.java:87)
> at 
> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:71)
> at 
> org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:237)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:357)
> ... 6 more

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Multiple shards for one collection on the same Solr server

[Mark Miller]

The nominators are generally the committers on the lucene and solr projects. 
There is a list of those on the website under "Who We Are". Yes, I am one. The 
PMC then votes - there is also a list of who is on the PMC on the website.

The history is, generally you just wait to be nominated. Everyone has their own 
criteria for nominating and voting. Because most were brought in using similar 
criteria, people tend to use the same criteria when they then start nominating 
others. There is something of a culture we are attempting to guide. 

- Mark

On Nov 26, 2012, at 2:43 PM, Per Steffensen  wrote:

> Ok thanks. Guess you are not ready to nominate me for now. But who are the 
> nominators? Are you a nominator, Mark? And how to get nominated - do I have 
> to ask (as I did today) or do I just have to wait until a nominator gets the 
> idea about me as a committer by himself?
> 
> Regards, Per Steffensen
> 
> Mark Miller skrev:
>> On Nov 26, 2012, at 11:51 AM, Per Steffensen 
>>  wrote:
>> 
>>   
>> 
>>> I am a very experinced programmer with a long record as architect, 
>>> designer, mentor etc. Is there a chance that I could become committer? You 
>>> will end up happy - and so will I! :-)
>>> 
>>> 
>> 
>> You have to build up enough merit in the project to get someone to nominate 
>> you. Getting someone to nominate you can come about in a few different ways, 
>> but generally it requires seeing a good number of JIRA issues through, being 
>> around long enough to learn and integrate with the current community vibe, 
>> and sometimes other things like helping out on the mailing list, helping 
>> others get JIRA issues in, etc.
>> 
>> You have to build the merit in the community - merit doesn't transfer from 
>> outside of the project.
>> 
>> You can imagine why we are careful about this - we want to make sure the 
>> community grows in a healthy manner and that the others we bring into the 
>> community are a nice fit with the rest of us. We want to make sure some of 
>> the Apache way has seeped into your mind and certainly that a lot of the 
>> current community vibe has as well. IMO, the best committers tend to act 
>> like committers before they are voted in. Once you earn the trust of another 
>> committer or two, and contribute in digestible pieces, you can generally 
>> build a relationship that gets your patches committed quite quickly. Once a 
>> committer comes to trust your code, he tends to have an easier time 
>> reviewing. Then he starts committing more of your stuff. Then he thinks, 
>> man, this would just be easier if this guy was a committer.
>> 
>> In any event, simply continuing to contribute and getting issues into Solr 
>> over a decent length of time is generally enough to be nominated at some 
>> point. Though it really all depends on the nominators in the end. And then 
>> of course you must be voted in by the PMC.
>> 
>> - Mark
>> -
>> To unsubscribe, e-mail: 
>> dev-unsubscr...@lucene.apache.org
>> 
>> For additional commands, e-mail: 
>> dev-h...@lucene.apache.org
>> 
>> 
>> 
>>   
>> 
> 


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: ant precommit; annoyed with "checkout is dirty"

[David Smiley (@MITRE.org)]

If svn:ignore is set to '*' at the top level project, then ant precommit won't 
complain about dirty files that exist there, such as patch files, notes to 
myself, and whatever else.  The wildcard should not conflict with build.xml or 
the other few checked-in files and folders being in source control -- it should 
have no impact.  Likewise, I would also find it convenient to set svn:ignore to 
'*' on dev-tools/idea/.idea/  I recognize the point of dirty file checking in 
ant precommit but I would prefer it be reigned in a bit.

~ David

On Nov 26, 2012, at 2:38 PM, sarowe [via Lucene] wrote:

I guess there are two classes of things here: new non-ignored 
non-svn-controlled files; and modifications to svn-controlled files.

My point about svn:ignore was that we could use it to handle the non-ignored 
non-svn-controlled files case.

I guess you're talking about the svn-controlled modified files case?  And the 
fact that svn:ignore=* doesn't cover this case?  If so, okay, that makes sense. 
 If not, can you increase your word count a little and help me understand?

Steve

On Nov 26, 2012, at 2:25 PM, David Smiley (@MITRE.org) <[hidden 
email]> wrote:

> AFAIK if you set svn:ignore * then it won't ignore files and directories that 
> are in source control.
>
> ~ David
>
> On Nov 26, 2012, at 1:41 PM, sarowe [via Lucene] wrote:
>
>> I've always put patches up one level from checked out dirs: svn diff > 
>> ../PROJECT-.patch; patch -p0 < ../PROJECT-.patch.
>>
>> For stuff that should be ignored by everybody (or that wouldn't cause 
>> trouble for others), we could add them to the svn:ignore list for the 
>> directory they're in?
>>
>> Alternatively, I'd support a set of individually-settable wildcard 
>> exceptions to the dirty-is-bad rule, maybe via a sysprop?
>>
>> Steve
>>
>> On Nov 26, 2012, at 1:30 PM, "Smiley, David W." <> href="x-msg://149/user/SendEmail.jtp?type=node&node=4022419&i=0" 
>> target="_top" rel="nofollow" link="external">[hidden email]> wrote:
>>
>> > How about doing the dirty checks on sub-directories only?  This way I can 
>> > keep the random .patch files & miscellany around in the root.
>> >
>> > The IDE files are a special circumstance for my setup because I use 
>> > symbolic links to the IDE files in dev-tools so that I can easily see how 
>> > my IDE setup is different than the one in source control, and more easily 
>> > commit desirable changes.  I might stop doing this if this were the only 
>> > thing in my way from "ant precommit".
>> >
>> > FWIW my solution has been to modify the build script to not do the dirty 
>> > check :-)
>> >
>> > ~ David
>> >
>> > On Nov 26, 2012, at 11:09 AM, Michael McCandless wrote:
>> >
>> >> I think the idea is to catch files you forgot to svn add.
>> >>
>> >> For IDE files, you should just svn ignore them?
>> >>
>> >> Mike McCandless
>> >>
>> >> http://blog.mikemccandless.com
>> >>
>> >> On Mon, Nov 26, 2012 at 9:46 AM, David Smiley (@MITRE.org)
>> >> <> >> href="x-msg://149/user/SendEmail.jtp?type=node&node=4022419&i=1" 
>> >> target="_top" rel="nofollow" link="external">[hidden email]> wrote:
>> >>> "ant precommit" will check if the source tree is "dirty" (i.e. contains 
>> >>> files
>> >>> not in source control) and stop with a failure if so.  I find that rather
>> >>> annoying since I've usually got a variety of .patch files and IDE config
>> >>> changes.  What is the rationale behind this check?  How do people usually
>> >>> deal with it?  Perhaps if I do my real development on another checkout 
>> >>> (git
>> >>> based), I could then patch on the svn one for the commit.  Pretty 
>> >>> annoying
>> >>> though, and the 4x port is yet another step.  There sure is a lot of 
>> >>> burden
>> >>> to getting commits in.  I no longer care to improve little javadoc and 
>> >>> typo
>> >>> stuff.
>> >>>
>> >>> ~ David
>> >>>
>> >>>
>> >>>
>> >>> -
>> >>> Author: 
>> >>> http://www.packtpub.com/apache-solr-3-enterprise-search-server/book
>> >>> --
>> >>> View this message in context: 
>> >>> http://lucene.472066.n3.nabble.com/ant-precommit-annoyed-with-checkout-is-dirty-tp4022362.html
>> >>> Sent from the Lucene - Java Developer mailing list archive at 
>> >>> Nabble.com.
>> >>>
>> >>> -
>> >>> To unsubscribe, e-mail: > >>> href="x-msg://149/user/SendEmail.jtp?type=node&node=4022419&i=2" 
>> >>> target="_top" rel="nofollow" link="external"> [hidden email]
>> >>> For additional commands, e-mail: > >>> href="x-msg://149/user/SendEmail.jtp?type=node&node=4022419&i=3" 
>> >>> target="_top" rel="nofollow" link="external"> [hidden email]
>> >>>
>> >>
>> >> -
>> >> To unsubscribe, e-mail: > >> href="x-msg://149/user/SendEmail.jtp?type=node&node=4022419&i=4" 
>> >> target="_top" rel="nofollow" link="external"> [hidden email]
>> >> For additional commands, e-mail: 

[jira] [Commented] (SOLR-2592) Custom Hashing

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504033#comment-13504033
 ] 

Yonik Seeley commented on SOLR-2592:


FWIW, I'm trying out github for this patch.

My current changes to add a DocCollection abstraction to ClusterState are on 
the "custom_hash" branch in my lucene-solr fork:
https://github.com/yonik/lucene-solr/tree/custom_hash
https://github.com/yonik/lucene-solr/commit/5f82a7917862a1f9e70d6d268c44b23af18aca3b

Warning: it doesn't work yet... I just got it to compile, but tests are failing.


> Custom Hashing
> --
>
> Key: SOLR-2592
> URL: https://issues.apache.org/jira/browse/SOLR-2592
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Affects Versions: 4.0-ALPHA
>Reporter: Noble Paul
> Attachments: dbq_fix.patch, pluggable_sharding.patch, 
> pluggable_sharding_V2.patch, SOLR-2592.patch, SOLR-2592_r1373086.patch, 
> SOLR-2592_r1384367.patch, SOLR-2592_rev_2.patch, 
> SOLR_2592_solr_4_0_0_BETA_ShardPartitioner.patch
>
>
> If the data in a cloud can be partitioned on some criteria (say range, hash, 
> attribute value etc) It will be easy to narrow down the search to a smaller 
> subset of shards and in effect can achieve more efficient search.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Multiple shards for one collection on the same Solr server

[Per Steffensen]

Ok thanks. Guess you are not ready to nominate me for now. But who are 
the nominators? Are you a nominator, Mark? And how to get nominated - do 
I have to ask (as I did today) or do I just have to wait until a 
nominator gets the idea about me as a committer by himself?


Regards, Per Steffensen

Mark Miller skrev:

On Nov 26, 2012, at 11:51 AM, Per Steffensen  wrote:

  

I am a very experinced programmer with a long record as architect, designer, 
mentor etc. Is there a chance that I could become committer? You will end up 
happy - and so will I! :-)



You have to build up enough merit in the project to get someone to nominate 
you. Getting someone to nominate you can come about in a few different ways, 
but generally it requires seeing a good number of JIRA issues through, being 
around long enough to learn and integrate with the current community vibe, and 
sometimes other things like helping out on the mailing list, helping others get 
JIRA issues in, etc.

You have to build the merit in the community - merit doesn't transfer from 
outside of the project.

You can imagine why we are careful about this - we want to make sure the 
community grows in a healthy manner and that the others we bring into the 
community are a nice fit with the rest of us. We want to make sure some of the 
Apache way has seeped into your mind and certainly that a lot of the current 
community vibe has as well. IMO, the best committers tend to act like 
committers before they are voted in. Once you earn the trust of another 
committer or two, and contribute in digestible pieces, you can generally build 
a relationship that gets your patches committed quite quickly. Once a committer 
comes to trust your code, he tends to have an easier time reviewing. Then he 
starts committing more of your stuff. Then he thinks, man, this would just be 
easier if this guy was a committer.

In any event, simply continuing to contribute and getting issues into Solr over 
a decent length of time is generally enough to be nominated at some point. 
Though it really all depends on the nominators in the end. And then of course 
you must be voted in by the PMC.

- Mark
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org


  

[jira] [Resolved] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile

[Alan Woodward (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward resolved SOLR-1972.
-

Resolution: Fixed

Committed to branch 4x, r1413797

Thanks for prodding, Shawn!

> Need additional query stats in admin interface - median, 95th and 99th 
> percentile
> -
>
> Key: SOLR-1972
> URL: https://issues.apache.org/jira/browse/SOLR-1972
> Project: Solr
>  Issue Type: Improvement
>  Components: web gui
>Affects Versions: 1.4
>Reporter: Shawn Heisey
>Assignee: Alan Woodward
>Priority: Minor
> Fix For: 4.1
>
> Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch, 
> elyograg-1972-trunk.patch, elyograg-1972-trunk.patch, 
> SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch, 
> SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, 
> SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, 
> SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, 
> solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch, 
> SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch
>
>
> I would like to see more detailed query statistics from the admin GUI.  This 
> is what you can get now:
> requests : 809
> errors : 0
> timeouts : 0
> totalTime : 70053
> avgTimePerRequest : 86.59209
> avgRequestsPerSecond : 0.8148785 
> I'd like to see more data on the time per request - median, 95th percentile, 
> 99th percentile, and any other statistical function that makes sense to 
> include.  In my environment, the first bunch of queries after startup tend to 
> take several seconds each.  I find that the average value tends to be useless 
> until it has several thousand queries under its belt and the caches are 
> thoroughly warmed.  The statistical functions I have mentioned would quickly 
> eliminate the influence of those initial slow queries.
> The system will have to store individual data about each query.  I don't know 
> if this is something Solr does already.  It would be nice to have a 
> configurable count of how many of the most recent data points are kept, to 
> control the amount of memory the feature uses.  The default value could be 
> something like 1024 or 4096.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: ant precommit; annoyed with "checkout is dirty"

[Steve Rowe]

I guess there are two classes of things here: new non-ignored 
non-svn-controlled files; and modifications to svn-controlled files.

My point about svn:ignore was that we could use it to handle the non-ignored 
non-svn-controlled files case.

I guess you're talking about the svn-controlled modified files case?  And the 
fact that svn:ignore=* doesn't cover this case?  If so, okay, that makes sense. 
 If not, can you increase your word count a little and help me understand?   

Steve

On Nov 26, 2012, at 2:25 PM, David Smiley (@MITRE.org)  
wrote:

> AFAIK if you set svn:ignore * then it won't ignore files and directories that 
> are in source control.
> 
> ~ David
> 
> On Nov 26, 2012, at 1:41 PM, sarowe [via Lucene] wrote:
> 
>> I've always put patches up one level from checked out dirs: svn diff > 
>> ../PROJECT-.patch; patch -p0 < ../PROJECT-.patch. 
>> 
>> For stuff that should be ignored by everybody (or that wouldn't cause 
>> trouble for others), we could add them to the svn:ignore list for the 
>> directory they're in? 
>> 
>> Alternatively, I'd support a set of individually-settable wildcard 
>> exceptions to the dirty-is-bad rule, maybe via a sysprop? 
>> 
>> Steve 
>> 
>> On Nov 26, 2012, at 1:30 PM, "Smiley, David W." <> href="x-msg://149/user/SendEmail.jtp?type=node&node=4022419&i=0" 
>> target="_top" rel="nofollow" link="external">[hidden email]> wrote: 
>> 
>> > How about doing the dirty checks on sub-directories only?  This way I can 
>> > keep the random .patch files & miscellany around in the root. 
>> > 
>> > The IDE files are a special circumstance for my setup because I use 
>> > symbolic links to the IDE files in dev-tools so that I can easily see how 
>> > my IDE setup is different than the one in source control, and more easily 
>> > commit desirable changes.  I might stop doing this if this were the only 
>> > thing in my way from "ant precommit". 
>> > 
>> > FWIW my solution has been to modify the build script to not do the dirty 
>> > check :-) 
>> > 
>> > ~ David   
>> > 
>> > On Nov 26, 2012, at 11:09 AM, Michael McCandless wrote: 
>> > 
>> >> I think the idea is to catch files you forgot to svn add. 
>> >> 
>> >> For IDE files, you should just svn ignore them? 
>> >> 
>> >> Mike McCandless 
>> >> 
>> >> http://blog.mikemccandless.com
>> >> 
>> >> On Mon, Nov 26, 2012 at 9:46 AM, David Smiley (@MITRE.org) 
>> >> <> >> href="x-msg://149/user/SendEmail.jtp?type=node&node=4022419&i=1" 
>> >> target="_top" rel="nofollow" link="external">[hidden email]> wrote: 
>> >>> "ant precommit" will check if the source tree is "dirty" (i.e. contains 
>> >>> files 
>> >>> not in source control) and stop with a failure if so.  I find that 
>> >>> rather 
>> >>> annoying since I've usually got a variety of .patch files and IDE config 
>> >>> changes.  What is the rationale behind this check?  How do people 
>> >>> usually 
>> >>> deal with it?  Perhaps if I do my real development on another checkout 
>> >>> (git 
>> >>> based), I could then patch on the svn one for the commit.  Pretty 
>> >>> annoying 
>> >>> though, and the 4x port is yet another step.  There sure is a lot of 
>> >>> burden 
>> >>> to getting commits in.  I no longer care to improve little javadoc and 
>> >>> typo 
>> >>> stuff. 
>> >>> 
>> >>> ~ David 
>> >>> 
>> >>> 
>> >>> 
>> >>> - 
>> >>> Author: 
>> >>> http://www.packtpub.com/apache-solr-3-enterprise-search-server/book
>> >>> -- 
>> >>> View this message in context: 
>> >>> http://lucene.472066.n3.nabble.com/ant-precommit-annoyed-with-checkout-is-dirty-tp4022362.html
>> >>> Sent from the Lucene - Java Developer mailing list archive at 
>> >>> Nabble.com. 
>> >>> 
>> >>> - 
>> >>> To unsubscribe, e-mail: > >>> href="x-msg://149/user/SendEmail.jtp?type=node&node=4022419&i=2" 
>> >>> target="_top" rel="nofollow" link="external"> [hidden email] 
>> >>> For additional commands, e-mail: > >>> href="x-msg://149/user/SendEmail.jtp?type=node&node=4022419&i=3" 
>> >>> target="_top" rel="nofollow" link="external"> [hidden email] 
>> >>> 
>> >> 
>> >> - 
>> >> To unsubscribe, e-mail: > >> href="x-msg://149/user/SendEmail.jtp?type=node&node=4022419&i=4" 
>> >> target="_top" rel="nofollow" link="external"> [hidden email] 
>> >> For additional commands, e-mail: > >> href="x-msg://149/user/SendEmail.jtp?type=node&node=4022419&i=5" 
>> >> target="_top" rel="nofollow" link="external"> [hidden email] 
>> >> 
>> > 
>> > 
>> > - 
>> > To unsubscribe, e-mail: > > href="x-msg://149/user/SendEmail.jtp?type=node&node=4022419&i=6" 
>> > target="_top" rel="nofollow" link="external"> [hidden email] 
>> > For additional commands, e-mail: > > href="x-msg://149/user/SendEmail.jtp?type=node&node=4022419&i=7" 
>> > target="_top" rel="nofollow" link="external"> [hidden email] 
>> >
>> 
>> 
>> 

Re: Commit-free ExternalFileField

[Mikhail Khludnev]

Alan and everybody,

How do you feel about it?
Feedback is quite appreciated!


On Fri, Nov 16, 2012 at 1:02 AM, Mikhail Khludnev <
mkhlud...@griddynamics.com> wrote:

> Alan,
>
> Please find the patch attached
>
> https://issues.apache.org/jira/browse/SOLR-4085
>
> Regards
>
>
>
> On Wed, Nov 14, 2012 at 12:42 AM, Alan Woodward <
> alan.woodw...@romseysoftware.co.uk> wrote:
>
>> Hi Mikhail,
>>
>> I would definitely be interested!  Open a JIRA, and let's see what you've
>> come up with :-)
>>
>> -Alan
>>
>> On 13 Nov 2012, at 18:48, Mikhail Khludnev wrote:
>>
>> Community,
>> I have an implementation scratch of the subj (with great test coverage,
>> btw), but before attach it into jira for ages. I want to know your opinion:
>> - do you think it will be useful and widely adopted by many users?
>> - do you think it's necessary to provide consistency&atomicity for case
>> like: {!func}sum(foo_extf, foo_extf) - where same field is mentioned twice
>> in a query, first reference can be evaluated differently than second one if
>> field is reloaded during query parsing/evaluation
>>
>> Looking forward for your feedback!
>> --
>> Sincerely yours
>> Mikhail Khludnev
>> Principal Engineer,
>> Grid Dynamics
>>
>> 
>>  
>>
>>
>>
>
>
> --
> Sincerely yours
> Mikhail Khludnev
> Principal Engineer,
> Grid Dynamics
>
> 
>  
>
>


-- 
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics


 

Re: ant precommit; annoyed with "checkout is dirty"

[David Smiley (@MITRE.org)]

AFAIK if you set svn:ignore * then it won't ignore files and directories that 
are in source control.

~ David

On Nov 26, 2012, at 1:41 PM, sarowe [via Lucene] wrote:

I've always put patches up one level from checked out dirs: svn diff > 
../PROJECT-.patch; patch -p0 < ../PROJECT-.patch.

For stuff that should be ignored by everybody (or that wouldn't cause trouble 
for others), we could add them to the svn:ignore list for the directory they're 
in?

Alternatively, I'd support a set of individually-settable wildcard exceptions 
to the dirty-is-bad rule, maybe via a sysprop?

Steve

On Nov 26, 2012, at 1:30 PM, "Smiley, David W." <[hidden 
email]> wrote:

> How about doing the dirty checks on sub-directories only?  This way I can 
> keep the random .patch files & miscellany around in the root.
>
> The IDE files are a special circumstance for my setup because I use symbolic 
> links to the IDE files in dev-tools so that I can easily see how my IDE setup 
> is different than the one in source control, and more easily commit desirable 
> changes.  I might stop doing this if this were the only thing in my way from 
> "ant precommit".
>
> FWIW my solution has been to modify the build script to not do the dirty 
> check :-)
>
> ~ David
>
> On Nov 26, 2012, at 11:09 AM, Michael McCandless wrote:
>
>> I think the idea is to catch files you forgot to svn add.
>>
>> For IDE files, you should just svn ignore them?
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>> On Mon, Nov 26, 2012 at 9:46 AM, David Smiley (@MITRE.org)
>> <[hidden email]> 
>> wrote:
>>> "ant precommit" will check if the source tree is "dirty" (i.e. contains 
>>> files
>>> not in source control) and stop with a failure if so.  I find that rather
>>> annoying since I've usually got a variety of .patch files and IDE config
>>> changes.  What is the rationale behind this check?  How do people usually
>>> deal with it?  Perhaps if I do my real development on another checkout (git
>>> based), I could then patch on the svn one for the commit.  Pretty annoying
>>> though, and the 4x port is yet another step.  There sure is a lot of burden
>>> to getting commits in.  I no longer care to improve little javadoc and typo
>>> stuff.
>>>
>>> ~ David
>>>
>>>
>>>
>>> -
>>> Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book
>>> --
>>> View this message in context: 
>>> http://lucene.472066.n3.nabble.com/ant-precommit-annoyed-with-checkout-is-dirty-tp4022362.html
>>> Sent from the Lucene - Java Developer mailing list archive at 
>>> Nabble.com.
>>>
>>> -
>>> To unsubscribe, e-mail: [hidden 
>>> email]
>>> For additional commands, e-mail: [hidden 
>>> email]
>>>
>>
>> -
>> To unsubscribe, e-mail: [hidden 
>> email]
>> For additional commands, e-mail: [hidden 
>> email]
>>
>
>
> -
> To unsubscribe, e-mail: [hidden 
> email]
> For additional commands, e-mail: [hidden 
> email]
>


-
To unsubscribe, e-mail: [hidden 
email]
For additional commands, e-mail: [hidden 
email]




If you reply to this email, your message will be added to the discussion below:
http://lucene.472066.n3.nabble.com/ant-precommit-annoyed-with-checkout-is-dirty-tp4022362p4022419.html
To unsubscribe from ant precommit; annoyed with "checkout is dirty", click 
here.
NAML





-
 Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book
--
View this message in context: 
http://lucene.472066.n3.nabble.com/ant-precommit-annoyed-with-checkout-is-dirty-tp4022362p4022431.html
Sent from the Lucene - Java Developer mailing list archive at Nabble.com.

[jira] [Commented] (SOLR-2592) Custom Hashing

[Michael Garski (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503978#comment-13503978
 ] 

Michael Garski commented on SOLR-2592:
--

bq. I realized that we want something that clients (like cloud aware SolrJ) can 
easily get to. So instead of config in solrconfig.xml that defines a parser for 
the core, it seems like this should be a property of the collection?

Agreed - that makes perfect sense and was an oversight on my part. Persisting 
that information in /collections/ makes sense as it is a 
collection level option.

One thing about my patch that has been on my mind has been the parsing of the 
value to hash on from a delete by query on the client side. Currently it is 
very simple and uses a regex to extract the value as using any of the query 
parsers would require the cloud aware SolrJ client to also have the Lucene jar, 
which I'm not sure is desired. 

> Custom Hashing
> --
>
> Key: SOLR-2592
> URL: https://issues.apache.org/jira/browse/SOLR-2592
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Affects Versions: 4.0-ALPHA
>Reporter: Noble Paul
> Attachments: dbq_fix.patch, pluggable_sharding.patch, 
> pluggable_sharding_V2.patch, SOLR-2592.patch, SOLR-2592_r1373086.patch, 
> SOLR-2592_r1384367.patch, SOLR-2592_rev_2.patch, 
> SOLR_2592_solr_4_0_0_BETA_ShardPartitioner.patch
>
>
> If the data in a cloud can be partitioned on some criteria (say range, hash, 
> attribute value etc) It will be easy to narrow down the search to a smaller 
> subset of shards and in effect can achieve more efficient search.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: ant precommit; annoyed with "checkout is dirty"

[Steve Rowe]

I've always put patches up one level from checked out dirs: svn diff > 
../PROJECT-.patch; patch -p0 < ../PROJECT-.patch.

For stuff that should be ignored by everybody (or that wouldn't cause trouble 
for others), we could add them to the svn:ignore list for the directory they're 
in?

Alternatively, I'd support a set of individually-settable wildcard exceptions 
to the dirty-is-bad rule, maybe via a sysprop?

Steve

On Nov 26, 2012, at 1:30 PM, "Smiley, David W."  wrote:

> How about doing the dirty checks on sub-directories only?  This way I can 
> keep the random .patch files & miscellany around in the root.
> 
> The IDE files are a special circumstance for my setup because I use symbolic 
> links to the IDE files in dev-tools so that I can easily see how my IDE setup 
> is different than the one in source control, and more easily commit desirable 
> changes.  I might stop doing this if this were the only thing in my way from 
> "ant precommit".
> 
> FWIW my solution has been to modify the build script to not do the dirty 
> check :-)
> 
> ~ David  
> 
> On Nov 26, 2012, at 11:09 AM, Michael McCandless wrote:
> 
>> I think the idea is to catch files you forgot to svn add.
>> 
>> For IDE files, you should just svn ignore them?
>> 
>> Mike McCandless
>> 
>> http://blog.mikemccandless.com
>> 
>> On Mon, Nov 26, 2012 at 9:46 AM, David Smiley (@MITRE.org)
>>  wrote:
>>> "ant precommit" will check if the source tree is "dirty" (i.e. contains 
>>> files
>>> not in source control) and stop with a failure if so.  I find that rather
>>> annoying since I've usually got a variety of .patch files and IDE config
>>> changes.  What is the rationale behind this check?  How do people usually
>>> deal with it?  Perhaps if I do my real development on another checkout (git
>>> based), I could then patch on the svn one for the commit.  Pretty annoying
>>> though, and the 4x port is yet another step.  There sure is a lot of burden
>>> to getting commits in.  I no longer care to improve little javadoc and typo
>>> stuff.
>>> 
>>> ~ David
>>> 
>>> 
>>> 
>>> -
>>> Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book
>>> --
>>> View this message in context: 
>>> http://lucene.472066.n3.nabble.com/ant-precommit-annoyed-with-checkout-is-dirty-tp4022362.html
>>> Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
>>> 
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>> 
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>> 
> 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
> 


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: ant precommit; annoyed with "checkout is dirty"

[Uwe Schindler]

Hi David,

there are actually 2 reasons for this check:
- It prevents that you forget to add files before committing. We had this quite 
often, causing unbuildable sources and Jenkins failures.
- Prevent test cases from leaving files around at wrong places. This was the 
original idea behind the task, now obsoleted by the SecurityManager installed 
by the test-framework. Lots of Solr tests did not set the working directory 
correctly and started to write files into the src/ (!!!) folder. The check was 
added, so Jenkins could complain about those problems. This is now solved in a 
better way, but the check is still good for non-tests. The installed 
SecurityManager with a very strict security profile prevents wrong-behaving 
test - a test that writes files outside the build directory will fail with a 
SecurityException (raised on fileopen with write/append).

Uwe

-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de


> -Original Message-
> From: David Smiley (@MITRE.org) [mailto:dsmi...@mitre.org]
> Sent: Monday, November 26, 2012 3:46 PM
> To: dev@lucene.apache.org
> Subject: ant precommit; annoyed with "checkout is dirty"
> 
> "ant precommit" will check if the source tree is "dirty" (i.e. contains files 
> not
> in source control) and stop with a failure if so.  I find that rather annoying
> since I've usually got a variety of .patch files and IDE config changes.  
> What is
> the rationale behind this check?  How do people usually deal with it?
> Perhaps if I do my real development on another checkout (git based), I could
> then patch on the svn one for the commit.  Pretty annoying though, and the
> 4x port is yet another step.  There sure is a lot of burden to getting commits
> in.  I no longer care to improve little javadoc and typo stuff.
> 
> ~ David
> 
> 
> 
> -
>  Author: http://www.packtpub.com/apache-solr-3-enterprise-search-
> server/book
> --
> View this message in context: http://lucene.472066.n3.nabble.com/ant-
> precommit-annoyed-with-checkout-is-dirty-tp4022362.html
> Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional
> commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: ant precommit; annoyed with "checkout is dirty"

[Smiley, David W.]

How about doing the dirty checks on sub-directories only?  This way I can keep 
the random .patch files & miscellany around in the root.

The IDE files are a special circumstance for my setup because I use symbolic 
links to the IDE files in dev-tools so that I can easily see how my IDE setup 
is different than the one in source control, and more easily commit desirable 
changes.  I might stop doing this if this were the only thing in my way from 
"ant precommit".

FWIW my solution has been to modify the build script to not do the dirty check 
:-)

~ David  

On Nov 26, 2012, at 11:09 AM, Michael McCandless wrote:

> I think the idea is to catch files you forgot to svn add.
> 
> For IDE files, you should just svn ignore them?
> 
> Mike McCandless
> 
> http://blog.mikemccandless.com
> 
> On Mon, Nov 26, 2012 at 9:46 AM, David Smiley (@MITRE.org)
>  wrote:
>> "ant precommit" will check if the source tree is "dirty" (i.e. contains files
>> not in source control) and stop with a failure if so.  I find that rather
>> annoying since I've usually got a variety of .patch files and IDE config
>> changes.  What is the rationale behind this check?  How do people usually
>> deal with it?  Perhaps if I do my real development on another checkout (git
>> based), I could then patch on the svn one for the commit.  Pretty annoying
>> though, and the 4x port is yet another step.  There sure is a lot of burden
>> to getting commits in.  I no longer care to improve little javadoc and typo
>> stuff.
>> 
>> ~ David
>> 
>> 
>> 
>> -
>> Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book
>> --
>> View this message in context: 
>> http://lucene.472066.n3.nabble.com/ant-precommit-annoyed-with-checkout-is-dirty-tp4022362.html
>> Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>> 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
> 


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: ant precommit; annoyed with "checkout is dirty"

[Dawid Weiss]

This is still a problem for people working with git-svn, but I admit
it makes sense to have a precommit target that assumes a "supported"
configuration, whatever it may be, and enforces strict checks rather
than support multiple configurations with next to nothing.

Dawid

On Mon, Nov 26, 2012 at 5:09 PM, Michael McCandless
 wrote:
> I think the idea is to catch files you forgot to svn add.
>
> For IDE files, you should just svn ignore them?
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Mon, Nov 26, 2012 at 9:46 AM, David Smiley (@MITRE.org)
>  wrote:
>> "ant precommit" will check if the source tree is "dirty" (i.e. contains files
>> not in source control) and stop with a failure if so.  I find that rather
>> annoying since I've usually got a variety of .patch files and IDE config
>> changes.  What is the rationale behind this check?  How do people usually
>> deal with it?  Perhaps if I do my real development on another checkout (git
>> based), I could then patch on the svn one for the commit.  Pretty annoying
>> though, and the 4x port is yet another step.  There sure is a lot of burden
>> to getting commits in.  I no longer care to improve little javadoc and typo
>> stuff.
>>
>> ~ David
>>
>>
>>
>> -
>>  Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book
>> --
>> View this message in context: 
>> http://lucene.472066.n3.nabble.com/ant-precommit-annoyed-with-checkout-is-dirty-tp4022362.html
>> Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Question about SOLR soft commit

[Illu]

 

Hello there,

 I'm confused about soft commit. There is very little explanation about 
this on wiki, I hope to know some more details.



 Thanks in advance.

 

Best Regards,

Illu

[jira] [Created] (LUCENE-4575) Allow IndexWriter to commit, even just commitData

[Shai Erera (JIRA)]

Shai Erera created LUCENE-4575:
--

 Summary: Allow IndexWriter to commit, even just commitData
 Key: LUCENE-4575
 URL: https://issues.apache.org/jira/browse/LUCENE-4575
 Project: Lucene - Core
  Issue Type: Improvement
  Components: core/index
Reporter: Shai Erera
Priority: Minor


Spinoff from here 
http://lucene.472066.n3.nabble.com/commit-with-only-commitData-td4022155.html.

In some cases, it is valuable to be able to commit changes to the index, even 
if the changes are just commitData. Such data is sometimes used by applications 
to register in the index some global application information/state.

The proposal is:

* Add a setCommitData() API and separate it from commit() and prepareCommit() 
(simplify their API)

* When that API is called, flip on the dirty/changes bit, so that this gets 
committed even if no other changes were made to the index.

I will work on a patch a post.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Multiple shards for one collection on the same Solr server

[Mark Miller]

On Nov 26, 2012, at 11:51 AM, Per Steffensen  wrote:

> I am a very experinced programmer with a long record as architect, designer, 
> mentor etc. Is there a chance that I could become committer? You will end up 
> happy - and so will I! :-)

You have to build up enough merit in the project to get someone to nominate 
you. Getting someone to nominate you can come about in a few different ways, 
but generally it requires seeing a good number of JIRA issues through, being 
around long enough to learn and integrate with the current community vibe, and 
sometimes other things like helping out on the mailing list, helping others get 
JIRA issues in, etc.

You have to build the merit in the community - merit doesn't transfer from 
outside of the project.

You can imagine why we are careful about this - we want to make sure the 
community grows in a healthy manner and that the others we bring into the 
community are a nice fit with the rest of us. We want to make sure some of the 
Apache way has seeped into your mind and certainly that a lot of the current 
community vibe has as well. IMO, the best committers tend to act like 
committers before they are voted in. Once you earn the trust of another 
committer or two, and contribute in digestible pieces, you can generally build 
a relationship that gets your patches committed quite quickly. Once a committer 
comes to trust your code, he tends to have an easier time reviewing. Then he 
starts committing more of your stuff. Then he thinks, man, this would just be 
easier if this guy was a committer.

In any event, simply continuing to contribute and getting issues into Solr over 
a decent length of time is generally enough to be nominated at some point. 
Though it really all depends on the nominators in the end. And then of course 
you must be voted in by the PMC.

- Mark
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Multiple shards for one collection on the same Solr server

[Per Steffensen]

Of course I understand everything you say. We are just using Solr very 
intensively in my current project and we want to do big changes and when 
we make them we test them very thoroughly - so we are pretty sure they 
work as they are supposed to, but of course that is hard to see on your 
side, because we do not make that many tests in the Solr code, because 
that would make our contribs even bigger. But we test them thoroughly 
through our own test-suite. I have worked with Solr for a long time now 
and there is a million things that I (we in my project in general) would 
like to do, we just have a hard time to really get some progress - and 
my POs face turns green whenever I say that a feature/fix of some issue 
will require a Solr modification, because he knows how hard it is for us 
to get contribs through, and he knowns that the amount of time we use 
merging every time we want to get the newest code from Apache will 
increase by the amount of code we have different from yours.


I am a very experinced programmer with a long record as architect, 
designer, mentor etc. Is there a chance that I could become committer? 
You will end up happy - and so will I! :-)


Regards, Per Steffensen

Mark Miller skrev:

On Mon, Nov 26, 2012 at 9:55 AM, Per Steffensen  wrote:
  

Well I would like to jump in with respect to making support for running
several shards of the same collection on the same instance, it is just so
damn hard to get you to commit stuff :-)



A lot of the problem is time limitations for me personally. Every day
I go to sleep late wishing I could stay up just a little later :) I
never get everything I want done and I have a back log list of things
I'm interested in a mile long. So I sympathize, but I'm stretched thin
personally.

I do think though, if you look at CHANGES.txt, you will see that I do
commit a lot of other peoples patches! They just tend to be targeted
things that are relatively easy to digest. Also, things that are on my
itch list.

The main patches you have supplied so far are quite large! And touch
areas that involve other committers. Large patches and multiple
parties do not generally do well as we have mentioned in the past. It
depends on the committers, their time, their agenda, their itch list,
timing, etc.

I wouldn't give up trying to contribute though. Many of the things
tied up in your larger issues are things I'd like to get too. Once or
twice I've already gone into your work to 'steal' a bit when
addressing issues (once I got the time).

I know you'd prefer we were looser about committing if tests just
pass, but it comes down to what each committer is comfortable taking
responsibility for - and I'm not willing to generally commit larger
patches like that unless I have thoroughly vetted them - something
that takes a lot of my time.

I hope you continue contributing in any case!

  

[jira] [Updated] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile

[Shawn Heisey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shawn Heisey updated SOLR-1972:
---

Attachment: SOLR-1972_metrics.patch

The r1413047 commit caused a minor hiccup for this patch against branch_4x.  
Attached updated patch.  Double-checked that it works against a clean branch_4x 
checkout and that all solr tests pass.

> Need additional query stats in admin interface - median, 95th and 99th 
> percentile
> -
>
> Key: SOLR-1972
> URL: https://issues.apache.org/jira/browse/SOLR-1972
> Project: Solr
>  Issue Type: Improvement
>  Components: web gui
>Affects Versions: 1.4
>Reporter: Shawn Heisey
>Assignee: Alan Woodward
>Priority: Minor
> Fix For: 4.1
>
> Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch, 
> elyograg-1972-trunk.patch, elyograg-1972-trunk.patch, 
> SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch, 
> SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, 
> SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, 
> SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, 
> solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch, 
> SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch
>
>
> I would like to see more detailed query statistics from the admin GUI.  This 
> is what you can get now:
> requests : 809
> errors : 0
> timeouts : 0
> totalTime : 70053
> avgTimePerRequest : 86.59209
> avgRequestsPerSecond : 0.8148785 
> I'd like to see more data on the time per request - median, 95th percentile, 
> 99th percentile, and any other statistical function that makes sense to 
> include.  In my environment, the first bunch of queries after startup tend to 
> take several seconds each.  I find that the average value tends to be useless 
> until it has several thousand queries under its belt and the caches are 
> thoroughly warmed.  The statistical functions I have mentioned would quickly 
> eliminate the influence of those initial slow queries.
> The system will have to store individual data about each query.  I don't know 
> if this is something Solr does already.  It would be nice to have a 
> configurable count of how many of the most recent data points are kept, to 
> control the amount of memory the feature uses.  The default value could be 
> something like 1024 or 4096.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2141) NullPointerException when using escapeSql function

[Dominik Siebel (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503879#comment-13503879
 ] 

Dominik Siebel commented on SOLR-2141:
--

Checked with

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1413079 
13f79535-47bb-0310-9956-ffa450edef68

from github trunk.
Everything seems to be running, at least the fields are present. 
On optimize I get a "_SEVERE: null:java.lang.NoSuchMethodError: 
org.apache.lucene.document.Document.add(Lorg/apache/lucene/index/IndexableField;)V_"
 from the SpellChecker component, though. But this might be related to some 
other issue.

I could not verify with the whole dataset yet since indexing, commit and 
optimize (required for spellcheckers) take around 5h. I'll get back to you once 
the import finished on our staging system.


> NullPointerException when using escapeSql function
> --
>
> Key: SOLR-2141
> URL: https://issues.apache.org/jira/browse/SOLR-2141
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - DataImportHandler
>Affects Versions: 1.4.1
> Environment: openjdk 1.6.0 b12
>Reporter: Edward Rudd
>Assignee: Koji Sekiguchi
> Attachments: dih-config.xml, dih-file.xml, SOLR-2141.b341f5b.patch, 
> SOLR-2141.patch, SOLR-2141.patch, SOLR-2141-sample.patch, SOLR-2141-test.patch
>
>
> I have two entities defined, nested in each other..
> 
>  
>
> 
> Now, when I run that it bombs on any article where subcategory = '' (it's a 
> NOT NULL column so empty string is there)  If i do where subcategory!='' in 
> the article query it works fine (aside from not pulling in all of the 
> articles).
> org.apache.solr.handler.dataimport.DataImportHandlerException: 
> java.lang.NullPointerException
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:424)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:383)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:242)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:180)
> at 
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:331)
> at 
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:389)
> at 
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:370)
> Caused by: java.lang.NullPointerException
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$1.evaluate(EvaluatorBag.java:75)
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$5.get(EvaluatorBag.java:216)
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$5.get(EvaluatorBag.java:204)
> at 
> org.apache.solr.handler.dataimport.VariableResolverImpl.resolve(VariableResolverImpl.java:107)
> at 
> org.apache.solr.handler.dataimport.TemplateString.fillTokens(TemplateString.java:81)
> at 
> org.apache.solr.handler.dataimport.TemplateString.replaceTokens(TemplateString.java:75)
> at 
> org.apache.solr.handler.dataimport.VariableResolverImpl.replaceTokens(VariableResolverImpl.java:87)
> at 
> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:71)
> at 
> org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:237)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:357)
> ... 6 more

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: commit with only commitData

[Shai Erera]

I will open an issue so we can continue discussing.

Shai

On Mon, Nov 26, 2012 at 6:17 PM, Michael McCandless <
luc...@mikemccandless.com> wrote:

> I agree this (skipping a commit if there "seems to be" no changes) is
> annoying.
>
> I think a separate API would make sense?  Then we'd just set the
> changed bit in IW so that the next commit we always write it.
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Sat, Nov 24, 2012 at 12:08 AM, Shai Erera  wrote:
> > Hi
> >
> > Today, you cannot call IW.commit(userData) twice, even if the userData's
> > content is not null (or different) in the two calls.
> > Is there any particular reason why we prevent someone from doing that?
> >
> > For instance, when one works with a search and taxonomy indexes, we
> found it
> > useful to store some commit
> > data in both indexes to keep them in sync, so that e.g. when you reopen
> > both, you can make sure the two actually
> > match.
> > However, for some indexing sessions, no new categories will be added to
> the
> > index, therefore any commit that
> > will be called on TaxoWriter will silently be ignored, even if
> commitData is
> > passed.
> >
> > I've asked around and discovered that more people had a need for that -
> > storing some global-application information which
> > e.g. denotes the state of this index in the overall app. Because
> commitData
> > cannot be used like that, they add a dummy
> > document to the index with that info, which they always update, and also
> > make sure to filter it out during search.
> >
> > I don't think that adding dummy documents to the index is good,
> especially
> > not if you need to ensure they're filtered
> > out. Also, it's currently not possible to add dummy documents to the
> > taxonomy index, but let's leave that aside for now.
> >
> > So, why shouldn't we let someone commit by only changing userData? What
> > would be the harm? I can see two ways to allow that:
> >
> > 1) If commit() is called and nothing has changed, don't create a new
> commit
> > point, only if commit(userData) is called.
> >
> > 2) Alternatively, remove userData from the commit() API (that will
> simplify
> > prpeareCommit API too !), and exchange with an
> >IndexWriter.setCommitData() API, which will also mark that IW has
> pending
> > changes, and therefore must commit.
> >
> > Maybe option #2 will make it clear to both users of IW (and us
> developers)
> > that the application requests to make a transaction
> > to this IW instance. It also removes the duplicate commit and
> prepareCommit
> > API.
> >
> > Thoughts?
> >
> > Shai
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: commit with only commitData

[Michael McCandless]

I agree this (skipping a commit if there "seems to be" no changes) is annoying.

I think a separate API would make sense?  Then we'd just set the
changed bit in IW so that the next commit we always write it.

Mike McCandless

http://blog.mikemccandless.com

On Sat, Nov 24, 2012 at 12:08 AM, Shai Erera  wrote:
> Hi
>
> Today, you cannot call IW.commit(userData) twice, even if the userData's
> content is not null (or different) in the two calls.
> Is there any particular reason why we prevent someone from doing that?
>
> For instance, when one works with a search and taxonomy indexes, we found it
> useful to store some commit
> data in both indexes to keep them in sync, so that e.g. when you reopen
> both, you can make sure the two actually
> match.
> However, for some indexing sessions, no new categories will be added to the
> index, therefore any commit that
> will be called on TaxoWriter will silently be ignored, even if commitData is
> passed.
>
> I've asked around and discovered that more people had a need for that -
> storing some global-application information which
> e.g. denotes the state of this index in the overall app. Because commitData
> cannot be used like that, they add a dummy
> document to the index with that info, which they always update, and also
> make sure to filter it out during search.
>
> I don't think that adding dummy documents to the index is good, especially
> not if you need to ensure they're filtered
> out. Also, it's currently not possible to add dummy documents to the
> taxonomy index, but let's leave that aside for now.
>
> So, why shouldn't we let someone commit by only changing userData? What
> would be the harm? I can see two ways to allow that:
>
> 1) If commit() is called and nothing has changed, don't create a new commit
> point, only if commit(userData) is called.
>
> 2) Alternatively, remove userData from the commit() API (that will simplify
> prpeareCommit API too !), and exchange with an
>IndexWriter.setCommitData() API, which will also mark that IW has pending
> changes, and therefore must commit.
>
> Maybe option #2 will make it clear to both users of IW (and us developers)
> that the application requests to make a transaction
> to this IW instance. It also removes the duplicate commit and prepareCommit
> API.
>
> Thoughts?
>
> Shai
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-4112) Dataimporting with SolrCloud Fails

[James Dyer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Dyer updated SOLR-4112:
-

Attachment: SOLR-4112.patch

Please try the attached patch and let me know if this fixes your issue.  This 
may have been broken with SOLR-4051 (Trunk r1408873).  You might also want to 
try Trunk prior to this r1408873 and verify it works there.

Unfortunately I am behind the times and do not know much about solrcloud.  
Perhaps Sami can help out?  

Eventually I would also like to figure out how to write a good test for 
ZKPropertiesWriter.  I think it has no coverage and thus easily broken.

> Dataimporting with SolrCloud Fails
> --
>
> Key: SOLR-4112
> URL: https://issues.apache.org/jira/browse/SOLR-4112
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 5.0
>Reporter: Deniz Durmus
> Attachments: SOLR-4112.patch
>
>
> While trying to import data from db on cloud, it shows this in logs:
> SEVERE: Full Import 
> failed:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable 
> to PropertyWriter implementation:ZKPropertiesWriter 
> at 
> org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:336)
>  
> at 
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:418)
>  
> at 
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) 
> at 
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) 
> Caused by: org.apache.solr.common.cloud.ZooKeeperException: 
> ZkSolrResourceLoader does not support getConfigDir() - likely, what you are 
> trying to do is not supported in ZooKeeper mode 
> at 
> org.apache.solr.cloud.ZkSolrResourceLoader.getConfigDir(ZkSolrResourceLoader.java:100)
>  
> at 
> org.apache.solr.handler.dataimport.SimplePropertiesWriter.init(SimplePropertiesWriter.java:91)
>  
> at 
> org.apache.solr.handler.dataimport.ZKPropertiesWriter.init(ZKPropertiesWriter.java:45)
>  
> at 
> org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:334)
>  
> ... 3 more 
> Exception in thread "Thread-306" java.lang.NullPointerException 
> at 
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:427)
>  
> at 
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) 
> at 
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: ant precommit; annoyed with "checkout is dirty"

[Michael McCandless]

I think the idea is to catch files you forgot to svn add.

For IDE files, you should just svn ignore them?

Mike McCandless

http://blog.mikemccandless.com

On Mon, Nov 26, 2012 at 9:46 AM, David Smiley (@MITRE.org)
 wrote:
> "ant precommit" will check if the source tree is "dirty" (i.e. contains files
> not in source control) and stop with a failure if so.  I find that rather
> annoying since I've usually got a variety of .patch files and IDE config
> changes.  What is the rationale behind this check?  How do people usually
> deal with it?  Perhaps if I do my real development on another checkout (git
> based), I could then patch on the svn one for the commit.  Pretty annoying
> though, and the 4x port is yet another step.  There sure is a lot of burden
> to getting commits in.  I no longer care to improve little javadoc and typo
> stuff.
>
> ~ David
>
>
>
> -
>  Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/ant-precommit-annoyed-with-checkout-is-dirty-tp4022362.html
> Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Multiple shards for one collection on the same Solr server

[Mark Miller]

On Mon, Nov 26, 2012 at 9:55 AM, Per Steffensen  wrote:
> Well I would like to jump in with respect to making support for running
> several shards of the same collection on the same instance, it is just so
> damn hard to get you to commit stuff :-)

A lot of the problem is time limitations for me personally. Every day
I go to sleep late wishing I could stay up just a little later :) I
never get everything I want done and I have a back log list of things
I'm interested in a mile long. So I sympathize, but I'm stretched thin
personally.

I do think though, if you look at CHANGES.txt, you will see that I do
commit a lot of other peoples patches! They just tend to be targeted
things that are relatively easy to digest. Also, things that are on my
itch list.

The main patches you have supplied so far are quite large! And touch
areas that involve other committers. Large patches and multiple
parties do not generally do well as we have mentioned in the past. It
depends on the committers, their time, their agenda, their itch list,
timing, etc.

I wouldn't give up trying to contribute though. Many of the things
tied up in your larger issues are things I'd like to get too. Once or
twice I've already gone into your work to 'steal' a bit when
addressing issues (once I got the time).

I know you'd prefer we were looser about committing if tests just
pass, but it comes down to what each committer is comfortable taking
responsibility for - and I'm not willing to generally commit larger
patches like that unless I have thoroughly vetted them - something
that takes a lot of my time.

I hope you continue contributing in any case!

-- 
- Mark

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2592) Custom Hashing

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503839#comment-13503839
 ] 

Yonik Seeley commented on SOLR-2592:


bq. the Overseer could probably grab this from the collection node and add it 
to the cluster state json file?

Yeah, that's the other way, and might be initially easier.  To keep back 
compat, we would need to add a magic name to the map that we know isn't a shard.
"properties"?

> Custom Hashing
> --
>
> Key: SOLR-2592
> URL: https://issues.apache.org/jira/browse/SOLR-2592
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Affects Versions: 4.0-ALPHA
>Reporter: Noble Paul
> Attachments: dbq_fix.patch, pluggable_sharding.patch, 
> pluggable_sharding_V2.patch, SOLR-2592.patch, SOLR-2592_r1373086.patch, 
> SOLR-2592_r1384367.patch, SOLR-2592_rev_2.patch, 
> SOLR_2592_solr_4_0_0_BETA_ShardPartitioner.patch
>
>
> If the data in a cloud can be partitioned on some criteria (say range, hash, 
> attribute value etc) It will be easy to narrow down the search to a smaller 
> subset of shards and in effect can achieve more efficient search.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2592) Custom Hashing

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503837#comment-13503837
 ] 

Mark Miller commented on SOLR-2592:
---

bq. Now, currently the ClusterState object is created by reading 
clusterstate.json, but I don't believe it reads /collections/, 
but perhaps we should change this?

Depending on the number of collections, it may add a lot of reads on cluster 
changes, but the Overseer could probably grab this from the collection node and 
add it to the cluster state json file? It could perhaps even use a watcher per 
collection and not read when it does not have to.

Or it could be read separately like live nodes - that gets a bit messy though.

> Custom Hashing
> --
>
> Key: SOLR-2592
> URL: https://issues.apache.org/jira/browse/SOLR-2592
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Affects Versions: 4.0-ALPHA
>Reporter: Noble Paul
> Attachments: dbq_fix.patch, pluggable_sharding.patch, 
> pluggable_sharding_V2.patch, SOLR-2592.patch, SOLR-2592_r1373086.patch, 
> SOLR-2592_r1384367.patch, SOLR-2592_rev_2.patch, 
> SOLR_2592_solr_4_0_0_BETA_ShardPartitioner.patch
>
>
> If the data in a cloud can be partitioned on some criteria (say range, hash, 
> attribute value etc) It will be easy to narrow down the search to a smaller 
> subset of shards and in effect can achieve more efficient search.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2141) NullPointerException when using escapeSql function

[James Dyer (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503828#comment-13503828
 ] 

James Dyer commented on SOLR-2141:
--

Dominik,

Can you confirm for me that with the current Trunk that the functionality is 
working so long you use the "dataimporter.functions." namespace?

Removing support for the "dih.functions." namespace was indeed a mistake.  I 
can fix that too (and its a simple change).  But I really want to know for sure 
if your bigger problem is solved first??

> NullPointerException when using escapeSql function
> --
>
> Key: SOLR-2141
> URL: https://issues.apache.org/jira/browse/SOLR-2141
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - DataImportHandler
>Affects Versions: 1.4.1
> Environment: openjdk 1.6.0 b12
>Reporter: Edward Rudd
>Assignee: Koji Sekiguchi
> Attachments: dih-config.xml, dih-file.xml, SOLR-2141.b341f5b.patch, 
> SOLR-2141.patch, SOLR-2141.patch, SOLR-2141-sample.patch, SOLR-2141-test.patch
>
>
> I have two entities defined, nested in each other..
> 
>  
>
> 
> Now, when I run that it bombs on any article where subcategory = '' (it's a 
> NOT NULL column so empty string is there)  If i do where subcategory!='' in 
> the article query it works fine (aside from not pulling in all of the 
> articles).
> org.apache.solr.handler.dataimport.DataImportHandlerException: 
> java.lang.NullPointerException
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:424)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:383)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:242)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:180)
> at 
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:331)
> at 
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:389)
> at 
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:370)
> Caused by: java.lang.NullPointerException
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$1.evaluate(EvaluatorBag.java:75)
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$5.get(EvaluatorBag.java:216)
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$5.get(EvaluatorBag.java:204)
> at 
> org.apache.solr.handler.dataimport.VariableResolverImpl.resolve(VariableResolverImpl.java:107)
> at 
> org.apache.solr.handler.dataimport.TemplateString.fillTokens(TemplateString.java:81)
> at 
> org.apache.solr.handler.dataimport.TemplateString.replaceTokens(TemplateString.java:75)
> at 
> org.apache.solr.handler.dataimport.VariableResolverImpl.replaceTokens(VariableResolverImpl.java:87)
> at 
> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:71)
> at 
> org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:237)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:357)
> ... 6 more

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Multiple shards for one collection on the same Solr server

[Per Steffensen]

Mark Miller skrev:

The Collections API was fairly rushed - so that 4.0 had something easier than 
the CoreAdmin API.
  
Yes I see. Our collection-creation code is more sophisticated than 
yours. We probably would like to migrate to the Solr Collection API now 
anyway - to be using it already when features are added later.

Due to that, it has a variety of limitations:

1. It only picks instances for a collection one way - randomly from the list of 
live instances. This means it's no good for multiple shards on the same 
instance. You should have enough instances to satisfy numShards X 
replicationFactor (although just being short on replicationFactor will 
currently just use what is there)
  
Well I think it shuffles the list of live-nodes and the begin assigning 
shard from one end. That is ok for us for now. But it will not start 
over in the list of live-nodes when there are more shards (shards * 
replica) than instances. This could easily be acheived, without making a 
very fancy allocation algorithm

2. It randomly chooses which instances to use rather than allowing manual 
specification or looking at existing cores.
  
A manual spec would be nice to be able to control everything if you 
really want to. But you probably also want to make different built-in 
shard-allocation-strategies that can be used out-of-the-box. E.g. a 
"AlwaysAssignNextShardToInstanceWithFewestShardsAlready"-strategy, but 
there are also other concerns that might be more interesting for people 
to have build into assignment algorithms - e.g. a rack-aware algorithm 
that assign replica of the same slice to instances running on different 
"racks".

3. You cannot get responses of success or failure other than polling for the 
expected results later.
  

Well we do that anyway, and will keep doing that in our own code for now.


Someone has a patch up for 3 that I hope to look at soon - others have 
contributed bug fixes that will be in 4.1. We still need to add the ability to 
control placement in other ways though.

I would say there are def plans, but I don't personally know exactly when I'll 
find the time for it, if others don't jump in.
  
Well I would like to jump in with respect to making support for running 
several shards of the same collection on the same instance, it is just 
so damn hard to get you to commit stuff :-) and we really dont want to 
have too many differences in our Solr compared to Apache Solr (and we 
have enough already - SOLR-3178 etc.). It seems like this feature with 
several shards on same instance is the only missing feature of the 
Collection API before we can "live with it".

- Mark
  

Regards, Per Steffensen

On Nov 26, 2012, at 4:57 AM, Per Steffensen  wrote:

  

Hi

Before upgrading to Solr 4.0.0 we used to handle our collection creation 
ourselves, by creating each shards through the low-level CoreAdmin API. We used 
to create multiple shards under the same collection on each Solr server. 
Performance tests has shown that this is a good idea, and it is also a good 
idea for easy elasticity later on - it is much easier to move an entire 
existing shards from one Solr server to another one that just joined the cluter 
than it is to split an exsiting shard among the Solr that used to run it and 
the new Solr.

Now we are trying to migrate to the Solr Collection API for creation of collections, but 
it seems like it will not accept multiple shards under the same collection running on the 
same Solr server. E.g. if we have 4 Solr servers and ask to have a collection created 
with 8 shards, all 8 shards will be "created" but only 4 of them will acutally 
run - one on each Solr server.

Is the a good reason why Solr does not allow multiple shards under the same collection to 
run on the same Solr server, or is it just made this way "by coincidence"? In 
general I seek info on the matter - is it planed for later? etc.

Thanks!

Regards, Per Steffensen

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org





-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org


  

[jira] [Commented] (SOLR-2141) NullPointerException when using escapeSql function

[Dominik Siebel (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503815#comment-13503815
 ] 

Dominik Siebel commented on SOLR-2141:
--

Hi [~jdyer],
I just worked through the changes in the DIH. Turns out my fields were null, 
because I used the 'dih.functions.' namespace since the 
'dataimporter.functions.' was marked *_deprecated_* before the change. Did you 
change that back on purpose, because it might cause some confusion (like in my 
case).

> NullPointerException when using escapeSql function
> --
>
> Key: SOLR-2141
> URL: https://issues.apache.org/jira/browse/SOLR-2141
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - DataImportHandler
>Affects Versions: 1.4.1
> Environment: openjdk 1.6.0 b12
>Reporter: Edward Rudd
>Assignee: Koji Sekiguchi
> Attachments: dih-config.xml, dih-file.xml, SOLR-2141.b341f5b.patch, 
> SOLR-2141.patch, SOLR-2141.patch, SOLR-2141-sample.patch, SOLR-2141-test.patch
>
>
> I have two entities defined, nested in each other..
> 
>  
>
> 
> Now, when I run that it bombs on any article where subcategory = '' (it's a 
> NOT NULL column so empty string is there)  If i do where subcategory!='' in 
> the article query it works fine (aside from not pulling in all of the 
> articles).
> org.apache.solr.handler.dataimport.DataImportHandlerException: 
> java.lang.NullPointerException
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:424)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:383)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:242)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:180)
> at 
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:331)
> at 
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:389)
> at 
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:370)
> Caused by: java.lang.NullPointerException
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$1.evaluate(EvaluatorBag.java:75)
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$5.get(EvaluatorBag.java:216)
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$5.get(EvaluatorBag.java:204)
> at 
> org.apache.solr.handler.dataimport.VariableResolverImpl.resolve(VariableResolverImpl.java:107)
> at 
> org.apache.solr.handler.dataimport.TemplateString.fillTokens(TemplateString.java:81)
> at 
> org.apache.solr.handler.dataimport.TemplateString.replaceTokens(TemplateString.java:75)
> at 
> org.apache.solr.handler.dataimport.VariableResolverImpl.replaceTokens(VariableResolverImpl.java:87)
> at 
> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:71)
> at 
> org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:237)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:357)
> ... 6 more

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-4112) Dataimporting with SolrCloud Fails

[Deniz Durmus (JIRA)]

Deniz Durmus created SOLR-4112:
--

 Summary: Dataimporting with SolrCloud Fails
 Key: SOLR-4112
 URL: https://issues.apache.org/jira/browse/SOLR-4112
 Project: Solr
  Issue Type: Bug
Affects Versions: 5.0
Reporter: Deniz Durmus


While trying to import data from db on cloud, it shows this in logs:

SEVERE: Full Import 
failed:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to 
PropertyWriter implementation:ZKPropertiesWriter 
at 
org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:336)
 
at 
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:418)
 
at 
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) 
at 
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) 
Caused by: org.apache.solr.common.cloud.ZooKeeperException: 
ZkSolrResourceLoader does not support getConfigDir() - likely, what you are 
trying to do is not supported in ZooKeeper mode 
at 
org.apache.solr.cloud.ZkSolrResourceLoader.getConfigDir(ZkSolrResourceLoader.java:100)
 
at 
org.apache.solr.handler.dataimport.SimplePropertiesWriter.init(SimplePropertiesWriter.java:91)
 
at 
org.apache.solr.handler.dataimport.ZKPropertiesWriter.init(ZKPropertiesWriter.java:45)
 
at 
org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:334)
 
... 3 more 

Exception in thread "Thread-306" java.lang.NullPointerException 
at 
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:427)
 
at 
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) 
at 
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) 





--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

ant precommit; annoyed with "checkout is dirty"

[David Smiley (@MITRE.org)]

"ant precommit" will check if the source tree is "dirty" (i.e. contains files
not in source control) and stop with a failure if so.  I find that rather
annoying since I've usually got a variety of .patch files and IDE config
changes.  What is the rationale behind this check?  How do people usually
deal with it?  Perhaps if I do my real development on another checkout (git
based), I could then patch on the svn one for the commit.  Pretty annoying
though, and the 4x port is yet another step.  There sure is a lot of burden
to getting commits in.  I no longer care to improve little javadoc and typo
stuff.

~ David



-
 Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book
--
View this message in context: 
http://lucene.472066.n3.nabble.com/ant-precommit-annoyed-with-checkout-is-dirty-tp4022362.html
Sent from the Lucene - Java Developer mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-2592) Custom Hashing

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503807#comment-13503807
 ] 

Yonik Seeley edited comment on SOLR-2592 at 11/26/12 2:44 PM:
--

Hi Michael, while I was reviewing your patch, I realized that we want something 
that clients (like cloud aware SolrJ) can easily get to.
So instead of config in solrconfig.xml that defines a parser for the core, it 
seems like this should be a property of the collection?

The HashPartitioner object is currently on the ClusterState object, but it 
really seems like this needs to be per-collection (or per config-set?).

Currently, there is a /collections/collection1 node with
{code}
{"configName":"myconf"}
{code}

We could add a "partitioner" or "shardPartitioner" attribute.

And of course there is /clusterstate.json with
{code}
{"collection1":{
"shard1":{
  "range":"8000-",
  "replicas":{"192.168.1.109:8983_solr_collection1":{
  "shard":"shard1",
  "roles":null,
  "state":"active",
  "core":"collection1",
  "collection":"collection1",
  "node_name":"192.168.1.109:8983_solr",
  "base_url":"http://192.168.1.109:8983/solr";,
  "leader":"true"}}},
"shard2":{
  "range":"0-7fff",
  "replicas":{
{code}

Now, currently the ClusterState object is created by reading clusterstate.json, 
but I don't believe it reads /collections/, but perhaps we 
should change this?

The other issue is that there is currently no java Collection object (it's just 
a map of the contained shards) to expose info like what the hash partitioner 
used is.  I think I'll start by trying to refactor some of the ClusterState 
code to introduce that.

  was (Author: ysee...@gmail.com):
Hi Michael, while I was reviewing your patch, I realized that we want 
something that clients (like cloud aware SolrJ) can easily get to.
So instead of config in solrconfig.xml that defines a parser for the core, it 
seems like this should be a property of the collection?

The HashPartitioner object is currently on the ClusterState object, but it 
really seems like this needs to be per-collection (or per config-set?).

Currently, there is a /collections/collection1 node with
{code}
{"configName":"myconf"}
{code}

We could add a "partitioner" or "shardPartitioner" attribute.

And of course there is /clusterstate.json with
{code}
{"collection1":{
"shard1":{
  "range":"8000-",
  "replicas":{"192.168.1.109:8983_solr_collection1":{
  "shard":"shard1",
  "roles":null,
  "state":"active",
  "core":"collection1",
  "collection":"collection1",
  "node_name":"192.168.1.109:8983_solr",
  "base_url":"http://192.168.1.109:8983/solr";,
  "leader":"true"}}},
"shard2":{
  "range":"0-7fff",
  "replicas":{
{code}

Now, currently the ClusterState object is created by reading clusterstate.json, 
but I don't believe it reads /collections/, but perhaps we 
should change this?


  
> Custom Hashing
> --
>
> Key: SOLR-2592
> URL: https://issues.apache.org/jira/browse/SOLR-2592
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Affects Versions: 4.0-ALPHA
>Reporter: Noble Paul
> Attachments: dbq_fix.patch, pluggable_sharding.patch, 
> pluggable_sharding_V2.patch, SOLR-2592.patch, SOLR-2592_r1373086.patch, 
> SOLR-2592_r1384367.patch, SOLR-2592_rev_2.patch, 
> SOLR_2592_solr_4_0_0_BETA_ShardPartitioner.patch
>
>
> If the data in a cloud can be partitioned on some criteria (say range, hash, 
> attribute value etc) It will be easy to narrow down the search to a smaller 
> subset of shards and in effect can achieve more efficient search.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2592) Custom Hashing

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503807#comment-13503807
 ] 

Yonik Seeley commented on SOLR-2592:


Hi Michael, while I was reviewing your patch, I realized that we want something 
that clients (like cloud aware SolrJ) can easily get to.
So instead of config in solrconfig.xml that defines a parser for the core, it 
seems like this should be a property of the collection?

The HashPartitioner object is currently on the ClusterState object, but it 
really seems like this needs to be per-collection (or per config-set?).

Currently, there is a /collections/collection1 node with
{code}
{"configName":"myconf"}
{code}

We could add a "partitioner" or "shardPartitioner" attribute.

And of course there is /clusterstate.json with
{code}
{"collection1":{
"shard1":{
  "range":"8000-",
  "replicas":{"192.168.1.109:8983_solr_collection1":{
  "shard":"shard1",
  "roles":null,
  "state":"active",
  "core":"collection1",
  "collection":"collection1",
  "node_name":"192.168.1.109:8983_solr",
  "base_url":"http://192.168.1.109:8983/solr";,
  "leader":"true"}}},
"shard2":{
  "range":"0-7fff",
  "replicas":{
{code}

Now, currently the ClusterState object is created by reading clusterstate.json, 
but I don't believe it reads /collections/, but perhaps we 
should change this?



> Custom Hashing
> --
>
> Key: SOLR-2592
> URL: https://issues.apache.org/jira/browse/SOLR-2592
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Affects Versions: 4.0-ALPHA
>Reporter: Noble Paul
> Attachments: dbq_fix.patch, pluggable_sharding.patch, 
> pluggable_sharding_V2.patch, SOLR-2592.patch, SOLR-2592_r1373086.patch, 
> SOLR-2592_r1384367.patch, SOLR-2592_rev_2.patch, 
> SOLR_2592_solr_4_0_0_BETA_ShardPartitioner.patch
>
>
> If the data in a cloud can be partitioned on some criteria (say range, hash, 
> attribute value etc) It will be easy to narrow down the search to a smaller 
> subset of shards and in effect can achieve more efficient search.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Possible sharded and replicated replacement for ExternalFileFields in SolrCloud

[Simone Gianni]

2012/11/23 Simone Gianni 

> 3) how does creation/synchronization of a new replica happens? (so where
> should I have to plug to replicate also the "external files")
>
>
As far as i understood, SolrCloud forwards add/update to connected
replicas, but when a replica starts up it uses the ReplicationHandler
together with RecoveryStrategy.

Is there any event triggered during this phase, to plug replication of
"external files"?

Simone

Re: Multiple shards for one collection on the same Solr server

[Mark Miller]

The Collections API was fairly rushed - so that 4.0 had something easier than 
the CoreAdmin API.

Due to that, it has a variety of limitations:

1. It only picks instances for a collection one way - randomly from the list of 
live instances. This means it's no good for multiple shards on the same 
instance. You should have enough instances to satisfy numShards X 
replicationFactor (although just being short on replicationFactor will 
currently just use what is there)

2. It randomly chooses which instances to use rather than allowing manual 
specification or looking at existing cores.

3. You cannot get responses of success or failure other than polling for the 
expected results later.


Someone has a patch up for 3 that I hope to look at soon - others have 
contributed bug fixes that will be in 4.1. We still need to add the ability to 
control placement in other ways though.

I would say there are def plans, but I don't personally know exactly when I'll 
find the time for it, if others don't jump in.

- Mark

On Nov 26, 2012, at 4:57 AM, Per Steffensen  wrote:

> Hi
> 
> Before upgrading to Solr 4.0.0 we used to handle our collection creation 
> ourselves, by creating each shards through the low-level CoreAdmin API. We 
> used to create multiple shards under the same collection on each Solr server. 
> Performance tests has shown that this is a good idea, and it is also a good 
> idea for easy elasticity later on - it is much easier to move an entire 
> existing shards from one Solr server to another one that just joined the 
> cluter than it is to split an exsiting shard among the Solr that used to run 
> it and the new Solr.
> 
> Now we are trying to migrate to the Solr Collection API for creation of 
> collections, but it seems like it will not accept multiple shards under the 
> same collection running on the same Solr server. E.g. if we have 4 Solr 
> servers and ask to have a collection created with 8 shards, all 8 shards will 
> be "created" but only 4 of them will acutally run - one on each Solr server.
> 
> Is the a good reason why Solr does not allow multiple shards under the same 
> collection to run on the same Solr server, or is it just made this way "by 
> coincidence"? In general I seek info on the matter - is it planed for later? 
> etc.
> 
> Thanks!
> 
> Regards, Per Steffensen
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
> 


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2141) NullPointerException when using escapeSql function

[Dominik Siebel (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503763#comment-13503763
 ] 

Dominik Siebel commented on SOLR-2141:
--

[~jdyer]
After I finally had some time to check whether the indexed results (after the 
update) are as I expect them to be I had to find out that the importer did not 
throw any NPE's now, instead the fields are just missing.
I have 7 fields defined in my nested entity that use escapeSql. Those are the 
exact fields that don't get indexed now :-/
Any other suggestions? ;)

I will have a look at the changed DIH code, maybe i can figure out where this 
is coming from.

> NullPointerException when using escapeSql function
> --
>
> Key: SOLR-2141
> URL: https://issues.apache.org/jira/browse/SOLR-2141
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - DataImportHandler
>Affects Versions: 1.4.1
> Environment: openjdk 1.6.0 b12
>Reporter: Edward Rudd
>Assignee: Koji Sekiguchi
> Attachments: dih-config.xml, dih-file.xml, SOLR-2141.b341f5b.patch, 
> SOLR-2141.patch, SOLR-2141.patch, SOLR-2141-sample.patch, SOLR-2141-test.patch
>
>
> I have two entities defined, nested in each other..
> 
>  
>
> 
> Now, when I run that it bombs on any article where subcategory = '' (it's a 
> NOT NULL column so empty string is there)  If i do where subcategory!='' in 
> the article query it works fine (aside from not pulling in all of the 
> articles).
> org.apache.solr.handler.dataimport.DataImportHandlerException: 
> java.lang.NullPointerException
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:424)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:383)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:242)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:180)
> at 
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:331)
> at 
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:389)
> at 
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:370)
> Caused by: java.lang.NullPointerException
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$1.evaluate(EvaluatorBag.java:75)
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$5.get(EvaluatorBag.java:216)
> at 
> org.apache.solr.handler.dataimport.EvaluatorBag$5.get(EvaluatorBag.java:204)
> at 
> org.apache.solr.handler.dataimport.VariableResolverImpl.resolve(VariableResolverImpl.java:107)
> at 
> org.apache.solr.handler.dataimport.TemplateString.fillTokens(TemplateString.java:81)
> at 
> org.apache.solr.handler.dataimport.TemplateString.replaceTokens(TemplateString.java:75)
> at 
> org.apache.solr.handler.dataimport.VariableResolverImpl.replaceTokens(VariableResolverImpl.java:87)
> at 
> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:71)
> at 
> org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:237)
> at 
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:357)
> ... 6 more

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Multiple shards for one collection on the same Solr server

[Per Steffensen]

Hi

Before upgrading to Solr 4.0.0 we used to handle our collection creation 
ourselves, by creating each shards through the low-level CoreAdmin API. 
We used to create multiple shards under the same collection on each Solr 
server. Performance tests has shown that this is a good idea, and it is 
also a good idea for easy elasticity later on - it is much easier to 
move an entire existing shards from one Solr server to another one that 
just joined the cluter than it is to split an exsiting shard among the 
Solr that used to run it and the new Solr.


Now we are trying to migrate to the Solr Collection API for creation of 
collections, but it seems like it will not accept multiple shards under 
the same collection running on the same Solr server. E.g. if we have 4 
Solr servers and ask to have a collection created with 8 shards, all 8 
shards will be "created" but only 4 of them will acutally run - one on 
each Solr server.


Is the a good reason why Solr does not allow multiple shards under the 
same collection to run on the same Solr server, or is it just made this 
way "by coincidence"? In general I seek info on the matter - is it 
planed for later? etc.


Thanks!

Regards, Per Steffensen

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3304) Add Solr support for the new Lucene spatial module

[Ari Maniatis (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503660#comment-13503660
 ] 

Ari Maniatis commented on SOLR-3304:


Thanks David. I've asked my developer to follow up on the mailing list with his 
detailed questions.

http://apache.markmail.org/thread/af5kg26rdbdhkftw

> Add Solr support for the new Lucene spatial module
> --
>
> Key: SOLR-3304
> URL: https://issues.apache.org/jira/browse/SOLR-3304
> Project: Solr
>  Issue Type: New Feature
>Affects Versions: 4.0-ALPHA
>Reporter: Bill Bell
>Assignee: David Smiley
>Priority: Critical
>  Labels: spatial
> Fix For: 4.0
>
> Attachments: SOLR-3304_Solr_fields_for_Lucene_spatial_module 
> (fieldName in Strategy) - indexableFields.patch, 
> SOLR-3304_Solr_fields_for_Lucene_spatial_module (fieldName in 
> Strategy).patch, SOLR-3304_Solr_fields_for_Lucene_spatial_module.patch, 
> SOLR-3304_Solr_fields_for_Lucene_spatial_module.patch, 
> SOLR-3304_Solr_fields_for_Lucene_spatial_module.patch, 
> SOLR-3304_Solr_fields_for_Lucene_spatial_module.patch, 
> SOLR-3304-strategy-getter-fixed.patch
>
>
> Get the Solr spatial module integrated with the lucene spatial module.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org