[jira] Resolved: (SOLR-977) version=2.2 is not necessary for wt=javabin
[ https://issues.apache.org/jira/browse/SOLR-977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noble Paul resolved SOLR-977. - Resolution: Fixed committed revision : 776961 version=2.2 is not necessary for wt=javabin --- Key: SOLR-977 URL: https://issues.apache.org/jira/browse/SOLR-977 Project: Solr Issue Type: Improvement Components: clients - java Reporter: Noble Paul Assignee: Noble Paul Priority: Trivial Fix For: 1.4 Attachments: SOLR-977.patch CommonsHttpSolrServer can drop the version=2.2 if the wt=javabin -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (SOLR-935) DataImportHandler: Add logging to record failure to acquire lock by DataImporter for a given request
[ https://issues.apache.org/jira/browse/SOLR-935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noble Paul reassigned SOLR-935: --- Assignee: Noble Paul DataImportHandler: Add logging to record failure to acquire lock by DataImporter for a given request - Key: SOLR-935 URL: https://issues.apache.org/jira/browse/SOLR-935 Project: Solr Issue Type: Improvement Components: contrib - DataImportHandler Environment: Java 6, Tomcat 6.0.18 Reporter: Kay Kay Assignee: Noble Paul Fix For: 1.4 Attachments: SOLR-935.patch, SOLR-935.patch Original Estimate: 1h Remaining Estimate: 1h There is a possibility of 2 threads to be in DataImporter:runCmd, until before importLock.tryLock() method and then depending on the scheduling - one of them is allowed to pass through from then . We need to log the failure of the other as to unable to start because of the failure to acquire the mutex, to distinguish between successful start of import and failure to do so. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-935) DataImportHandler: Add logging to record failure to acquire lock by DataImporter for a given request
[ https://issues.apache.org/jira/browse/SOLR-935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12711486#action_12711486 ] Noble Paul commented on SOLR-935: - committed revision : 776965 DataImportHandler: Add logging to record failure to acquire lock by DataImporter for a given request - Key: SOLR-935 URL: https://issues.apache.org/jira/browse/SOLR-935 Project: Solr Issue Type: Improvement Components: contrib - DataImportHandler Environment: Java 6, Tomcat 6.0.18 Reporter: Kay Kay Assignee: Noble Paul Fix For: 1.4 Attachments: SOLR-935.patch, SOLR-935.patch, SOLR-935.patch Original Estimate: 1h Remaining Estimate: 1h There is a possibility of 2 threads to be in DataImporter:runCmd, until before importLock.tryLock() method and then depending on the scheduling - one of them is allowed to pass through from then . We need to log the failure of the other as to unable to start because of the failure to acquire the mutex, to distinguish between successful start of import and failure to do so. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-935) DataImportHandler: Add logging to record failure to acquire lock by DataImporter for a given request
[ https://issues.apache.org/jira/browse/SOLR-935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noble Paul updated SOLR-935: Fix Version/s: (was: 1.4) 1.5 I feel that the original concern is addressed. I am pushing it to 1.5 so that we can have a detailed look at this later DataImportHandler: Add logging to record failure to acquire lock by DataImporter for a given request - Key: SOLR-935 URL: https://issues.apache.org/jira/browse/SOLR-935 Project: Solr Issue Type: Improvement Components: contrib - DataImportHandler Environment: Java 6, Tomcat 6.0.18 Reporter: Kay Kay Assignee: Noble Paul Fix For: 1.5 Attachments: SOLR-935.patch, SOLR-935.patch, SOLR-935.patch Original Estimate: 1h Remaining Estimate: 1h There is a possibility of 2 threads to be in DataImporter:runCmd, until before importLock.tryLock() method and then depending on the scheduling - one of them is allowed to pass through from then . We need to log the failure of the other as to unable to start because of the failure to acquire the mutex, to distinguish between successful start of import and failure to do so. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (SOLR-1175) disable replication on master side
[ https://issues.apache.org/jira/browse/SOLR-1175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noble Paul reassigned SOLR-1175: Assignee: Noble Paul disable replication on master side -- Key: SOLR-1175 URL: https://issues.apache.org/jira/browse/SOLR-1175 Project: Solr Issue Type: New Feature Components: replication (java) Affects Versions: 1.4 Environment: any Reporter: Jianhan Assignee: Noble Paul Priority: Minor Fix For: 1.4 Original Estimate: 4h Remaining Estimate: 4h In an environment where one master and several slaves are deployed, it usually takes more effort to discover all the slaves and disable replication on slave side (which is available right now), and it would be much easier to disable it on master instance (when, for example, there is a need to rebuild the index, yet search has to continue). The following is the original email describing a scenario when this feature is needed. Hi, Occasionally, we want to build our indexes from scratch, and during this period we want our search continue to work. Here are the steps that I think will do it 1. on all slaves: disable replication 2. on master: stop the server 3. on master: delete all the documents 4. on master: restart the server 5. on master: index all documents 6. on slaves: enable replication The only problem is: step 1 and 6. We may schedule any time to rebuild indexes and it is an automated process. It is possible to let the master to disable replication on all slaves, but then we have to discover all the slaves automatically, also exceptions may happen, e.g. a slave may be down at the time and then restarted later on. Anyhow it becomes an unreliable process. So I am thinking of somehow disabling replication on the master side during reindex, i.e. set a state on master so that any request for replication will be ignored. That all the steps described above will be on master side only. Is that possible? By the way, I am talking about solr 1.4. I looked at how 1.3 works, and noticed that in 1.3 there is a way to disable replication on master side: shutdown rsyncd, so I guess it would be nice to have something equivalent in solr 1.4. Thanks, Jianhan -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1175) disable replication on master side
[ https://issues.apache.org/jira/browse/SOLR-1175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noble Paul updated SOLR-1175: - Attachment: SOLR-1175.patch disable replication on master side -- Key: SOLR-1175 URL: https://issues.apache.org/jira/browse/SOLR-1175 Project: Solr Issue Type: New Feature Components: replication (java) Affects Versions: 1.4 Environment: any Reporter: Jianhan Assignee: Noble Paul Priority: Minor Fix For: 1.4 Attachments: SOLR-1175.patch Original Estimate: 4h Remaining Estimate: 4h In an environment where one master and several slaves are deployed, it usually takes more effort to discover all the slaves and disable replication on slave side (which is available right now), and it would be much easier to disable it on master instance (when, for example, there is a need to rebuild the index, yet search has to continue). The following is the original email describing a scenario when this feature is needed. Hi, Occasionally, we want to build our indexes from scratch, and during this period we want our search continue to work. Here are the steps that I think will do it 1. on all slaves: disable replication 2. on master: stop the server 3. on master: delete all the documents 4. on master: restart the server 5. on master: index all documents 6. on slaves: enable replication The only problem is: step 1 and 6. We may schedule any time to rebuild indexes and it is an automated process. It is possible to let the master to disable replication on all slaves, but then we have to discover all the slaves automatically, also exceptions may happen, e.g. a slave may be down at the time and then restarted later on. Anyhow it becomes an unreliable process. So I am thinking of somehow disabling replication on the master side during reindex, i.e. set a state on master so that any request for replication will be ignored. That all the steps described above will be on master side only. Is that possible? By the way, I am talking about solr 1.4. I looked at how 1.3 works, and noticed that in 1.3 there is a way to disable replication on master side: shutdown rsyncd, so I guess it would be nice to have something equivalent in solr 1.4. Thanks, Jianhan -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1175) disable replication on master side
[ https://issues.apache.org/jira/browse/SOLR-1175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12711492#action_12711492 ] Noble Paul commented on SOLR-1175: -- two commands are added 'disableReplication' and 'enableReplication' disable replication on master side -- Key: SOLR-1175 URL: https://issues.apache.org/jira/browse/SOLR-1175 Project: Solr Issue Type: New Feature Components: replication (java) Affects Versions: 1.4 Environment: any Reporter: Jianhan Assignee: Noble Paul Priority: Minor Fix For: 1.4 Attachments: SOLR-1175.patch Original Estimate: 4h Remaining Estimate: 4h In an environment where one master and several slaves are deployed, it usually takes more effort to discover all the slaves and disable replication on slave side (which is available right now), and it would be much easier to disable it on master instance (when, for example, there is a need to rebuild the index, yet search has to continue). The following is the original email describing a scenario when this feature is needed. Hi, Occasionally, we want to build our indexes from scratch, and during this period we want our search continue to work. Here are the steps that I think will do it 1. on all slaves: disable replication 2. on master: stop the server 3. on master: delete all the documents 4. on master: restart the server 5. on master: index all documents 6. on slaves: enable replication The only problem is: step 1 and 6. We may schedule any time to rebuild indexes and it is an automated process. It is possible to let the master to disable replication on all slaves, but then we have to discover all the slaves automatically, also exceptions may happen, e.g. a slave may be down at the time and then restarted later on. Anyhow it becomes an unreliable process. So I am thinking of somehow disabling replication on the master side during reindex, i.e. set a state on master so that any request for replication will be ignored. That all the steps described above will be on master side only. Is that possible? By the way, I am talking about solr 1.4. I looked at how 1.3 works, and noticed that in 1.3 there is a way to disable replication on master side: shutdown rsyncd, so I guess it would be nice to have something equivalent in solr 1.4. Thanks, Jianhan -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-1175) disable replication on master side
[ https://issues.apache.org/jira/browse/SOLR-1175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noble Paul resolved SOLR-1175. -- Resolution: Fixed committed revision: 776978 disable replication on master side -- Key: SOLR-1175 URL: https://issues.apache.org/jira/browse/SOLR-1175 Project: Solr Issue Type: New Feature Components: replication (java) Affects Versions: 1.4 Environment: any Reporter: Jianhan Assignee: Noble Paul Priority: Minor Fix For: 1.4 Attachments: SOLR-1175.patch Original Estimate: 4h Remaining Estimate: 4h In an environment where one master and several slaves are deployed, it usually takes more effort to discover all the slaves and disable replication on slave side (which is available right now), and it would be much easier to disable it on master instance (when, for example, there is a need to rebuild the index, yet search has to continue). The following is the original email describing a scenario when this feature is needed. Hi, Occasionally, we want to build our indexes from scratch, and during this period we want our search continue to work. Here are the steps that I think will do it 1. on all slaves: disable replication 2. on master: stop the server 3. on master: delete all the documents 4. on master: restart the server 5. on master: index all documents 6. on slaves: enable replication The only problem is: step 1 and 6. We may schedule any time to rebuild indexes and it is an automated process. It is possible to let the master to disable replication on all slaves, but then we have to discover all the slaves automatically, also exceptions may happen, e.g. a slave may be down at the time and then restarted later on. Anyhow it becomes an unreliable process. So I am thinking of somehow disabling replication on the master side during reindex, i.e. set a state on master so that any request for replication will be ignored. That all the steps described above will be on master side only. Is that possible? By the way, I am talking about solr 1.4. I looked at how 1.3 works, and noticed that in 1.3 there is a way to disable replication on master side: shutdown rsyncd, so I guess it would be nice to have something equivalent in solr 1.4. Thanks, Jianhan -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (SOLR-1129) SolrJ cannot bind dynamic fields to beans
[
https://issues.apache.org/jira/browse/SOLR-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noble Paul reassigned SOLR-1129:
Assignee: Noble Paul
SolrJ cannot bind dynamic fields to beans
-
Key: SOLR-1129
URL: https://issues.apache.org/jira/browse/SOLR-1129
Project: Solr
Issue Type: Improvement
Components: clients - java
Affects Versions: 1.3
Reporter: Noble Paul
Assignee: Noble Paul
Fix For: 1.4
Attachments: SOLR-1129.patch
SolrJ does not support binding of dynamic fields to bean fields
The field declaration could be as follows
{code:java}
@Field(*_s)
public String anyString;
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-993) VariableResolverImpl addNamespace overwrites entire namespace instead of adding
[ https://issues.apache.org/jira/browse/SOLR-993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noble Paul updated SOLR-993: Fix Version/s: (was: 1.4) 1.5 there is no compelling reason to change the behavior now. we can take a relook at this in 1.5 VariableResolverImpl addNamespace overwrites entire namespace instead of adding --- Key: SOLR-993 URL: https://issues.apache.org/jira/browse/SOLR-993 Project: Solr Issue Type: Bug Components: contrib - DataImportHandler Affects Versions: 1.4 Reporter: Jared Flatow Assignee: Noble Paul Fix For: 1.5 Attachments: SOLR-993.patch, SOLR-993.patch, SOLR-993b.patch, SOLR-993c.patch, SOLR-993c.patch, SOLR-993c.patch Original Estimate: 0.08h Remaining Estimate: 0.08h The addNamespace method in VariableResolverImpl does not so much add the namespace as overwrite it. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (SOLR-1021) NullPointerException(s) in SolrJ when using stats and result set is empty
[ https://issues.apache.org/jira/browse/SOLR-1021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noble Paul reassigned SOLR-1021: Assignee: Noble Paul NullPointerException(s) in SolrJ when using stats and result set is empty - Key: SOLR-1021 URL: https://issues.apache.org/jira/browse/SOLR-1021 Project: Solr Issue Type: Bug Components: clients - java Affects Versions: 1.4 Environment: Tomcat 6, Windows Reporter: Arie Fishler Assignee: Noble Paul Priority: Minor Fix For: 1.4 Attachments: QueryResponse.java, SOLR-1021.patch A NullPointerException is thrown at line 51 of class FieldStatsInfo since the returned stats information for the fname (ctor variable) is null and not checked for being so. Seems like this occurs when the search returns with no results but there were some setGetFieldStatistics defined for some fields. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1174) Logging Admin Servlet broken for multicore, cannot submit form
[ https://issues.apache.org/jira/browse/SOLR-1174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-1174: Affects Version/s: 1.3 Assignee: Shalin Shekhar Mangar Logging Admin Servlet broken for multicore, cannot submit form -- Key: SOLR-1174 URL: https://issues.apache.org/jira/browse/SOLR-1174 Project: Solr Issue Type: Bug Components: web gui Affects Versions: 1.3 Reporter: Jacob Singh Assignee: Shalin Shekhar Mangar Fix For: 1.4 Attachments: logging_servlet_multicore.diff See SOLR-554 for the original improvement. This has a bug in it. If I submit the form using multicore, it goes to /solr/admin/logging (not the right path). I'm guessing this is because of the path trickery in SolrDispathFilter but I can't follow what the best procedure is to fix it. The simplest thing to do is to not use request.getRequestURI() because a form with no action submits to itself. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1021) NullPointerException(s) in SolrJ when using stats and result set is empty
[ https://issues.apache.org/jira/browse/SOLR-1021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12711499#action_12711499 ] Noble Paul commented on SOLR-1021: -- there is a fix that is already checked in Rohit ,is the fix good enough ? NullPointerException(s) in SolrJ when using stats and result set is empty - Key: SOLR-1021 URL: https://issues.apache.org/jira/browse/SOLR-1021 Project: Solr Issue Type: Bug Components: clients - java Affects Versions: 1.4 Environment: Tomcat 6, Windows Reporter: Arie Fishler Assignee: Noble Paul Priority: Minor Fix For: 1.4 Attachments: QueryResponse.java, SOLR-1021.patch A NullPointerException is thrown at line 51 of class FieldStatsInfo since the returned stats information for the fname (ctor variable) is null and not checked for being so. Seems like this occurs when the search returns with no results but there were some setGetFieldStatistics defined for some fields. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1154) allow specifying solr configuration file through system property to simplify deployment procedure in certain cases
[
https://issues.apache.org/jira/browse/SOLR-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12711500#action_12711500
]
Noble Paul commented on SOLR-1154:
--
do we still need to pursue this?
allow specifying solr configuration file through system property to simplify
deployment procedure in certain cases
--
Key: SOLR-1154
URL: https://issues.apache.org/jira/browse/SOLR-1154
Project: Solr
Issue Type: Improvement
Affects Versions: 1.4
Reporter: Jianhan
Priority: Minor
Fix For: 1.4
Attachments: SOLR-1154.patch, SOLR-1154.patch
Original Estimate: 5h
Remaining Estimate: 5h
Hi,
I wanted to use this parameter to specify different solr configuration files
for master and slave to simplify deployment procedure. Unfortunately, I can't
dynamically replace the value of this parameter. Basically, what I want is
filter
filter-nameSolrRequestFilter/filter-name
filter-classorg.apache.solr.servlet.SolrDispatchFilter/filter-class
init-param
param-namesolrconfig-filename/param-name
param-valuesolrconfig-master.xml/param-value
/init-param
/filter
for master instance, and
filter
filter-nameSolrRequestFilter/filter-name
filter-classorg.apache.solr.servlet.SolrDispatchFilter/filter-class
init-param
param-namesolrconfig-filename/param-name
param-valuesolrconfig-slave.xml/param-value
/init-param
/filter
for slave instance.
Ideally, if I can use system property for its value like in solrconfig.xml.
For example,
filter
filter-nameSolrRequestFilter/filter-name
filter-classorg.apache.solr.servlet.SolrDispatchFilter/filter-class
init-param
param-namesolrconfig-filename/param-name
param-value${solr.config.filename: solrconfig.xml}/param-value
/init-param
/filter
but I learned that in general we can't use system property in web.xml.
I realize that I can use replication of config file to achieve this, but I
thought that creates unnecessary dependencies for slaves on master instance.
So here is my proposal:
make SolrDispatchFilter look up another init parameter, say
'solrconfig-filename-property', and its value is a system property name, and
if this property is set, we get the file name, otherwise nothing happens (of
course, if both exist, 'solrconfig-filename' takes precedence). This will
give us maximum flexibility of specifying configuration files for different
instances.
Your thoughts?
Thanks,
Jianhan
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (SOLR-1154) allow specifying solr configuration file through system property to simplify deployment procedure in certain cases
[
https://issues.apache.org/jira/browse/SOLR-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noble Paul reassigned SOLR-1154:
Assignee: Noble Paul
allow specifying solr configuration file through system property to simplify
deployment procedure in certain cases
--
Key: SOLR-1154
URL: https://issues.apache.org/jira/browse/SOLR-1154
Project: Solr
Issue Type: Improvement
Affects Versions: 1.4
Reporter: Jianhan
Assignee: Noble Paul
Priority: Minor
Fix For: 1.4
Attachments: SOLR-1154.patch, SOLR-1154.patch
Original Estimate: 5h
Remaining Estimate: 5h
Hi,
I wanted to use this parameter to specify different solr configuration files
for master and slave to simplify deployment procedure. Unfortunately, I can't
dynamically replace the value of this parameter. Basically, what I want is
filter
filter-nameSolrRequestFilter/filter-name
filter-classorg.apache.solr.servlet.SolrDispatchFilter/filter-class
init-param
param-namesolrconfig-filename/param-name
param-valuesolrconfig-master.xml/param-value
/init-param
/filter
for master instance, and
filter
filter-nameSolrRequestFilter/filter-name
filter-classorg.apache.solr.servlet.SolrDispatchFilter/filter-class
init-param
param-namesolrconfig-filename/param-name
param-valuesolrconfig-slave.xml/param-value
/init-param
/filter
for slave instance.
Ideally, if I can use system property for its value like in solrconfig.xml.
For example,
filter
filter-nameSolrRequestFilter/filter-name
filter-classorg.apache.solr.servlet.SolrDispatchFilter/filter-class
init-param
param-namesolrconfig-filename/param-name
param-value${solr.config.filename: solrconfig.xml}/param-value
/init-param
/filter
but I learned that in general we can't use system property in web.xml.
I realize that I can use replication of config file to achieve this, but I
thought that creates unnecessary dependencies for slaves on master instance.
So here is my proposal:
make SolrDispatchFilter look up another init parameter, say
'solrconfig-filename-property', and its value is a system property name, and
if this property is set, we get the file name, otherwise nothing happens (of
course, if both exist, 'solrconfig-filename' takes precedence). This will
give us maximum flexibility of specifying configuration files for different
instances.
Your thoughts?
Thanks,
Jianhan
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-1174) Logging Admin Servlet broken for multicore, cannot submit form
[ https://issues.apache.org/jira/browse/SOLR-1174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar resolved SOLR-1174. - Resolution: Fixed Committed revision 776984. Thanks Jacob! Logging Admin Servlet broken for multicore, cannot submit form -- Key: SOLR-1174 URL: https://issues.apache.org/jira/browse/SOLR-1174 Project: Solr Issue Type: Bug Components: web gui Affects Versions: 1.3 Reporter: Jacob Singh Assignee: Shalin Shekhar Mangar Fix For: 1.4 Attachments: logging_servlet_multicore.diff See SOLR-554 for the original improvement. This has a bug in it. If I submit the form using multicore, it goes to /solr/admin/logging (not the right path). I'm guessing this is because of the path trickery in SolrDispathFilter but I can't follow what the best procedure is to fix it. The simplest thing to do is to not use request.getRequestURI() because a form with no action submits to itself. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1116) Add a Binary FieldType
[ https://issues.apache.org/jira/browse/SOLR-1116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12711502#action_12711502 ] Noble Paul commented on SOLR-1116: -- hi Andrzej from the wikipedia documentation what I understand is that browsers support standard base64 not the url-safe version Add a Binary FieldType -- Key: SOLR-1116 URL: https://issues.apache.org/jira/browse/SOLR-1116 Project: Solr Issue Type: New Feature Components: search Affects Versions: 1.3 Reporter: Noble Paul Assignee: Noble Paul Fix For: 1.4 Attachments: SOLR-1116.patch, SOLR-1116.patch Lucene supports binary data for field but Solr has no corresponding field type. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1023) StatsComponent should support dates (and other non-numeric fields)
[ https://issues.apache.org/jira/browse/SOLR-1023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-1023: Fix Version/s: (was: 1.4) 1.5 StatsComponent should support dates (and other non-numeric fields) -- Key: SOLR-1023 URL: https://issues.apache.org/jira/browse/SOLR-1023 Project: Solr Issue Type: New Feature Affects Versions: 1.4 Environment: Mac OS 10.5, java version 1.5.0_16 Reporter: Peter Wolanin Fix For: 1.5 Currently, the StatsComponent only supports single-value numeric fields: http://wiki.apache.org/solr/StatsComponent trying to use it with a date field I get an exception like: java.lang.NumberFormatException: For input string: 2009-01-27T20:04:04Z trying to use it with a string I get an error 400 Stats are valid for single valued numeric values. For constructing date facets it would be very useful to be able to get the minimum and maximum date from a DateField within a set of documents. In general, it could be useful to get the minimum and maximum from any field type that can be compared, though that's of less importance. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1023) StatsComponent should support dates (and other non-numeric fields)
[ https://issues.apache.org/jira/browse/SOLR-1023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-1023: Description: Currently, the StatsComponent only supports single-value numeric fields: http://wiki.apache.org/solr/StatsComponent trying to use it with a date field I get an exception like: java.lang.NumberFormatException: For input string: 2009-01-27T20:04:04Z trying to use it with a string I get an error 400 Stats are valid for single valued numeric values. For constructing date facets it would be very useful to be able to get the minimum and maximum date from a DateField within a set of documents. In general, it could be useful to get the minimum and maximum from any field type that can be compared, though that's of less importance. was: Currently, the StatsComponent only supports single-value numeric fields: http://wiki.apache.org/solr/StatsComponent trying to use it with a date field I get an exception like: java.lang.NumberFormatException: For input string: 2009-01-27T20:04:04Z trying to use it with a string I get an error 400 Stats are valid for single valued numeric values. For constructing date facets it would be very useful to be able to get the minimum and maximum date from a DateField within a set of documents. In general, it could be useful to get the minimum and maximum from any field type that can be compared, though that's of less importance. Marking for 1.5 because although it is useful, there is no patch yet. StatsComponent should support dates (and other non-numeric fields) -- Key: SOLR-1023 URL: https://issues.apache.org/jira/browse/SOLR-1023 Project: Solr Issue Type: New Feature Affects Versions: 1.4 Environment: Mac OS 10.5, java version 1.5.0_16 Reporter: Peter Wolanin Fix For: 1.5 Currently, the StatsComponent only supports single-value numeric fields: http://wiki.apache.org/solr/StatsComponent trying to use it with a date field I get an exception like: java.lang.NumberFormatException: For input string: 2009-01-27T20:04:04Z trying to use it with a string I get an error 400 Stats are valid for single valued numeric values. For constructing date facets it would be very useful to be able to get the minimum and maximum date from a DateField within a set of documents. In general, it could be useful to get the minimum and maximum from any field type that can be compared, though that's of less importance. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Solr nightly build failure
init-forrest-entities: [mkdir] Created dir: /tmp/apache-solr-nightly/build [mkdir] Created dir: /tmp/apache-solr-nightly/build/web compile-solrj: [mkdir] Created dir: /tmp/apache-solr-nightly/build/solrj [javac] Compiling 82 source files to /tmp/apache-solr-nightly/build/solrj [javac] Note: Some input files use or override a deprecated API. [javac] Note: Recompile with -Xlint:deprecation for details. [javac] Note: Some input files use unchecked or unsafe operations. [javac] Note: Recompile with -Xlint:unchecked for details. compile: [mkdir] Created dir: /tmp/apache-solr-nightly/build/solr [javac] Compiling 370 source files to /tmp/apache-solr-nightly/build/solr [javac] Note: Some input files use or override a deprecated API. [javac] Note: Recompile with -Xlint:deprecation for details. [javac] Note: Some input files use unchecked or unsafe operations. [javac] Note: Recompile with -Xlint:unchecked for details. compileTests: [mkdir] Created dir: /tmp/apache-solr-nightly/build/tests [javac] Compiling 158 source files to /tmp/apache-solr-nightly/build/tests [javac] Note: Some input files use or override a deprecated API. [javac] Note: Recompile with -Xlint:deprecation for details. [javac] Note: Some input files use unchecked or unsafe operations. [javac] Note: Recompile with -Xlint:unchecked for details. junit: [mkdir] Created dir: /tmp/apache-solr-nightly/build/test-results [junit] Running org.apache.solr.BasicFunctionalityTest [junit] Tests run: 19, Failures: 0, Errors: 0, Time elapsed: 16.389 sec [junit] Running org.apache.solr.ConvertedLegacyTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 7.574 sec [junit] Running org.apache.solr.DisMaxRequestHandlerTest [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 5.171 sec [junit] Running org.apache.solr.EchoParamsTest [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 2.377 sec [junit] Running org.apache.solr.OutputWriterTest [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 2.471 sec [junit] Running org.apache.solr.SampleTest [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 2.197 sec [junit] Running org.apache.solr.SolrInfoMBeanTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.868 sec [junit] Running org.apache.solr.TestDistributedSearch [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 29.953 sec [junit] Running org.apache.solr.TestTrie [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 6.267 sec [junit] Running org.apache.solr.analysis.DoubleMetaphoneFilterFactoryTest [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.394 sec [junit] Running org.apache.solr.analysis.DoubleMetaphoneFilterTest [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 0.384 sec [junit] Running org.apache.solr.analysis.EnglishPorterFilterFactoryTest [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 1.047 sec [junit] Running org.apache.solr.analysis.HTMLStripReaderTest [junit] Tests run: 9, Failures: 0, Errors: 0, Time elapsed: 0.847 sec [junit] Running org.apache.solr.analysis.LengthFilterTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.965 sec [junit] Running org.apache.solr.analysis.SnowballPorterFilterFactoryTest [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 1.212 sec [junit] Running org.apache.solr.analysis.TestBufferedTokenStream [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 1.032 sec [junit] Running org.apache.solr.analysis.TestCapitalizationFilter [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 1.117 sec [junit] Running org.apache.solr.analysis.TestCharFilter [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.389 sec [junit] Running org.apache.solr.analysis.TestHyphenatedWordsFilter [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.909 sec [junit] Running org.apache.solr.analysis.TestKeepFilterFactory [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.855 sec [junit] Running org.apache.solr.analysis.TestKeepWordFilter [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.007 sec [junit] Running org.apache.solr.analysis.TestMappingCharFilter [junit] Tests run: 11, Failures: 0, Errors: 0, Time elapsed: 0.43 sec [junit] Running org.apache.solr.analysis.TestMappingCharFilterFactory [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.435 sec [junit] Running org.apache.solr.analysis.TestPatternReplaceFilter [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 1.818 sec [junit] Running org.apache.solr.analysis.TestPatternTokenizerFactory [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.013 sec [junit] Running org.apache.solr.analysis.TestPhoneticFilter
Build failed in Hudson: Solr-trunk #808
See http://hudson.zones.apache.org/hudson/job/Solr-trunk/808/changes Changes: [shalin] SOLR-1174 -- Fix Logging admin form submit url for multicore [noble] forgot to mention author name [noble] SOLR-1175 disable/enable replication from master [noble] SOLR-935 log a warning if it fails to acquire a lock [noble] SOLR-977 version must be set as 1 [noble] SOLR-1153 deltaImportQuery should be honored on child entities as well [koji] fix funny README.txt [yonik] SOLR-1169: SortedIntDocSet [gsingers] SOLR-769: forgot the CHANGES entry [gsingers] SOLR-769: Add clustering contrib capabilities to Solr [yonik] add performance test for range queries -- [...truncated 2148 lines...] [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 4.686 sec [junit] Running org.apache.solr.client.solrj.embedded.LargeVolumeEmbeddedTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 4.141 sec [junit] Running org.apache.solr.client.solrj.embedded.LargeVolumeJettyTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 5.223 sec [junit] Running org.apache.solr.client.solrj.embedded.MultiCoreEmbeddedTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 2.896 sec [junit] Running org.apache.solr.client.solrj.embedded.MultiCoreExampleJettyTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 5.453 sec [junit] Running org.apache.solr.client.solrj.embedded.SolrExampleEmbeddedTest [junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: 8.888 sec [junit] Running org.apache.solr.client.solrj.embedded.SolrExampleJettyTest [junit] Tests run: 9, Failures: 0, Errors: 0, Time elapsed: 15.616 sec [junit] Running org.apache.solr.client.solrj.embedded.SolrExampleStreamingTest [junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: 17.951 sec [junit] Running org.apache.solr.client.solrj.embedded.TestSolrProperties [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 6.056 sec [junit] Running org.apache.solr.client.solrj.request.TestUpdateRequestCodec [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.731 sec [junit] Running org.apache.solr.client.solrj.response.AnlysisResponseBaseTest [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.478 sec [junit] Running org.apache.solr.client.solrj.response.DocumentAnalysisResponseTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.437 sec [junit] Running org.apache.solr.client.solrj.response.FieldAnalysisResponseTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.504 sec [junit] Running org.apache.solr.client.solrj.response.QueryResponseTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.771 sec [junit] Running org.apache.solr.client.solrj.response.TestSpellCheckResponse [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 9.754 sec [junit] Running org.apache.solr.client.solrj.util.ClientUtilsTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.446 sec [junit] Running org.apache.solr.common.SolrDocumentTest [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 0.578 sec [junit] Running org.apache.solr.common.params.ModifiableSolrParamsTest [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 0.425 sec [junit] Running org.apache.solr.common.params.SolrParamTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.42 sec [junit] Running org.apache.solr.common.util.ContentStreamTest [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.467 sec [junit] Running org.apache.solr.common.util.DOMUtilTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.51 sec [junit] Running org.apache.solr.common.util.FileUtilsTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.412 sec [junit] Running org.apache.solr.common.util.IteratorChainTest [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 0.465 sec [junit] Running org.apache.solr.common.util.NamedListTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.464 sec [junit] Running org.apache.solr.common.util.TestFastInputStream [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.535 sec [junit] Running org.apache.solr.common.util.TestHash [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.6 sec [junit] Running org.apache.solr.common.util.TestNamedListCodec [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 2.093 sec [junit] Running org.apache.solr.common.util.TestXMLEscaping [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 0.465 sec [junit] Running org.apache.solr.core.AlternateDirectoryTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 4.802 sec [junit] Running org.apache.solr.core.RequestHandlersTest [junit]
[jira] Commented: (SOLR-1116) Add a Binary FieldType
[ https://issues.apache.org/jira/browse/SOLR-1116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12711528#action_12711528 ] Andrzej Bialecki commented on SOLR-1116: - Indeed! then it's not relevant here. +0 from me for the regular base64. Add a Binary FieldType -- Key: SOLR-1116 URL: https://issues.apache.org/jira/browse/SOLR-1116 Project: Solr Issue Type: New Feature Components: search Affects Versions: 1.3 Reporter: Noble Paul Assignee: Noble Paul Fix For: 1.4 Attachments: SOLR-1116.patch, SOLR-1116.patch Lucene supports binary data for field but Solr has no corresponding field type. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (SOLR-1149) Make QParserPlugin and related classes extendible
[ https://issues.apache.org/jira/browse/SOLR-1149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar reassigned SOLR-1149: --- Assignee: Shalin Shekhar Mangar Make QParserPlugin and related classes extendible - Key: SOLR-1149 URL: https://issues.apache.org/jira/browse/SOLR-1149 Project: Solr Issue Type: Improvement Components: search Reporter: Kaktu Chakarabati Assignee: Shalin Shekhar Mangar Fix For: 1.4 Attachments: SOLR-1149.patch In a recent attempt to create a QParserPlugin which extends DisMaxQParser/FunctionQParser functionality, it became apparent that in the current state of these classes, it is not straight forward and in fact impossible to seriously build upon the existing code. To this end, I've refactored some of the involved classes which enabled me to reuse existing logic to great results. I thought I will share these changes and comment on their nature in the hope these will make sense to other solr developers/users, and at the very least cultivate a fruitful discussion about this particular area of the solr codebase. The relevant changes are as follows: * Renamed DismaxQParser class to DisMaxQParser ( in accordance with the apparent naming convention, e.g DisMaxQParserPlugin ) * Moved DisMaxQParser to its own .java file, making it a public class rather than its previous package-private visibility. This makes it possible for users to build upon its logic, which is considerable, and to my mind is a good place to start alot of custom QParser implementations. * Changed access modifiers for the QParser abstract base class to protected (were package-private). Again as above, it makes this object usable by user-defined classes that wish to define custom QParser classes. More generally, and on the philosophy-of-code side of things, it seems misleading to define some class members as having the default access modifier (package-private) and then letting other package-scope derived classes use these while not explicitly allowing user-defined derived classes to make use of these members. In specific i'm thinking of how DisMaxQParser makes use of these members: **not because it is derived from QParser, but because it simply resides in the same namespace** * Changed access modifier for the QueryParsing.StrParser inner class and its constructors to public. Again as in above, same issue of having same-package classes enjoy the benefit of being in the same namespace (FunctionQParser.parse() uses it like so), while user defined classes cannot. Particulary in this case it is pretty bad since this class advertises itself as a collection of utilities for query parsing in general - great resource, should probably even live elsewhere (common.utils?) * Changed Function.FunctionWeight inner class data member modifiers to protected (were default - package-private). This allowed me to inherit from FunctionQuery as well as make use of its original FunctionWeight inner class while overriding some of the latter's methods. This is in the same spirit of the changes above. Please also note this follows the common Query/Weight implementation pattern in the lucene codebase, see for example the BooleanQuery/BooleanWeight code. All in all these are relatively minor changes which unlock a great deal of functionality to 3rd party developers, which i think is ultimately a big part of what solr is all about - extendability. It is also perhaps a cue for a more serious refactoring of the QParserPlugin hierarchy, although i will leave such bold exclamations to another occasion. Attached is a patch file, having passed the usual coding-style/unit testing cycle. -Chak -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1149) Make QParserPlugin and related classes extendible
[ https://issues.apache.org/jira/browse/SOLR-1149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-1149: Attachment: SOLR-1149.patch Added an experimental API note to all the changed classes. I'll commit in a day or two. Make QParserPlugin and related classes extendible - Key: SOLR-1149 URL: https://issues.apache.org/jira/browse/SOLR-1149 Project: Solr Issue Type: Improvement Components: search Reporter: Kaktu Chakarabati Assignee: Shalin Shekhar Mangar Fix For: 1.4 Attachments: SOLR-1149.patch, SOLR-1149.patch In a recent attempt to create a QParserPlugin which extends DisMaxQParser/FunctionQParser functionality, it became apparent that in the current state of these classes, it is not straight forward and in fact impossible to seriously build upon the existing code. To this end, I've refactored some of the involved classes which enabled me to reuse existing logic to great results. I thought I will share these changes and comment on their nature in the hope these will make sense to other solr developers/users, and at the very least cultivate a fruitful discussion about this particular area of the solr codebase. The relevant changes are as follows: * Renamed DismaxQParser class to DisMaxQParser ( in accordance with the apparent naming convention, e.g DisMaxQParserPlugin ) * Moved DisMaxQParser to its own .java file, making it a public class rather than its previous package-private visibility. This makes it possible for users to build upon its logic, which is considerable, and to my mind is a good place to start alot of custom QParser implementations. * Changed access modifiers for the QParser abstract base class to protected (were package-private). Again as above, it makes this object usable by user-defined classes that wish to define custom QParser classes. More generally, and on the philosophy-of-code side of things, it seems misleading to define some class members as having the default access modifier (package-private) and then letting other package-scope derived classes use these while not explicitly allowing user-defined derived classes to make use of these members. In specific i'm thinking of how DisMaxQParser makes use of these members: **not because it is derived from QParser, but because it simply resides in the same namespace** * Changed access modifier for the QueryParsing.StrParser inner class and its constructors to public. Again as in above, same issue of having same-package classes enjoy the benefit of being in the same namespace (FunctionQParser.parse() uses it like so), while user defined classes cannot. Particulary in this case it is pretty bad since this class advertises itself as a collection of utilities for query parsing in general - great resource, should probably even live elsewhere (common.utils?) * Changed Function.FunctionWeight inner class data member modifiers to protected (were default - package-private). This allowed me to inherit from FunctionQuery as well as make use of its original FunctionWeight inner class while overriding some of the latter's methods. This is in the same spirit of the changes above. Please also note this follows the common Query/Weight implementation pattern in the lucene codebase, see for example the BooleanQuery/BooleanWeight code. All in all these are relatively minor changes which unlock a great deal of functionality to 3rd party developers, which i think is ultimately a big part of what solr is all about - extendability. It is also perhaps a cue for a more serious refactoring of the QParserPlugin hierarchy, although i will leave such bold exclamations to another occasion. Attached is a patch file, having passed the usual coding-style/unit testing cycle. -Chak -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: Compile error from new contrib clustering?
Thanks, Stanislaw, I will hook that in to the downloads area. -Grant On May 21, 2009, at 4:37 AM, Stanislaw Osinski wrote: Hi there, compile: [javac] Compiling 7 source files to /trunk/asf_solr_src/contrib/clustering/build/classes [javac] An exception has occurred in the compiler (1.5.0_16). Please file a bug at the Java Developer Connection ( http://java.sun.com/webapps/bugreport) after checking the Bug Parade for duplicates. Include your program and the following diagnostic in your report. Thank you. [javac] com.sun.tools.javac.code.Symbol$CompletionFailure: file org/simpleframework/xml/Root.class not found I think I know what's going on. Carrot2 has a dependency on Simple XML framework, but the JAR is not needed at runtime in Solr. Javac 1.6 issues warnings about missing annotations during compilation, but compilation completes fine. Javac 1.5 seems to fail, so the build process will probably need to download Simple XML dependency (it's LGPL) at least for compilation. For the peace of mind we may keep Simple XML in runtime as well, though I've tested that when using Carrot2 in Solr Simple XML classes are not needed. The download link for Simple XML we need would be: http://mirrors.ibiblio.org/pub/mirrors/maven2/org/simpleframework/simple-xml/1.7.3/simple-xml-1.7.3.jar Cheers, S. -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene: http://www.lucidimagination.com/search
Re: Compile error from new contrib clustering?
OK, committed. I also tried it on the JDK 1.5 and it works. On May 21, 2009, at 7:18 AM, Grant Ingersoll wrote: Thanks, Stanislaw, I will hook that in to the downloads area. -Grant On May 21, 2009, at 4:37 AM, Stanislaw Osinski wrote: Hi there, compile: [javac] Compiling 7 source files to /trunk/asf_solr_src/contrib/clustering/build/classes [javac] An exception has occurred in the compiler (1.5.0_16). Please file a bug at the Java Developer Connection ( http://java.sun.com/webapps/bugreport) after checking the Bug Parade for duplicates. Include your program and the following diagnostic in your report. Thank you. [javac] com.sun.tools.javac.code.Symbol$CompletionFailure: file org/simpleframework/xml/Root.class not found I think I know what's going on. Carrot2 has a dependency on Simple XML framework, but the JAR is not needed at runtime in Solr. Javac 1.6 issues warnings about missing annotations during compilation, but compilation completes fine. Javac 1.5 seems to fail, so the build process will probably need to download Simple XML dependency (it's LGPL) at least for compilation. For the peace of mind we may keep Simple XML in runtime as well, though I've tested that when using Carrot2 in Solr Simple XML classes are not needed. The download link for Simple XML we need would be: http://mirrors.ibiblio.org/pub/mirrors/maven2/org/simpleframework/simple-xml/1.7.3/simple-xml-1.7.3.jar Cheers, S. -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene: http://www.lucidimagination.com/search -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene: http://www.lucidimagination.com/search
[jira] Commented: (SOLR-1142) faster example schema
[ https://issues.apache.org/jira/browse/SOLR-1142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12711574#action_12711574 ] Shalin Shekhar Mangar commented on SOLR-1142: - So what are the things we need to do? Remove the following from the schema: # Default field values # TermVectors # CopyFields # uniqueKey? What other settings can we use? Some pointers from the recent java-dev discussion on default settings: http://www.lucidimagination.com/search/document/2ac9d8236873a53a/lucene_s_default_settings_back_compatibility faster example schema - Key: SOLR-1142 URL: https://issues.apache.org/jira/browse/SOLR-1142 Project: Solr Issue Type: Improvement Reporter: Yonik Seeley Fix For: 1.4 need faster example schema: http://www.lucidimagination.com/search/document/d46ea3fa441b6d94 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-1088) newSearcher(String name, boolean readOnly) is broken
[ https://issues.apache.org/jira/browse/SOLR-1088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller resolved SOLR-1088. --- Resolution: Fixed newSearcher(String name, boolean readOnly) is broken Key: SOLR-1088 URL: https://issues.apache.org/jira/browse/SOLR-1088 Project: Solr Issue Type: Bug Reporter: Hoss Man Assignee: Mark Miller Fix For: 1.4 Attachments: SOLR-1088.patch Noted on the mailing list while investigating something semi-related, the method SolrCore.newSearcher(String name, boolean readOnly) doesn't work as advertised (the readOnly option is completely ignored) the method isn't used anywhere in Solr (that i could find) but it might be used by clients ... we should deal with this in some way prior to the 1.4 release (either fix; fix deprecate; or document problem and deprecate) http://www.nabble.com/Re%3A-Lucene-sync-bottleneck--p22237748.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1150) OutofMemoryError on enabling highlighting
[ https://issues.apache.org/jira/browse/SOLR-1150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12711582#action_12711582 ] Mark Miller commented on SOLR-1150: --- Patch looks right - my only concern would be a performance impact, but that does look unlikely. There is likely to be some benefit in loading them all at once, but I can't imagine one at a time is much of a loss. OutofMemoryError on enabling highlighting - Key: SOLR-1150 URL: https://issues.apache.org/jira/browse/SOLR-1150 Project: Solr Issue Type: Improvement Components: highlighter Affects Versions: 1.4 Reporter: Siddharth Gargate Fix For: 1.4 Attachments: SOLR-1150.patch Please refer to following mail thread http://markmail.org/message/5nhkm5h3ongqlput I am testing with 2MB document size and just 500 documents. Indexing is working fine even with 128MB heap size. But on searching Solr throws OOM error. This issue is observed only when we enable highlighting. While indexing I am storing 1 MB text. While searching Solr reads all the 500 documents in the memory. It also reads the complete 1 MB stored field in the memory for all 500 documents. Due to this 500 docs * 1 MB * 2 (2 bytes per char) = 1000 MB memory is required for searching. This memory usage can be reduced by reading one document at a time. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (SOLR-831) revist jar packaging prior to next release
[
https://issues.apache.org/jira/browse/SOLR-831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shalin Shekhar Mangar reassigned SOLR-831:
--
Assignee: Shalin Shekhar Mangar
revist jar packaging prior to next release
--
Key: SOLR-831
URL: https://issues.apache.org/jira/browse/SOLR-831
Project: Solr
Issue Type: Bug
Components: clients - java
Affects Versions: 1.3
Reporter: Hoss Man
Assignee: Shalin Shekhar Mangar
Fix For: 1.4
as noted on the mailing list...
http://www.nabble.com/Solr-1.3-Maven-Artifact-Problem-to20132273.html#a20132273
{noformat}
I am using the Solr 1.3 mave nartifacts from [1]. It seems that these
artifacts are
not correct. I have noticed that:
1) solr-core artifact contains org.apache.solr.client.solrj packages, and at
the same
time, the solr-core artifact depends on the solr-solrj artifact.
{noformat}
at a minimum we should resolve this discrepancy, but more generally we should
consider moving anything currently in solrj that solr core has a
compile/run dependency on into solr-common ... possibly to the point of
eliminating the solr-solrj jar altogether.
marking fix for 1.4 as a reminder to consider/decide before the next release.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-916) CoreContainer :: register(String, SolrCore, boolean) documentation clarification about returnPrev argument
[ https://issues.apache.org/jira/browse/SOLR-916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12711599#action_12711599 ] Mark Miller commented on SOLR-916: -- I would say that if returnPrev is false, what is returned is undefined. The closed core is returned, but it just as well could be null. If the user expects to get and be able to use the prevCore, he must pass returnPrev = true. It still makes sense to me, but I wouldn't object to the change if someone wants to put it in. CoreContainer :: register(String, SolrCore, boolean) documentation clarification about returnPrev argument --- Key: SOLR-916 URL: https://issues.apache.org/jira/browse/SOLR-916 Project: Solr Issue Type: Improvement Affects Versions: 1.3 Environment: Tomcat 6, JRE 6 Reporter: Kay Kay Priority: Minor Fix For: 1.4 Attachments: SOLR-916.patch Original Estimate: 2h Remaining Estimate: 2h In CoreContainer.java :: register(name, core, returnPrev) - the documentation says it would return a previous core having the same name if it existed *and returnPrev = true*. * @return a previous core having the same name if it existed and returnPrev==true */ public SolrCore register(String name, SolrCore core, boolean returnPrev) .. But as per the code towards the end - the previous core is returned anyway, irrespective of the value of returnPrev. The difference, though, seems to be that when returnPrev is false, the previous core (of the same name, if exists) is closed. Which one of them is correct . If the code were correct , would the variable be better renamed as closePrevious , as opposed to returnPrevious. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-914) Presence of finalize() in the codebase
[
https://issues.apache.org/jira/browse/SOLR-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12711610#action_12711610
]
Mark Miller commented on SOLR-914:
--
I agree with Kay Kay and Lance here - I don't think we should be doing any
closing/shutdown with finalize. Logging a warning seems like the right way to
go to me. This is the type of thing that hides problems, and it just doesn't do
it cleanly.
Presence of finalize() in the codebase
---
Key: SOLR-914
URL: https://issues.apache.org/jira/browse/SOLR-914
Project: Solr
Issue Type: Improvement
Components: clients - java
Affects Versions: 1.3
Environment: Tomcat 6, JRE 6
Reporter: Kay Kay
Priority: Minor
Fix For: 1.4
Original Estimate: 480h
Remaining Estimate: 480h
There seems to be a number of classes - that implement finalize() method.
Given that it is perfectly ok for a Java VM to not to call it - may be -
there has to some other way { try .. finally - when they are created to
destroy them } to destroy them and the presence of finalize() method , (
depending on implementation ) might not serve what we want and in some cases
can end up delaying the gc process, depending on the algorithms.
$ find . -name *.java | xargs grep finalize
./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/JdbcDataSource.java:
protected void finalize() {
./src/java/org/apache/solr/update/SolrIndexWriter.java: protected void
finalize() {
./src/java/org/apache/solr/core/CoreContainer.java: protected void
finalize() {
./src/java/org/apache/solr/core/SolrCore.java: protected void finalize() {
./src/common/org/apache/solr/common/util/ConcurrentLRUCache.java: protected
void finalize() throws Throwable {
May be we need to revisit these occurences from a design perspective to see
if they are necessary / if there is an alternate way of managing guaranteed
destruction of resources.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1179) create lucene Filters from DocSets that translate ids
[ https://issues.apache.org/jira/browse/SOLR-1179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yonik Seeley updated SOLR-1179: --- Attachment: SOLR-1179.patch OK, this patch adds a DocSet.getTopFilter that returns a lucene Filter that will work for the reader the DocSet was generated with, or any of it's sub-readers (as Lucene now does per-segment searching and filtering). SortedIntDocSet has optimizations for when segments are sorted in order. It also looks up the end index once instead of comparing to the max value on every next() or skipTo() I plan on committing shortly. create lucene Filters from DocSets that translate ids - Key: SOLR-1179 URL: https://issues.apache.org/jira/browse/SOLR-1179 Project: Solr Issue Type: Improvement Reporter: Yonik Seeley Assignee: Yonik Seeley Attachments: SOLR-1179.patch In order to use Lucene Filtering, we need to efficiently translate top-level DocSet filters into per-segment filters. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
RE: Help Writing a Distributed Component
Matt,
I applied you patch to the trunk and tossed it in the debugger. I'm not
sure how your solrconfig.xml looks, but if your using the out of the box
autosuggest shards don't work.
The reason is that the default search handler has the QueryComponent in
it's list and the autosuggest doesn't.
Try stealing these lines from the end QueryComponent.prepare
// TODO: temporary... this should go in a different component.
String shards = params.get(ShardParams.SHARDS);
if (shards != null) {
ListString lst = StrUtils.splitSmart(shards, ,, true);
rb.shards = lst.toArray(new String[lst.size()]);
}
Matt Woytowitz
-Original Message-
From: Matt Weber [mailto:m...@mattweber.org]
Sent: Wednesday, May 20, 2009 12:07 PM
To: solr-dev@lucene.apache.org
Subject: Re: Help Writing a Distributed Component
Hey Grant,
I have opened a new ticket with the current version of my patch at
https://issues.apache.org/jira/browse/SOLR-1177
. What do you mean by configuring my shards? I implemented the
distributed methods (prepare, process, modifyRequest, etc) as is done
in the other distributed components, then I just specify a shards
parameter in the query. I looked though the other distributed
components and did not see them doing any special shard configuration
other than modifying the query that is sent to the shards. I must be
missing something, but I can't seem to figure out due to my lack of
knowledge on distributed components and the solr codebase in general.
Thanks for your help!
For my testing, I have 2 instances of Solr running in tomcat called
solr1 and solr2. My test query looks like:
http://localhost:8080/solr1/terms?shards=localhost:8080/solr1,localhost:
8080/solr2terms=trueterms.fl=spellterms.prefix=fterms.lower=f
Thanks,
Matt Weber
eSr Technologies
http://www.esr-technologies.com
On May 20, 2009, at 7:24 AM, Grant Ingersoll wrote:
Matt,
Perhaps you can post your patch even in it's current state on a JIRA
issue and then we can work through it? Also, how are you
configuring your shards?
I started that Wiki page as a way to make sure I understood what was
going on in distributed. I'm not 100% sure it is correct, so it
would be good if others review/edit as well.
-Grant
On May 20, 2009, at 1:36 AM, Matt Weber wrote:
Hi all,
I am working on a patch to get TermsComponent distributed and have
run into a problem. I have overridden the prepare, process,
modifyRequest, handleResponses, and finishStage methods as
described at
http://wiki.apache.org/solr/WritingDistributedSearchComponents
. My problem is that only the prepare and process methods are
called just as if it was non-distributed. It looks like the shards
parameter is not being honored for this handler. For some reason
rb.shards is always null. I looked though all the other
distributed components code (facets, stats, highlighting, etc) and
did not notice them doing anything special that my handler is not.
Is there some setting I need to enable for the shards parameter to
be honored?
Re: Help Writing a Distributed Component
Thanks Matt that worked! My patch still doesn't work correctly, but
at least I can actually get it calling the distributed methods now.
Do you mind explaining how you run solr though a debugger? That would
sure make things easier for me! Thanks again, I really appreciate
your help.
Thanks,
Matt Weber
eSr Technologies
http://www.esr-technologies.com
On May 21, 2009, at 1:28 PM, Woytowitz, Matthew wrote:
Matt,
I applied you patch to the trunk and tossed it in the debugger. I'm
not
sure how your solrconfig.xml looks, but if your using the out of the
box
autosuggest shards don't work.
The reason is that the default search handler has the QueryComponent
in
it's list and the autosuggest doesn't.
Try stealing these lines from the end QueryComponent.prepare
// TODO: temporary... this should go in a different component.
String shards = params.get(ShardParams.SHARDS);
if (shards != null) {
ListString lst = StrUtils.splitSmart(shards, ,, true);
rb.shards = lst.toArray(new String[lst.size()]);
}
Matt Woytowitz
-Original Message-
From: Matt Weber [mailto:m...@mattweber.org]
Sent: Wednesday, May 20, 2009 12:07 PM
To: solr-dev@lucene.apache.org
Subject: Re: Help Writing a Distributed Component
Hey Grant,
I have opened a new ticket with the current version of my patch at
https://issues.apache.org/jira/browse/SOLR-1177
. What do you mean by configuring my shards? I implemented the
distributed methods (prepare, process, modifyRequest, etc) as is done
in the other distributed components, then I just specify a shards
parameter in the query. I looked though the other distributed
components and did not see them doing any special shard configuration
other than modifying the query that is sent to the shards. I must be
missing something, but I can't seem to figure out due to my lack of
knowledge on distributed components and the solr codebase in general.
Thanks for your help!
For my testing, I have 2 instances of Solr running in tomcat called
solr1 and solr2. My test query looks like:
http://localhost:8080/solr1/terms?shards=localhost:8080/solr1,localhost
:
8080/solr2terms=trueterms.fl=spellterms.prefix=fterms.lower=f
Thanks,
Matt Weber
eSr Technologies
http://www.esr-technologies.com
On May 20, 2009, at 7:24 AM, Grant Ingersoll wrote:
Matt,
Perhaps you can post your patch even in it's current state on a JIRA
issue and then we can work through it? Also, how are you
configuring your shards?
I started that Wiki page as a way to make sure I understood what was
going on in distributed. I'm not 100% sure it is correct, so it
would be good if others review/edit as well.
-Grant
On May 20, 2009, at 1:36 AM, Matt Weber wrote:
Hi all,
I am working on a patch to get TermsComponent distributed and have
run into a problem. I have overridden the prepare, process,
modifyRequest, handleResponses, and finishStage methods as
described at
http://wiki.apache.org/solr/WritingDistributedSearchComponents
. My problem is that only the prepare and process methods are
called just as if it was non-distributed. It looks like the shards
parameter is not being honored for this handler. For some reason
rb.shards is always null. I looked though all the other
distributed components code (facets, stats, highlighting, etc) and
did not notice them doing anything special that my handler is not.
Is there some setting I need to enable for the shards parameter to
be honored?
[jira] Commented: (SOLR-1142) faster example schema
[ https://issues.apache.org/jira/browse/SOLR-1142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12711814#action_12711814 ] Otis Gospodnetic commented on SOLR-1142: I'd comment-out dynamic fields and I'd leave uniqueKey as I bet 99% of users need it. faster example schema - Key: SOLR-1142 URL: https://issues.apache.org/jira/browse/SOLR-1142 Project: Solr Issue Type: Improvement Reporter: Yonik Seeley Fix For: 1.4 need faster example schema: http://www.lucidimagination.com/search/document/d46ea3fa441b6d94 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-1179) create lucene Filters from DocSets that translate ids
[ https://issues.apache.org/jira/browse/SOLR-1179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yonik Seeley resolved SOLR-1179. Resolution: Fixed Fix Version/s: 1.4 committed. create lucene Filters from DocSets that translate ids - Key: SOLR-1179 URL: https://issues.apache.org/jira/browse/SOLR-1179 Project: Solr Issue Type: Improvement Reporter: Yonik Seeley Assignee: Yonik Seeley Fix For: 1.4 Attachments: SOLR-1179.patch In order to use Lucene Filtering, we need to efficiently translate top-level DocSet filters into per-segment filters. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1165) use skipping on filters to improve search performance
[ https://issues.apache.org/jira/browse/SOLR-1165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12711822#action_12711822 ] Yonik Seeley commented on SOLR-1165: OK, now that the two sub-issues related to DocSets and Filters are ironed out, we can move on to restructuring the search code in SolrIndexSearcher to use lucene Filters and the new Lucene Collector classes. use skipping on filters to improve search performance - Key: SOLR-1165 URL: https://issues.apache.org/jira/browse/SOLR-1165 Project: Solr Issue Type: Improvement Reporter: Yonik Seeley Assignee: Yonik Seeley Fix For: 1.4 Solr should use filters to skip scorers to improve search performance. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: [jira] Commented: (SOLR-1154) allow specifying solr configuration file through system property to simplify deployment procedure in certain cases
In my opinion, we don't need do anything, at least for now. It is not ideal
but we have a work around.
Thanks,
Jianhan
On Thu, May 21, 2009 at 12:18 AM, Noble Paul (JIRA) j...@apache.org wrote:
[
https://issues.apache.org/jira/browse/SOLR-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12711500#action_12711500]
Noble Paul commented on SOLR-1154:
--
do we still need to pursue this?
allow specifying solr configuration file through system property to
simplify deployment procedure in certain cases
--
Key: SOLR-1154
URL: https://issues.apache.org/jira/browse/SOLR-1154
Project: Solr
Issue Type: Improvement
Affects Versions: 1.4
Reporter: Jianhan
Priority: Minor
Fix For: 1.4
Attachments: SOLR-1154.patch, SOLR-1154.patch
Original Estimate: 5h
Remaining Estimate: 5h
Hi,
I wanted to use this parameter to specify different solr configuration
files for master and slave to simplify deployment procedure. Unfortunately,
I can't dynamically replace the value of this parameter. Basically, what I
want is
filter
filter-nameSolrRequestFilter/filter-name
filter-classorg.apache.solr.servlet.SolrDispatchFilter/filter-class
init-param
param-namesolrconfig-filename/param-name
param-valuesolrconfig-master.xml/param-value
/init-param
/filter
for master instance, and
filter
filter-nameSolrRequestFilter/filter-name
filter-classorg.apache.solr.servlet.SolrDispatchFilter/filter-class
init-param
param-namesolrconfig-filename/param-name
param-valuesolrconfig-slave.xml/param-value
/init-param
/filter
for slave instance.
Ideally, if I can use system property for its value like in
solrconfig.xml. For example,
filter
filter-nameSolrRequestFilter/filter-name
filter-classorg.apache.solr.servlet.SolrDispatchFilter/filter-class
init-param
param-namesolrconfig-filename/param-name
param-value${solr.config.filename: solrconfig.xml}/param-value
/init-param
/filter
but I learned that in general we can't use system property in web.xml.
I realize that I can use replication of config file to achieve this, but
I thought that creates unnecessary dependencies for slaves on master
instance.
So here is my proposal:
make SolrDispatchFilter look up another init parameter, say
'solrconfig-filename-property', and its value is a system property name, and
if this property is set, we get the file name, otherwise nothing happens (of
course, if both exist, 'solrconfig-filename' takes precedence). This will
give us maximum flexibility of specifying configuration files for different
instances.
Your thoughts?
Thanks,
Jianhan
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
RE: Help Writing a Distributed Component
bin/catalina.sh jpda start
Starts tomcat in remote debugging mode.
Then I connect with eclipse.
Run -- Debug configs
Click Remote Java Application.
Click the New Lauch Config button. Hover to find it
Click the debug button should work.
-Original Message-
From: Matt Weber [mailto:m...@mattweber.org]
Sent: Thursday, May 21, 2009 5:03 PM
To: solr-dev@lucene.apache.org
Subject: Re: Help Writing a Distributed Component
Thanks Matt that worked! My patch still doesn't work correctly, but
at least I can actually get it calling the distributed methods now.
Do you mind explaining how you run solr though a debugger? That would
sure make things easier for me! Thanks again, I really appreciate
your help.
Thanks,
Matt Weber
eSr Technologies
http://www.esr-technologies.com
On May 21, 2009, at 1:28 PM, Woytowitz, Matthew wrote:
Matt,
I applied you patch to the trunk and tossed it in the debugger. I'm
not
sure how your solrconfig.xml looks, but if your using the out of the
box
autosuggest shards don't work.
The reason is that the default search handler has the QueryComponent
in
it's list and the autosuggest doesn't.
Try stealing these lines from the end QueryComponent.prepare
// TODO: temporary... this should go in a different component.
String shards = params.get(ShardParams.SHARDS);
if (shards != null) {
ListString lst = StrUtils.splitSmart(shards, ,, true);
rb.shards = lst.toArray(new String[lst.size()]);
}
Matt Woytowitz
-Original Message-
From: Matt Weber [mailto:m...@mattweber.org]
Sent: Wednesday, May 20, 2009 12:07 PM
To: solr-dev@lucene.apache.org
Subject: Re: Help Writing a Distributed Component
Hey Grant,
I have opened a new ticket with the current version of my patch at
https://issues.apache.org/jira/browse/SOLR-1177
. What do you mean by configuring my shards? I implemented the
distributed methods (prepare, process, modifyRequest, etc) as is done
in the other distributed components, then I just specify a shards
parameter in the query. I looked though the other distributed
components and did not see them doing any special shard configuration
other than modifying the query that is sent to the shards. I must be
missing something, but I can't seem to figure out due to my lack of
knowledge on distributed components and the solr codebase in general.
Thanks for your help!
For my testing, I have 2 instances of Solr running in tomcat called
solr1 and solr2. My test query looks like:
http://localhost:8080/solr1/terms?shards=localhost:8080/solr1,localhost
:
8080/solr2terms=trueterms.fl=spellterms.prefix=fterms.lower=f
Thanks,
Matt Weber
eSr Technologies
http://www.esr-technologies.com
On May 20, 2009, at 7:24 AM, Grant Ingersoll wrote:
Matt,
Perhaps you can post your patch even in it's current state on a JIRA
issue and then we can work through it? Also, how are you
configuring your shards?
I started that Wiki page as a way to make sure I understood what was
going on in distributed. I'm not 100% sure it is correct, so it
would be good if others review/edit as well.
-Grant
On May 20, 2009, at 1:36 AM, Matt Weber wrote:
Hi all,
I am working on a patch to get TermsComponent distributed and have
run into a problem. I have overridden the prepare, process,
modifyRequest, handleResponses, and finishStage methods as
described at
http://wiki.apache.org/solr/WritingDistributedSearchComponents
. My problem is that only the prepare and process methods are
called just as if it was non-distributed. It looks like the shards
parameter is not being honored for this handler. For some reason
rb.shards is always null. I looked though all the other
distributed components code (facets, stats, highlighting, etc) and
did not notice them doing anything special that my handler is not.
Is there some setting I need to enable for the shards parameter to
be honored?
Re: Help Writing a Distributed Component
Thank you, this will help a lot.
Thanks,
Matt Weber
eSr Technologies
http://www.esr-technologies.com
On May 21, 2009, at 2:43 PM, Woytowitz, Matthew wrote:
bin/catalina.sh jpda start
Starts tomcat in remote debugging mode.
Then I connect with eclipse.
Run -- Debug configs
Click Remote Java Application.
Click the New Lauch Config button. Hover to find it
Click the debug button should work.
-Original Message-
From: Matt Weber [mailto:m...@mattweber.org]
Sent: Thursday, May 21, 2009 5:03 PM
To: solr-dev@lucene.apache.org
Subject: Re: Help Writing a Distributed Component
Thanks Matt that worked! My patch still doesn't work correctly, but
at least I can actually get it calling the distributed methods now.
Do you mind explaining how you run solr though a debugger? That would
sure make things easier for me! Thanks again, I really appreciate
your help.
Thanks,
Matt Weber
eSr Technologies
http://www.esr-technologies.com
On May 21, 2009, at 1:28 PM, Woytowitz, Matthew wrote:
Matt,
I applied you patch to the trunk and tossed it in the debugger. I'm
not
sure how your solrconfig.xml looks, but if your using the out of the
box
autosuggest shards don't work.
The reason is that the default search handler has the QueryComponent
in
it's list and the autosuggest doesn't.
Try stealing these lines from the end QueryComponent.prepare
// TODO: temporary... this should go in a different component.
String shards = params.get(ShardParams.SHARDS);
if (shards != null) {
ListString lst = StrUtils.splitSmart(shards, ,, true);
rb.shards = lst.toArray(new String[lst.size()]);
}
Matt Woytowitz
-Original Message-
From: Matt Weber [mailto:m...@mattweber.org]
Sent: Wednesday, May 20, 2009 12:07 PM
To: solr-dev@lucene.apache.org
Subject: Re: Help Writing a Distributed Component
Hey Grant,
I have opened a new ticket with the current version of my patch at
https://issues.apache.org/jira/browse/SOLR-1177
. What do you mean by configuring my shards? I implemented the
distributed methods (prepare, process, modifyRequest, etc) as is done
in the other distributed components, then I just specify a shards
parameter in the query. I looked though the other distributed
components and did not see them doing any special shard configuration
other than modifying the query that is sent to the shards. I must be
missing something, but I can't seem to figure out due to my lack of
knowledge on distributed components and the solr codebase in general.
Thanks for your help!
For my testing, I have 2 instances of Solr running in tomcat called
solr1 and solr2. My test query looks like:
http://localhost:8080/solr1/terms?shards=localhost:8080/
solr1,localhost
:
8080/solr2terms=trueterms.fl=spellterms.prefix=fterms.lower=f
Thanks,
Matt Weber
eSr Technologies
http://www.esr-technologies.com
On May 20, 2009, at 7:24 AM, Grant Ingersoll wrote:
Matt,
Perhaps you can post your patch even in it's current state on a JIRA
issue and then we can work through it? Also, how are you
configuring your shards?
I started that Wiki page as a way to make sure I understood what was
going on in distributed. I'm not 100% sure it is correct, so it
would be good if others review/edit as well.
-Grant
On May 20, 2009, at 1:36 AM, Matt Weber wrote:
Hi all,
I am working on a patch to get TermsComponent distributed and have
run into a problem. I have overridden the prepare, process,
modifyRequest, handleResponses, and finishStage methods as
described at
http://wiki.apache.org/solr/WritingDistributedSearchComponents
. My problem is that only the prepare and process methods are
called just as if it was non-distributed. It looks like the shards
parameter is not being honored for this handler. For some reason
rb.shards is always null. I looked though all the other
distributed components code (facets, stats, highlighting, etc) and
did not notice them doing anything special that my handler is not.
Is there some setting I need to enable for the shards parameter to
be honored?
[jira] Updated: (SOLR-1145) Patch to set IndexWriter.defaultInfoStream from solr.xml
[ https://issues.apache.org/jira/browse/SOLR-1145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Harris updated SOLR-1145: --- Attachment: SOLR-1145.patch I found it confusing not to have any timestamp data in the infostream log. This new version prints a timestamp along with each infoStream message. You can configure the timestamp format, though there's currently no way to disable timestamping. Patch to set IndexWriter.defaultInfoStream from solr.xml Key: SOLR-1145 URL: https://issues.apache.org/jira/browse/SOLR-1145 Project: Solr Issue Type: Improvement Reporter: Chris Harris Fix For: 1.4 Attachments: SOLR-1145.patch, SOLR-1145.patch Lucene IndexWriters use an infoStream to log detailed info about indexing operations for debugging purpose. This patch is an extremely simple way to allow logging this info to a file from within Solr: After applying the patch, set the new defaultInfoStreamFilePath attribute of the solr element in solr.xml to the path of the file where you'd like to save the logging information. Note that, in a multi-core setup, all cores will end up logging to the same infoStream log file. This may not be desired. (But it does justify putting the setting in solr.xml rather than solrconfig.xml.) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1145) Patch to set IndexWriter.defaultInfoStream from solr.xml
[ https://issues.apache.org/jira/browse/SOLR-1145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Harris updated SOLR-1145: --- Attachment: SOLR-1145.patch Oops. Infostream logging should not be enabled by default in example/multicore Patch to set IndexWriter.defaultInfoStream from solr.xml Key: SOLR-1145 URL: https://issues.apache.org/jira/browse/SOLR-1145 Project: Solr Issue Type: Improvement Reporter: Chris Harris Fix For: 1.4 Attachments: SOLR-1145.patch, SOLR-1145.patch, SOLR-1145.patch Lucene IndexWriters use an infoStream to log detailed info about indexing operations for debugging purpose. This patch is an extremely simple way to allow logging this info to a file from within Solr: After applying the patch, set the new defaultInfoStreamFilePath attribute of the solr element in solr.xml to the path of the file where you'd like to save the logging information. Note that, in a multi-core setup, all cores will end up logging to the same infoStream log file. This may not be desired. (But it does justify putting the setting in solr.xml rather than solrconfig.xml.) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1145) Patch to set IndexWriter.defaultInfoStream from solr.xml
[ https://issues.apache.org/jira/browse/SOLR-1145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Harris updated SOLR-1145: --- Attachment: (was: SOLR-1145.patch) Patch to set IndexWriter.defaultInfoStream from solr.xml Key: SOLR-1145 URL: https://issues.apache.org/jira/browse/SOLR-1145 Project: Solr Issue Type: Improvement Reporter: Chris Harris Fix For: 1.4 Attachments: SOLR-1145.patch, SOLR-1145.patch Lucene IndexWriters use an infoStream to log detailed info about indexing operations for debugging purpose. This patch is an extremely simple way to allow logging this info to a file from within Solr: After applying the patch, set the new defaultInfoStreamFilePath attribute of the solr element in solr.xml to the path of the file where you'd like to save the logging information. Note that, in a multi-core setup, all cores will end up logging to the same infoStream log file. This may not be desired. (But it does justify putting the setting in solr.xml rather than solrconfig.xml.) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
RE: Help Writing a Distributed Component
I found a couple more things for you.
Your modifyRequest method won't get called, because in the default
/autosuggest the termsComp is the only component.
/** Called after another component adds a request */
public void modifyRequest(ResponseBuilder rb, SearchComponent who,
ShardRequest sreq) {
}
searchComponent name=termsComp
class=org.apache.solr.handler.component.TermsComponent/
requestHandler name=/autoSuggest
class=org.apache.solr.handler.component.SearchHandler
arr name=components
strtermsComp/str
/arr
/requestHandler
You should move the logic from modifyRequest (since there isn't going to
be another component creating a shard request for you to modify) to
distributeProcess
I lifted this from QueryComponent.
@Override
public int distributedProcess(ResponseBuilder rb) throws IOException {
if (rb.stage == ResponseBuilder.STAGE_EXECUTE_QUERY) {
TermsInfo ti = rb._termsInfo;
if (ti == null) {
ti = rb._termsInfo = new TermsInfo();
ti.init(rb.req.getParams());
}
createMainQuery(rb);
}
if (rb.stage ResponseBuilder.STAGE_EXECUTE_QUERY){
return ResponseBuilder.STAGE_EXECUTE_QUERY;
}else {
return ResponseBuilder.STAGE_DONE;
}
}
private void createMainQuery(ResponseBuilder rb) {
ShardRequest sreq = new ShardRequest();
sreq.purpose = ShardRequest.PURPOSE_GET_TERMS;
sreq.params = new ModifiableSolrParams(rb.req.getParams());
// TODO: base on current params or original params?
// don't pass through any shards param
sreq.params.remove(ShardParams.SHARDS);
//Add what ever else you need.
rb.addRequest(this, sreq);
}
Make sure you add shards.qt to your original request. I set it to
shards.qt=/autosuggest. If you don't the shard request will be send to
/select instead of /autoSuggest.
Matt Woytowitz
-Original Message-
From: Matt Weber [mailto:m...@mattweber.org]
Sent: Thursday, May 21, 2009 5:03 PM
To: solr-dev@lucene.apache.org
Subject: Re: Help Writing a Distributed Component
Thanks Matt that worked! My patch still doesn't work correctly, but
at least I can actually get it calling the distributed methods now.
Do you mind explaining how you run solr though a debugger? That would
sure make things easier for me! Thanks again, I really appreciate
your help.
Thanks,
Matt Weber
eSr Technologies
http://www.esr-technologies.com
On May 21, 2009, at 1:28 PM, Woytowitz, Matthew wrote:
Matt,
I applied you patch to the trunk and tossed it in the debugger. I'm
not
sure how your solrconfig.xml looks, but if your using the out of the
box
autosuggest shards don't work.
The reason is that the default search handler has the QueryComponent
in
it's list and the autosuggest doesn't.
Try stealing these lines from the end QueryComponent.prepare
// TODO: temporary... this should go in a different component.
String shards = params.get(ShardParams.SHARDS);
if (shards != null) {
ListString lst = StrUtils.splitSmart(shards, ,, true);
rb.shards = lst.toArray(new String[lst.size()]);
}
Matt Woytowitz
-Original Message-
From: Matt Weber [mailto:m...@mattweber.org]
Sent: Wednesday, May 20, 2009 12:07 PM
To: solr-dev@lucene.apache.org
Subject: Re: Help Writing a Distributed Component
Hey Grant,
I have opened a new ticket with the current version of my patch at
https://issues.apache.org/jira/browse/SOLR-1177
. What do you mean by configuring my shards? I implemented the
distributed methods (prepare, process, modifyRequest, etc) as is done
in the other distributed components, then I just specify a shards
parameter in the query. I looked though the other distributed
components and did not see them doing any special shard configuration
other than modifying the query that is sent to the shards. I must be
missing something, but I can't seem to figure out due to my lack of
knowledge on distributed components and the solr codebase in general.
Thanks for your help!
For my testing, I have 2 instances of Solr running in tomcat called
solr1 and solr2. My test query looks like:
http://localhost:8080/solr1/terms?shards=localhost:8080/solr1,localhost
:
8080/solr2terms=trueterms.fl=spellterms.prefix=fterms.lower=f
Thanks,
Matt Weber
eSr Technologies
http://www.esr-technologies.com
On May 20, 2009, at 7:24 AM, Grant Ingersoll wrote:
Matt,
Perhaps you can post your patch even in it's current state on a JIRA
issue and then we can work through it? Also, how are you
configuring your shards?
I started that Wiki page as a way to make sure I understood what was
going on in distributed. I'm not 100% sure it is correct, so it
would be good if others review/edit as well.
-Grant
On May 20, 2009, at 1:36 AM, Matt Weber wrote:
Hi all,
I am working on a patch to get TermsComponent distributed and have
run into a problem. I have overridden the prepare, process,
RE: Help Writing a Distributed Component
Another trick I used when debugging is to setup the shards to point to
it's self.
Example:
http://localhost:8080/solr?q=iran?shards=localhost:8080/solr
-Original Message-
From: Woytowitz, Matthew [mailto:matthew.woytow...@mantech.com]
Sent: Thursday, May 21, 2009 5:43 PM
To: solr-dev@lucene.apache.org
Subject: RE: Help Writing a Distributed Component
bin/catalina.sh jpda start
Starts tomcat in remote debugging mode.
Then I connect with eclipse.
Run -- Debug configs
Click Remote Java Application.
Click the New Lauch Config button. Hover to find it
Click the debug button should work.
-Original Message-
From: Matt Weber [mailto:m...@mattweber.org]
Sent: Thursday, May 21, 2009 5:03 PM
To: solr-dev@lucene.apache.org
Subject: Re: Help Writing a Distributed Component
Thanks Matt that worked! My patch still doesn't work correctly, but
at least I can actually get it calling the distributed methods now.
Do you mind explaining how you run solr though a debugger? That would
sure make things easier for me! Thanks again, I really appreciate
your help.
Thanks,
Matt Weber
eSr Technologies
http://www.esr-technologies.com
On May 21, 2009, at 1:28 PM, Woytowitz, Matthew wrote:
Matt,
I applied you patch to the trunk and tossed it in the debugger. I'm
not
sure how your solrconfig.xml looks, but if your using the out of the
box
autosuggest shards don't work.
The reason is that the default search handler has the QueryComponent
in
it's list and the autosuggest doesn't.
Try stealing these lines from the end QueryComponent.prepare
// TODO: temporary... this should go in a different component.
String shards = params.get(ShardParams.SHARDS);
if (shards != null) {
ListString lst = StrUtils.splitSmart(shards, ,, true);
rb.shards = lst.toArray(new String[lst.size()]);
}
Matt Woytowitz
-Original Message-
From: Matt Weber [mailto:m...@mattweber.org]
Sent: Wednesday, May 20, 2009 12:07 PM
To: solr-dev@lucene.apache.org
Subject: Re: Help Writing a Distributed Component
Hey Grant,
I have opened a new ticket with the current version of my patch at
https://issues.apache.org/jira/browse/SOLR-1177
. What do you mean by configuring my shards? I implemented the
distributed methods (prepare, process, modifyRequest, etc) as is done
in the other distributed components, then I just specify a shards
parameter in the query. I looked though the other distributed
components and did not see them doing any special shard configuration
other than modifying the query that is sent to the shards. I must be
missing something, but I can't seem to figure out due to my lack of
knowledge on distributed components and the solr codebase in general.
Thanks for your help!
For my testing, I have 2 instances of Solr running in tomcat called
solr1 and solr2. My test query looks like:
http://localhost:8080/solr1/terms?shards=localhost:8080/solr1,localhost
:
8080/solr2terms=trueterms.fl=spellterms.prefix=fterms.lower=f
Thanks,
Matt Weber
eSr Technologies
http://www.esr-technologies.com
On May 20, 2009, at 7:24 AM, Grant Ingersoll wrote:
Matt,
Perhaps you can post your patch even in it's current state on a JIRA
issue and then we can work through it? Also, how are you
configuring your shards?
I started that Wiki page as a way to make sure I understood what was
going on in distributed. I'm not 100% sure it is correct, so it
would be good if others review/edit as well.
-Grant
On May 20, 2009, at 1:36 AM, Matt Weber wrote:
Hi all,
I am working on a patch to get TermsComponent distributed and have
run into a problem. I have overridden the prepare, process,
modifyRequest, handleResponses, and finishStage methods as
described at
http://wiki.apache.org/solr/WritingDistributedSearchComponents
. My problem is that only the prepare and process methods are
called just as if it was non-distributed. It looks like the shards
parameter is not being honored for this handler. For some reason
rb.shards is always null. I looked though all the other
distributed components code (facets, stats, highlighting, etc) and
did not notice them doing anything special that my handler is not.
Is there some setting I need to enable for the shards parameter to
be honored?
RE: Help Writing a Distributed Component
I got it working, your we close. I attached the java file. Still needs
more testing. My only test was:
http://localhost:8080/apache-solr-1.4-dev/autoSuggest?terms=trueterms.f
l=text_allterms.lower=helshards=localhost:8080/apache-solr-1.4-devsha
rds.qt=/autoSuggest
I'm interested in using this in our project so please email me when you
have a more final patch.
Thanks,
Matt Woytowitz
-Original Message-
From: Woytowitz, Matthew [mailto:matthew.woytow...@mantech.com]
Sent: Thursday, May 21, 2009 6:12 PM
To: solr-dev@lucene.apache.org
Subject: RE: Help Writing a Distributed Component
Another trick I used when debugging is to setup the shards to point to
it's self.
Example:
http://localhost:8080/solr?q=iran?shards=localhost:8080/solr
-Original Message-
From: Woytowitz, Matthew [mailto:matthew.woytow...@mantech.com]
Sent: Thursday, May 21, 2009 5:43 PM
To: solr-dev@lucene.apache.org
Subject: RE: Help Writing a Distributed Component
bin/catalina.sh jpda start
Starts tomcat in remote debugging mode.
Then I connect with eclipse.
Run -- Debug configs
Click Remote Java Application.
Click the New Lauch Config button. Hover to find it
Click the debug button should work.
-Original Message-
From: Matt Weber [mailto:m...@mattweber.org]
Sent: Thursday, May 21, 2009 5:03 PM
To: solr-dev@lucene.apache.org
Subject: Re: Help Writing a Distributed Component
Thanks Matt that worked! My patch still doesn't work correctly, but
at least I can actually get it calling the distributed methods now.
Do you mind explaining how you run solr though a debugger? That would
sure make things easier for me! Thanks again, I really appreciate
your help.
Thanks,
Matt Weber
eSr Technologies
http://www.esr-technologies.com
On May 21, 2009, at 1:28 PM, Woytowitz, Matthew wrote:
Matt,
I applied you patch to the trunk and tossed it in the debugger. I'm
not
sure how your solrconfig.xml looks, but if your using the out of the
box
autosuggest shards don't work.
The reason is that the default search handler has the QueryComponent
in
it's list and the autosuggest doesn't.
Try stealing these lines from the end QueryComponent.prepare
// TODO: temporary... this should go in a different component.
String shards = params.get(ShardParams.SHARDS);
if (shards != null) {
ListString lst = StrUtils.splitSmart(shards, ,, true);
rb.shards = lst.toArray(new String[lst.size()]);
}
Matt Woytowitz
-Original Message-
From: Matt Weber [mailto:m...@mattweber.org]
Sent: Wednesday, May 20, 2009 12:07 PM
To: solr-dev@lucene.apache.org
Subject: Re: Help Writing a Distributed Component
Hey Grant,
I have opened a new ticket with the current version of my patch at
https://issues.apache.org/jira/browse/SOLR-1177
. What do you mean by configuring my shards? I implemented the
distributed methods (prepare, process, modifyRequest, etc) as is done
in the other distributed components, then I just specify a shards
parameter in the query. I looked though the other distributed
components and did not see them doing any special shard configuration
other than modifying the query that is sent to the shards. I must be
missing something, but I can't seem to figure out due to my lack of
knowledge on distributed components and the solr codebase in general.
Thanks for your help!
For my testing, I have 2 instances of Solr running in tomcat called
solr1 and solr2. My test query looks like:
http://localhost:8080/solr1/terms?shards=localhost:8080/solr1,localhost
:
8080/solr2terms=trueterms.fl=spellterms.prefix=fterms.lower=f
Thanks,
Matt Weber
eSr Technologies
http://www.esr-technologies.com
On May 20, 2009, at 7:24 AM, Grant Ingersoll wrote:
Matt,
Perhaps you can post your patch even in it's current state on a JIRA
issue and then we can work through it? Also, how are you
configuring your shards?
I started that Wiki page as a way to make sure I understood what was
going on in distributed. I'm not 100% sure it is correct, so it
would be good if others review/edit as well.
-Grant
On May 20, 2009, at 1:36 AM, Matt Weber wrote:
Hi all,
I am working on a patch to get TermsComponent distributed and have
run into a problem. I have overridden the prepare, process,
modifyRequest, handleResponses, and finishStage methods as
described at
http://wiki.apache.org/solr/WritingDistributedSearchComponents
. My problem is that only the prepare and process methods are
called just as if it was non-distributed. It looks like the shards
parameter is not being honored for this handler. For some reason
rb.shards is always null. I looked though all the other
distributed components code (facets, stats, highlighting, etc) and
did not notice them doing anything special that my handler is not.
Is there some setting I need to enable for the shards parameter to
be honored?
[jira] Updated: (SOLR-1177) Distributed TermsComponent
[ https://issues.apache.org/jira/browse/SOLR-1177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Woytowitz updated SOLR-1177: Attachment: TermsComponent.java TermsComponent.patch I got the previous patch working. It was we close. I attached the java file and a patch for just the TermsComponent Distributed TermsComponent -- Key: SOLR-1177 URL: https://issues.apache.org/jira/browse/SOLR-1177 Project: Solr Issue Type: Improvement Affects Versions: 1.4 Reporter: Matt Weber Priority: Minor Fix For: 1.5 Attachments: SOLR-1177.patch, TermsComponent.java, TermsComponent.patch TermsComponent should be distributed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: Help Writing a Distributed Component
On May 21, 2009, at 4:28 PM, Woytowitz, Matthew wrote:
Matt,
I applied you patch to the trunk and tossed it in the debugger. I'm
not
sure how your solrconfig.xml looks, but if your using the out of the
box
autosuggest shards don't work.
The reason is that the default search handler has the QueryComponent
in
it's list and the autosuggest doesn't.
Does this imply that the QC must be present for most components to be
distro ready? Perhaps we need to abstract some of that up, so that
other components can better take advantage w/o requiring the QC or
having to repeat the same code.
Try stealing these lines from the end QueryComponent.prepare
// TODO: temporary... this should go in a different component.
String shards = params.get(ShardParams.SHARDS);
if (shards != null) {
ListString lst = StrUtils.splitSmart(shards, ,, true);
rb.shards = lst.toArray(new String[lst.size()]);
}
Matt Woytowitz
-Original Message-
From: Matt Weber [mailto:m...@mattweber.org]
Sent: Wednesday, May 20, 2009 12:07 PM
To: solr-dev@lucene.apache.org
Subject: Re: Help Writing a Distributed Component
Hey Grant,
I have opened a new ticket with the current version of my patch at
https://issues.apache.org/jira/browse/SOLR-1177
. What do you mean by configuring my shards? I implemented the
distributed methods (prepare, process, modifyRequest, etc) as is done
in the other distributed components, then I just specify a shards
parameter in the query. I looked though the other distributed
components and did not see them doing any special shard configuration
other than modifying the query that is sent to the shards. I must be
missing something, but I can't seem to figure out due to my lack of
knowledge on distributed components and the solr codebase in general.
Thanks for your help!
For my testing, I have 2 instances of Solr running in tomcat called
solr1 and solr2. My test query looks like:
http://localhost:8080/solr1/terms?shards=localhost:8080/solr1,localhost
:
8080/solr2terms=trueterms.fl=spellterms.prefix=fterms.lower=f
Thanks,
Matt Weber
eSr Technologies
http://www.esr-technologies.com
On May 20, 2009, at 7:24 AM, Grant Ingersoll wrote:
Matt,
Perhaps you can post your patch even in it's current state on a JIRA
issue and then we can work through it? Also, how are you
configuring your shards?
I started that Wiki page as a way to make sure I understood what was
going on in distributed. I'm not 100% sure it is correct, so it
would be good if others review/edit as well.
-Grant
On May 20, 2009, at 1:36 AM, Matt Weber wrote:
Hi all,
I am working on a patch to get TermsComponent distributed and have
run into a problem. I have overridden the prepare, process,
modifyRequest, handleResponses, and finishStage methods as
described at
http://wiki.apache.org/solr/WritingDistributedSearchComponents
. My problem is that only the prepare and process methods are
called just as if it was non-distributed. It looks like the shards
parameter is not being honored for this handler. For some reason
rb.shards is always null. I looked though all the other
distributed components code (facets, stats, highlighting, etc) and
did not notice them doing anything special that my handler is not.
Is there some setting I need to enable for the shards parameter to
be honored?
--
Grant Ingersoll
http://www.lucidimagination.com/
Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)
using Solr/Lucene:
http://www.lucidimagination.com/search
[jira] Commented: (SOLR-920) Cache and reuse IndexSchema
[ https://issues.apache.org/jira/browse/SOLR-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12711856#action_12711856 ] Otis Gospodnetic commented on SOLR-920: --- Looks good to me. What happens when a core has a copy of schema.xml in its conf/ dir and that schema.xml is potentially different from the shared one? Cache and reuse IndexSchema --- Key: SOLR-920 URL: https://issues.apache.org/jira/browse/SOLR-920 Project: Solr Issue Type: Improvement Reporter: Noble Paul Assignee: Noble Paul if there are 1000's of cores then the cost of loading unloading schema.xml can be prohibitive similar to SOLR-919 we can also cache the DOM object of schema.xml if the location on disk is same. All the dynamic properties can be replaced lazily when they are read. We can go one step ahead in this case. Th IndexSchema object is immutable . So if there are no core properties then the same IndexSchema object can be used across all the cores -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: Help Writing a Distributed Component
Thank you Matt, this seems to be working as expected. I am going to
clean it up a bit, write some unit tests, and we should be good to
go. Thanks again!
Grant, yes that is exactly what this implies. I had no idea this was
the cause because the other distributed components (facets, stats,
debug, etc) are all default components that follow the query
component. Because they follow the query component, they don't do
this manual parsing of the shards and I never knew this is why they
worked and my code did not. Anyways, I think abstracting this logic
as you suggested is the way to go because it will define a standard
way to write a distributed component.
Thanks,
Matt Weber
eSr Technologies
http://www.esr-technologies.com
On May 21, 2009, at 3:54 PM, Grant Ingersoll wrote:
On May 21, 2009, at 4:28 PM, Woytowitz, Matthew wrote:
Matt,
I applied you patch to the trunk and tossed it in the debugger.
I'm not
sure how your solrconfig.xml looks, but if your using the out of
the box
autosuggest shards don't work.
The reason is that the default search handler has the
QueryComponent in
it's list and the autosuggest doesn't.
Does this imply that the QC must be present for most components to
be distro ready? Perhaps we need to abstract some of that up, so
that other components can better take advantage w/o requiring the QC
or having to repeat the same code.
Try stealing these lines from the end QueryComponent.prepare
// TODO: temporary... this should go in a different component.
String shards = params.get(ShardParams.SHARDS);
if (shards != null) {
ListString lst = StrUtils.splitSmart(shards, ,, true);
rb.shards = lst.toArray(new String[lst.size()]);
}
Matt Woytowitz
-Original Message-
From: Matt Weber [mailto:m...@mattweber.org]
Sent: Wednesday, May 20, 2009 12:07 PM
To: solr-dev@lucene.apache.org
Subject: Re: Help Writing a Distributed Component
Hey Grant,
I have opened a new ticket with the current version of my patch at
https://issues.apache.org/jira/browse/SOLR-1177
. What do you mean by configuring my shards? I implemented the
distributed methods (prepare, process, modifyRequest, etc) as is done
in the other distributed components, then I just specify a shards
parameter in the query. I looked though the other distributed
components and did not see them doing any special shard configuration
other than modifying the query that is sent to the shards. I must be
missing something, but I can't seem to figure out due to my lack of
knowledge on distributed components and the solr codebase in general.
Thanks for your help!
For my testing, I have 2 instances of Solr running in tomcat called
solr1 and solr2. My test query looks like:
http://localhost:8080/solr1/terms?shards=localhost:8080/solr1,localhost
:
8080/solr2terms=trueterms.fl=spellterms.prefix=fterms.lower=f
Thanks,
Matt Weber
eSr Technologies
http://www.esr-technologies.com
On May 20, 2009, at 7:24 AM, Grant Ingersoll wrote:
Matt,
Perhaps you can post your patch even in it's current state on a JIRA
issue and then we can work through it? Also, how are you
configuring your shards?
I started that Wiki page as a way to make sure I understood what was
going on in distributed. I'm not 100% sure it is correct, so it
would be good if others review/edit as well.
-Grant
On May 20, 2009, at 1:36 AM, Matt Weber wrote:
Hi all,
I am working on a patch to get TermsComponent distributed and have
run into a problem. I have overridden the prepare, process,
modifyRequest, handleResponses, and finishStage methods as
described at
http://wiki.apache.org/solr/WritingDistributedSearchComponents
. My problem is that only the prepare and process methods are
called just as if it was non-distributed. It looks like the shards
parameter is not being honored for this handler. For some reason
rb.shards is always null. I looked though all the other
distributed components code (facets, stats, highlighting, etc) and
did not notice them doing anything special that my handler is not.
Is there some setting I need to enable for the shards parameter to
be honored?
--
Grant Ingersoll
http://www.lucidimagination.com/
Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)
using Solr/Lucene:
http://www.lucidimagination.com/search
Re: [jira] Resolved: (SOLR-1175) disable replication on master side
That is great, and I tried it and it looks working great. Now how do we disable/enable replication programmatically? Here is what I tried: ModifiableSolrParams params = new ModifiableSolrParams(); params.set(qt, /replication); params.set(command, disablereplication); solrServer.query(params); where solrServer is an instance of SolrServer. It seems working fine (meaning I didn't get any exceptions). The questions are: 1. is this the right way to do it? It looks odd to call the function query to disable replication, but I don't find a better way to do it. SolrServer.request seems a better choice, but I don't know which subclass of SolrRequest to use and the comments of the class indicates these classes may change. 2. how can we verify that replication is actually disabled/enabled? launching a slave and starting replication is a way to verify, but it would be nice if we can see the status in replication details page. Thanks, Jianhan On Wed, May 20, 2009 at 11:58 PM, Noble Paul (JIRA) j...@apache.org wrote: [ https://issues.apache.org/jira/browse/SOLR-1175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel] Noble Paul resolved SOLR-1175. -- Resolution: Fixed committed revision: 776978 disable replication on master side -- Key: SOLR-1175 URL: https://issues.apache.org/jira/browse/SOLR-1175 Project: Solr Issue Type: New Feature Components: replication (java) Affects Versions: 1.4 Environment: any Reporter: Jianhan Assignee: Noble Paul Priority: Minor Fix For: 1.4 Attachments: SOLR-1175.patch Original Estimate: 4h Remaining Estimate: 4h In an environment where one master and several slaves are deployed, it usually takes more effort to discover all the slaves and disable replication on slave side (which is available right now), and it would be much easier to disable it on master instance (when, for example, there is a need to rebuild the index, yet search has to continue). The following is the original email describing a scenario when this feature is needed. Hi, Occasionally, we want to build our indexes from scratch, and during this period we want our search continue to work. Here are the steps that I think will do it 1. on all slaves: disable replication 2. on master: stop the server 3. on master: delete all the documents 4. on master: restart the server 5. on master: index all documents 6. on slaves: enable replication The only problem is: step 1 and 6. We may schedule any time to rebuild indexes and it is an automated process. It is possible to let the master to disable replication on all slaves, but then we have to discover all the slaves automatically, also exceptions may happen, e.g. a slave may be down at the time and then restarted later on. Anyhow it becomes an unreliable process. So I am thinking of somehow disabling replication on the master side during reindex, i.e. set a state on master so that any request for replication will be ignored. That all the steps described above will be on master side only. Is that possible? By the way, I am talking about solr 1.4. I looked at how 1.3 works, and noticed that in 1.3 there is a way to disable replication on master side: shutdown rsyncd, so I guess it would be nice to have something equivalent in solr 1.4. Thanks, Jianhan -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-769) Support Document and Search Result clustering
[ https://issues.apache.org/jira/browse/SOLR-769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brad Giaccio updated SOLR-769: -- Attachment: clustering-componet-shard.patch This is a patch to add shard support to the ClusteringComponent. Much like the recently posted spell check shard patch it simply implements finishStage and stitches the response together. A second option would have been to move the body of the process method to finishStage. This would have the benefit of only needing to do the clustering on the final set of responses. After the QueryComponent does its job of creating the final result set. This would also not make finishStage be so dependent on what is happening in the engines when they create their cluster response. I'm still trying to wrap my head around TestDistributedSearch so see how I can provide test methods. If option 2 that I laid out is preferred I should be able to provide a patch for that as well. Support Document and Search Result clustering - Key: SOLR-769 URL: https://issues.apache.org/jira/browse/SOLR-769 Project: Solr Issue Type: New Feature Reporter: Grant Ingersoll Assignee: Grant Ingersoll Priority: Minor Fix For: 1.4 Attachments: clustering-componet-shard.patch, clustering-libs.tar, clustering-libs.tar, SOLR-769-lib.zip, SOLR-769.patch, SOLR-769.patch, SOLR-769.patch, SOLR-769.patch, SOLR-769.patch, SOLR-769.patch, SOLR-769.patch, SOLR-769.patch, SOLR-769.patch, SOLR-769.patch, SOLR-769.patch, SOLR-769.tar, SOLR-769.zip Clustering is a useful tool for working with documents and search results, similar to the notion of dynamic faceting. Carrot2 (http://project.carrot2.org/) is a nice, BSD-licensed, library for doing search results clustering. Mahout (http://lucene.apache.org/mahout) is well suited for whole-corpus clustering. The patch I lays out a contrib module that starts off w/ an integration of a SearchComponent for doing clustering and an implementation using Carrot. In search results mode, it will use the DocList as the input for the cluster. While Carrot2 comes w/ a Solr input component, it is not the same as the SearchComponent that I have in that the Carrot example actually submits a query to Solr, whereas my SearchComponent is just chained into the Component list and uses the ResponseBuilder to add in the cluster results. While not fully fleshed out yet, the collection based mode will take in a list of ids or just use the whole collection and will produce clusters. Since this is a longer, typically offline task, there will need to be some type of storage mechanism (and replication??) for the clusters. I _may_ push this off to a separate JIRA issue, but I at least want to present the use case as part of the design of this component/contrib. It may even make sense that we split this out, such that the building piece is something like an UpdateProcessor and then the SearchComponent just acts as a lookup mechanism. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (SOLR-1180) Delete only delta import does not commit index
Delete only delta import does not commit index
--
Key: SOLR-1180
URL: https://issues.apache.org/jira/browse/SOLR-1180
Project: Solr
Issue Type: Bug
Components: contrib - DataImportHandler
Affects Versions: 1.3
Environment: OS: Windows Linux.
Java: 1.6
DB: MySQL SQL Server
Reporter: Ali Syed
Priority: Minor
Entity is set up in db-data-config.xml as follows
Ientity dataSource=mysqlDS name=contact pk=id
query=SELECT o.* FROM Contact o WHERE (o.deleted is null OR o.deleted = 0)
deltaQuery=select id from Contact o where (o.deleted is null OR o.deleted
= 0) AND (o.createDate gt; '${dataimporter.last_index_time}' OR o.updateDate
gt; '${dataimporter.last_index_time}')
deletedPkQuery=select id FROM Contact o WHERE o.deleted = 1 AND
(o.createDate gt; '${dataimporter.last_index_time}' OR o.updateDate gt;
'${dataimporter.last_index_time}')
...
/entity
If a delta import is trigger which causes only documents to be deleted the
index is not commit.
Problem seems to be in DocBuilder.execute() method checks for deletedDocCount
before commiting.
if (importStatistics.docCount.get() 0 ||
importStatistics.deletedDocCount.get() 0) {
commit(lastIndexTimeProps);
}
doDelta() method sets the count as follows
importStatistics.deletedDocCount.addAndGet(deletedKeys.size());
but after deleteAll(Set) method removes the each key from iterator
while (...) ...
iter.remove();
Simply remove iter.remove() line should fix the problem. I am not sure what is
the point of removing the keys from the iterator.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (SOLR-1180) Delete only delta import does not commit index
[
https://issues.apache.org/jira/browse/SOLR-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noble Paul reassigned SOLR-1180:
Assignee: Noble Paul
Delete only delta import does not commit index
--
Key: SOLR-1180
URL: https://issues.apache.org/jira/browse/SOLR-1180
Project: Solr
Issue Type: Bug
Components: contrib - DataImportHandler
Affects Versions: 1.3
Environment: OS: Windows Linux.
Java: 1.6
DB: MySQL SQL Server
Reporter: Ali Syed
Assignee: Noble Paul
Priority: Minor
Fix For: 1.4
Entity is set up in db-data-config.xml as follows
Ientity dataSource=mysqlDS name=contact pk=id
query=SELECT o.* FROM Contact o WHERE (o.deleted is null OR o.deleted =
0)
deltaQuery=select id from Contact o where (o.deleted is null OR
o.deleted = 0) AND (o.createDate gt; '${dataimporter.last_index_time}' OR
o.updateDate gt; '${dataimporter.last_index_time}')
deletedPkQuery=select id FROM Contact o WHERE o.deleted = 1 AND
(o.createDate gt; '${dataimporter.last_index_time}' OR o.updateDate gt;
'${dataimporter.last_index_time}')
...
/entity
If a delta import is trigger which causes only documents to be deleted the
index is not commit.
Problem seems to be in DocBuilder.execute() method checks for deletedDocCount
before commiting.
if (importStatistics.docCount.get() 0 ||
importStatistics.deletedDocCount.get() 0) {
commit(lastIndexTimeProps);
}
doDelta() method sets the count as follows
importStatistics.deletedDocCount.addAndGet(deletedKeys.size());
but after deleteAll(Set) method removes the each key from iterator
while (...) ...
iter.remove();
Simply remove iter.remove() line should fix the problem. I am not sure what
is the point of removing the keys from the iterator.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1180) Delete only delta import does not commit index
[
https://issues.apache.org/jira/browse/SOLR-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noble Paul updated SOLR-1180:
-
Fix Version/s: 1.4
Delete only delta import does not commit index
--
Key: SOLR-1180
URL: https://issues.apache.org/jira/browse/SOLR-1180
Project: Solr
Issue Type: Bug
Components: contrib - DataImportHandler
Affects Versions: 1.3
Environment: OS: Windows Linux.
Java: 1.6
DB: MySQL SQL Server
Reporter: Ali Syed
Assignee: Noble Paul
Priority: Minor
Fix For: 1.4
Entity is set up in db-data-config.xml as follows
Ientity dataSource=mysqlDS name=contact pk=id
query=SELECT o.* FROM Contact o WHERE (o.deleted is null OR o.deleted =
0)
deltaQuery=select id from Contact o where (o.deleted is null OR
o.deleted = 0) AND (o.createDate gt; '${dataimporter.last_index_time}' OR
o.updateDate gt; '${dataimporter.last_index_time}')
deletedPkQuery=select id FROM Contact o WHERE o.deleted = 1 AND
(o.createDate gt; '${dataimporter.last_index_time}' OR o.updateDate gt;
'${dataimporter.last_index_time}')
...
/entity
If a delta import is trigger which causes only documents to be deleted the
index is not commit.
Problem seems to be in DocBuilder.execute() method checks for deletedDocCount
before commiting.
if (importStatistics.docCount.get() 0 ||
importStatistics.deletedDocCount.get() 0) {
commit(lastIndexTimeProps);
}
doDelta() method sets the count as follows
importStatistics.deletedDocCount.addAndGet(deletedKeys.size());
but after deleteAll(Set) method removes the each key from iterator
while (...) ...
iter.remove();
Simply remove iter.remove() line should fix the problem. I am not sure what
is the point of removing the keys from the iterator.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1180) Delete only delta import does not commit index
[
https://issues.apache.org/jira/browse/SOLR-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12711943#action_12711943
]
Noble Paul commented on SOLR-1180:
--
bq.I am not sure what is the point of removing the keys from the iterator.
If the no:of items in that list is huge, we want to leave the objects to GC
Delete only delta import does not commit index
--
Key: SOLR-1180
URL: https://issues.apache.org/jira/browse/SOLR-1180
Project: Solr
Issue Type: Bug
Components: contrib - DataImportHandler
Affects Versions: 1.3
Environment: OS: Windows Linux.
Java: 1.6
DB: MySQL SQL Server
Reporter: Ali Syed
Priority: Minor
Fix For: 1.4
Entity is set up in db-data-config.xml as follows
Ientity dataSource=mysqlDS name=contact pk=id
query=SELECT o.* FROM Contact o WHERE (o.deleted is null OR o.deleted =
0)
deltaQuery=select id from Contact o where (o.deleted is null OR
o.deleted = 0) AND (o.createDate gt; '${dataimporter.last_index_time}' OR
o.updateDate gt; '${dataimporter.last_index_time}')
deletedPkQuery=select id FROM Contact o WHERE o.deleted = 1 AND
(o.createDate gt; '${dataimporter.last_index_time}' OR o.updateDate gt;
'${dataimporter.last_index_time}')
...
/entity
If a delta import is trigger which causes only documents to be deleted the
index is not commit.
Problem seems to be in DocBuilder.execute() method checks for deletedDocCount
before commiting.
if (importStatistics.docCount.get() 0 ||
importStatistics.deletedDocCount.get() 0) {
commit(lastIndexTimeProps);
}
doDelta() method sets the count as follows
importStatistics.deletedDocCount.addAndGet(deletedKeys.size());
but after deleteAll(Set) method removes the each key from iterator
while (...) ...
iter.remove();
Simply remove iter.remove() line should fix the problem. I am not sure what
is the point of removing the keys from the iterator.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1129) SolrJ cannot bind dynamic fields to beans
[
https://issues.apache.org/jira/browse/SOLR-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Avlesh Singh updated SOLR-1129:
---
Attachment: SOLR-1129.patch
All the changes as discussed earlier, have been implemented.
SolrJ can now be used to bind dynamic fields using any one of these
{code:java}
@Field(*_random)
public MapString, Integer random;
@Field(categories_*)
public MapString, String[] categories;
@Field(oem_*)
public MapString, ListString oem;
@Field(supplier_*)
public ListString allSuppliers;
@Field(supplier_*)
public String supplier;
@Field(supplier_*)
public void setSuppliers(String[] allSuppliers){}
{code}
The attached patch also contains a test case for the above mentioned usages.
Cheers
Avlesh
SolrJ cannot bind dynamic fields to beans
-
Key: SOLR-1129
URL: https://issues.apache.org/jira/browse/SOLR-1129
Project: Solr
Issue Type: Improvement
Components: clients - java
Affects Versions: 1.3
Reporter: Noble Paul
Assignee: Noble Paul
Fix For: 1.4
Attachments: SOLR-1129.patch, SOLR-1129.patch
SolrJ does not support binding of dynamic fields to bean fields
The field declaration could be as follows
{code:java}
@Field(*_s)
public String anyString;
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1150) OutofMemoryError on enabling highlighting
[ https://issues.apache.org/jira/browse/SOLR-1150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12711952#action_12711952 ] Siddharth Gargate commented on SOLR-1150: - Thanks Mark. SolrIndexSearcher.readDocs method internally reads one doc at a time. So there shouldn't be any performance loss. OutofMemoryError on enabling highlighting - Key: SOLR-1150 URL: https://issues.apache.org/jira/browse/SOLR-1150 Project: Solr Issue Type: Improvement Components: highlighter Affects Versions: 1.4 Reporter: Siddharth Gargate Fix For: 1.4 Attachments: SOLR-1150.patch Please refer to following mail thread http://markmail.org/message/5nhkm5h3ongqlput I am testing with 2MB document size and just 500 documents. Indexing is working fine even with 128MB heap size. But on searching Solr throws OOM error. This issue is observed only when we enable highlighting. While indexing I am storing 1 MB text. While searching Solr reads all the 500 documents in the memory. It also reads the complete 1 MB stored field in the memory for all 500 documents. Due to this 500 docs * 1 MB * 2 (2 bytes per char) = 1000 MB memory is required for searching. This memory usage can be reduced by reading one document at a time. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
