[jira] [Commented] (SOLR-1729) Date Facet now override time parameter
[ https://issues.apache.org/jira/browse/SOLR-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13186914#comment-13186914 ] Simon Willnauer commented on SOLR-1729: --- I am planning to backport SOLR-1709 to 3.x and apparently its a good idea to do the same for this issue. Any objections & issues what I should be aware of with this issue? > Date Facet now override time parameter > -- > > Key: SOLR-1729 > URL: https://issues.apache.org/jira/browse/SOLR-1729 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.4 > Environment: Solr 1.4 >Reporter: Peter Sturge >Priority: Minor > Fix For: 4.0 > > Attachments: FacetParams.java, SOLR-1729.patch, SOLR-1729.patch, > SOLR-1729.patch, SimpleFacets.java, UnInvertedField.java, > solr-1.4.0-solr-1729.patch > > > This PATCH introduces a new query parameter that tells a (typically, but not > necessarily) remote server what time to use as 'NOW' when calculating date > facets for a query (and, for the moment, date facets *only*) - overriding the > default behaviour of using the local server's current time. > This gets 'round a problem whereby an explicit time range is specified in a > query (e.g. timestamp:[then0 TO then1]), and date facets are required for the > given time range (in fact, any explicit time range). > Because DateMathParser performs all its calculations from 'NOW', remote > callers have to work out how long ago 'then0' and 'then1' are from 'now', and > use the relative-to-now values in the facet.date.xxx parameters. If a remote > server has a different opinion of NOW compared to the caller, the results > will be skewed (e.g. they are in a different time-zone, not time-synced etc.). > This becomes particularly salient when performing distributed date faceting > (see SOLR-1709), where multiple shards may all be running with different > times, and the faceting needs to be aligned. > The new parameter is called 'facet.date.now', and takes as a parameter a > (stringified) long that is the number of milliseconds from the epoch (1 Jan > 1970 00:00) - i.e. the returned value from a System.currentTimeMillis() call. > This was chosen over a formatted date to delineate it from a 'searchable' > time and to avoid superfluous date parsing. This makes the value generally a > programatically-set value, but as that is where the use-case is for this type > of parameter, this should be ok. > NOTE: This parameter affects date facet timing only. If there are other areas > of a query that rely on 'NOW', these will not interpret this value. This is a > broader issue about setting a 'query-global' NOW that all parts of query > analysis can share. > Source files affected: > FacetParams.java (holds the new constant FACET_DATE_NOW) > SimpleFacets.java getFacetDateCounts() NOW parameter modified > This PATCH is mildly related to SOLR-1709 (Distributed Date Faceting), but as > it's a general change for date faceting, it was deemed deserving of its own > patch. I will be updating SOLR-1709 in due course to include the use of this > new parameter, after some rfc acceptance. > A possible enhancement to this is to detect facet.date fields, look for and > match these fields in queries (if they exist), and potentially determine > automatically the required time skew, if any. There are a whole host of > reasons why this could be problematic to implement, so an explicit > facet.date.now parameter is the safest route. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1729) Date Facet now override time parameter
[ https://issues.apache.org/jira/browse/SOLR-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13186984#comment-13186984 ] David Smiley commented on SOLR-1729: Fantastic Simon! I'm glad someone with clout (i.e. a committer) is willing to make this happen. Users need SOLR-1709. > Date Facet now override time parameter > -- > > Key: SOLR-1729 > URL: https://issues.apache.org/jira/browse/SOLR-1729 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.4 > Environment: Solr 1.4 >Reporter: Peter Sturge >Priority: Minor > Fix For: 4.0 > > Attachments: FacetParams.java, SOLR-1729.patch, SOLR-1729.patch, > SOLR-1729.patch, SimpleFacets.java, UnInvertedField.java, > solr-1.4.0-solr-1729.patch > > > This PATCH introduces a new query parameter that tells a (typically, but not > necessarily) remote server what time to use as 'NOW' when calculating date > facets for a query (and, for the moment, date facets *only*) - overriding the > default behaviour of using the local server's current time. > This gets 'round a problem whereby an explicit time range is specified in a > query (e.g. timestamp:[then0 TO then1]), and date facets are required for the > given time range (in fact, any explicit time range). > Because DateMathParser performs all its calculations from 'NOW', remote > callers have to work out how long ago 'then0' and 'then1' are from 'now', and > use the relative-to-now values in the facet.date.xxx parameters. If a remote > server has a different opinion of NOW compared to the caller, the results > will be skewed (e.g. they are in a different time-zone, not time-synced etc.). > This becomes particularly salient when performing distributed date faceting > (see SOLR-1709), where multiple shards may all be running with different > times, and the faceting needs to be aligned. > The new parameter is called 'facet.date.now', and takes as a parameter a > (stringified) long that is the number of milliseconds from the epoch (1 Jan > 1970 00:00) - i.e. the returned value from a System.currentTimeMillis() call. > This was chosen over a formatted date to delineate it from a 'searchable' > time and to avoid superfluous date parsing. This makes the value generally a > programatically-set value, but as that is where the use-case is for this type > of parameter, this should be ok. > NOTE: This parameter affects date facet timing only. If there are other areas > of a query that rely on 'NOW', these will not interpret this value. This is a > broader issue about setting a 'query-global' NOW that all parts of query > analysis can share. > Source files affected: > FacetParams.java (holds the new constant FACET_DATE_NOW) > SimpleFacets.java getFacetDateCounts() NOW parameter modified > This PATCH is mildly related to SOLR-1709 (Distributed Date Faceting), but as > it's a general change for date faceting, it was deemed deserving of its own > patch. I will be updating SOLR-1709 in due course to include the use of this > new parameter, after some rfc acceptance. > A possible enhancement to this is to detect facet.date fields, look for and > match these fields in queries (if they exist), and potentially determine > automatically the required time skew, if any. There are a whole host of > reasons why this could be problematic to implement, so an explicit > facet.date.now parameter is the safest route. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1729) Date Facet now override time parameter
[ https://issues.apache.org/jira/browse/SOLR-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13187533#comment-13187533 ] Simon Willnauer commented on SOLR-1729: --- I am going to commit this soon if nobody objects. > Date Facet now override time parameter > -- > > Key: SOLR-1729 > URL: https://issues.apache.org/jira/browse/SOLR-1729 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.4 > Environment: Solr 1.4 >Reporter: Peter Sturge >Assignee: Simon Willnauer >Priority: Minor > Fix For: 3.6, 4.0 > > Attachments: FacetParams.java, SOLR-1729.patch, SOLR-1729.patch, > SOLR-1729.patch, SOLR-1729_3x.patch, SOLR-1729_3x.patch, SimpleFacets.java, > UnInvertedField.java, solr-1.4.0-solr-1729.patch > > > This PATCH introduces a new query parameter that tells a (typically, but not > necessarily) remote server what time to use as 'NOW' when calculating date > facets for a query (and, for the moment, date facets *only*) - overriding the > default behaviour of using the local server's current time. > This gets 'round a problem whereby an explicit time range is specified in a > query (e.g. timestamp:[then0 TO then1]), and date facets are required for the > given time range (in fact, any explicit time range). > Because DateMathParser performs all its calculations from 'NOW', remote > callers have to work out how long ago 'then0' and 'then1' are from 'now', and > use the relative-to-now values in the facet.date.xxx parameters. If a remote > server has a different opinion of NOW compared to the caller, the results > will be skewed (e.g. they are in a different time-zone, not time-synced etc.). > This becomes particularly salient when performing distributed date faceting > (see SOLR-1709), where multiple shards may all be running with different > times, and the faceting needs to be aligned. > The new parameter is called 'facet.date.now', and takes as a parameter a > (stringified) long that is the number of milliseconds from the epoch (1 Jan > 1970 00:00) - i.e. the returned value from a System.currentTimeMillis() call. > This was chosen over a formatted date to delineate it from a 'searchable' > time and to avoid superfluous date parsing. This makes the value generally a > programatically-set value, but as that is where the use-case is for this type > of parameter, this should be ok. > NOTE: This parameter affects date facet timing only. If there are other areas > of a query that rely on 'NOW', these will not interpret this value. This is a > broader issue about setting a 'query-global' NOW that all parts of query > analysis can share. > Source files affected: > FacetParams.java (holds the new constant FACET_DATE_NOW) > SimpleFacets.java getFacetDateCounts() NOW parameter modified > This PATCH is mildly related to SOLR-1709 (Distributed Date Faceting), but as > it's a general change for date faceting, it was deemed deserving of its own > patch. I will be updating SOLR-1709 in due course to include the use of this > new parameter, after some rfc acceptance. > A possible enhancement to this is to detect facet.date fields, look for and > match these fields in queries (if they exist), and potentially determine > automatically the required time skew, if any. There are a whole host of > reasons why this could be problematic to implement, so an explicit > facet.date.now parameter is the safest route. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (SOLR-1729) Date Facet now override time parameter
[ https://issues.apache.org/jira/browse/SOLR-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12966181#action_12966181 ] Peter Karich commented on SOLR-1729: Peter Sturge, in SOLR-1709 you said that you are working with branch3x I checked it out from here: https://svn.apache.org/repos/asf/lucene/dev/branches/branch_3x but this 1729 patch didn't apply cleanly*. When I tried the 1.4.1 release it is ok, but the tests fail due to** What could be wrong? Regards, Peter. * solr_branch_3x/solr$ patch -p0 < solr-1.4.0-solr-1729.patch patching file src/java/org/apache/solr/request/SimpleFacets.java Hunk #1 succeeded at 245 (offset 28 lines). Hunk #2 succeeded at 280 (offset 28 lines). Hunk #3 FAILED at 582. Hunk #4 FAILED at 652. 2 out of 4 hunks FAILED -- saving rejects to file src/java/org/apache/solr/request/SimpleFacets.java.rej patching file src/java/org/apache/solr/request/UnInvertedField.java Hunk #2 succeeded at 40 with fuzz 1 (offset 1 line). Hunk #3 succeeded at 440 (offset 5 lines). Hunk #4 succeeded at 557 (offset 5 lines). patching file src/common/org/apache/solr/common/params/FacetParams.java Hunk #1 FAILED at 175. 1 out of 1 hunk FAILED -- saving rejects to file src/common/org/apache/solr/common/params/FacetParams.java.rej ** [junit] Running org.apache.solr.TestTrie [junit] xml response was: [junit] [junit] 01570.02010-12-02T00:00:00Z0.00.0021474836471.02010-12-03T00:00:00Z2.3331.11121474836482.02010-12-04T00:00:00Z4.66124.44221474836493.02010-12-05T00:00:00Z6.99279.99321474836504.02010-12-06T00:00:00Z9.32497.76421474836515.02010-12-07T00:00:00Z11.65777.75521474836526.02010-12-08T00:00:00Z13.981119.96621474836537.02010-12-09T00:00:00Z16.3121524.39721474836548.02010-12-10T00:00:00Z18.641991.04821474836559.02010-12-11T00:00:00Z20.972519.91029214748365610.02010-12-02T00:00:00Z0.00.00214748364720.02010-12-03T00:00:00Z2.3331.111214748364830.02010-12-04T00:00:00Z4.66124.442214748364940.02010-12-05T00:00:00Z6.99279.993214748365050.02010-12-06T00:00:00Z9.32497.76421474836512121212111+1DAY2010-12-08T00:00:00Z [junit] [junit] [junit] request was: facet.date.start=NOW/DAY&facet=true&q=*:*&facet.date=tdate&facet.date.gap=%2B1DAY&facet.field=tint&facet.field=tlong&facet.field=tfloat&facet.field=tdouble&facet.date.end=NOW/DAY%2B6DAYS&rows=15) [junit] Tests run: 8, Failures: 1, Errors: 0, Time elapsed: 4,948 sec [junit] Test org.apache.solr.TestTrie FAILED [junit] Running org.apache.solr.request.SimpleFacetsTest [junit] xml response was: [junit] [junit] 011002210011020110+1DAY1976-08-01T00:00:00Z2111 [junit] [junit] [junit] request was: facet.date.start=1976-07-01T00:00:00.000Z&facet=true&q=*:*&facet.date=bday&facet.date.other=all&facet.date.gap=%2B1DAY&facet.date.end=1976-07-01T00:00:00.000Z%2B1MONTH&rows=0) [junit] Tests run: 6, Failures: 1, Errors: 0, Time elapsed: 6,996 sec [junit] Test org.apache.solr.request.SimpleFacetsTest FAILED > Date Facet now override time parameter > -- > > Key: SOLR-1729 > URL: https://issues.apache.org/jira/browse/SOLR-1729 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.4 > Environment: Solr 1.4 >Reporter: Peter Sturge >Priority: Minor > Attachments: FacetParams.java, SimpleFacets.java, > solr-1.4.0-solr-1729.patch, UnInvertedField.java > > > This PATCH introduces a new query parameter that tells a (typically, but not > necessarily) remote server what time to use as 'NOW' when calculating date > facets for a query (and, for the moment, date facets *only*) - overriding the > default behaviour of using the local server's current time. > This gets 'round a problem whereby an explicit time range is specified in a > query (e.g. timestamp:[then0 TO then1]), and date facets are required for the > given time range (in fact, any explicit time range). > Because DateMathParser performs all its calculations from 'NOW', remote > callers have to work out how long ago 'then0' and 'then1' are from 'now', and > use the relative-to-now values in the facet.date.xxx parameters. If a remote > server has a different opinion of NOW compared to the caller, the results > will be skewed (e.g. they are in a different time-zone, not time-synced etc.). > This becomes particularly salient when performing distributed date faceting > (see SOLR-1709), where multiple shards may all be running with different > times, and the faceting needs to be aligned. > The new parameter is called 'facet.date.now', and takes as a parameter a > (stringified) long that is the number of milliseconds from the epoch (1 Jan > 1970 00:00) - i.e. the returned value from a System.currentTimeMi
[jira] Commented: (SOLR-1729) Date Facet now override time parameter
[ https://issues.apache.org/jira/browse/SOLR-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12966238#action_12966238 ] Peter Sturge commented on SOLR-1729: So is 1709 ok, but 1729 isn't? > Date Facet now override time parameter > -- > > Key: SOLR-1729 > URL: https://issues.apache.org/jira/browse/SOLR-1729 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.4 > Environment: Solr 1.4 >Reporter: Peter Sturge >Priority: Minor > Attachments: FacetParams.java, SimpleFacets.java, > solr-1.4.0-solr-1729.patch, UnInvertedField.java > > > This PATCH introduces a new query parameter that tells a (typically, but not > necessarily) remote server what time to use as 'NOW' when calculating date > facets for a query (and, for the moment, date facets *only*) - overriding the > default behaviour of using the local server's current time. > This gets 'round a problem whereby an explicit time range is specified in a > query (e.g. timestamp:[then0 TO then1]), and date facets are required for the > given time range (in fact, any explicit time range). > Because DateMathParser performs all its calculations from 'NOW', remote > callers have to work out how long ago 'then0' and 'then1' are from 'now', and > use the relative-to-now values in the facet.date.xxx parameters. If a remote > server has a different opinion of NOW compared to the caller, the results > will be skewed (e.g. they are in a different time-zone, not time-synced etc.). > This becomes particularly salient when performing distributed date faceting > (see SOLR-1709), where multiple shards may all be running with different > times, and the faceting needs to be aligned. > The new parameter is called 'facet.date.now', and takes as a parameter a > (stringified) long that is the number of milliseconds from the epoch (1 Jan > 1970 00:00) - i.e. the returned value from a System.currentTimeMillis() call. > This was chosen over a formatted date to delineate it from a 'searchable' > time and to avoid superfluous date parsing. This makes the value generally a > programatically-set value, but as that is where the use-case is for this type > of parameter, this should be ok. > NOTE: This parameter affects date facet timing only. If there are other areas > of a query that rely on 'NOW', these will not interpret this value. This is a > broader issue about setting a 'query-global' NOW that all parts of query > analysis can share. > Source files affected: > FacetParams.java (holds the new constant FACET_DATE_NOW) > SimpleFacets.java getFacetDateCounts() NOW parameter modified > This PATCH is mildly related to SOLR-1709 (Distributed Date Faceting), but as > it's a general change for date faceting, it was deemed deserving of its own > patch. I will be updating SOLR-1709 in due course to include the use of this > new parameter, after some rfc acceptance. > A possible enhancement to this is to detect facet.date fields, look for and > match these fields in queries (if they exist), and potentially determine > automatically the required time skew, if any. There are a whole host of > reasons why this could be problematic to implement, so an explicit > facet.date.now parameter is the safest route. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (SOLR-1729) Date Facet now override time parameter
[ https://issues.apache.org/jira/browse/SOLR-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12966488#action_12966488 ] Peter Karich commented on SOLR-1729: *regarding: 1.4.1* Hmmh, today download.carrot2.org is down and I had to delete contrib/clustering to do the build after the patch. which does not apply cleanly (strange that it appled yesterday): solr1.4.1$ patch -p0 < solr-1.4.0-solr-1729.patch patching file src/java/org/apache/solr/handler/component/FacetComponent.java patching file src/java/org/apache/solr/handler/component/ResponseBuilder.java solr1.4.1$ patch -p0 < solr-1.4.0-solr-1709.patch patching file src/java/org/apache/solr/handler/component/FacetComponent.java Reversed (or previously applied) patch detected! Assume -R? [n] y patching file src/java/org/apache/solr/handler/component/ResponseBuilder.java Reversed (or previously applied) patch detected! Assume -R? [n] y Hunk #3 succeeded at 251 (offset -1 lines). Or is this ok?? Because then, all tests would pass ... *regarding branch3x* both patches do not apply cleanly. SOLR-1709 fails also without SOLR-1729 solr_branch_3x/solr$ patch -p0 < solr-1.4.0-solr-1709.patch patching file src/java/org/apache/solr/handler/component/FacetComponent.java Hunk #1 succeeded at 240 (offset 2 lines). Hunk #2 succeeded at 267 with fuzz 2 (offset 7 lines). Hunk #3 FAILED at 436. 1 out of 3 hunks FAILED -- saving rejects to file src/java/org/apache/solr/handler/component/FacetComponent.java.rej patching file src/java/org/apache/solr/handler/component/ResponseBuilder.java Reversed (or previously applied) patch detected! Assume -R? [n] y Hunk #2 FAILED at 61. Hunk #3 FAILED at 252. 2 out of 3 hunks FAILED -- saving rejects to file src/java/org/apache/solr/handler/component/ResponseBuilder.java.rej > Date Facet now override time parameter > -- > > Key: SOLR-1729 > URL: https://issues.apache.org/jira/browse/SOLR-1729 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.4 > Environment: Solr 1.4 >Reporter: Peter Sturge >Priority: Minor > Attachments: FacetParams.java, SimpleFacets.java, > solr-1.4.0-solr-1729.patch, UnInvertedField.java > > > This PATCH introduces a new query parameter that tells a (typically, but not > necessarily) remote server what time to use as 'NOW' when calculating date > facets for a query (and, for the moment, date facets *only*) - overriding the > default behaviour of using the local server's current time. > This gets 'round a problem whereby an explicit time range is specified in a > query (e.g. timestamp:[then0 TO then1]), and date facets are required for the > given time range (in fact, any explicit time range). > Because DateMathParser performs all its calculations from 'NOW', remote > callers have to work out how long ago 'then0' and 'then1' are from 'now', and > use the relative-to-now values in the facet.date.xxx parameters. If a remote > server has a different opinion of NOW compared to the caller, the results > will be skewed (e.g. they are in a different time-zone, not time-synced etc.). > This becomes particularly salient when performing distributed date faceting > (see SOLR-1709), where multiple shards may all be running with different > times, and the faceting needs to be aligned. > The new parameter is called 'facet.date.now', and takes as a parameter a > (stringified) long that is the number of milliseconds from the epoch (1 Jan > 1970 00:00) - i.e. the returned value from a System.currentTimeMillis() call. > This was chosen over a formatted date to delineate it from a 'searchable' > time and to avoid superfluous date parsing. This makes the value generally a > programatically-set value, but as that is where the use-case is for this type > of parameter, this should be ok. > NOTE: This parameter affects date facet timing only. If there are other areas > of a query that rely on 'NOW', these will not interpret this value. This is a > broader issue about setting a 'query-global' NOW that all parts of query > analysis can share. > Source files affected: > FacetParams.java (holds the new constant FACET_DATE_NOW) > SimpleFacets.java getFacetDateCounts() NOW parameter modified > This PATCH is mildly related to SOLR-1709 (Distributed Date Faceting), but as > it's a general change for date faceting, it was deemed deserving of its own > patch. I will be updating SOLR-1709 in due course to include the use of this > new parameter, after some rfc acceptance. > A possible enhancement to this is to detect facet.date fields, look for and > match these fields in queries (if they exist), and potentially determine > automatically the required time skew, if any. There are a whole host of > reasons why this could be problematic to imple
[jira] Commented: (SOLR-1729) Date Facet now override time parameter
[ https://issues.apache.org/jira/browse/SOLR-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12966494#action_12966494 ] Peter Sturge commented on SOLR-1729: Hi Peter, Not sure why it would work, then not... Both these patches were submitted just before all the version name changes (which I'm still getting to grips with). At the time, I think 1.4.1 was the latest release train. For 3.x recently we've done some manual merging due to some other changes (forwarding http credentials to remote shards). I'll have a look at building a separate 'branch3x' patch version, as there may have been some separate back porting changes in the affected files that's breaking the current patch. Are you using the latest release, or the latest trunk version? Thanks, Peter > Date Facet now override time parameter > -- > > Key: SOLR-1729 > URL: https://issues.apache.org/jira/browse/SOLR-1729 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.4 > Environment: Solr 1.4 >Reporter: Peter Sturge >Priority: Minor > Attachments: FacetParams.java, SimpleFacets.java, > solr-1.4.0-solr-1729.patch, UnInvertedField.java > > > This PATCH introduces a new query parameter that tells a (typically, but not > necessarily) remote server what time to use as 'NOW' when calculating date > facets for a query (and, for the moment, date facets *only*) - overriding the > default behaviour of using the local server's current time. > This gets 'round a problem whereby an explicit time range is specified in a > query (e.g. timestamp:[then0 TO then1]), and date facets are required for the > given time range (in fact, any explicit time range). > Because DateMathParser performs all its calculations from 'NOW', remote > callers have to work out how long ago 'then0' and 'then1' are from 'now', and > use the relative-to-now values in the facet.date.xxx parameters. If a remote > server has a different opinion of NOW compared to the caller, the results > will be skewed (e.g. they are in a different time-zone, not time-synced etc.). > This becomes particularly salient when performing distributed date faceting > (see SOLR-1709), where multiple shards may all be running with different > times, and the faceting needs to be aligned. > The new parameter is called 'facet.date.now', and takes as a parameter a > (stringified) long that is the number of milliseconds from the epoch (1 Jan > 1970 00:00) - i.e. the returned value from a System.currentTimeMillis() call. > This was chosen over a formatted date to delineate it from a 'searchable' > time and to avoid superfluous date parsing. This makes the value generally a > programatically-set value, but as that is where the use-case is for this type > of parameter, this should be ok. > NOTE: This parameter affects date facet timing only. If there are other areas > of a query that rely on 'NOW', these will not interpret this value. This is a > broader issue about setting a 'query-global' NOW that all parts of query > analysis can share. > Source files affected: > FacetParams.java (holds the new constant FACET_DATE_NOW) > SimpleFacets.java getFacetDateCounts() NOW parameter modified > This PATCH is mildly related to SOLR-1709 (Distributed Date Faceting), but as > it's a general change for date faceting, it was deemed deserving of its own > patch. I will be updating SOLR-1709 in due course to include the use of this > new parameter, after some rfc acceptance. > A possible enhancement to this is to detect facet.date fields, look for and > match these fields in queries (if they exist), and potentially determine > automatically the required time skew, if any. There are a whole host of > reasons why this could be problematic to implement, so an explicit > facet.date.now parameter is the safest route. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (SOLR-1729) Date Facet now override time parameter
[ https://issues.apache.org/jira/browse/SOLR-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12966520#action_12966520 ] Peter Karich commented on SOLR-1729: Hi Peter, 1.4.1 would be fine (I asked Jake from solandra, before I thought he uses the trunk) Now in my last comment I made a stupid mistake: the patches didn't cleanly apply for 1.4.1 because I accidentially overwrote solr-1729.patch with solr-1709 when copying from branch3x and got two identical 1709 patches :-/ So: for 1.4.1 the patches apply cleanly. But the question remains why the following tests are failing: Test org.apache.solr.TestTrie FAILED Test org.apache.solr.request.SimpleFacetsTest FAILED > Date Facet now override time parameter > -- > > Key: SOLR-1729 > URL: https://issues.apache.org/jira/browse/SOLR-1729 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.4 > Environment: Solr 1.4 >Reporter: Peter Sturge >Priority: Minor > Attachments: FacetParams.java, SimpleFacets.java, > solr-1.4.0-solr-1729.patch, UnInvertedField.java > > > This PATCH introduces a new query parameter that tells a (typically, but not > necessarily) remote server what time to use as 'NOW' when calculating date > facets for a query (and, for the moment, date facets *only*) - overriding the > default behaviour of using the local server's current time. > This gets 'round a problem whereby an explicit time range is specified in a > query (e.g. timestamp:[then0 TO then1]), and date facets are required for the > given time range (in fact, any explicit time range). > Because DateMathParser performs all its calculations from 'NOW', remote > callers have to work out how long ago 'then0' and 'then1' are from 'now', and > use the relative-to-now values in the facet.date.xxx parameters. If a remote > server has a different opinion of NOW compared to the caller, the results > will be skewed (e.g. they are in a different time-zone, not time-synced etc.). > This becomes particularly salient when performing distributed date faceting > (see SOLR-1709), where multiple shards may all be running with different > times, and the faceting needs to be aligned. > The new parameter is called 'facet.date.now', and takes as a parameter a > (stringified) long that is the number of milliseconds from the epoch (1 Jan > 1970 00:00) - i.e. the returned value from a System.currentTimeMillis() call. > This was chosen over a formatted date to delineate it from a 'searchable' > time and to avoid superfluous date parsing. This makes the value generally a > programatically-set value, but as that is where the use-case is for this type > of parameter, this should be ok. > NOTE: This parameter affects date facet timing only. If there are other areas > of a query that rely on 'NOW', these will not interpret this value. This is a > broader issue about setting a 'query-global' NOW that all parts of query > analysis can share. > Source files affected: > FacetParams.java (holds the new constant FACET_DATE_NOW) > SimpleFacets.java getFacetDateCounts() NOW parameter modified > This PATCH is mildly related to SOLR-1709 (Distributed Date Faceting), but as > it's a general change for date faceting, it was deemed deserving of its own > patch. I will be updating SOLR-1709 in due course to include the use of this > new parameter, after some rfc acceptance. > A possible enhancement to this is to detect facet.date fields, look for and > match these fields in queries (if they exist), and potentially determine > automatically the required time skew, if any. There are a whole host of > reasons why this could be problematic to implement, so an explicit > facet.date.now parameter is the safest route. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (SOLR-1729) Date Facet now override time parameter
[ https://issues.apache.org/jira/browse/SOLR-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12966533#action_12966533 ] Peter Sturge commented on SOLR-1729: Hi Peter, So, the patches are clean (for 1.4.1), but the tests are failing for 1.4.1? Or is the failure in 3.x? Sorry, but I'm a bit confused which bit isn't working now. Thanks, Peter On Fri, Dec 3, 2010 at 1:05 PM, Peter Karich (JIRA) wrote: > Date Facet now override time parameter > -- > > Key: SOLR-1729 > URL: https://issues.apache.org/jira/browse/SOLR-1729 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.4 > Environment: Solr 1.4 >Reporter: Peter Sturge >Priority: Minor > Attachments: FacetParams.java, SimpleFacets.java, > solr-1.4.0-solr-1729.patch, UnInvertedField.java > > > This PATCH introduces a new query parameter that tells a (typically, but not > necessarily) remote server what time to use as 'NOW' when calculating date > facets for a query (and, for the moment, date facets *only*) - overriding the > default behaviour of using the local server's current time. > This gets 'round a problem whereby an explicit time range is specified in a > query (e.g. timestamp:[then0 TO then1]), and date facets are required for the > given time range (in fact, any explicit time range). > Because DateMathParser performs all its calculations from 'NOW', remote > callers have to work out how long ago 'then0' and 'then1' are from 'now', and > use the relative-to-now values in the facet.date.xxx parameters. If a remote > server has a different opinion of NOW compared to the caller, the results > will be skewed (e.g. they are in a different time-zone, not time-synced etc.). > This becomes particularly salient when performing distributed date faceting > (see SOLR-1709), where multiple shards may all be running with different > times, and the faceting needs to be aligned. > The new parameter is called 'facet.date.now', and takes as a parameter a > (stringified) long that is the number of milliseconds from the epoch (1 Jan > 1970 00:00) - i.e. the returned value from a System.currentTimeMillis() call. > This was chosen over a formatted date to delineate it from a 'searchable' > time and to avoid superfluous date parsing. This makes the value generally a > programatically-set value, but as that is where the use-case is for this type > of parameter, this should be ok. > NOTE: This parameter affects date facet timing only. If there are other areas > of a query that rely on 'NOW', these will not interpret this value. This is a > broader issue about setting a 'query-global' NOW that all parts of query > analysis can share. > Source files affected: > FacetParams.java (holds the new constant FACET_DATE_NOW) > SimpleFacets.java getFacetDateCounts() NOW parameter modified > This PATCH is mildly related to SOLR-1709 (Distributed Date Faceting), but as > it's a general change for date faceting, it was deemed deserving of its own > patch. I will be updating SOLR-1709 in due course to include the use of this > new parameter, after some rfc acceptance. > A possible enhancement to this is to detect facet.date fields, look for and > match these fields in queries (if they exist), and potentially determine > automatically the required time skew, if any. There are a whole host of > reasons why this could be problematic to implement, so an explicit > facet.date.now parameter is the safest route. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (SOLR-1729) Date Facet now override time parameter
[ https://issues.apache.org/jira/browse/SOLR-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12966562#action_12966562 ] Peter Karich commented on SOLR-1729: Hi Peter, sorry for the confusion :-/ I was speaking of 1.4.1: the two patches apply. 2 tests fail. Regards, Peter. > Date Facet now override time parameter > -- > > Key: SOLR-1729 > URL: https://issues.apache.org/jira/browse/SOLR-1729 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.4 > Environment: Solr 1.4 >Reporter: Peter Sturge >Priority: Minor > Attachments: FacetParams.java, SimpleFacets.java, > solr-1.4.0-solr-1729.patch, UnInvertedField.java > > > This PATCH introduces a new query parameter that tells a (typically, but not > necessarily) remote server what time to use as 'NOW' when calculating date > facets for a query (and, for the moment, date facets *only*) - overriding the > default behaviour of using the local server's current time. > This gets 'round a problem whereby an explicit time range is specified in a > query (e.g. timestamp:[then0 TO then1]), and date facets are required for the > given time range (in fact, any explicit time range). > Because DateMathParser performs all its calculations from 'NOW', remote > callers have to work out how long ago 'then0' and 'then1' are from 'now', and > use the relative-to-now values in the facet.date.xxx parameters. If a remote > server has a different opinion of NOW compared to the caller, the results > will be skewed (e.g. they are in a different time-zone, not time-synced etc.). > This becomes particularly salient when performing distributed date faceting > (see SOLR-1709), where multiple shards may all be running with different > times, and the faceting needs to be aligned. > The new parameter is called 'facet.date.now', and takes as a parameter a > (stringified) long that is the number of milliseconds from the epoch (1 Jan > 1970 00:00) - i.e. the returned value from a System.currentTimeMillis() call. > This was chosen over a formatted date to delineate it from a 'searchable' > time and to avoid superfluous date parsing. This makes the value generally a > programatically-set value, but as that is where the use-case is for this type > of parameter, this should be ok. > NOTE: This parameter affects date facet timing only. If there are other areas > of a query that rely on 'NOW', these will not interpret this value. This is a > broader issue about setting a 'query-global' NOW that all parts of query > analysis can share. > Source files affected: > FacetParams.java (holds the new constant FACET_DATE_NOW) > SimpleFacets.java getFacetDateCounts() NOW parameter modified > This PATCH is mildly related to SOLR-1709 (Distributed Date Faceting), but as > it's a general change for date faceting, it was deemed deserving of its own > patch. I will be updating SOLR-1709 in due course to include the use of this > new parameter, after some rfc acceptance. > A possible enhancement to this is to detect facet.date fields, look for and > match these fields in queries (if they exist), and potentially determine > automatically the required time skew, if any. There are a whole host of > reasons why this could be problematic to implement, so an explicit > facet.date.now parameter is the safest route. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (SOLR-1729) Date Facet now override time parameter
[ https://issues.apache.org/jira/browse/SOLR-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12970544#action_12970544 ] Peter Karich commented on SOLR-1729: Yonik, thanks for the update. I refreshed my sources (now trunk) to rev 1044745. But the patch does not cleanly apply* for SearchHandler. Am I doing something stupid here? Regards, Peter. * pathxy/solr_branch_3x$ patch -p0 < SOLR-1729.patch patching file solr/src/test/test-files/solr/conf/schema12.xml patching file solr/src/test/org/apache/solr/search/function/TestFunctionQuery.java Hunk #1 succeeded at 301 (offset -17 lines). patching file solr/src/test/org/apache/solr/handler/component/SpellCheckComponentTest.java patching file solr/src/test/org/apache/solr/handler/component/TermVectorComponentTest.java patching file solr/src/java/org/apache/solr/core/QuerySenderListener.java patching file solr/src/java/org/apache/solr/request/SimpleFacets.java Hunk #1 succeeded at 64 (offset -9 lines). Hunk #2 succeeded at 620 (offset -200 lines). Hunk #3 succeeded at 630 (offset -200 lines). Hunk #4 succeeded at 645 (offset -200 lines). Hunk #5 succeeded at 803 (offset -200 lines). patching file solr/src/java/org/apache/solr/handler/component/SearchHandler.java Hunk #1 FAILED at 192. Hunk #2 succeeded at 255 (offset -36 lines). 1 out of 2 hunks FAILED -- saving rejects to file solr/src/java/org/apache/solr/handler/component/SearchHandler.java.rej patching file solr/src/java/org/apache/solr/handler/component/ResponseBuilder.java Hunk #2 succeeded at 67 (offset -1 lines). patching file solr/src/java/org/apache/solr/spelling/SpellCheckCollator.java patching file solr/src/java/org/apache/solr/util/TestHarness.java Hunk #2 succeeded at 320 (offset -9 lines). Hunk #3 succeeded at 335 (offset -9 lines). patching file solr/src/java/org/apache/solr/util/DateMathParser.java patching file solr/src/webapp/src/org/apache/solr/servlet/SolrServlet.java patching file solr/src/webapp/src/org/apache/solr/servlet/SolrDispatchFilter.java Hunk #1 succeeded at 241 (offset 4 lines). Hunk #2 succeeded at 255 (offset 4 lines). Hunk #3 succeeded at 283 (offset 4 lines). patching file solr/src/webapp/src/org/apache/solr/servlet/DirectSolrConnection.java Hunk #2 succeeded at 170 (offset -16 lines). Hunk #3 succeeded at 185 with fuzz 1 (offset -16 lines). patching file solr/src/webapp/src/org/apache/solr/client/solrj/embedded/EmbeddedSolrServer.java Hunk #1 succeeded at 32 with fuzz 1 (offset -9 lines). Hunk #2 succeeded at 138 (offset -11 lines). Hunk #3 succeeded at 156 (offset -77 lines). > Date Facet now override time parameter > -- > > Key: SOLR-1729 > URL: https://issues.apache.org/jira/browse/SOLR-1729 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.4 > Environment: Solr 1.4 >Reporter: Peter Sturge >Priority: Minor > Attachments: FacetParams.java, SimpleFacets.java, > solr-1.4.0-solr-1729.patch, SOLR-1729.patch, SOLR-1729.patch, > UnInvertedField.java > > > This PATCH introduces a new query parameter that tells a (typically, but not > necessarily) remote server what time to use as 'NOW' when calculating date > facets for a query (and, for the moment, date facets *only*) - overriding the > default behaviour of using the local server's current time. > This gets 'round a problem whereby an explicit time range is specified in a > query (e.g. timestamp:[then0 TO then1]), and date facets are required for the > given time range (in fact, any explicit time range). > Because DateMathParser performs all its calculations from 'NOW', remote > callers have to work out how long ago 'then0' and 'then1' are from 'now', and > use the relative-to-now values in the facet.date.xxx parameters. If a remote > server has a different opinion of NOW compared to the caller, the results > will be skewed (e.g. they are in a different time-zone, not time-synced etc.). > This becomes particularly salient when performing distributed date faceting > (see SOLR-1709), where multiple shards may all be running with different > times, and the faceting needs to be aligned. > The new parameter is called 'facet.date.now', and takes as a parameter a > (stringified) long that is the number of milliseconds from the epoch (1 Jan > 1970 00:00) - i.e. the returned value from a System.currentTimeMillis() call. > This was chosen over a formatted date to delineate it from a 'searchable' > time and to avoid superfluous date parsing. This makes the value generally a > programatically-set value, but as that is where the use-case is for this type > of parameter, this should be ok. > NOTE: This parameter affects date facet timing only. If there are other areas > of a query that rely on 'NOW', these will not interpret this value. This is a >
[jira] Commented: (SOLR-1729) Date Facet now override time parameter
[ https://issues.apache.org/jira/browse/SOLR-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12970547#action_12970547 ] Yonik Seeley commented on SOLR-1729: bq. pathxy/solr_branch_3x$ patch -p0 < SOLR-1729.patch The patch was for trunk, not 3x? > Date Facet now override time parameter > -- > > Key: SOLR-1729 > URL: https://issues.apache.org/jira/browse/SOLR-1729 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.4 > Environment: Solr 1.4 >Reporter: Peter Sturge >Priority: Minor > Attachments: FacetParams.java, SimpleFacets.java, > solr-1.4.0-solr-1729.patch, SOLR-1729.patch, SOLR-1729.patch, > UnInvertedField.java > > > This PATCH introduces a new query parameter that tells a (typically, but not > necessarily) remote server what time to use as 'NOW' when calculating date > facets for a query (and, for the moment, date facets *only*) - overriding the > default behaviour of using the local server's current time. > This gets 'round a problem whereby an explicit time range is specified in a > query (e.g. timestamp:[then0 TO then1]), and date facets are required for the > given time range (in fact, any explicit time range). > Because DateMathParser performs all its calculations from 'NOW', remote > callers have to work out how long ago 'then0' and 'then1' are from 'now', and > use the relative-to-now values in the facet.date.xxx parameters. If a remote > server has a different opinion of NOW compared to the caller, the results > will be skewed (e.g. they are in a different time-zone, not time-synced etc.). > This becomes particularly salient when performing distributed date faceting > (see SOLR-1709), where multiple shards may all be running with different > times, and the faceting needs to be aligned. > The new parameter is called 'facet.date.now', and takes as a parameter a > (stringified) long that is the number of milliseconds from the epoch (1 Jan > 1970 00:00) - i.e. the returned value from a System.currentTimeMillis() call. > This was chosen over a formatted date to delineate it from a 'searchable' > time and to avoid superfluous date parsing. This makes the value generally a > programatically-set value, but as that is where the use-case is for this type > of parameter, this should be ok. > NOTE: This parameter affects date facet timing only. If there are other areas > of a query that rely on 'NOW', these will not interpret this value. This is a > broader issue about setting a 'query-global' NOW that all parts of query > analysis can share. > Source files affected: > FacetParams.java (holds the new constant FACET_DATE_NOW) > SimpleFacets.java getFacetDateCounts() NOW parameter modified > This PATCH is mildly related to SOLR-1709 (Distributed Date Faceting), but as > it's a general change for date faceting, it was deemed deserving of its own > patch. I will be updating SOLR-1709 in due course to include the use of this > new parameter, after some rfc acceptance. > A possible enhancement to this is to detect facet.date fields, look for and > match these fields in queries (if they exist), and potentially determine > automatically the required time skew, if any. There are a whole host of > reasons why this could be problematic to implement, so an explicit > facet.date.now parameter is the safest route. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (SOLR-1729) Date Facet now override time parameter
[ https://issues.apache.org/jira/browse/SOLR-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12970612#action_12970612 ] Peter Karich commented on SOLR-1729: Hi Yonik, so, sorry for another misposting: yes, you were right. it was the wrong solr version. it was too late yesterday :-/ All is fine now with this patch. But the org.apache.solr.request.SolrRequestInfo class is missing or am I completely crazy now? (I checked out solr twice and applied the patch again but it didn't compile) Regards, Peter. > Date Facet now override time parameter > -- > > Key: SOLR-1729 > URL: https://issues.apache.org/jira/browse/SOLR-1729 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.4 > Environment: Solr 1.4 >Reporter: Peter Sturge >Priority: Minor > Attachments: FacetParams.java, SimpleFacets.java, > solr-1.4.0-solr-1729.patch, SOLR-1729.patch, SOLR-1729.patch, > UnInvertedField.java > > > This PATCH introduces a new query parameter that tells a (typically, but not > necessarily) remote server what time to use as 'NOW' when calculating date > facets for a query (and, for the moment, date facets *only*) - overriding the > default behaviour of using the local server's current time. > This gets 'round a problem whereby an explicit time range is specified in a > query (e.g. timestamp:[then0 TO then1]), and date facets are required for the > given time range (in fact, any explicit time range). > Because DateMathParser performs all its calculations from 'NOW', remote > callers have to work out how long ago 'then0' and 'then1' are from 'now', and > use the relative-to-now values in the facet.date.xxx parameters. If a remote > server has a different opinion of NOW compared to the caller, the results > will be skewed (e.g. they are in a different time-zone, not time-synced etc.). > This becomes particularly salient when performing distributed date faceting > (see SOLR-1709), where multiple shards may all be running with different > times, and the faceting needs to be aligned. > The new parameter is called 'facet.date.now', and takes as a parameter a > (stringified) long that is the number of milliseconds from the epoch (1 Jan > 1970 00:00) - i.e. the returned value from a System.currentTimeMillis() call. > This was chosen over a formatted date to delineate it from a 'searchable' > time and to avoid superfluous date parsing. This makes the value generally a > programatically-set value, but as that is where the use-case is for this type > of parameter, this should be ok. > NOTE: This parameter affects date facet timing only. If there are other areas > of a query that rely on 'NOW', these will not interpret this value. This is a > broader issue about setting a 'query-global' NOW that all parts of query > analysis can share. > Source files affected: > FacetParams.java (holds the new constant FACET_DATE_NOW) > SimpleFacets.java getFacetDateCounts() NOW parameter modified > This PATCH is mildly related to SOLR-1709 (Distributed Date Faceting), but as > it's a general change for date faceting, it was deemed deserving of its own > patch. I will be updating SOLR-1709 in due course to include the use of this > new parameter, after some rfc acceptance. > A possible enhancement to this is to detect facet.date fields, look for and > match these fields in queries (if they exist), and potentially determine > automatically the required time skew, if any. There are a whole host of > reasons why this could be problematic to implement, so an explicit > facet.date.now parameter is the safest route. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (SOLR-1729) Date Facet now override time parameter
[ https://issues.apache.org/jira/browse/SOLR-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12970671#action_12970671 ] Peter Karich commented on SOLR-1729: Nice, now this patch 1729 applies + compiles + run tests successfully (I'm using rev 1044942 of trunk) One further question: Would facet queries (with dates) work in the distributed setup without the date-patches? To get a quick(er) workaround. because I would need the patch for 1.4.1 (->solandra) > Date Facet now override time parameter > -- > > Key: SOLR-1729 > URL: https://issues.apache.org/jira/browse/SOLR-1729 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.4 > Environment: Solr 1.4 >Reporter: Peter Sturge >Priority: Minor > Attachments: FacetParams.java, SimpleFacets.java, > solr-1.4.0-solr-1729.patch, SOLR-1729.patch, SOLR-1729.patch, > SOLR-1729.patch, UnInvertedField.java > > > This PATCH introduces a new query parameter that tells a (typically, but not > necessarily) remote server what time to use as 'NOW' when calculating date > facets for a query (and, for the moment, date facets *only*) - overriding the > default behaviour of using the local server's current time. > This gets 'round a problem whereby an explicit time range is specified in a > query (e.g. timestamp:[then0 TO then1]), and date facets are required for the > given time range (in fact, any explicit time range). > Because DateMathParser performs all its calculations from 'NOW', remote > callers have to work out how long ago 'then0' and 'then1' are from 'now', and > use the relative-to-now values in the facet.date.xxx parameters. If a remote > server has a different opinion of NOW compared to the caller, the results > will be skewed (e.g. they are in a different time-zone, not time-synced etc.). > This becomes particularly salient when performing distributed date faceting > (see SOLR-1709), where multiple shards may all be running with different > times, and the faceting needs to be aligned. > The new parameter is called 'facet.date.now', and takes as a parameter a > (stringified) long that is the number of milliseconds from the epoch (1 Jan > 1970 00:00) - i.e. the returned value from a System.currentTimeMillis() call. > This was chosen over a formatted date to delineate it from a 'searchable' > time and to avoid superfluous date parsing. This makes the value generally a > programatically-set value, but as that is where the use-case is for this type > of parameter, this should be ok. > NOTE: This parameter affects date facet timing only. If there are other areas > of a query that rely on 'NOW', these will not interpret this value. This is a > broader issue about setting a 'query-global' NOW that all parts of query > analysis can share. > Source files affected: > FacetParams.java (holds the new constant FACET_DATE_NOW) > SimpleFacets.java getFacetDateCounts() NOW parameter modified > This PATCH is mildly related to SOLR-1709 (Distributed Date Faceting), but as > it's a general change for date faceting, it was deemed deserving of its own > patch. I will be updating SOLR-1709 in due course to include the use of this > new parameter, after some rfc acceptance. > A possible enhancement to this is to detect facet.date fields, look for and > match these fields in queries (if they exist), and potentially determine > automatically the required time skew, if any. There are a whole host of > reasons why this could be problematic to implement, so an explicit > facet.date.now parameter is the safest route. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (SOLR-1729) Date Facet now override time parameter
[ https://issues.apache.org/jira/browse/SOLR-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12970691#action_12970691 ] Yonik Seeley commented on SOLR-1729: bq. Would facet queries (with dates) work in the distributed setup without the date-patches? I'm not sure which date-patches you refer to, but this should fix all issues related to NOW being different in different places (both distributed and non distributed). > Date Facet now override time parameter > -- > > Key: SOLR-1729 > URL: https://issues.apache.org/jira/browse/SOLR-1729 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.4 > Environment: Solr 1.4 >Reporter: Peter Sturge >Priority: Minor > Attachments: FacetParams.java, SimpleFacets.java, > solr-1.4.0-solr-1729.patch, SOLR-1729.patch, SOLR-1729.patch, > SOLR-1729.patch, UnInvertedField.java > > > This PATCH introduces a new query parameter that tells a (typically, but not > necessarily) remote server what time to use as 'NOW' when calculating date > facets for a query (and, for the moment, date facets *only*) - overriding the > default behaviour of using the local server's current time. > This gets 'round a problem whereby an explicit time range is specified in a > query (e.g. timestamp:[then0 TO then1]), and date facets are required for the > given time range (in fact, any explicit time range). > Because DateMathParser performs all its calculations from 'NOW', remote > callers have to work out how long ago 'then0' and 'then1' are from 'now', and > use the relative-to-now values in the facet.date.xxx parameters. If a remote > server has a different opinion of NOW compared to the caller, the results > will be skewed (e.g. they are in a different time-zone, not time-synced etc.). > This becomes particularly salient when performing distributed date faceting > (see SOLR-1709), where multiple shards may all be running with different > times, and the faceting needs to be aligned. > The new parameter is called 'facet.date.now', and takes as a parameter a > (stringified) long that is the number of milliseconds from the epoch (1 Jan > 1970 00:00) - i.e. the returned value from a System.currentTimeMillis() call. > This was chosen over a formatted date to delineate it from a 'searchable' > time and to avoid superfluous date parsing. This makes the value generally a > programatically-set value, but as that is where the use-case is for this type > of parameter, this should be ok. > NOTE: This parameter affects date facet timing only. If there are other areas > of a query that rely on 'NOW', these will not interpret this value. This is a > broader issue about setting a 'query-global' NOW that all parts of query > analysis can share. > Source files affected: > FacetParams.java (holds the new constant FACET_DATE_NOW) > SimpleFacets.java getFacetDateCounts() NOW parameter modified > This PATCH is mildly related to SOLR-1709 (Distributed Date Faceting), but as > it's a general change for date faceting, it was deemed deserving of its own > patch. I will be updating SOLR-1709 in due course to include the use of this > new parameter, after some rfc acceptance. > A possible enhancement to this is to detect facet.date fields, look for and > match these fields in queries (if they exist), and potentially determine > automatically the required time skew, if any. There are a whole host of > reasons why this could be problematic to implement, so an explicit > facet.date.now parameter is the safest route. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (SOLR-1729) Date Facet now override time parameter
[ https://issues.apache.org/jira/browse/SOLR-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12970785#action_12970785 ] Peter Sturge commented on SOLR-1729: Many thanks for finishing off this patch. Sorry I didn't get time to fix this, been swamped with so many projects at the moment. That's great you got the thread local NOW included as well. Thanks! > Date Facet now override time parameter > -- > > Key: SOLR-1729 > URL: https://issues.apache.org/jira/browse/SOLR-1729 > Project: Solr > Issue Type: Improvement > Components: search >Affects Versions: 1.4 > Environment: Solr 1.4 >Reporter: Peter Sturge >Priority: Minor > Fix For: 4.0 > > Attachments: FacetParams.java, SimpleFacets.java, > solr-1.4.0-solr-1729.patch, SOLR-1729.patch, SOLR-1729.patch, > SOLR-1729.patch, UnInvertedField.java > > > This PATCH introduces a new query parameter that tells a (typically, but not > necessarily) remote server what time to use as 'NOW' when calculating date > facets for a query (and, for the moment, date facets *only*) - overriding the > default behaviour of using the local server's current time. > This gets 'round a problem whereby an explicit time range is specified in a > query (e.g. timestamp:[then0 TO then1]), and date facets are required for the > given time range (in fact, any explicit time range). > Because DateMathParser performs all its calculations from 'NOW', remote > callers have to work out how long ago 'then0' and 'then1' are from 'now', and > use the relative-to-now values in the facet.date.xxx parameters. If a remote > server has a different opinion of NOW compared to the caller, the results > will be skewed (e.g. they are in a different time-zone, not time-synced etc.). > This becomes particularly salient when performing distributed date faceting > (see SOLR-1709), where multiple shards may all be running with different > times, and the faceting needs to be aligned. > The new parameter is called 'facet.date.now', and takes as a parameter a > (stringified) long that is the number of milliseconds from the epoch (1 Jan > 1970 00:00) - i.e. the returned value from a System.currentTimeMillis() call. > This was chosen over a formatted date to delineate it from a 'searchable' > time and to avoid superfluous date parsing. This makes the value generally a > programatically-set value, but as that is where the use-case is for this type > of parameter, this should be ok. > NOTE: This parameter affects date facet timing only. If there are other areas > of a query that rely on 'NOW', these will not interpret this value. This is a > broader issue about setting a 'query-global' NOW that all parts of query > analysis can share. > Source files affected: > FacetParams.java (holds the new constant FACET_DATE_NOW) > SimpleFacets.java getFacetDateCounts() NOW parameter modified > This PATCH is mildly related to SOLR-1709 (Distributed Date Faceting), but as > it's a general change for date faceting, it was deemed deserving of its own > patch. I will be updating SOLR-1709 in due course to include the use of this > new parameter, after some rfc acceptance. > A possible enhancement to this is to detect facet.date fields, look for and > match these fields in queries (if they exist), and potentially determine > automatically the required time skew, if any. There are a whole host of > reasons why this could be problematic to implement, so an explicit > facet.date.now parameter is the safest route. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] Commented: (SOLR-1729) Date Facet now override time parameter
Hi Peter K, Sorry for the delay - I'm a bit swamped with projects at the moment... Are you ok with your environment for the moment, until I can get time to look at this properly? Thanks, Peter On Fri, Dec 3, 2010 at 3:57 PM, Peter Karich (JIRA) wrote: > > [ > https://issues.apache.org/jira/browse/SOLR-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12966562#action_12966562 > ] > > Peter Karich commented on SOLR-1729: > > > Hi Peter, > > sorry for the confusion :-/ > > I was speaking of 1.4.1: the two patches apply. 2 tests fail. > > Regards, > Peter. > >> Date Facet now override time parameter >> -- >> >> Key: SOLR-1729 >> URL: https://issues.apache.org/jira/browse/SOLR-1729 >> Project: Solr >> Issue Type: Improvement >> Components: search >> Affects Versions: 1.4 >> Environment: Solr 1.4 >> Reporter: Peter Sturge >> Priority: Minor >> Attachments: FacetParams.java, SimpleFacets.java, >> solr-1.4.0-solr-1729.patch, UnInvertedField.java >> >> >> This PATCH introduces a new query parameter that tells a (typically, but not >> necessarily) remote server what time to use as 'NOW' when calculating date >> facets for a query (and, for the moment, date facets *only*) - overriding >> the default behaviour of using the local server's current time. >> This gets 'round a problem whereby an explicit time range is specified in a >> query (e.g. timestamp:[then0 TO then1]), and date facets are required for >> the given time range (in fact, any explicit time range). >> Because DateMathParser performs all its calculations from 'NOW', remote >> callers have to work out how long ago 'then0' and 'then1' are from 'now', >> and use the relative-to-now values in the facet.date.xxx parameters. If a >> remote server has a different opinion of NOW compared to the caller, the >> results will be skewed (e.g. they are in a different time-zone, not >> time-synced etc.). >> This becomes particularly salient when performing distributed date faceting >> (see SOLR-1709), where multiple shards may all be running with different >> times, and the faceting needs to be aligned. >> The new parameter is called 'facet.date.now', and takes as a parameter a >> (stringified) long that is the number of milliseconds from the epoch (1 Jan >> 1970 00:00) - i.e. the returned value from a System.currentTimeMillis() >> call. This was chosen over a formatted date to delineate it from a >> 'searchable' time and to avoid superfluous date parsing. This makes the >> value generally a programatically-set value, but as that is where the >> use-case is for this type of parameter, this should be ok. >> NOTE: This parameter affects date facet timing only. If there are other >> areas of a query that rely on 'NOW', these will not interpret this value. >> This is a broader issue about setting a 'query-global' NOW that all parts of >> query analysis can share. >> Source files affected: >> FacetParams.java (holds the new constant FACET_DATE_NOW) >> SimpleFacets.java getFacetDateCounts() NOW parameter modified >> This PATCH is mildly related to SOLR-1709 (Distributed Date Faceting), but >> as it's a general change for date faceting, it was deemed deserving of its >> own patch. I will be updating SOLR-1709 in due course to include the use of >> this new parameter, after some rfc acceptance. >> A possible enhancement to this is to detect facet.date fields, look for and >> match these fields in queries (if they exist), and potentially determine >> automatically the required time skew, if any. There are a whole host of >> reasons why this could be problematic to implement, so an explicit >> facet.date.now parameter is the safest route. > > -- > This message is automatically generated by JIRA. > - > You can reply to this email to add a comment to the issue online. > > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > > - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
