[jira] [Commented] (CASSANDRA-13218) Duration validation error is unclear in case of overflow.
[ https://issues.apache.org/jira/browse/CASSANDRA-13218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15983870#comment-15983870 ] Sandeep Tamhankar commented on CASSANDRA-13218: --- Found a few other bad side effects of the current behavior: 1. The validated (truncated) values are not actually re-encoded to vint's and stored; the user-provided bytes are stored. This is bad because it introduces an inconsistency of behavior -- a client can insert/select 64-bit values for months and days, but when C* exports the data as json (for example), the Duration.deserialize method will produce an object with truncated values. 2. Validation doesn't check for truncation error and reject the insert. 3. I think validation will succeed for this illegal set of values provided by a driver: (-9223372036854775807, 1, 1) because the months attribute will be coerced to 1 (since the highest 4 bytes will be dropped). *Proposal 1*: accept and handle 64-bit values for months and days consistently. I don't know how often these values are used in C*, but an extra 8 bytes doesn't sound like it's worth the hassle of treating months and days differently than nanos. It makes C* code a little more complex; it makes driver code a little more complex (for drivers that are type-aware at least). *Proposal 2*: beef up the validation logic to reject values that would overflow/underflow during long->int truncation. Write a tool that users can use to identify illegal rows they inserted because of the holes in the validation logic and give users an opportunity to clean up such bad data. Admittedly, the chances that users have inserted illegal duration's is relatively small, particularly since this new type hasn't been around very long. But it seems much simpler to make those values legal, so I'd lean towards my first proposal. > Duration validation error is unclear in case of overflow. > - > > Key: CASSANDRA-13218 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13218 > Project: Cassandra > Issue Type: Bug > Components: CQL >Reporter: Benjamin Lerer >Assignee: Benjamin Lerer > Fix For: 3.11.x > > > If a user try to insert a {{duration}} with a number of months or days that > cannot fit in an {{int}} (for example: {{9223372036854775807mo1d}}), the > error message is confusing. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13218) Duration validation error is unclear in case of overflow.
[ https://issues.apache.org/jira/browse/CASSANDRA-13218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16004679#comment-16004679 ] Benjamin Lerer commented on CASSANDRA-13218: Unfortunately, there are other factors that force us to use 32-bit values for months and days. Java utilities like {{Calendar}} accept only {{int}} for months or days value and we already use {{Calendar}} for data arithmetics. Due to that the solution is simply to reject to values at write time. I pushed a patch for that [here|https://github.com/apache/cassandra/compare/trunk...blerer:13128-3.11]. I think that the chances that users have inserted illegal durations is rather null. The 2 reasons for that are that: # duration is not really usefull in itself and was mainly added for simplifying the life of user querying timeseries data # the invalid values are greater than the age of mankind and by consequence have not much sense in the real life [~adelapena] could you review? > Duration validation error is unclear in case of overflow. > - > > Key: CASSANDRA-13218 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13218 > Project: Cassandra > Issue Type: Bug > Components: CQL >Reporter: Benjamin Lerer >Assignee: Benjamin Lerer > Fix For: 3.11.x > > > If a user try to insert a {{duration}} with a number of months or days that > cannot fit in an {{int}} (for example: {{9223372036854775807mo1d}}), the > error message is confusing. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13218) Duration validation error is unclear in case of overflow.
[ https://issues.apache.org/jira/browse/CASSANDRA-13218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16004796#comment-16004796 ] Andrés de la Peña commented on CASSANDRA-13218: --- [~blerer] sure, can you set me as reviewer? > Duration validation error is unclear in case of overflow. > - > > Key: CASSANDRA-13218 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13218 > Project: Cassandra > Issue Type: Bug > Components: CQL >Reporter: Benjamin Lerer >Assignee: Benjamin Lerer > Fix For: 3.11.x > > > If a user try to insert a {{duration}} with a number of months or days that > cannot fit in an {{int}} (for example: {{9223372036854775807mo1d}}), the > error message is confusing. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13218) Duration validation error is unclear in case of overflow.
[ https://issues.apache.org/jira/browse/CASSANDRA-13218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16004960#comment-16004960 ] Andrés de la Peña commented on CASSANDRA-13218: --- Indeed, a 32-bit integer seems more than enough for months. I either can't figure out any use case involving time intervals over 5,883,516 years and a precision higher than millennia. The patch looks good to me, +1. ||[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...blerer:13128-3.11.patch]|[utests|http://cassci.datastax.com/view/Dev/view/blerer/job/adelapena-13128-3.11-testall/]| > Duration validation error is unclear in case of overflow. > - > > Key: CASSANDRA-13218 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13218 > Project: Cassandra > Issue Type: Bug > Components: CQL >Reporter: Benjamin Lerer >Assignee: Benjamin Lerer > Fix For: 3.11.x > > > If a user try to insert a {{duration}} with a number of months or days that > cannot fit in an {{int}} (for example: {{9223372036854775807mo1d}}), the > error message is confusing. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13218) Duration validation error is unclear in case of overflow.
[ https://issues.apache.org/jira/browse/CASSANDRA-13218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16006103#comment-16006103 ] Benjamin Lerer commented on CASSANDRA-13218: Thanks for the review. > Duration validation error is unclear in case of overflow. > - > > Key: CASSANDRA-13218 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13218 > Project: Cassandra > Issue Type: Bug > Components: CQL >Reporter: Benjamin Lerer >Assignee: Benjamin Lerer > Fix For: 3.11.x > > > If a user try to insert a {{duration}} with a number of months or days that > cannot fit in an {{int}} (for example: {{9223372036854775807mo1d}}), the > error message is confusing. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13218) Duration validation error is unclear in case of overflow.
[ https://issues.apache.org/jira/browse/CASSANDRA-13218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15866074#comment-15866074 ] Sandeep Tamhankar commented on CASSANDRA-13218: --- Where is it specified that these are supposed to be int's? I was under the impression that since we're transmitting zigzag-encoded vint values and that format supports a 64-bit range, these attributes are intended to have signed 64-bit range. Is there a particular reason to restrict it? > Duration validation error is unclear in case of overflow. > - > > Key: CASSANDRA-13218 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13218 > Project: Cassandra > Issue Type: Bug > Components: CQL >Reporter: Benjamin Lerer >Assignee: Benjamin Lerer > Fix For: 3.11.x > > > If a user try to insert a {{duration}} with a number of months or days that > cannot fit in an {{int}} (for example: {{9223372036854775807mo1d}}), the > error message is confusing. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13218) Duration validation error is unclear in case of overflow.
[ https://issues.apache.org/jira/browse/CASSANDRA-13218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15867759#comment-15867759 ] Benjamin Lerer commented on CASSANDRA-13218: The encoding format support effectively a 64-bit range but {{months}} and {{days}} are stored as {{int}} to limit the memory usage and because larger numbers do not really make sense. The duration format is not intended to store things like the age of the Earth as you do not need a nanoseconds precision for those type of information. A {{days}} value of 2^32 (roughly 5,883,516 year) is far more than the age of mankind. Now, I fully agree that it should have been documented properly and that the validation message should be more expicit about the problem. > Duration validation error is unclear in case of overflow. > - > > Key: CASSANDRA-13218 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13218 > Project: Cassandra > Issue Type: Bug > Components: CQL >Reporter: Benjamin Lerer >Assignee: Benjamin Lerer > Fix For: 3.11.x > > > If a user try to insert a {{duration}} with a number of months or days that > cannot fit in an {{int}} (for example: {{9223372036854775807mo1d}}), the > error message is confusing. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13218) Duration validation error is unclear in case of overflow.
[ https://issues.apache.org/jira/browse/CASSANDRA-13218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15868130#comment-15868130 ] Sandeep Tamhankar commented on CASSANDRA-13218: --- Ok, sounds reasonable. Thanks. > Duration validation error is unclear in case of overflow. > - > > Key: CASSANDRA-13218 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13218 > Project: Cassandra > Issue Type: Bug > Components: CQL >Reporter: Benjamin Lerer >Assignee: Benjamin Lerer > Fix For: 3.11.x > > > If a user try to insert a {{duration}} with a number of months or days that > cannot fit in an {{int}} (for example: {{9223372036854775807mo1d}}), the > error message is confusing. -- This message was sent by Atlassian JIRA (v6.3.15#6346)