[jira] [Commented] (CASSANDRA-13218) Duration validation error is unclear in case of overflow.
[
https://issues.apache.org/jira/browse/CASSANDRA-13218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15983870#comment-15983870
]
Sandeep Tamhankar commented on CASSANDRA-13218:
---
Found a few other bad side effects of the current behavior:
1. The validated (truncated) values are not actually re-encoded to vint's and
stored; the user-provided bytes are stored. This is bad because it introduces
an inconsistency of behavior -- a client can insert/select 64-bit values for
months and days, but when C* exports the data as json (for example), the
Duration.deserialize method will produce an object with truncated values.
2. Validation doesn't check for truncation error and reject the insert.
3. I think validation will succeed for this illegal set of values provided by a
driver: (-9223372036854775807, 1, 1) because the months attribute will be
coerced to 1 (since the highest 4 bytes will be dropped).
*Proposal 1*: accept and handle 64-bit values for months and days consistently.
I don't know how often these values are used in C*, but an extra 8 bytes
doesn't sound like it's worth the hassle of treating months and days
differently than nanos. It makes C* code a little more complex; it makes driver
code a little more complex (for drivers that are type-aware at least).
*Proposal 2*: beef up the validation logic to reject values that would
overflow/underflow during long->int truncation. Write a tool that users can use
to identify illegal rows they inserted because of the holes in the validation
logic and give users an opportunity to clean up such bad data.
Admittedly, the chances that users have inserted illegal duration's is
relatively small, particularly since this new type hasn't been around very
long. But it seems much simpler to make those values legal, so I'd lean towards
my first proposal.
> Duration validation error is unclear in case of overflow.
> -
>
> Key: CASSANDRA-13218
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13218
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.11.x
>
>
> If a user try to insert a {{duration}} with a number of months or days that
> cannot fit in an {{int}} (for example: {{9223372036854775807mo1d}}), the
> error message is confusing.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13218) Duration validation error is unclear in case of overflow.
[
https://issues.apache.org/jira/browse/CASSANDRA-13218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16004679#comment-16004679
]
Benjamin Lerer commented on CASSANDRA-13218:
Unfortunately, there are other factors that force us to use 32-bit values for
months and days. Java utilities like {{Calendar}} accept only {{int}} for
months or days value and we already use {{Calendar}} for data arithmetics. Due
to that the solution is simply to reject to values at write time.
I pushed a patch for that
[here|https://github.com/apache/cassandra/compare/trunk...blerer:13128-3.11].
I think that the chances that users have inserted illegal durations is rather
null.
The 2 reasons for that are that:
# duration is not really usefull in itself and was mainly added for simplifying
the life of user querying timeseries data
# the invalid values are greater than the age of mankind and by consequence
have not much sense in the real life
[~adelapena] could you review?
> Duration validation error is unclear in case of overflow.
> -
>
> Key: CASSANDRA-13218
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13218
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.11.x
>
>
> If a user try to insert a {{duration}} with a number of months or days that
> cannot fit in an {{int}} (for example: {{9223372036854775807mo1d}}), the
> error message is confusing.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13218) Duration validation error is unclear in case of overflow.
[
https://issues.apache.org/jira/browse/CASSANDRA-13218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16004796#comment-16004796
]
Andrés de la Peña commented on CASSANDRA-13218:
---
[~blerer] sure, can you set me as reviewer?
> Duration validation error is unclear in case of overflow.
> -
>
> Key: CASSANDRA-13218
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13218
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.11.x
>
>
> If a user try to insert a {{duration}} with a number of months or days that
> cannot fit in an {{int}} (for example: {{9223372036854775807mo1d}}), the
> error message is confusing.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13218) Duration validation error is unclear in case of overflow.
[
https://issues.apache.org/jira/browse/CASSANDRA-13218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16004960#comment-16004960
]
Andrés de la Peña commented on CASSANDRA-13218:
---
Indeed, a 32-bit integer seems more than enough for months. I either can't
figure out any use case involving time intervals over 5,883,516 years and a
precision higher than millennia.
The patch looks good to me, +1.
||[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...blerer:13128-3.11.patch]|[utests|http://cassci.datastax.com/view/Dev/view/blerer/job/adelapena-13128-3.11-testall/]|
> Duration validation error is unclear in case of overflow.
> -
>
> Key: CASSANDRA-13218
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13218
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.11.x
>
>
> If a user try to insert a {{duration}} with a number of months or days that
> cannot fit in an {{int}} (for example: {{9223372036854775807mo1d}}), the
> error message is confusing.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13218) Duration validation error is unclear in case of overflow.
[
https://issues.apache.org/jira/browse/CASSANDRA-13218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16006103#comment-16006103
]
Benjamin Lerer commented on CASSANDRA-13218:
Thanks for the review.
> Duration validation error is unclear in case of overflow.
> -
>
> Key: CASSANDRA-13218
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13218
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.11.x
>
>
> If a user try to insert a {{duration}} with a number of months or days that
> cannot fit in an {{int}} (for example: {{9223372036854775807mo1d}}), the
> error message is confusing.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13218) Duration validation error is unclear in case of overflow.
[
https://issues.apache.org/jira/browse/CASSANDRA-13218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15866074#comment-15866074
]
Sandeep Tamhankar commented on CASSANDRA-13218:
---
Where is it specified that these are supposed to be int's? I was under the
impression that since we're transmitting zigzag-encoded vint values and that
format supports a 64-bit range, these attributes are intended to have signed
64-bit range. Is there a particular reason to restrict it?
> Duration validation error is unclear in case of overflow.
> -
>
> Key: CASSANDRA-13218
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13218
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.11.x
>
>
> If a user try to insert a {{duration}} with a number of months or days that
> cannot fit in an {{int}} (for example: {{9223372036854775807mo1d}}), the
> error message is confusing.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13218) Duration validation error is unclear in case of overflow.
[
https://issues.apache.org/jira/browse/CASSANDRA-13218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15867759#comment-15867759
]
Benjamin Lerer commented on CASSANDRA-13218:
The encoding format support effectively a 64-bit range but {{months}} and
{{days}} are stored as {{int}} to limit the memory usage and because larger
numbers do not really make sense.
The duration format is not intended to store things like the age of the Earth
as you do not need a nanoseconds precision for those type of information.
A {{days}} value of 2^32 (roughly 5,883,516 year) is far more than the age of
mankind.
Now, I fully agree that it should have been documented properly and that the
validation message should be more expicit about the problem.
> Duration validation error is unclear in case of overflow.
> -
>
> Key: CASSANDRA-13218
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13218
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.11.x
>
>
> If a user try to insert a {{duration}} with a number of months or days that
> cannot fit in an {{int}} (for example: {{9223372036854775807mo1d}}), the
> error message is confusing.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13218) Duration validation error is unclear in case of overflow.
[
https://issues.apache.org/jira/browse/CASSANDRA-13218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15868130#comment-15868130
]
Sandeep Tamhankar commented on CASSANDRA-13218:
---
Ok, sounds reasonable. Thanks.
> Duration validation error is unclear in case of overflow.
> -
>
> Key: CASSANDRA-13218
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13218
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.11.x
>
>
> If a user try to insert a {{duration}} with a number of months or days that
> cannot fit in an {{int}} (for example: {{9223372036854775807mo1d}}), the
> error message is confusing.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
