[jira] [Commented] (CASSANDRA-14683) Pagestate is null after 2^31 rows
[
https://issues.apache.org/jira/browse/CASSANDRA-14683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937268#comment-16937268
]
Jeremiah Jordan commented on CASSANDRA-14683:
-
+1 for interpreting MAX as “all the rows”. Someone sending that value in the
limit is most likely trying to get everything.
> Pagestate is null after 2^31 rows
> -
>
> Key: CASSANDRA-14683
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14683
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Abhishek
>Priority: Normal
>
> I am using the nodejs driver to take a dump of my table via
> [pagination|http://datastax.github.io/nodejs-driver/features/paging/#manual-paging]
> for a simple query.
> My query is \{{select * from mytable}}
> The table has close to 4 billion rows and cassandra stops returning results
> exactly after 2147483647 rows. The pagestate is not returned after this.
> Cassandra version - 3.0.9
> Nodejs cassandra driver version - 3.5.0
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14683) Pagestate is null after 2^31 rows
[
https://issues.apache.org/jira/browse/CASSANDRA-14683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16604239#comment-16604239
]
Sylvain Lebresne commented on CASSANDRA-14683:
--
Thinking about this, while we certainly should up/remove the limit supported
for this, I think the sub-part that really matter the most here is solving the
case of queries without no limit at all, which doesn't require increasing the
resolution of the limit in the {{PagingState}}, at least not in theory.
In fact, it might be worth fixing that case in 3.x as well, which I think we
can do by starting to interpret {{Integer.MAX_VALUE}} as "no limit". That is,
all we'd change here is that when a new page request has {{Integer.MAX_VALUE}}
as {{remaining}} in his {{PagingState}}, we'll execute the query exactly like
today, but we'll simply make sure that the newly returned {{PagingState}} still
has {{Integer.MAX_VALUE}} as {{remaining}}.
I'll note that this would strictly speaking break backward compatibility in the
sense that if some user was using {{Integer.MAX_VALUE}} as concrete user LIMIT
*expecting* the query to not return anymore rows, we would now return more rows
after this change. It's a wacky enough case that it's imo worth breaking so the
*much* more common case of "I want all my rows" actually works.
> Pagestate is null after 2^31 rows
> -
>
> Key: CASSANDRA-14683
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14683
> Project: Cassandra
> Issue Type: Bug
>Reporter: Abhishek
>Priority: Major
>
> I am using the nodejs driver to take a dump of my table via
> [pagination|http://datastax.github.io/nodejs-driver/features/paging/#manual-paging]
> for a simple query.
> My query is \{{select * from mytable}}
> The table has close to 4 billion rows and cassandra stops returning results
> exactly after 2147483647 rows. The pagestate is not returned after this.
> Cassandra version - 3.0.9
> Nodejs cassandra driver version - 3.5.0
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14683) Pagestate is null after 2^31 rows
[
https://issues.apache.org/jira/browse/CASSANDRA-14683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16599030#comment-16599030
]
Abhishek commented on CASSANDRA-14683:
--
[~iamaleksey]
I'm willing to fix this but I'm new to the codebase. Can you point me to the
class which needs changes?
> Pagestate is null after 2^31 rows
> -
>
> Key: CASSANDRA-14683
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14683
> Project: Cassandra
> Issue Type: Bug
>Reporter: Abhishek
>Priority: Major
>
> I am using the nodejs driver to take a dump of my table via
> [pagination|http://datastax.github.io/nodejs-driver/features/paging/#manual-paging]
> for a simple query.
> My query is \{{select * from mytable}}
> The table has close to 4 billion rows and cassandra stops returning results
> exactly after 2147483647 rows. The pagestate is not returned after this.
> Cassandra version - 3.0.9
> Nodejs cassandra driver version - 3.5.0
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14683) Pagestate is null after 2^31 rows
[
https://issues.apache.org/jira/browse/CASSANDRA-14683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16598957#comment-16598957
]
Aleksey Yeschenko commented on CASSANDRA-14683:
---
Essentially that, except we could use varints instead. And the freeze doesn't
apply to bug fixes, testing and stabilisation related work - so this would be
fine, if anyone's willing to do the work.
> Pagestate is null after 2^31 rows
> -
>
> Key: CASSANDRA-14683
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14683
> Project: Cassandra
> Issue Type: Bug
>Reporter: Abhishek
>Priority: Major
>
> I am using the nodejs driver to take a dump of my table via
> [pagination|http://datastax.github.io/nodejs-driver/features/paging/#manual-paging]
> for a simple query.
> My query is \{{select * from mytable}}
> The table has close to 4 billion rows and cassandra stops returning results
> exactly after 2147483647 rows. The pagestate is not returned after this.
> Cassandra version - 3.0.9
> Nodejs cassandra driver version - 3.5.0
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14683) Pagestate is null after 2^31 rows
[
https://issues.apache.org/jira/browse/CASSANDRA-14683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16598813#comment-16598813
]
Sylvain Lebresne commented on CASSANDRA-14683:
--
No mystery here, the paging state uses a 4-byte integer (currently interpreted
signed) to encode the limit of rows yet to return. A query (paged or not) can
thus currently not return more than ~2 billion rows. That's definitively an
issue that should be fixed, and it's not really hard to do so in principle (the
paging state should use a 8-byte long), but it will require a native protocol
change. Translation, this cannot be fixed in current versions, and given the
freeze date for 4.0, this won't be fixed there either.
So short term, you will have to complicate your dumping code a bit I'm afraid
to use more than one query to do this.
> Pagestate is null after 2^31 rows
> -
>
> Key: CASSANDRA-14683
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14683
> Project: Cassandra
> Issue Type: Bug
>Reporter: Abhishek
>Priority: Major
>
> I am using the nodejs driver to take a dump of my table via
> [pagination|http://datastax.github.io/nodejs-driver/features/paging/#manual-paging]
> for a simple query.
> My query is \{{select * from mytable}}
> The table has close to 4 billion rows and cassandra stops returning results
> exactly after 2147483647 rows. The pagestate is not returned after this.
> Cassandra version - 3.0.9
> Nodejs cassandra driver version - 3.5.0
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
