[ https://issues.apache.org/jira/browse/SPARK-30852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039730#comment-17039730 ]
Damianos Christophides commented on SPARK-30852: ------------------------------------------------ thanks [~rakson] and [~cloud_fan] for looking into it. I understand the limitations. Not sure how I can mark it as "won't fix", I can't see the option > Use Long instead of Int as argument type in Dataset limit method > ---------------------------------------------------------------- > > Key: SPARK-30852 > URL: https://issues.apache.org/jira/browse/SPARK-30852 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 2.4.4 > Reporter: Damianos Christophides > Priority: Minor > > The Dataset limit method takes an input of type Int, which is a 32bit > integer. The numerical upper limit of this type is 2,147,483,647. I found in > my work to need to apply a limit to a Dataset higher than that which gives an > error: > "py4j.Py4JException: Method limit([class java.lang.Long]) does not exist" > > Could the input type of the limit method be changed to a Long (64bit)? > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org