[jira] [Commented] (SPARK-24662) Structured Streaming should support LIMIT
[ https://issues.apache.org/jira/browse/SPARK-24662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526859#comment-16526859 ] Apache Spark commented on SPARK-24662: -- User 'mukulmurthy' has created a pull request for this issue: https://github.com/apache/spark/pull/21662 > Structured Streaming should support LIMIT > - > > Key: SPARK-24662 > URL: https://issues.apache.org/jira/browse/SPARK-24662 > Project: Spark > Issue Type: New Feature > Components: Structured Streaming >Affects Versions: 2.3.1 >Reporter: Mukul Murthy >Priority: Major > > Make structured streams support the LIMIT operator. > This will undo SPARK-24525 as the limit operator would be a superior solution. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24662) Structured Streaming should support LIMIT
[ https://issues.apache.org/jira/browse/SPARK-24662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526848#comment-16526848 ] Mukul Murthy commented on SPARK-24662: -- Calling .limit(n) on a DataFrame (or in SQL, SELECT ... LIMIT n) returns only n rows from that query. limit is currently not supported on streaming dataframes; my fix is going to support it for streams writing in append and complete output modes. It's still going to be unsupported for streams in update output mode, because for updating streams, limit doesn't make sense. > Structured Streaming should support LIMIT > - > > Key: SPARK-24662 > URL: https://issues.apache.org/jira/browse/SPARK-24662 > Project: Spark > Issue Type: New Feature > Components: Structured Streaming >Affects Versions: 2.3.1 >Reporter: Mukul Murthy >Priority: Major > > Make structured streams support the LIMIT operator. > This will undo SPARK-24525 as the limit operator would be a superior solution. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24662) Structured Streaming should support LIMIT
[ https://issues.apache.org/jira/browse/SPARK-24662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526065#comment-16526065 ] Richard Yu commented on SPARK-24662: Just to be clear on the function of the limit operator, could you explain what it does? > Structured Streaming should support LIMIT > - > > Key: SPARK-24662 > URL: https://issues.apache.org/jira/browse/SPARK-24662 > Project: Spark > Issue Type: New Feature > Components: Structured Streaming >Affects Versions: 2.3.1 >Reporter: Mukul Murthy >Priority: Major > > Make structured streams support the LIMIT operator. > This will undo SPARK-24525 as the limit operator would be a superior solution. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
