[jira] [Comment Edited] (SPARK-13705) UpdateStateByKey Operation documentation incorrectly refers to StatefulNetworkWordCount
[
https://issues.apache.org/jira/browse/SPARK-13705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15182763#comment-15182763
]
Rishi edited comment on SPARK-13705 at 3/7/16 8:50 AM:
---
Thanx Sean, Missed that part
was (Author: rishitesh):
Thhnx Sean, Missed that part
> UpdateStateByKey Operation documentation incorrectly refers to
> StatefulNetworkWordCount
> ---
>
> Key: SPARK-13705
> URL: https://issues.apache.org/jira/browse/SPARK-13705
> Project: Spark
> Issue Type: Bug
> Components: Documentation
>Affects Versions: 1.6.0
>Reporter: Rishi
>Priority: Trivial
> Labels: documentation
>
> Doc Issue : Apache Spark Streaming guide elaborates UpdateStateByKey
> Operation and goes onto give an example. However for complete code sample it
> referes to
> {{site.SPARK_GITHUB_URL}}/blob/master/examples/src/main/scala/org/apache
> /spark/examples/streaming/StatefulNetworkWordCount.scala).
> StatefulNetworkWordCount.scala has changed to demonstrate more recent API
> mapWithState.
> This creates confusion in the document.
> Till the time more detailed explanation of mapWIthState is added the
> reference to StatefulNetworkWordCount.scala should be removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13705) UpdateStateByKey Operation documentation incorrectly refers to StatefulNetworkWordCount
[
https://issues.apache.org/jira/browse/SPARK-13705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15182763#comment-15182763
]
Rishi commented on SPARK-13705:
---
Thhnx Sean, Missed that part
> UpdateStateByKey Operation documentation incorrectly refers to
> StatefulNetworkWordCount
> ---
>
> Key: SPARK-13705
> URL: https://issues.apache.org/jira/browse/SPARK-13705
> Project: Spark
> Issue Type: Bug
> Components: Documentation
>Affects Versions: 1.6.0
>Reporter: Rishi
>Priority: Trivial
> Labels: documentation
>
> Doc Issue : Apache Spark Streaming guide elaborates UpdateStateByKey
> Operation and goes onto give an example. However for complete code sample it
> referes to
> {{site.SPARK_GITHUB_URL}}/blob/master/examples/src/main/scala/org/apache
> /spark/examples/streaming/StatefulNetworkWordCount.scala).
> StatefulNetworkWordCount.scala has changed to demonstrate more recent API
> mapWithState.
> This creates confusion in the document.
> Till the time more detailed explanation of mapWIthState is added the
> reference to StatefulNetworkWordCount.scala should be removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13705) UpdateStateByKey Operation documentation incorrectly refers to StatefulNetworkWordCount
[
https://issues.apache.org/jira/browse/SPARK-13705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15182120#comment-15182120
]
Rishi commented on SPARK-13705:
---
Pull request created.
> UpdateStateByKey Operation documentation incorrectly refers to
> StatefulNetworkWordCount
> ---
>
> Key: SPARK-13705
> URL: https://issues.apache.org/jira/browse/SPARK-13705
> Project: Spark
> Issue Type: Bug
> Components: Documentation
>Affects Versions: 1.6.0
>Reporter: Rishi
>Priority: Minor
> Labels: documentation
> Fix For: 1.6.0
>
>
> Doc Issue : Apache Spark Streaming guide elaborates UpdateStateByKey
> Operation and goes onto give an example. However for complete code sample it
> referes to
> {{site.SPARK_GITHUB_URL}}/blob/master/examples/src/main/scala/org/apache
> /spark/examples/streaming/StatefulNetworkWordCount.scala).
> StatefulNetworkWordCount.scala has changed to demonstrate more recent API
> mapWithState.
> This creates confusion in the document.
> Till the time more detailed explanation of mapWIthState is added the
> reference to StatefulNetworkWordCount.scala should be removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-13705) UpdateStateByKey Operation documentation incorrectly refers to StatefulNetworkWordCount
[
https://issues.apache.org/jira/browse/SPARK-13705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rishi updated SPARK-13705:
--
Description:
Doc Issue : Apache Spark Streaming guide elaborates UpdateStateByKey Operation
and goes onto give an example. However for complete code sample it referes to
{{site.SPARK_GITHUB_URL}}/blob/master/examples/src/main/scala/org/apache
/spark/examples/streaming/StatefulNetworkWordCount.scala).
StatefulNetworkWordCount.scala has changed to demonstrate more recent API
mapWithState.
This creates confusion in the document.
Till the time more detailed explanation of mapWIthState is added the reference
to StatefulNetworkWordCount.scala should be removed.
> UpdateStateByKey Operation documentation incorrectly refers to
> StatefulNetworkWordCount
> ---
>
> Key: SPARK-13705
> URL: https://issues.apache.org/jira/browse/SPARK-13705
> Project: Spark
> Issue Type: Bug
> Components: Documentation
>Affects Versions: 1.6.0
>Reporter: Rishi
>Priority: Minor
> Labels: documentation
> Fix For: 1.6.0
>
>
> Doc Issue : Apache Spark Streaming guide elaborates UpdateStateByKey
> Operation and goes onto give an example. However for complete code sample it
> referes to
> {{site.SPARK_GITHUB_URL}}/blob/master/examples/src/main/scala/org/apache
> /spark/examples/streaming/StatefulNetworkWordCount.scala).
> StatefulNetworkWordCount.scala has changed to demonstrate more recent API
> mapWithState.
> This creates confusion in the document.
> Till the time more detailed explanation of mapWIthState is added the
> reference to StatefulNetworkWordCount.scala should be removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-13705) UpdateStateByKey Operation documentation incorrectly refers to StatefulNetworkWordCount
Rishi created SPARK-13705: - Summary: UpdateStateByKey Operation documentation incorrectly refers to StatefulNetworkWordCount Key: SPARK-13705 URL: https://issues.apache.org/jira/browse/SPARK-13705 Project: Spark Issue Type: Bug Components: Documentation Affects Versions: 1.6.0 Reporter: Rishi Priority: Minor Fix For: 1.6.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-9379) org.apache.spark.sql.catalyst.SqlParser should be extensible easily
[ https://issues.apache.org/jira/browse/SPARK-9379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643951#comment-14643951 ] Rishi commented on SPARK-9379: -- Thanks Michael for the reply. Precisely for plug-able SQL dialects one might only intend to add to existing SqlParser's functionality by extending this class. Its a separate matter to have a completely new dialect like HiveQLDialect. > org.apache.spark.sql.catalyst.SqlParser should be extensible easily > > > Key: SPARK-9379 > URL: https://issues.apache.org/jira/browse/SPARK-9379 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 1.4.1 >Reporter: Rishi > Original Estimate: 48h > Remaining Estimate: 48h > > When extending org.apache.spark.sql.catalyst.SqlParser we are usually stuck > with Scala limitation of not able to override lazy vals and refer super > values from it . (See http://www.scala-lang.org/old/node/11315.html) . This > can be avoided with a simple alteration to SqlPasrer.scala. See patch below. > - protected lazy val start: Parser[LogicalPlan] = > -start1 | insert | cte > + protected lazy val start: Parser[LogicalPlan] = allParsers > + > + protected def allParsers = start1 | insert | cte > This will allow subclasses of SqlParser to override only "allParsers" . This > will also ease of upgrading through spark revisions -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-9379) org.apache.spark.sql.catalyst.SqlParser should be extensible easily
Rishi created SPARK-9379: Summary: org.apache.spark.sql.catalyst.SqlParser should be extensible easily Key: SPARK-9379 URL: https://issues.apache.org/jira/browse/SPARK-9379 Project: Spark Issue Type: Improvement Components: SQL Affects Versions: 1.4.1 Reporter: Rishi When extending org.apache.spark.sql.catalyst.SqlParser we are usually stuck with Scala limitation of not able to override lazy vals and refer super values from it . (See http://www.scala-lang.org/old/node/11315.html) . This can be avoided with a simple alteration to SqlPasrer.scala. See patch below. - protected lazy val start: Parser[LogicalPlan] = -start1 | insert | cte + protected lazy val start: Parser[LogicalPlan] = allParsers + + protected def allParsers = start1 | insert | cte This will allow subclasses of SqlParser to override only "allParsers" . This will also ease of upgrading through spark revisions -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
