[jira] [Commented] (SPARK-18799) Spark SQL expose interface for plug-gable parser extension
[ https://issues.apache.org/jira/browse/SPARK-18799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15739210#comment-15739210 ] Hyukjin Kwon commented on SPARK-18799: -- Could we resolve this as a duplicate if it does? > Spark SQL expose interface for plug-gable parser extension > --- > > Key: SPARK-18799 > URL: https://issues.apache.org/jira/browse/SPARK-18799 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 2.0.0 >Reporter: Jihong MA > > There used to be an interface to plug a parser extension through > ParserDialect in HiveContext in all Spark 1.x version. Starting Spark 2.x > release, Apache Spark moved to the new parser (Antlr4), there is no longer a > way to extend the default SQL parser through SparkSession interface, however > this is really a pain and hard to work around it when integrating other data > source with Spark with extended support such as Insert, Update, Delete > statement or any other data management statement. > It would be very nice to continue to expose an interface for parser extension > to make data source integration easier and smoother. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18799) Spark SQL expose interface for plug-gable parser extension
[ https://issues.apache.org/jira/browse/SPARK-18799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15736179#comment-15736179 ] Herman van Hovell commented on SPARK-18799: --- It is way too late to put this in Spark 2.1. So yeah 2.2. > Spark SQL expose interface for plug-gable parser extension > --- > > Key: SPARK-18799 > URL: https://issues.apache.org/jira/browse/SPARK-18799 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 2.0.0 >Reporter: Jihong MA > > There used to be an interface to plug a parser extension through > ParserDialect in HiveContext in all Spark 1.x version. Starting Spark 2.x > release, Apache Spark moved to the new parser (Antlr4), there is no longer a > way to extend the default SQL parser through SparkSession interface, however > this is really a pain and hard to work around it when integrating other data > source with Spark with extended support such as Insert, Update, Delete > statement or any other data management statement. > It would be very nice to continue to expose an interface for parser extension > to make data source integration easier and smoother. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18799) Spark SQL expose interface for plug-gable parser extension
[ https://issues.apache.org/jira/browse/SPARK-18799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15736118#comment-15736118 ] Jihong MA commented on SPARK-18799: --- are we looking at first quarter of 2017 for Spark 2.2? is now too late to squeeze this in Spark 2.1 release? thanks! > Spark SQL expose interface for plug-gable parser extension > --- > > Key: SPARK-18799 > URL: https://issues.apache.org/jira/browse/SPARK-18799 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 2.0.0 >Reporter: Jihong MA > > There used to be an interface to plug a parser extension through > ParserDialect in HiveContext in all Spark 1.x version. Starting Spark 2.x > release, Apache Spark moved to the new parser (Antlr4), there is no longer a > way to extend the default SQL parser through SparkSession interface, however > this is really a pain and hard to work around it when integrating other data > source with Spark with extended support such as Insert, Update, Delete > statement or any other data management statement. > It would be very nice to continue to expose an interface for parser extension > to make data source integration easier and smoother. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18799) Spark SQL expose interface for plug-gable parser extension
[ https://issues.apache.org/jira/browse/SPARK-18799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15736100#comment-15736100 ] Jihong MA commented on SPARK-18799: --- DML statement support for instance > Spark SQL expose interface for plug-gable parser extension > --- > > Key: SPARK-18799 > URL: https://issues.apache.org/jira/browse/SPARK-18799 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 2.0.0 >Reporter: Jihong MA > > There used to be an interface to plug a parser extension through > ParserDialect in HiveContext in all Spark 1.x version. Starting Spark 2.x > release, Apache Spark moved to the new parser (Antlr4), there is no longer a > way to extend the default SQL parser through SparkSession interface, however > this is really a pain and hard to work around it when integrating other data > source with Spark with extended support such as Insert, Update, Delete > statement or any other data management statement. > It would be very nice to continue to expose an interface for parser extension > to make data source integration easier and smoother. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18799) Spark SQL expose interface for plug-gable parser extension
[ https://issues.apache.org/jira/browse/SPARK-18799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15736074#comment-15736074 ] Herman van Hovell commented on SPARK-18799: --- Spark has its own parser. What kind of syntax do you want to support? > Spark SQL expose interface for plug-gable parser extension > --- > > Key: SPARK-18799 > URL: https://issues.apache.org/jira/browse/SPARK-18799 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 2.0.0 >Reporter: Jihong MA > > There used to be an interface to plug a parser extension through > ParserDialect in HiveContext in all Spark 1.x version. Starting Spark 2.x > release, Apache Spark moved to the new parser (Antlr4), there is no longer a > way to extend the default SQL parser through SparkSession interface, however > this is really a pain and hard to work around it when integrating other data > source with Spark with extended support such as Insert, Update, Delete > statement or any other data management statement. > It would be very nice to continue to expose an interface for parser extension > to make data source integration easier and smoother. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18799) Spark SQL expose interface for plug-gable parser extension
[ https://issues.apache.org/jira/browse/SPARK-18799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15736070#comment-15736070 ] Herman van Hovell commented on SPARK-18799: --- We are going to reintroduce parser plugability in Spark 2.2. With a bunch of other extensions. > Spark SQL expose interface for plug-gable parser extension > --- > > Key: SPARK-18799 > URL: https://issues.apache.org/jira/browse/SPARK-18799 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 2.0.0 >Reporter: Jihong MA > > There used to be an interface to plug a parser extension through > ParserDialect in HiveContext in all Spark 1.x version. Starting Spark 2.x > release, Apache Spark moved to the new parser (Antlr4), there is no longer a > way to extend the default SQL parser through SparkSession interface, however > this is really a pain and hard to work around it when integrating other data > source with Spark with extended support such as Insert, Update, Delete > statement or any other data management statement. > It would be very nice to continue to expose an interface for parser extension > to make data source integration easier and smoother. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18799) Spark SQL expose interface for plug-gable parser extension
[ https://issues.apache.org/jira/browse/SPARK-18799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15736003#comment-15736003 ] Jihong MA commented on SPARK-18799: --- [~hyukjin.kwon] the intention to remove it at that time is different as Spark doesn't have its own parser yet and would prefer community to contribute parser changes directly. for Spark 2.x, it is essential to provide an interface for extension for data source integration, especially for those syntax/statements Spark has no intention to support even in the future. > Spark SQL expose interface for plug-gable parser extension > --- > > Key: SPARK-18799 > URL: https://issues.apache.org/jira/browse/SPARK-18799 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 2.0.0 >Reporter: Jihong MA > > There used to be an interface to plug a parser extension through > ParserDialect in HiveContext in all Spark 1.x version. Starting Spark 2.x > release, Apache Spark moved to the new parser (Antlr4), there is no longer a > way to extend the default SQL parser through SparkSession interface, however > this is really a pain and hard to work around it when integrating other data > source with Spark with extended support such as Insert, Update, Delete > statement or any other data management statement. > It would be very nice to continue to expose an interface for parser extension > to make data source integration easier and smoother. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18799) Spark SQL expose interface for plug-gable parser extension
[ https://issues.apache.org/jira/browse/SPARK-18799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15734130#comment-15734130 ] Hyukjin Kwon commented on SPARK-18799: -- Ah, there it is - https://github.com/apache/spark/pull/10801 > Spark SQL expose interface for plug-gable parser extension > --- > > Key: SPARK-18799 > URL: https://issues.apache.org/jira/browse/SPARK-18799 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 2.0.0 >Reporter: Jihong MA > > There used to be an interface to plug a parser extension through > ParserDialect in HiveContext in all Spark 1.x version. Starting Spark 2.x > release, Apache Spark moved to the new parser (Antlr4), there is no longer a > way to extend the default SQL parser through SparkSession interface, however > this is really a pain and hard to work around it when integrating other data > source with Spark with extended support such as Insert, Update, Delete > statement or any other data management statement. > It would be very nice to continue to expose an interface for parser extension > to make data source integration easier and smoother. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18799) Spark SQL expose interface for plug-gable parser extension
[ https://issues.apache.org/jira/browse/SPARK-18799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15734128#comment-15734128 ] Hyukjin Kwon commented on SPARK-18799: -- Less than roughly about a year ago, I saw a PR that says Spark does not support plug-gable parser and it was removed to prevent illusion of supporting this while tracking down the code path about SQL parser *If I remember this correctly*. cc [~rxin] I hope it was not my wrong memory manipulated by myself. > Spark SQL expose interface for plug-gable parser extension > --- > > Key: SPARK-18799 > URL: https://issues.apache.org/jira/browse/SPARK-18799 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 2.0.0 >Reporter: Jihong MA > > There used to be an interface to plug a parser extension through > ParserDialect in HiveContext in all Spark 1.x version. Starting Spark 2.x > release, Apache Spark moved to the new parser (Antlr4), there is no longer a > way to extend the default SQL parser through SparkSession interface, however > this is really a pain and hard to work around it when integrating other data > source with Spark with extended support such as Insert, Update, Delete > statement or any other data management statement. > It would be very nice to continue to expose an interface for parser extension > to make data source integration easier and smoother. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
