Hi Ajith,
1. Now if the config includes spark extensions can we configure the carbon store path and metastoreDB path while creating spark session instance 2. Impact on the existing carbonsession user for spark 2.3.x user mentions that there will be "no profiler and no refresh across driver". Could these points be elaborated in detail ? Regards Chetan On 2019/08/22 04:58:53, Ajith shetty <[email protected]> wrote: > Hi Community > > From https://issues.apache.org/jira/browse/SPARK-18127 Spark provides > SparkSessionExtensions in order to extended capabilities of spark. Carbon can > use this in order to avoid the tight coupling due to CarbonSession in spark > environment. > https://spark.apache.org/docs/2.4.3/api/java/org/apache/spark/sql/SparkSessionExtensions.html > > Main Scope: > 1. Compatible with Spark 2.3.2+ > 2. Make Carbon Parser Pluggable > a. Move to antlr4 based parser > 3. Make Analyzer Rules Pluggable > 4. Make Optimizer Rules Pluggable > 5. Make Planning Strategies Pluggable > > We can have Sub jiras in order to cover all the scenarios due to this. Please > input your thoughts. > > Regards >
