[ https://issues.apache.org/jira/browse/SPARK-3880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14167006#comment-14167006 ]
Yan commented on SPARK-3880: ---------------------------- The new context is intended to be very light-weighted. We noticed that SparkSQL is a very active project and there have been talks/Jiras about SQLContext and data sources. As mentioned in the design, we are aware of the PR, and the need to have a universal mechanism to access different types of data stores, and will a keep close watch on the latest movements and will definitely fit our efforts to those latest features and interfaces when they are ready and reasonably stable. In the meanwhile, the design is intended to be heavy of HBase-specific data model, data access mechanisms and query optimizations, and keep the integration part light-weighted so it can be easily adjusted to future changes. The point is that we need to find some compromise between a rapidly changing project and the need to have a more or less stable "context" to base a new feature on. Chasing a constantly moving target is never easy, I guess. > HBase as data source to SparkSQL > -------------------------------- > > Key: SPARK-3880 > URL: https://issues.apache.org/jira/browse/SPARK-3880 > Project: Spark > Issue Type: New Feature > Components: SQL > Reporter: Yan > Assignee: Yan > Attachments: HBaseOnSpark.docx > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org