[jira] [Updated] (HBASE-14796) Enhance the Gets in the connector
[ https://issues.apache.org/jira/browse/HBASE-14796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-14796: --- Attachment: HBASE-14796-1.patch solve review comments > Enhance the Gets in the connector > - > > Key: HBASE-14796 > URL: https://issues.apache.org/jira/browse/HBASE-14796 > Project: HBase > Issue Type: Improvement >Reporter: Ted Malaska >Assignee: Zhan Zhang >Priority: Minor > Attachments: HBASE-14796-1.patch, HBASE-14976.patch > > > Current the Spark-Module Spark SQL implementation gets records from HBase > from the driver if there is something like the following found in the SQL. > rowkey = 123 > The reason for this original was normal sql will not have many equal > operations in a single where clause. > Zhan, had brought up too points that have value. > 1. The SQL may be generated and may have many many equal statements in it so > moving the work to an executor protects the driver from load > 2. In the correct implementation the drive is connecting to HBase and > exceptions may cause trouble with the Spark application and not just with the > a single task execution -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14796) Enhance the Gets in the connector
[ https://issues.apache.org/jira/browse/HBASE-14796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-14796: --- Hadoop Flags: Reviewed Fix Version/s: 2.0.0 > Enhance the Gets in the connector > - > > Key: HBASE-14796 > URL: https://issues.apache.org/jira/browse/HBASE-14796 > Project: HBase > Issue Type: Improvement >Reporter: Ted Malaska >Assignee: Zhan Zhang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-14796-1.patch, HBASE-14976.patch > > > Current the Spark-Module Spark SQL implementation gets records from HBase > from the driver if there is something like the following found in the SQL. > rowkey = 123 > The reason for this original was normal sql will not have many equal > operations in a single where clause. > Zhan, had brought up too points that have value. > 1. The SQL may be generated and may have many many equal statements in it so > moving the work to an executor protects the driver from load > 2. In the correct implementation the drive is connecting to HBase and > exceptions may cause trouble with the Spark application and not just with the > a single task execution -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14796) Enhance the Gets in the connector
[ https://issues.apache.org/jira/browse/HBASE-14796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-14796: --- Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the patch, Zhan. Thanks for the review, Ted. > Enhance the Gets in the connector > - > > Key: HBASE-14796 > URL: https://issues.apache.org/jira/browse/HBASE-14796 > Project: HBase > Issue Type: Improvement >Reporter: Ted Malaska >Assignee: Zhan Zhang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-14796-1.patch, HBASE-14976.patch > > > Current the Spark-Module Spark SQL implementation gets records from HBase > from the driver if there is something like the following found in the SQL. > rowkey = 123 > The reason for this original was normal sql will not have many equal > operations in a single where clause. > Zhan, had brought up too points that have value. > 1. The SQL may be generated and may have many many equal statements in it so > moving the work to an executor protects the driver from load > 2. In the correct implementation the drive is connecting to HBase and > exceptions may cause trouble with the Spark application and not just with the > a single task execution -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14796) Enhance the Gets in the connector
[ https://issues.apache.org/jira/browse/HBASE-14796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-14796: --- Attachment: HBASE-14976.patch We have use case where bulkget may consists of thousands of gets. Move BulkGet to executor side from driver, which will improve the failure recovery, and potentially improve the performance as well when the gets number is big. > Enhance the Gets in the connector > - > > Key: HBASE-14796 > URL: https://issues.apache.org/jira/browse/HBASE-14796 > Project: HBase > Issue Type: Improvement >Reporter: Ted Malaska >Assignee: Zhan Zhang >Priority: Minor > Attachments: HBASE-14976.patch > > > Current the Spark-Module Spark SQL implementation gets records from HBase > from the driver if there is something like the following found in the SQL. > rowkey = 123 > The reason for this original was normal sql will not have many equal > operations in a single where clause. > Zhan, had brought up too points that have value. > 1. The SQL may be generated and may have many many equal statements in it so > moving the work to an executor protects the driver from load > 2. In the correct implementation the drive is connecting to HBase and > exceptions may cause trouble with the Spark application and not just with the > a single task execution -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14796) Enhance the Gets in the connector
[ https://issues.apache.org/jira/browse/HBASE-14796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-14796: --- Release Note: spark.hbase.bulkGetSize in HBaseSparkConf is for grouping bulkGet, and default value is 1000. Status: Patch Available (was: Open) > Enhance the Gets in the connector > - > > Key: HBASE-14796 > URL: https://issues.apache.org/jira/browse/HBASE-14796 > Project: HBase > Issue Type: Improvement >Reporter: Ted Malaska >Assignee: Zhan Zhang >Priority: Minor > Attachments: HBASE-14976.patch > > > Current the Spark-Module Spark SQL implementation gets records from HBase > from the driver if there is something like the following found in the SQL. > rowkey = 123 > The reason for this original was normal sql will not have many equal > operations in a single where clause. > Zhan, had brought up too points that have value. > 1. The SQL may be generated and may have many many equal statements in it so > moving the work to an executor protects the driver from load > 2. In the correct implementation the drive is connecting to HBase and > exceptions may cause trouble with the Spark application and not just with the > a single task execution -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14796) Enhance the Gets in the connector
[ https://issues.apache.org/jira/browse/HBASE-14796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-14796: --- Summary: Enhance the Gets in the connector (was: Provide an alternative spark-hbase SQL implementations for Gets) > Enhance the Gets in the connector > - > > Key: HBASE-14796 > URL: https://issues.apache.org/jira/browse/HBASE-14796 > Project: HBase > Issue Type: Improvement >Reporter: Ted Malaska >Assignee: Zhan Zhang >Priority: Minor > > Current the Spark-Module Spark SQL implementation gets records from HBase > from the driver if there is something like the following found in the SQL. > rowkey = 123 > The reason for this original was normal sql will not have many equal > operations in a single where clause. > Zhan, had brought up too points that have value. > 1. The SQL may be generated and may have many many equal statements in it so > moving the work to an executor protects the driver from load > 2. In the correct implementation the drive is connecting to HBase and > exceptions may cause trouble with the Spark application and not just with the > a single task execution -- This message was sent by Atlassian JIRA (v6.3.4#6332)
