[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427763#comment-16427763 ] stack commented on HBASE-14849: --- Reverted from branch-2/2.0.0 by HBASE-18817 > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Theodore michael Malaska >Assignee: Zhan Zhang >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-14849-1.patch, HBASE-14849-2.patch, > HBASE-14849.patch > > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065605#comment-15065605 ] Hudson commented on HBASE-14849: FAILURE: Integrated in HBase-Trunk_matrix #572 (See [https://builds.apache.org/job/HBase-Trunk_matrix/572/]) HBASE-14849 Add option to set block cache to false on SparkSQL (tedyu: rev e75e26e3c6cfe7fd378081839d60fc711c1e095f) * hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableScanRDD.scala * hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/HBaseTestSource.scala * hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseSparkConf.scala * hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/SerializableConfiguration.scala * hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/Bound.scala * hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/DefaultSource.scala * hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > Fix For: 2.0.0 > > Attachments: HBASE-14849-1.patch, HBASE-14849-2.patch, > HBASE-14849.patch > > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15064402#comment-15064402 ] Sean Busbey commented on HBASE-14849: - filed HBASE-15011 for the javadoc issue. > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > Attachments: HBASE-14849-1.patch, HBASE-14849-2.patch, > HBASE-14849.patch > > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15064656#comment-15064656 ] Zhan Zhang commented on HBASE-14849: [~ted_yu] Not very familiar with it. Could you clarify which doc I need to update? Thanks. > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > Attachments: HBASE-14849-1.patch, HBASE-14849-2.patch, > HBASE-14849.patch > > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15064662#comment-15064662 ] Ted Yu commented on HBASE-14849: When you click on Edit button, you should see 'Release Note' field in the middle. > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > Attachments: HBASE-14849-1.patch, HBASE-14849-2.patch, > HBASE-14849.patch > > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15064555#comment-15064555 ] Ted Yu commented on HBASE-14849: lgtm Please fill in release notes w.r.t. the parameters > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > Attachments: HBASE-14849-1.patch, HBASE-14849-2.patch, > HBASE-14849.patch > > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15064491#comment-15064491 ] Ted Malaska commented on HBASE-14849: - I reviewed the your changes. I'm liking what you did. So I give it a +1 But let one more person review Thanks Zhan > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > Attachments: HBASE-14849-1.patch, HBASE-14849-2.patch, > HBASE-14849.patch > > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[
https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15064784#comment-15064784
]
Hadoop QA commented on HBASE-14849:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12778531/HBASE-14849-2.patch
against master branch at commit 58342a85fa77b0d415889dd1994c55c60525b83b.
ATTACHMENT ID: 12778531
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16926//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16926//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16926//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16926//console
This message is automatically generated.
> Add option to set block cache to false on SparkSQL executions
> -
>
> Key: HBASE-14849
> URL: https://issues.apache.org/jira/browse/HBASE-14849
> Project: HBase
> Issue Type: New Feature
> Components: spark
>Reporter: Ted Malaska
>Assignee: Zhan Zhang
> Attachments: HBASE-14849-1.patch, HBASE-14849-2.patch,
> HBASE-14849.patch
>
>
> I was working at a client with a ported down version of the Spark module for
> HBase and realized we didn't add an option to turn of block cache for the
> scans.
> At the client I just disabled all caching with Spark SQL, this is an easy but
> very impactful fix.
> The fix for this patch will make this configurable
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062634#comment-15062634 ] Ted Malaska commented on HBASE-14849: - I added comments > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > Attachments: HBASE-14849-1.patch, HBASE-14849.patch > > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062684#comment-15062684 ] Ted Malaska commented on HBASE-14849: - Thanks Zhan, I updated again. Thank you for the work > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > Attachments: HBASE-14849-1.patch, HBASE-14849.patch > > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060290#comment-15060290 ] stack commented on HBASE-14849: --- [~zhanzhang] If you click on the javadoc warnings link above and dig down passing out the warnings about our use of unsafe features, you'll find this: 1 warning [WARNING] Javadoc Warnings [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java:430: warning - @param argument "hfilesDir" is not a parameter name. [INFO] It may not be because of you but a warning left over by anothers' commit. > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > Attachments: HBASE-14849.patch > > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15059253#comment-15059253 ] Ted Malaska commented on HBASE-14849: - Can you make the review board? Thanks > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > Attachments: HBASE-14849.patch > > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[
https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15059468#comment-15059468
]
Hadoop QA commented on HBASE-14849:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12777865/HBASE-14849.patch
against master branch at commit 60d33ce34191533bb858852584bd9bddfeb16a23.
ATTACHMENT ID: 12777865
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+sqlContext.sparkContext.getConf.getInt(HBaseSparkConf.CACHE_SIZE,
HBaseSparkConf.defaultCachingSize))
+
scan.setCaching(sqlContext.sparkContext.getConf.getInt(HBaseSparkConf.BATCH_NUM,
HBaseSparkConf.defaultBatchNum))
+ sparkConf.getBoolean(HBaseSparkConf.BLOCK_CACHE_ENABLE,
HBaseSparkConf.defaultBlockCacheEnable))
+ Map("cacheSize" -> "100", "batchNum" -> "100", "blockCacheingEnable" ->
"true", "rowNum" -> "10"))
+ Map("cacheSize" -> "1000", "batchNum" -> "100", "blockCacheingEnable" ->
"true", "rowNum" -> "10"))
+ Map("cacheSize" -> "100", "batchNum" -> "1000", "blockCacheingEnable" ->
"true", "rowNum" -> "10"))
+ Map("cacheSize" -> "100", "batchNum" -> "100", "blockCacheingEnable" ->
"false", "rowNum" -> "10"))
+case class DummyScan(cacheSize: Int, batchNum: Int, blockCachingEnable:
Boolean, rowNum: Int)(@transient val sqlContext: SQLContext)
+ override def buildScan(): RDD[Row] = sqlContext.sparkContext.parallelize(0
until rowNum).map(Row(_)).map{
+sparkConf.getInt(HBaseSparkConf.CACHE_SIZE,
HBaseSparkConf.defaultCachingSize) != cacheSize ||
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16870//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16870//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16870//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16870//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16870//console
This message is automatically generated.
> Add option to set block cache to false on SparkSQL executions
> -
>
> Key: HBASE-14849
> URL: https://issues.apache.org/jira/browse/HBASE-14849
> Project: HBase
> Issue Type: New Feature
> Components: spark
>Reporter: Ted Malaska
>Assignee: Zhan Zhang
> Attachments: HBASE-14849.patch
>
>
> I was working at a client with a ported down version of the Spark module for
> HBase and realized we didn't add an option to turn of block cache for the
> scans.
> At the client I just disabled all caching with Spark SQL, this is an easy but
> very impactful fix.
> The fix for this patch will make this configurable
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15059555#comment-15059555 ] Zhan Zhang commented on HBASE-14849: I use following command, but didn't find any javadoc warning. Will fix other issue after gathering review comments. mvn clean package javadoc:javadoc -DskipTests -DHBasePatchProcess > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > Attachments: HBASE-14849.patch > > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057071#comment-15057071 ] Ted Malaska commented on HBASE-14849: - Thanks Zhan. I will be able to review as soon as you finish > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15053169#comment-15053169 ] Zhan Zhang commented on HBASE-14849: [~ted.m] Please feel free to assign to me. > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Ted Malaska > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15053168#comment-15053168 ] Zhan Zhang commented on HBASE-14849: I suggest to put this type of configuration into SparkConf, for example spark.hbase.blockcache.enable and we can also migrate existing configurations following the similar way. spark.hbase.blockcache.size spark.hbase.batchnum I also have not think out a good way to test it. One way is to create a new hbase default sources dedicated for testing (with buildScan overridden), and based on the configuration we return different result to verify the configuration is correctly pushed. But it does not test the feature itself. > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Ted Malaska > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15029922#comment-15029922 ] Ted Malaska commented on HBASE-14849: - If it is ok I'm going to start this, with the hopes of getting a patch in the next 5 days. It should be an easy pass. My only worry is how to unit test something like this. hmm > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > This is an easy but very impactful fix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
