[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-02-10 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10965 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-02-10 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10965#issuecomment-182591038 Thanks - I've merged this in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10965#issuecomment-181965541 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10965#issuecomment-181965539 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-02-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10965#issuecomment-181964783 **[Test build #50979 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50979/consoleFull)** for PR 10965 at commit [`fab3fb2`](https://g

[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-02-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10965#issuecomment-181927110 **[Test build #50979 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50979/consoleFull)** for PR 10965 at commit [`fab3fb2`](https://gi

[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-02-09 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/10965#issuecomment-181784411 @nongli Okay, I'll let you know the plan first. plz give me some time to look around similar codes in `Parquet` and `Orc`. --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-02-08 Thread nongli
Github user nongli commented on the pull request: https://github.com/apache/spark/pull/10965#issuecomment-181520467 The benchmark LGTM and I think this is useful. @maropu Before you make significant changes to this, can you write up what you plan to do? --- If your project

[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-02-07 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/10965#issuecomment-181177811 the benchmark infra is updated, I think we need to rerun it and update the results. --- If your project is set up for it, you can reply to this email and have your r

[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-02-07 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/10965#issuecomment-181048306 @nongli ping --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-02-04 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10965#issuecomment-180240077 cc @nongli is this useful? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-02-04 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/10965#issuecomment-180233817 @rxin Could you give me any comment on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-02-04 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/10965#issuecomment-180233354 I tried to use `DeltaBinaryPackingValuesReader` and `DeltaBinaryPackingValuesWriter` in `parquet-column` package. ``` Benchmark Running benchmark: INT De

[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-02-04 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/10965#issuecomment-180226842 The size of in-memory columnar cache is much bigger than parquet data on disk because Spark uses simpler compression algorithms than parquet does in `CompressionSchemes`

[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10965#issuecomment-178438278 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-02-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10965#issuecomment-178438079 **[Test build #50546 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50546/consoleFull)** for PR 10965 at commit [`cc58f20`](https://g

[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10965#issuecomment-178438281 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-02-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10965#issuecomment-178412175 **[Test build #50546 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50546/consoleFull)** for PR 10965 at commit [`cc58f20`](https://gi

[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10965#issuecomment-176028625 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10965#issuecomment-176028621 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10965#issuecomment-176027860 **[Test build #50254 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50254/consoleFull)** for PR 10965 at commit [`b3bf70c`](https://g

[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10965#issuecomment-176000161 **[Test build #50254 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50254/consoleFull)** for PR 10965 at commit [`b3bf70c`](https://gi

[GitHub] spark pull request: [SPARK-13057][SQL] Add benchmark codes and the...

2016-01-27 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/10965 [SPARK-13057][SQL] Add benchmark codes and the performance results for implemented compression schemes for InMemoryRelation This pr adds benchmark codes for in-memory cache compression to make futur