[GitHub] spark issue #10292: SPARK-11882: Custom scheduler support

2016-06-19 Thread cerisier
Github user cerisier commented on the issue: https://github.com/apache/spark/pull/10292 This is pure awesome. Any chance of this being revisited someday ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #13769: [SPARK-16030] [SQL] Allow specifying static parti...

2016-06-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13769#discussion_r67638513 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -313,13 +313,32 @@ trait CheckAnalysis extends

[GitHub] spark pull request #13769: [SPARK-16030] [SQL] Allow specifying static parti...

2016-06-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13769#discussion_r67638318 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -43,8 +43,127 @@ import

[GitHub] spark pull request #13769: [SPARK-16030] [SQL] Allow specifying static parti...

2016-06-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13769#discussion_r67638211 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -43,8 +43,127 @@ import

[GitHub] spark issue #13769: [SPARK-16030] [SQL] Allow specifying static partitions w...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13769 **[Test build #60833 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60833/consoleFull)** for PR 13769 at commit

[GitHub] spark issue #13770: [SPARK-16054] [SQL] Verification of Multiple DataFrameRe...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13770 **[Test build #60832 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60832/consoleFull)** for PR 13770 at commit

[GitHub] spark issue #13761: [SPARK-12197] [SparkCore] Kryo & Avro - Support Schema R...

2016-06-19 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13761 Don't `Dataset`s and `Encoder`s make this less relevant? What would be the use case here? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #13776: [SPARK-16050][Tests]Remove the flaky test: ConsoleSinkSu...

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13776 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60829/ Test PASSed. ---

[GitHub] spark issue #13776: [SPARK-16050][Tests]Remove the flaky test: ConsoleSinkSu...

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13776 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13776: [SPARK-16050][Tests]Remove the flaky test: ConsoleSinkSu...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13776 **[Test build #60829 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60829/consoleFull)** for PR 13776 at commit

[GitHub] spark issue #13770: [SPARK-16054] [SQL] Verification of Multiple DataFrameRe...

2016-06-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13770 @rxin So far, I think we do not provide such a capability for table-level configuration. For `parquet`, the `DataFrameReader`'s option `mergeSchema` has a higher priority than the global

[GitHub] spark pull request #13769: [SPARK-16030] [SQL] Allow specifying static parti...

2016-06-19 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/13769#discussion_r67637488 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -43,8 +43,128 @@ import

[GitHub] spark pull request #13676: [SPARK-15956] [SQL] When unwrapping ORC avoid pat...

2016-06-19 Thread dafrista
Github user dafrista commented on a diff in the pull request: https://github.com/apache/spark/pull/13676#discussion_r67637381 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala --- @@ -479,8 +340,299 @@ private[hive] trait HiveInspectors { }

[GitHub] spark pull request #13769: [SPARK-16030] [SQL] Allow specifying static parti...

2016-06-19 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/13769#discussion_r67637303 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -43,8 +43,128 @@ import

[GitHub] spark pull request #13777: [SPARK-16061][SQL][Minor] The property "spark.sql...

2016-06-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/13777#discussion_r67637263 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala --- @@ -115,7 +115,7 @@ case class KeyRemoved(key:

[GitHub] spark issue #13770: [SPARK-16054] [SQL] Verification of Multiple DataFrameRe...

2016-06-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13770 hm - what if i want to specify specific options when reading data from a table? e.g. whether to use the vectorized reader or not? --- If your project is set up for it, you can reply to this

[GitHub] spark issue #13676: [SPARK-15956] [SQL] When unwrapping ORC avoid pattern ma...

2016-06-19 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13676 This looks pretty good. What I am thinking is that generating an encoder could create another nice performance speedup here. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request #13769: [SPARK-16030] [SQL] Allow specifying static parti...

2016-06-19 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/13769#discussion_r67637019 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -43,8 +43,128 @@ import

[GitHub] spark pull request #13777: [SPARK-16061][SQL][Minor] The property "spark.sql...

2016-06-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13777#discussion_r67636827 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala --- @@ -115,7 +115,7 @@ case class KeyRemoved(key:

[GitHub] spark pull request #13706: [SPARK-15988] [SQL] Implement DDL commands: Creat...

2016-06-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13706#discussion_r67636641 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -590,6 +592,53 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark pull request #13706: [SPARK-15988] [SQL] Implement DDL commands: Creat...

2016-06-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13706#discussion_r67636452 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/macros.scala --- @@ -0,0 +1,69 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-06-19 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/13775 @hvanhovell @rxin Got it. Thanks! I will re-run the benchmark. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #13676: [SPARK-15956] [SQL] When unwrapping ORC avoid pat...

2016-06-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13676#discussion_r67636421 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala --- @@ -479,8 +340,299 @@ private[hive] trait HiveInspectors { }

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13775 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13775 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60828/ Test PASSed. ---

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13775 **[Test build #60828 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60828/consoleFull)** for PR 13775 at commit

[GitHub] spark issue #13769: [SPARK-16030] [SQL] Allow specifying static partitions w...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13769 **[Test build #60831 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60831/consoleFull)** for PR 13769 at commit

[GitHub] spark pull request #13706: [SPARK-15988] [SQL] Implement DDL commands: Creat...

2016-06-19 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/13706#discussion_r67635961 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -590,6 +592,53 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark pull request #13766: [SPARK-16036][SPARK-16037][SPARK-16034][SQL] Foll...

2016-06-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13766 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #13706: [SPARK-15988] [SQL] Implement DDL commands: Creat...

2016-06-19 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/13706#discussion_r67635827 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -590,6 +592,53 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark issue #13766: [SPARK-16036][SPARK-16037][SPARK-16034][SQL] Follow up c...

2016-06-19 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/13766 Thanks. I am merging this to master and branch 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #13706: [SPARK-15988] [SQL] Implement DDL commands: Creat...

2016-06-19 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/13706#discussion_r67635570 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/macros.scala --- @@ -0,0 +1,69 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #13772: [SPARK-16049][SQL] Make InsertIntoTable's expectedColumn...

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13772 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60826/ Test PASSed. ---

[GitHub] spark issue #13772: [SPARK-16049][SQL] Make InsertIntoTable's expectedColumn...

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13772 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13772: [SPARK-16049][SQL] Make InsertIntoTable's expectedColumn...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13772 **[Test build #60826 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60826/consoleFull)** for PR 13772 at commit

[GitHub] spark issue #13777: [SPARK-16061][SQL][Minor] The property "spark.sql.stateS...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13777 **[Test build #60830 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60830/consoleFull)** for PR 13777 at commit

[GitHub] spark pull request #13748: [SPARK-16031] Add debug-only socket source in Str...

2016-06-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13748 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #13777: [SPARK-16061][SQL] The property "spark.sql.stateS...

2016-06-19 Thread sarutak
GitHub user sarutak opened a pull request: https://github.com/apache/spark/pull/13777 [SPARK-16061][SQL] The property "spark.sql.stateStore.maintenanceInterval" should be renamed to "spark.streaming.stateStore.maintenanceInterval" ## What changes were proposed in this pull request?

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-06-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13775 @viirya when you construct a performance benchmark, you would want to minimize the overhead of things outside the code path you are testing. In this case, a lot of the time were spent in the collect

[GitHub] spark issue #13748: [SPARK-16031] Add debug-only socket source in Structured...

2016-06-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13748 LGTM - merging in master/2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-06-19 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13775 Would PR https://github.com/apache/spark/pull/13676 help to improve performance? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-06-19 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13775 @viirya could you re-run the benchmarks without calling collect(). Do a count or a simple aggregate instead, collect spends a tonne of time in serializing results from `InternalRow` to `Row`.

[GitHub] spark pull request #13706: [SPARK-15988] [SQL] Implement DDL commands: Creat...

2016-06-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13706#discussion_r67634631 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/macros.scala --- @@ -0,0 +1,69 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #13706: [SPARK-15988] [SQL] Implement DDL commands: Creat...

2016-06-19 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/13706#discussion_r67634511 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -590,6 +592,53 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark issue #13706: [SPARK-15988] [SQL] Implement DDL commands: Create/Drop ...

2016-06-19 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13706 @lianhuiwang thanks for updating the PR. Could you implement the Macro removal by pattern matching on a (to be created) `MacroFunctionBuilder` class. I feel this is simpler, and doesn't touch as

[GitHub] spark issue #13770: [SPARK-16054] [SQL] Verification of Multiple DataFrameRe...

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13770 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60825/ Test PASSed. ---

[GitHub] spark issue #13770: [SPARK-16054] [SQL] Verification of Multiple DataFrameRe...

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13770 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13770: [SPARK-16054] [SQL] Verification of Multiple DataFrameRe...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13770 **[Test build #60825 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60825/consoleFull)** for PR 13770 at commit

[GitHub] spark issue #13776: [SPARK-16050][Tests]Remove the flaky test: ConsoleSinkSu...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13776 **[Test build #60829 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60829/consoleFull)** for PR 13776 at commit

[GitHub] spark pull request #13706: [SPARK-15988] [SQL] Implement DDL commands: Creat...

2016-06-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13706#discussion_r67633556 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -590,6 +592,53 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark pull request #13706: [SPARK-15988] [SQL] Implement DDL commands: Creat...

2016-06-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13706#discussion_r67633550 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -590,6 +592,53 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark pull request #13776: [SPARK-16050][Tests]Remove the flaky test: Consol...

2016-06-19 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/13776 [SPARK-16050][Tests]Remove the flaky test: ConsoleSinkSuite ## What changes were proposed in this pull request? ConsoleSinkSuite just collects content from stdout and compare them with

[GitHub] spark issue #13776: [SPARK-16050][Tests]Remove the flaky test: ConsoleSinkSu...

2016-06-19 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/13776 /cc @marmbrus @brkyvz --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #13770: [SPARK-16054] [SQL] Verification of Multiple DataFrameRe...

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13770 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13770: [SPARK-16054] [SQL] Verification of Multiple DataFrameRe...

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13770 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60824/ Test PASSed. ---

[GitHub] spark issue #13770: [SPARK-16054] [SQL] Verification of Multiple DataFrameRe...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13770 **[Test build #60824 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60824/consoleFull)** for PR 13770 at commit

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13775 **[Test build #60828 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60828/consoleFull)** for PR 13775 at commit

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13775 **[Test build #60827 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60827/consoleFull)** for PR 13775 at commit

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13775 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60827/ Test FAILed. ---

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13775 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13775 **[Test build #60827 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60827/consoleFull)** for PR 13775 at commit

[GitHub] spark pull request #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-06-19 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/13775 [SPARK-16060][SQL] Vectorized Orc reader ## What changes were proposed in this pull request? Currently Orc reader in Spark SQL doesn't support vectorized reading. As Hive Orc already

[GitHub] spark issue #13772: [SPARK-16049][SQL] Make InsertIntoTable's expectedColumn...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13772 **[Test build #60826 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60826/consoleFull)** for PR 13772 at commit

[GitHub] spark issue #13196: [SPARK-15395][Core]Use getHostString to create RpcAddres...

2016-06-19 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/13196 @zzcclp see https://issues.apache.org/jira/browse/SPARK-16017 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #13631: [SPARK-15911][SQL] Remove the additional Project to be c...

2016-06-19 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/13631 ping @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #13196: [SPARK-15395][Core]Use getHostString to create RpcAddres...

2016-06-19 Thread zzcclp
Github user zzcclp commented on the issue: https://github.com/apache/spark/pull/13196 @zsxwing , why does this pr be reverted in branch-1.6? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #13770: [SPARK-16054] [SQL] Verification of Multiple DataFrameRe...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13770 **[Test build #60825 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60825/consoleFull)** for PR 13770 at commit

[GitHub] spark issue #13770: [SPARK-16054] [SQL] Verification of Multiple DataFrameRe...

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13770 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60823/ Test PASSed. ---

[GitHub] spark issue #13770: [SPARK-16054] [SQL] Verification of Multiple DataFrameRe...

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13770 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13770: [SPARK-16054] [SQL] Verification of Multiple DataFrameRe...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13770 **[Test build #60823 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60823/consoleFull)** for PR 13770 at commit

[GitHub] spark issue #13770: [SPARK-16054] [SQL] Verification of Multiple DataFrameRe...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13770 **[Test build #60824 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60824/consoleFull)** for PR 13770 at commit

[GitHub] spark issue #13766: [SPARK-16036][SPARK-16037][SPARK-16034][SQL] Follow up c...

2016-06-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13766 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #13770: [SPARK-16054] [SQL] Verification of Multiple DataFrameRe...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13770 **[Test build #60823 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60823/consoleFull)** for PR 13770 at commit

[GitHub] spark issue #13770: [SPARK-16054] [SQL] Verification of Multiple DataFrameRe...

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13770 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60822/ Test FAILed. ---

[GitHub] spark issue #13770: [SPARK-16054] [SQL] Verification of Multiple DataFrameRe...

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13770 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13770: [SPARK-16054] [SQL] Verification of Multiple DataFrameRe...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13770 **[Test build #60822 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60822/consoleFull)** for PR 13770 at commit

[GitHub] spark issue #13770: [SPARK-16054] [SQL] Verification of Multiple DataFrameRe...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13770 **[Test build #60822 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60822/consoleFull)** for PR 13770 at commit

[GitHub] spark issue #13770: [SPARK-16054] [SQL] Verification of Multiple DataFrameRe...

2016-06-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13770 Maybe I should move all the JDBC related API misuse issues into this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #13748: [SPARK-16031] Add debug-only socket source in Structured...

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13748 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60819/ Test PASSed. ---

[GitHub] spark issue #13748: [SPARK-16031] Add debug-only socket source in Structured...

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13748 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13748: [SPARK-16031] Add debug-only socket source in Structured...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13748 **[Test build #60819 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60819/consoleFull)** for PR 13748 at commit

[GitHub] spark issue #13773: [SPARK-16056] [SPARK-16057] [SPARK-16058] [SQL] Fix Mult...

2016-06-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13773 @rxin @liancheng @clockfly @yhuai Could you please also review this PR? I found all of you recently reviewed the JDBC-related PRs. Thanks! --- If your project is set up for it, you can reply to

[GitHub] spark issue #13773: [SPARK-16056] [SPARK-16057] [SPARK-16058] [SQL] Fix Mult...

2016-06-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13773 @srowen Will submit more PRs about `JDBC`. The interface of `DataFrameReader` and `DataFrameWriter` are not designed for `JDBC` data sources. For Spark SQL beginners, they might hit various

[GitHub] spark issue #12675: [SPARK-14894][PySpark] Add result summary api to Gaussia...

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12675 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60821/ Test PASSed. ---

[GitHub] spark issue #12675: [SPARK-14894][PySpark] Add result summary api to Gaussia...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12675 **[Test build #60821 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60821/consoleFull)** for PR 12675 at commit

[GitHub] spark issue #12675: [SPARK-14894][PySpark] Add result summary api to Gaussia...

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12675 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13760: [SPARK-16012][SparkR] GapplyCollect - applies a R functi...

2016-06-19 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13760 Thanks @NarineK -- cc @sun-rui for review --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #12675: [SPARK-14894][PySpark] Add result summary api to Gaussia...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12675 **[Test build #60821 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60821/consoleFull)** for PR 12675 at commit

[GitHub] spark issue #13760: [SPARK-16012][SparkR] GapplyCollect - applies a R functi...

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13760 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60818/ Test PASSed. ---

[GitHub] spark issue #13760: [SPARK-16012][SparkR] GapplyCollect - applies a R functi...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13760 **[Test build #60818 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60818/consoleFull)** for PR 13760 at commit

[GitHub] spark issue #13760: [SPARK-16012][SparkR] GapplyCollect - applies a R functi...

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13760 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #12675: [SPARK-14894][PySpark] Add result summary api to Gaussia...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12675 **[Test build #60820 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60820/consoleFull)** for PR 12675 at commit

[GitHub] spark issue #12675: [SPARK-14894][PySpark] Add result summary api to Gaussia...

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12675 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60820/ Test FAILed. ---

[GitHub] spark issue #12675: [SPARK-14894][PySpark] Add result summary api to Gaussia...

2016-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12675 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #12675: [SPARK-14894][PySpark] Add result summary api to Gaussia...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12675 **[Test build #60820 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60820/consoleFull)** for PR 12675 at commit

[GitHub] spark issue #13748: [SPARK-16031] Add debug-only socket source in Structured...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13748 **[Test build #60819 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60819/consoleFull)** for PR 13748 at commit

[GitHub] spark issue #13737: [SPARK-15954][SQL][PySpark][TEST] Fix TestHiveContext in...

2016-06-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13737 Why does Python need to load these test resources? I think the proper fix is to get rid of that dependency. Otherwise we are making the test harness more and more complicated and tighter coupling.

[GitHub] spark issue #12836: [SPARK-12922][SparkR][WIP] Implement gapply() on DataFra...

2016-06-19 Thread NarineK
Github user NarineK commented on the issue: https://github.com/apache/spark/pull/12836 Thanks for the quick response. I'll create one. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #13760: [SPARK-16012][SparkR] GapplyCollect - applies a R functi...

2016-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13760 **[Test build #60818 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60818/consoleFull)** for PR 13760 at commit

[GitHub] spark issue #12836: [SPARK-12922][SparkR][WIP] Implement gapply() on DataFra...

2016-06-19 Thread vectorijk
Github user vectorijk commented on the issue: https://github.com/apache/spark/pull/12836 @NarineK I am not quite sure. Maybe you could create a new JIRA for gapply's programming guide. --- If your project is set up for it, you can reply to this email and have your reply appear on

  1   2   3   >