[GitHub] spark pull request: [SPARK-12371][SQL] Runtime nullability check f...

2015-12-20 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10331#issuecomment-166227395 @yhuai Narrowed down the scope of this PR. As we discussed offline, will open another one for the analysis phase check. --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-12102][SQL] Cast a non-nullable struct ...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10156#discussion_r48123275 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala --- @@ -274,4 +274,12 @@ class AnalysisSuite extends Anal

[GitHub] spark pull request: [SPARK-12438][SQL] Add SQLUserDefinedType supp...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10390#issuecomment-166226967 **[Test build #48100 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48100/consoleFull)** for PR 10390 at commit [`d01c661`](https://gi

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10393#issuecomment-166225222 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10393#issuecomment-166225223 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12293][SQL] Support UnsafeRow in LocalT...

2015-12-20 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/10283#issuecomment-166225151 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10393#issuecomment-166225104 **[Test build #48098 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48098/consoleFull)** for PR 10393 at commit [`63058e3`](https://g

[GitHub] spark pull request: [SPARK-5479] [yarn] Handle --py-files correctl...

2015-12-20 Thread zjffdu
Github user zjffdu commented on the pull request: https://github.com/apache/spark/pull/6360#issuecomment-166224830 @vanzin I am reading the yarn related code specially on org.apache.spark.deploy.yarn.Client.scala Do you know where LOCAL_SCHEME("local") come from ? As I know we use

[GitHub] spark pull request: [SPARK-12438][SQL] Add SQLUserDefinedType supp...

2015-12-20 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/10390#discussion_r48122523 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/UserDefinedTypeSuite.scala --- @@ -89,6 +94,23 @@ class UserDefinedTypeSuite extends QueryTest with Sh

[GitHub] spark pull request: [SPARK-12444][SQL] A lightweight Scala DSL for...

2015-12-20 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10400#issuecomment-166221029 @markhamstra @rxin Thanks for the comments. I'm not surprised that you find `!` and `?` not super clear. @markhamstra You're right, my motivation was to make

[GitHub] spark pull request: [SPARK-12444][SQL] A lightweight Scala DSL for...

2015-12-20 Thread liancheng
Github user liancheng closed the pull request at: https://github.com/apache/spark/pull/10400 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-12293][SQL] Support UnsafeRow in LocalT...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10283#issuecomment-166219102 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12293][SQL] Support UnsafeRow in LocalT...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10283#issuecomment-166219103 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12293][SQL] Support UnsafeRow in LocalT...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10283#issuecomment-166219060 **[Test build #48094 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48094/consoleFull)** for PR 10283 at commit [`2500de3`](https://g

[GitHub] spark pull request: [SPARK-12102][SQL] Cast a non-nullable struct ...

2015-12-20 Thread dilipbiswal
Github user dilipbiswal commented on the pull request: https://github.com/apache/spark/pull/10156#issuecomment-166219070 @cloud-fan Hi Wenchen, have implemented your suggestion. Please take a look when you get a chance. Thanks !! --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-11394][SQL] Throw IllegalArgumentExcept...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9350#issuecomment-166218522 **[Test build #48099 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48099/consoleFull)** for PR 9350 at commit [`c2363aa`](https://gith

[GitHub] spark pull request: [SPARK-12371][SQL] Dataset nullability check

2015-12-20 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/10331#issuecomment-166215771 Yeah, let's use this PR for the runtime check. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-12153][SPARK-7617][MLlib]add support of...

2015-12-20 Thread MLnick
Github user MLnick commented on the pull request: https://github.com/apache/spark/pull/10152#issuecomment-166215753 Pinging @jkbradley @mengxr @MechCoder again for a final review - could you give this a look and confirm you're in agreement with my comments above. Also thought

[GitHub] spark pull request: [SPARK-12153][SPARK-7617][MLlib]add support of...

2015-12-20 Thread MLnick
Github user MLnick commented on the pull request: https://github.com/apache/spark/pull/10152#issuecomment-166215472 @ygcao just one final comment on the `setMaxSentenceLength` setter comment to address, thanks! --- If your project is set up for it, you can reply to this email and hav

[GitHub] spark pull request: [SPARK-12153][SPARK-7617][MLlib]add support of...

2015-12-20 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/10152#discussion_r48119953 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/Word2Vec.scala --- @@ -77,6 +77,20 @@ class Word2Vec extends Serializable with Logging {

[GitHub] spark pull request: [SPARK-12371][SQL] Dataset nullability check

2015-12-20 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/10331#discussion_r48119682 --- Diff: sql/catalyst/src/test/resources/log4j.properties --- @@ -16,9 +16,9 @@ # # Set everything to be logged to the file target/unit-tes

[GitHub] spark pull request: [SPARK-12371][SQL] Dataset nullability check

2015-12-20 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10331#issuecomment-166213937 @yhuai Do you think we should move analysis phase checking into another PR or just drop that part? This check does find other nullability bugs (revealed by the Jenkin

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10393#issuecomment-166213409 **[Test build #48098 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48098/consoleFull)** for PR 10393 at commit [`63058e3`](https://gi

[GitHub] spark pull request: [SPARK-6624][SQL]Convert filters into CNF for ...

2015-12-20 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/8200#issuecomment-166213204 @gatorsmile +1 and great work :)) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-12287] [SQL] Support UnsafeRow in MapPa...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10398#issuecomment-166212830 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-6624][SQL]Convert filters into CNF for ...

2015-12-20 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/8200#issuecomment-166212614 It sounds like multiple PRs are blocked by this PR. I will submit a PR for fixing it tomorrow. Thanks! --- If your project is set up for it, you can reply to this em

[GitHub] spark pull request: [SPARK-12287] [SQL] Support UnsafeRow in MapPa...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10398#issuecomment-166212601 **[Test build #48095 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48095/consoleFull)** for PR 10398 at commit [`4c745f5`](https://g

[GitHub] spark pull request: [SPARK-12085] [SQL] The join condition hidden ...

2015-12-20 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/10087#issuecomment-166212178 @flyson Great work though, you'd better to work with #8200 . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as we

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-20 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10393#discussion_r48119148 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -131,6 +131,7 @@ class DataFrameSuite extends QueryTest with SharedSQL

[GitHub] spark pull request: [SPARK-12446][SQL] Add unit tests for JDBCRDD ...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10409#issuecomment-166211512 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12446][SQL] Add unit tests for JDBCRDD ...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10409#issuecomment-166211511 **[Test build #48097 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48097/consoleFull)** for PR 10409 at commit [`ed94623`](https://g

[GitHub] spark pull request: [SPARK-12446][SQL] Add unit tests for JDBCRDD ...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10409#issuecomment-166211513 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12446][SQL] Add unit tests for JDBCRDD ...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10409#issuecomment-166211388 **[Test build #48097 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48097/consoleFull)** for PR 10409 at commit [`ed94623`](https://gi

[GitHub] spark pull request: [SPARK-11807] Remove support for Hadoop < 2.2

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10404#issuecomment-166210297 **[Test build #2242 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2242/consoleFull)** for PR 10404 at commit [`6c9fb80`](https://

[GitHub] spark pull request: [SPARK-12446][SQL] Add unit tests for JDBCRDD ...

2015-12-20 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/10409 [SPARK-12446][SQL] Add unit tests for JDBCRDD internal functions No tests done for JDBCRDD#compileFilter. You can merge this pull request into a Git repository by running: $ git pull https://gi

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10393#discussion_r48118001 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -131,6 +131,7 @@ class DataFrameSuite extends QueryTest with SharedSQLC

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10393#issuecomment-166203846 **[Test build #48096 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48096/consoleFull)** for PR 10393 at commit [`6b4ba74`](https://gi

[GitHub] spark pull request: [SPARK-12292] [SQL] Support UnsafeRow in Gener...

2015-12-20 Thread gatorsmile
Github user gatorsmile closed the pull request at: https://github.com/apache/spark/pull/10396 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-12292] [SQL] Support UnsafeRow in Gener...

2015-12-20 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/10396#issuecomment-166203268 After a few tries, I am unable to create a test case that can trigger the issue. I think I am not the right person to fix it. Thus, For not wasting the reviewers' ti

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-20 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10393#discussion_r48117166 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/Generate.scala --- @@ -51,9 +52,12 @@ case class Generate( join: Boolean,

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-20 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10393#discussion_r48117154 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -131,6 +131,7 @@ class DataFrameSuite extends QueryTest with SharedSQL

[GitHub] spark pull request: [SPARK-12437][SQL] [WIP] Encapsulate the table...

2015-12-20 Thread naveenminchu
Github user naveenminchu commented on the pull request: https://github.com/apache/spark/pull/10403#issuecomment-166203008 @rxin Agree 100% --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-12287] [SQL] Support UnsafeRow in MapPa...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10398#issuecomment-166202288 **[Test build #48095 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48095/consoleFull)** for PR 10398 at commit [`4c745f5`](https://gi

[GitHub] spark pull request: [SPARK-12371][SQL] Dataset nullability check

2015-12-20 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/10331#issuecomment-166201679 Should we just keep the runtime part of changes? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your p

[GitHub] spark pull request: [SPARK-12371][SQL] Dataset nullability check

2015-12-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/10331#discussion_r48116699 --- Diff: sql/core/src/test/resources/log4j.properties --- @@ -33,7 +33,7 @@ log4j.appender.FA.layout=org.apache.log4j.PatternLayout log4j.appender.FA.layo

[GitHub] spark pull request: [SPARK-12287] [SQL] Support UnsafeRow in MapPa...

2015-12-20 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/10398#issuecomment-166201463 Thank you! @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12371][SQL] Dataset nullability check

2015-12-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/10331#discussion_r48116475 --- Diff: sql/catalyst/src/test/resources/log4j.properties --- @@ -16,9 +16,9 @@ # # Set everything to be logged to the file target/unit-tests.l

[GitHub] spark pull request: [SPARK-12392][Core] Optimize a location order ...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10346#issuecomment-166199431 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12392][Core] Optimize a location order ...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10346#issuecomment-166199432 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12392][Core] Optimize a location order ...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10346#issuecomment-166199376 **[Test build #48093 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48093/consoleFull)** for PR 10346 at commit [`d962f15`](https://g

[GitHub] spark pull request: [SPARK-12444][SQL] A lightweight Scala DSL for...

2015-12-20 Thread markhamstra
Github user markhamstra commented on the pull request: https://github.com/apache/spark/pull/10400#issuecomment-166197805 @cloud-fan You're kind of hinting at my point: To me, this DSL seems to make life easier for Spark developers, not Spark users. In that kind of trade-off, we shoul

[GitHub] spark pull request: [SPARK-12444][SQL] A lightweight Scala DSL for...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/10400#issuecomment-166195988 @rxin you are right, an example is [`'b.struct('a.int, 'b.long)`](https://github.com/apache/spark/blob/master/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst

[GitHub] spark pull request: [SPARK-12444][SQL] A lightweight Scala DSL for...

2015-12-20 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10400#issuecomment-166195733 Note that we also have the "ColumnName" implicit. Using that you can already define a struct field using: `'fieldName.int`. --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-12444][SQL] A lightweight Scala DSL for...

2015-12-20 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10400#issuecomment-166195414 I'm with @markhamstra here. It is unclear what ! or ? mean. They are unintuitive, and are not general symbols for schema construction or nullability. --- If your projec

[GitHub] spark pull request: [SPARK-12444][SQL] A lightweight Scala DSL for...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/10400#issuecomment-166195514 hi @markhamstra , defining a schema is very frequent when writing test, do you feel it's a good idea to put this functionality only in test scope? --- If your projec

[GitHub] spark pull request: [SPARK-11807] Remove support for Hadoop < 2.2

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10404#issuecomment-166193507 **[Test build #2242 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2242/consoleFull)** for PR 10404 at commit [`6c9fb80`](https://g

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10393#discussion_r48115321 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -131,6 +131,7 @@ class DataFrameSuite extends QueryTest with SharedSQLC

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10393#discussion_r48115249 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/Generate.scala --- @@ -51,9 +52,12 @@ case class Generate( join: Boolean,

[GitHub] spark pull request: [SPARK-12287] [SQL] Support UnsafeRow in MapPa...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/10398#issuecomment-166191466 LGTM except a minor comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-12287] [SQL] Support UnsafeRow in MapPa...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10398#discussion_r48115141 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -253,6 +254,18 @@ class DatasetSuite extends QueryTest with SharedSQLCont

[GitHub] spark pull request: [SPARK-12168][SPARKR] Add automated tests for ...

2015-12-20 Thread felixcheung
Github user felixcheung commented on the pull request: https://github.com/apache/spark/pull/10171#issuecomment-166189995 @shivaram what do you think about adding `--vanilla` to `RRunner` [here](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/RRun

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-20 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/10393#discussion_r48114994 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/Generate.scala --- @@ -51,9 +52,12 @@ case class Generate( join: Boolean, o

[GitHub] spark pull request: [SPARK-12439][SQL] Fix toCatalystArray and Map...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/10391#issuecomment-166189382 Good catch! One mirror comment, can we write the test in `ExpressionEncoderSuite`? --- If your project is set up for it, you can reply to this email and have your re

[GitHub] spark pull request: [SPARK-12399] Display correct error message wh...

2015-12-20 Thread carsonwang
Github user carsonwang commented on a diff in the pull request: https://github.com/apache/spark/pull/10352#discussion_r48114578 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/HistoryServer.scala --- @@ -115,7 +117,17 @@ class HistoryServer( } def

[GitHub] spark pull request: [SPARK-12443][SQL] encoderFor should support D...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10399#discussion_r48114340 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/ScalaReflectionRelationSuite.scala --- @@ -138,4 +144,16 @@ class ScalaReflectionRelationSuite exte

[GitHub] spark pull request: [SPARK-11627] Add initial input rate limit for...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9593#issuecomment-166185197 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11627] Add initial input rate limit for...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9593#issuecomment-166185198 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-11627] Add initial input rate limit for...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9593#issuecomment-166185146 **[Test build #48091 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48091/consoleFull)** for PR 9593 at commit [`2d750c4`](https://git

[GitHub] spark pull request: [SPARK-12443][SQL] encoderFor should support D...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10399#discussion_r48114262 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -61,6 +61,7 @@ object ScalaReflection extends ScalaReflec

[GitHub] spark pull request: [SPARK-12327][SPARKR] fix code for lintr warni...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10408#issuecomment-166184144 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12293][SQL] Support UnsafeRow in LocalT...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10283#issuecomment-166184502 **[Test build #48094 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48094/consoleFull)** for PR 10283 at commit [`2500de3`](https://gi

[GitHub] spark pull request: [SPARK-12339] [WebUI] Added a null check that ...

2015-12-20 Thread carsonwang
Github user carsonwang commented on the pull request: https://github.com/apache/spark/pull/10405#issuecomment-166184298 Thanks for catching this. I think the null check here is necessary, and it seems the code that really pass a null taskMetrcis is from the `TaskSetManager` line 796 w

[GitHub] spark pull request: [SPARK-12327][SPARKR] fix code for lintr warni...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10408#issuecomment-166184145 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12327][SPARKR] fix code for lintr warni...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10408#issuecomment-166184083 **[Test build #48092 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48092/consoleFull)** for PR 10408 at commit [`a4f47a2`](https://g

[GitHub] spark pull request: [SPARK-12293][SQL] Support UnsafeRow in LocalT...

2015-12-20 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/10283#issuecomment-166184035 @cloud-fan I think I have addressed all your comments. These bugs found in implementing UnsafeRow support in LocalTableScan are submitted as other PRs with their tests,

[GitHub] spark pull request: [SPARK-12438][SQL] Add SQLUserDefinedType supp...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10390#discussion_r48113653 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/UserDefinedTypeSuite.scala --- @@ -89,6 +94,23 @@ class UserDefinedTypeSuite extends QueryTest with

[GitHub] spark pull request: [SPARK-12443][SQL] encoderFor should support D...

2015-12-20 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/10399#issuecomment-166181467 cc @cloud-fan @marmbrus @davies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-12439][SQL] Fix toCatalystArray and Map...

2015-12-20 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/10391#issuecomment-166181451 cc @cloud-fan @marmbrus @davies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-12438][SQL] Add SQLUserDefinedType supp...

2015-12-20 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/10390#issuecomment-166181434 cc @cloud-fan @marmbrus @davies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-12392][Core] Optimize a location order ...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10346#issuecomment-166181392 **[Test build #48093 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48093/consoleFull)** for PR 10346 at commit [`d962f15`](https://gi

[GitHub] spark pull request: [SPARK-12327][SPARKR] fix code for lintr warni...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10408#issuecomment-166181204 **[Test build #48092 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48092/consoleFull)** for PR 10408 at commit [`a4f47a2`](https://gi

[GitHub] spark pull request: [SPARK-12232][SPARKR] New R API for read.table...

2015-12-20 Thread sun-rui
Github user sun-rui commented on the pull request: https://github.com/apache/spark/pull/10406#issuecomment-166180732 How about "tableToDF" ? there are some API methods having table in their names, like "createExternalTable", "saveAsTable", "tables". "tableToDF" is shorter and consist

[GitHub] spark pull request: [SPARK-12392][Core] Optimize a location order ...

2015-12-20 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/10346#issuecomment-166179434 @andrewor14 Fixed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fea

[GitHub] spark pull request: [SPARK-11627] Add initial input rate limit for...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9593#issuecomment-166178142 **[Test build #48091 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48091/consoleFull)** for PR 9593 at commit [`2d750c4`](https://gith

[GitHub] spark pull request: [SPARK-12327][SPARKR] fix code for lintr warni...

2015-12-20 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/10408 [SPARK-12327][SPARKR] fix code for lintr warning for commented code @shivaram You can merge this pull request into a Git repository by running: $ git pull https://github.com/felixcheung/s

[GitHub] spark pull request: [SPARK-11627] Add initial input rate limit for...

2015-12-20 Thread junhaoMg
Github user junhaoMg commented on a diff in the pull request: https://github.com/apache/spark/pull/9593#discussion_r48112803 --- Diff: docs/configuration.md --- @@ -1523,6 +1523,15 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark pull request: [SPARK-12396][Core]Once driver connect to a ma...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10407#issuecomment-166175075 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your p

[GitHub] spark pull request: [SPARK-12396][Core]Once driver connect to a ma...

2015-12-20 Thread echoTomei
GitHub user echoTomei opened a pull request: https://github.com/apache/spark/pull/10407 [SPARK-12396][Core]Once driver connect to a master successfully, stop it connect to master again. You can merge this pull request into a Git repository by running: $ git pull https://githu

[GitHub] spark pull request: [SPARK-12339] [WebUI] Added a null check that ...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10405#issuecomment-166170887 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12339] [WebUI] Added a null check that ...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10405#issuecomment-166170888 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12339] [WebUI] Added a null check that ...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10405#issuecomment-166170840 **[Test build #48089 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48089/consoleFull)** for PR 10405 at commit [`0a46559`](https://g

[GitHub] spark pull request: [SPARK-12400][Shuffle] Avoid generating temp s...

2015-12-20 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/10376#issuecomment-166170360 Hi @JoshRosen , from performance point I don't think there's a big difference with this patch, since at most we will only open `200 * Cores` number of files simultane

[GitHub] spark pull request: [SPARK-12232] New R API for read.table to avoi...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10406#issuecomment-166169309 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12232] New R API for read.table to avoi...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10406#issuecomment-166169270 **[Test build #48090 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48090/consoleFull)** for PR 10406 at commit [`2e4b090`](https://g

[GitHub] spark pull request: [SPARK-12232] New R API for read.table to avoi...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10406#issuecomment-166169310 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-11807] Remove support for Hadoop < 2.2

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10404#issuecomment-166168830 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-11807] Remove support for Hadoop < 2.2

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10404#issuecomment-166168817 **[Test build #48088 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48088/consoleFull)** for PR 10404 at commit [`6c9fb80`](https://g

[GitHub] spark pull request: [SPARK-11807] Remove support for Hadoop < 2.2

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10404#issuecomment-166168831 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12232] New R API for read.table to avoi...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10406#issuecomment-166168381 **[Test build #48090 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48090/consoleFull)** for PR 10406 at commit [`2e4b090`](https://gi

[GitHub] spark pull request: [SPARK-12232] New R API for read.table to avoi...

2015-12-20 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/10406 [SPARK-12232] New R API for read.table to avoid name conflict @shivaram sorry it took longer to fix some conflicts, this is the change to add an alias for `table` You can merge this pull reque

  1   2   3   >