[GitHub] spark issue #16741: [SPARK-19402][DOCS] Support LaTex inline formula correct...

2017-02-06 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/16741 Thanks for these many cleanups! It's a shame to lose links. Do you think we should use fully qualified names rather than abandoning the links? --- If your project is set up for it, you can repl

[GitHub] spark pull request #16744: [SPARK-19405][STREAMING] Support for cross-accoun...

2017-02-06 Thread budde
Github user budde commented on a diff in the pull request: https://github.com/apache/spark/pull/16744#discussion_r99718950 --- Diff: external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReceiver.scala --- @@ -35,10 +36,65 @@ import org.apache.spark.stream

[GitHub] spark issue #16797: [SPARK-19455][SQL] Add option for case-insensitive Parqu...

2017-02-06 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16797 > Should we roll these behaviors into one flag? e.g. ```spark.sql.hive.mixedCaseSchemaSupport``` That sounds reasonable to me. The only thing I wonder about is if there's any use case where w

[GitHub] spark issue #16787: [WIP][SPARK-19448][SQL]optimize some duplication functio...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16787 **[Test build #72478 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72478/testReport)** for PR 16787 at commit [`d4155c2`](https://github.com/apache/spark/commit/d4

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 Retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishe

[GitHub] spark pull request #15505: [SPARK-18890][CORE] Move task serialization from ...

2017-02-06 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/15505#discussion_r99725005 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -243,27 +245,42 @@ class CoarseGrainedSche

[GitHub] spark pull request #15505: [SPARK-18890][CORE] Move task serialization from ...

2017-02-06 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/15505#discussion_r99724943 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskDescription.scala --- @@ -66,7 +100,8 @@ private[spark] object TaskDescription {

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16795 **[Test build #72479 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72479/testReport)** for PR 16795 at commit [`941bf00`](https://github.com/apache/spark/commit/94

[GitHub] spark issue #16825: [SPARK-19481][REPL][maven]Avoid to leak SparkContext in ...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16825 **[Test build #72472 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72472/testReport)** for PR 16825 at commit [`3554e33`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #16825: [SPARK-19481][REPL][maven]Avoid to leak SparkContext in ...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16825 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16825: [SPARK-19481][REPL][maven]Avoid to leak SparkContext in ...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16825 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72472/ Test PASSed. ---

[GitHub] spark pull request #16824: [SPARK-18069][PYTHON] Make PySpark doctests for S...

2017-02-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16824#discussion_r99727033 --- Diff: python/pyspark/sql/dataframe.py --- @@ -218,8 +228,10 @@ def writeStream(self): def schema(self): """Returns the schema of

[GitHub] spark issue #16715: [Spark-18080][ML] Python API & Examples for Locality Sen...

2017-02-06 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/16715 @Yunni I'm on travel at Spark Summit East these days, and will review after the summit. Thanks for your patience. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request #16824: [SPARK-18069][PYTHON] Make PySpark doctests for S...

2017-02-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16824#discussion_r99727122 --- Diff: python/pyspark/sql/dataframe.py --- @@ -233,11 +245,13 @@ def schema(self): def printSchema(self): """Prints out the schem

[GitHub] spark issue #16620: [SPARK-19263] DAGScheduler should avoid sending conflict...

2017-02-06 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/16620 @kayousterhout That sounds more clear, and I can see this being a problem (and probably explains some hung jobs I had seen a while earlier), thanks ! --- If your project is set up for it, you can r

[GitHub] spark pull request #16824: [SPARK-18069][PYTHON] Make PySpark doctests for S...

2017-02-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16824#discussion_r99728503 --- Diff: python/pyspark/sql/dataframe.py --- @@ -247,9 +261,11 @@ def explain(self, extended=False): :param extended: boolean, default

[GitHub] spark pull request #16824: [SPARK-18069][PYTHON] Make PySpark doctests for S...

2017-02-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16824#discussion_r99728558 --- Diff: python/pyspark/sql/dataframe.py --- @@ -296,8 +312,10 @@ def show(self, n=20, truncate=True): If set to a number greater than o

[GitHub] spark issue #16737: [SPARK-19397] [SQL] Make option names of LIBSVM and TEXT...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16737 **[Test build #72476 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72476/testReport)** for PR 16737 at commit [`07beaad`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #16737: [SPARK-19397] [SQL] Make option names of LIBSVM and TEXT...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16737 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16737: [SPARK-19397] [SQL] Make option names of LIBSVM and TEXT...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16737 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72476/ Test PASSed. ---

[GitHub] spark pull request #16824: [SPARK-18069][PYTHON] Make PySpark doctests for S...

2017-02-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16824#discussion_r99728911 --- Diff: python/pyspark/sql/dataframe.py --- @@ -553,10 +594,10 @@ def repartition(self, numPartitions, *cols): +---+-+ |ag

[GitHub] spark pull request #16824: [SPARK-18069][PYTHON] Make PySpark doctests for S...

2017-02-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16824#discussion_r99729087 --- Diff: python/pyspark/sql/session.py --- @@ -51,8 +51,10 @@ def toDF(self, schema=None, sampleRatio=None): :param samplingRatio: the sampl

[GitHub] spark issue #16783: [SPARK-19441] [SQL] Remove IN type coercion from Promote...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16783 **[Test build #72474 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72474/testReport)** for PR 16783 at commit [`127a114`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #16783: [SPARK-19441] [SQL] Remove IN type coercion from Promote...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16783 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72474/ Test PASSed. ---

[GitHub] spark issue #16783: [SPARK-19441] [SQL] Remove IN type coercion from Promote...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16783 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #11760: [SPARK-13931] Resolve stage hanging up problem in a part...

2017-02-06 Thread GavinGavinNo1
Github user GavinGavinNo1 commented on the issue: https://github.com/apache/spark/pull/11760 @kayousterhout Sorry that I tried before but internet connection is poor in my company and then I forget. I'll work on it back home today or tomorrow. Thank you for caring for this PR. ---

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 PR has been amended to reflect feedback. Thanks for taking a look, @brkyvz. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72480 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72480/testReport)** for PR 16744 at commit [`d23365b`](https://github.com/apache/spark/commit/d2

[GitHub] spark issue #16783: [SPARK-19441] [SQL] Remove IN type coercion from Promote...

2017-02-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16783 LGTM, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wish

[GitHub] spark pull request #16737: [SPARK-19397] [SQL] Make option names of LIBSVM a...

2017-02-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16737#discussion_r99730718 --- Diff: mllib/src/main/scala/org/apache/spark/ml/source/libsvm/LibSVMOptions.scala --- @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software F

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15009 **[Test build #72475 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72475/testReport)** for PR 15009 at commit [`4357107`](https://github.com/apache/spark/commit/4

[GitHub] spark pull request #16783: [SPARK-19441] [SQL] Remove IN type coercion from ...

2017-02-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16783 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15009 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15009 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72475/ Test PASSed. ---

[GitHub] spark issue #16747: SPARK-16636 Add CalendarIntervalType to documentation

2017-02-06 Thread terma
Github user terma commented on the issue: https://github.com/apache/spark/pull/16747 @srowen As I understood ```CalendarIntervalType``` only for compatibility with similar type in ```Hive```. So probably better to mark it as internal and close jira? --- If your project is set up for

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72481 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72481/testReport)** for PR 16744 at commit [`5823740`](https://github.com/apache/spark/commit/58

[GitHub] spark issue #16814: [SPARK-19467][ML][PYTHON]Remove cyclic imports from pysp...

2017-02-06 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/16814 LGTM Merging with master Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16787: [WIP][SPARK-19448][SQL]optimize some duplication functio...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16787 **[Test build #72478 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72478/testReport)** for PR 16787 at commit [`d4155c2`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #16814: [SPARK-19467][ML][PYTHON]Remove cyclic imports from pysp...

2017-02-06 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/16814 Btw, do you have a need to backport this to previous releases? Or is master sufficient? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #16787: [WIP][SPARK-19448][SQL]optimize some duplication functio...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16787 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72478/ Test FAILed. ---

[GitHub] spark issue #16787: [WIP][SPARK-19448][SQL]optimize some duplication functio...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16787 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #16814: [SPARK-19467][ML][PYTHON]Remove cyclic imports fr...

2017-02-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16814 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #16747: SPARK-16636 Add CalendarIntervalType to documentation

2017-02-06 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16747 ^ I want to be very sure if we are not going to expose this or not. Could any SQL committer guy or PMC confirm this? > `CalendarIntervalType` only for compatibility with similar type in

[GitHub] spark issue #16814: [SPARK-19467][ML][PYTHON]Remove cyclic imports from pysp...

2017-02-06 Thread zero323
Github user zero323 commented on the issue: https://github.com/apache/spark/pull/16814 @jkbradley, @srowen Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wis

[GitHub] spark pull request #16737: [SPARK-19397] [SQL] Make option names of LIBSVM a...

2017-02-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16737#discussion_r99732105 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/TextSuite.scala --- @@ -125,6 +124,25 @@ class TextSuite extends QueryTe

[GitHub] spark pull request #16737: [SPARK-19397] [SQL] Make option names of LIBSVM a...

2017-02-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16737#discussion_r99732168 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/TextSuite.scala --- @@ -125,6 +124,25 @@ class TextSuite extends QueryTe

[GitHub] spark issue #16495: SPARK-16920: Add a stress test for evaluateEachIteration...

2017-02-06 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/16495 @mhmoudr Will you be able to update this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16171: [SPARK-18739][ML][PYSPARK] Classification and regression...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16171 **[Test build #72482 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72482/testReport)** for PR 16171 at commit [`bc85645`](https://github.com/apache/spark/commit/bc

[GitHub] spark issue #16797: [SPARK-19455][SQL] Add option for case-insensitive Parqu...

2017-02-06 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16797 > I'll double check, but I don't think spark.sql.hive.manageFilesourcePartitions=false would solve this issue since we're still deriving the file relation's dataSchema parameter from the schema of M

[GitHub] spark issue #16620: [SPARK-19263] DAGScheduler should avoid sending conflict...

2017-02-06 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/16620 @mridulm yeah once I saw this it seemed like something that's probably been a lurking issue for a bunch of jobs!! Will be great to get this fixed -- thanks for finding it @jinxing64! --- If

[GitHub] spark issue #16797: [SPARK-19455][SQL] Add option for case-insensitive Parqu...

2017-02-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16797 how about we add a new SQL command to refresh the table schema in metastore by inferring schema with data files? This is a compatibility issue and we should have provided a way for users to migrat

[GitHub] spark issue #11760: [SPARK-13931] Resolve stage hanging up problem in a part...

2017-02-06 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/11760 Great thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16376: [SPARK-18967][SCHEDULER] compute locality levels even if...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16376 **[Test build #72477 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72477/testReport)** for PR 16376 at commit [`0852742`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #16376: [SPARK-18967][SCHEDULER] compute locality levels even if...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16376 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72477/ Test PASSed. ---

[GitHub] spark issue #16376: [SPARK-18967][SCHEDULER] compute locality levels even if...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16376 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16795 **[Test build #72473 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72473/testReport)** for PR 16795 at commit [`941bf00`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16795 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16795 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72473/ Test FAILed. ---

[GitHub] spark issue #16762: [SPARK-19419] [SPARK-19420] Fix the cross join detection

2017-02-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16762 is CROSS JOIN a logical or physical concept? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #16171: [SPARK-18739][ML][PYSPARK] Classification and regression...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16171 **[Test build #72482 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72482/testReport)** for PR 16171 at commit [`bc85645`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #16171: [SPARK-18739][ML][PYSPARK] Classification and regression...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16171 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16171: [SPARK-18739][ML][PYSPARK] Classification and regression...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16171 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72482/ Test PASSed. ---

[GitHub] spark issue #16797: [SPARK-19455][SQL] Add option for case-insensitive Parqu...

2017-02-06 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16797 If the use case where we want to infer the schema but not attempt to write it back as a property as suggested by @budde, is making sense, then the new SQL command approach might not work for it. But

[GitHub] spark issue #16747: SPARK-16636 Add CalendarIntervalType to documentation

2017-02-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16747 Actually `CalendarInterval` is already exposed to users, e.g. we can call `collect` on a DataFrame with `CalendarIntervalType` field, and get rows containing `CalendarInterval`. We don't support r

[GitHub] spark pull request #16827: [SPARK-19482][CORE] Fail it if 'spark.master' is ...

2017-02-06 Thread uncleGen
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/16827 [SPARK-19482][CORE] Fail it if 'spark.master' is set with different value ## What changes were proposed in this pull request? First, there is no need to set 'spark.master' multi-times with

[GitHub] spark pull request #16827: [SPARK-19482][CORE] Fail it if 'spark.master' is ...

2017-02-06 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/16827#discussion_r99738940 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -779,6 +781,30 @@ private[spark] object SparkConf extends Logging { }

[GitHub] spark issue #16656: [SPARK-18116][DStream] Report stream input information a...

2017-02-06 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/16656 cc @tdas also. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or i

[GitHub] spark issue #16376: [SPARK-18967][SCHEDULER] compute locality levels even if...

2017-02-06 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/16376 yes, I think this is ready (I just noticed a couple of minor nits with a fresh read but no real changes) --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #16827: [SPARK-19482][CORE] Fail it if 'spark.master' is set wit...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16827 **[Test build #72483 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72483/testReport)** for PR 16827 at commit [`0cc5997`](https://github.com/apache/spark/commit/0c

[GitHub] spark issue #16376: [SPARK-18967][SCHEDULER] compute locality levels even if...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16376 **[Test build #72484 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72484/testReport)** for PR 16376 at commit [`cf8271e`](https://github.com/apache/spark/commit/cf

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16795 **[Test build #72479 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72479/testReport)** for PR 16795 at commit [`941bf00`](https://github.com/apache/spark/commit/9

[GitHub] spark pull request #16827: [SPARK-19482][CORE] Fail it if 'spark.master' is ...

2017-02-06 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/16827#discussion_r99739390 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -779,6 +781,31 @@ private[spark] object SparkConf extends Logging { }

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16795 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72479/ Test PASSed. ---

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16795 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16827: [SPARK-19482][CORE] Fail it if 'spark.master' is set wit...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16827 **[Test build #72485 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72485/testReport)** for PR 16827 at commit [`b50a0bd`](https://github.com/apache/spark/commit/b5

[GitHub] spark issue #16815: [SPARK-19407][SS] defaultFS is used FileSystem.get inste...

2017-02-06 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16815 yea! - I found this earlier but forgot to track it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroC...

2017-02-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16795#discussion_r99740767 --- Diff: resource-managers/mesos/pom.xml --- @@ -49,6 +49,13 @@ + org.apache.spark + spark-tags_${scala.bin

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 @srowen and @liancheng Could you review this PR again? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #16737: [SPARK-19397] [SQL] Make option names of LIBSVM a...

2017-02-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16737#discussion_r99742158 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/TextSuite.scala --- @@ -125,6 +124,25 @@ class TextSuite extends QueryT

[GitHub] spark pull request #16737: [SPARK-19397] [SQL] Make option names of LIBSVM a...

2017-02-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16737#discussion_r99742193 --- Diff: mllib/src/main/scala/org/apache/spark/ml/source/libsvm/LibSVMOptions.scala --- @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #16827: [SPARK-19482][CORE] Fail it if 'spark.master' is ...

2017-02-06 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/16827#discussion_r99743099 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -779,6 +781,30 @@ private[spark] object SparkConf extends Logging { }

[GitHub] spark issue #16827: [SPARK-19482][CORE] Fail it if 'spark.master' is set wit...

2017-02-06 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/16827 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16827: [SPARK-19482][CORE] Fail it if 'spark.master' is set wit...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16827 **[Test build #72486 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72486/testReport)** for PR 16827 at commit [`8198db3`](https://github.com/apache/spark/commit/81

[GitHub] spark issue #16747: SPARK-16636 Add CalendarIntervalType to documentation

2017-02-06 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16747 Then, It looks okay to me as describing the current state and I just checked it after building the doc with this, and also we can already use it as below: ```scala scala> sq

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72480 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72480/testReport)** for PR 16744 at commit [`d23365b`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72480/ Test PASSed. ---

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72481 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72481/testReport)** for PR 16744 at commit [`5823740`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72481/ Test PASSed. ---

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16747: SPARK-16636 Add CalendarIntervalType to documentation

2017-02-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16747 CC @rxin, if we are going to expose `CalendarInterval` and `CalendarIntervalType` officially, shall we move `CalendarInterval` to the same package as `Decimal`, or create a new class as the extern

[GitHub] spark issue #16827: [SPARK-19482][CORE] Fail it if 'spark.master' is set wit...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16827 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72483/ Test FAILed. ---

[GitHub] spark issue #16827: [SPARK-19482][CORE] Fail it if 'spark.master' is set wit...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16827 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16827: [SPARK-19482][CORE] Fail it if 'spark.master' is set wit...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16827 **[Test build #72483 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72483/testReport)** for PR 16827 at commit [`0cc5997`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #16827: [SPARK-19482][CORE] Fail it if 'spark.master' is set wit...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16827 **[Test build #72485 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72485/testReport)** for PR 16827 at commit [`b50a0bd`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #16827: [SPARK-19482][CORE] Fail it if 'spark.master' is set wit...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16827 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72485/ Test FAILed. ---

[GitHub] spark issue #16827: [SPARK-19482][CORE] Fail it if 'spark.master' is set wit...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16827 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16815: [SPARK-19407][SS] defaultFS is used FileSystem.get inste...

2017-02-06 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16815 LGTM. Merging to master and 2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] spark issue #16740: [SPARK-19400][ML] Allow GLM to handle intercept only mod...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16740 **[Test build #72487 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72487/testReport)** for PR 16740 at commit [`37c41aa`](https://github.com/apache/spark/commit/37

[GitHub] spark issue #16740: [SPARK-19400][ML] Allow GLM to handle intercept only mod...

2017-02-06 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/16740 @sethah Thanks for the comments. OK, added more tests to cover all families. It's not possible to test all family and link combination if that's what you mean: the tweedie family supp

<    1   2   3   4   5   6   >