[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103056297 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriter.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #14412: [SPARK-15355] [CORE] Proactive block replication

2017-02-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14412 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #14412: [SPARK-15355] [CORE] Proactive block replication

2017-02-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14412 thanks, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wi

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103056078 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSinkSuite.scala --- @@ -0,0 +1,413 @@ +/* + * Licensed to the Apac

[GitHub] spark pull request #17059: Removed unnecessary castings and refactored check...

2017-02-24 Thread datumbox
Github user datumbox commented on a diff in the pull request: https://github.com/apache/spark/pull/17059#discussion_r103055703 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -82,12 +82,20 @@ private[recommendation] trait ALSModelParams extends P

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103043531 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceProvider.scala --- @@ -152,6 +157,56 @@ private[kafka010] class Kafk

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103039701 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSink.scala --- @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Sof

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103043927 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriter.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103033329 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala --- @@ -346,5 +346,5 @@ private[kafka010] object KafkaSource

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103045028 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriter.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103044512 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriter.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103044371 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriter.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103053838 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSinkSuite.scala --- @@ -0,0 +1,413 @@ +/* + * Licensed to the Apac

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103044757 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriter.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103052449 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSinkSuite.scala --- @@ -0,0 +1,413 @@ +/* + * Licensed to the Apac

[GitHub] spark issue #15666: [SPARK-11421] [Core][Python][R] Added ability for addJar...

2017-02-24 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/15666 Do you have a chance to update this to master @mariusvniekerk? In the meantime maybe @rxin or @brkyvz want to take a look as well? --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103053432 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSinkSuite.scala --- @@ -0,0 +1,413 @@ +/* + * Licensed to the Apac

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103043738 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceProvider.scala --- @@ -377,10 +432,11 @@ private[kafka010] class Kaf

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103039931 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceProvider.scala --- @@ -152,6 +157,56 @@ private[kafka010] class Kafk

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103043568 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceProvider.scala --- @@ -152,6 +157,56 @@ private[kafka010] class Kafk

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103046651 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSinkSuite.scala --- @@ -0,0 +1,413 @@ +/* + * Licensed to the Apac

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103033150 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSink.scala --- @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Sof

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103053768 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSinkSuite.scala --- @@ -0,0 +1,413 @@ +/* + * Licensed to the Apac

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103033292 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSink.scala --- @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Sof

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103039812 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSink.scala --- @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Sof

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103053627 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSinkSuite.scala --- @@ -0,0 +1,413 @@ +/* + * Licensed to the Apac

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r103055530 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -510,8 +510,13 @@ private[spark] class HiveExternalCatalog(co

[GitHub] spark issue #16959: [SPARK-19631][CORE] OutputCommitCoordinator should not a...

2017-02-24 Thread ash211
Github user ash211 commented on the issue: https://github.com/apache/spark/pull/16959 @vanzin are you right person to review this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17059: Removed unnecessary castings and refactored checked cast...

2017-02-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17059 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #16845: [SPARK-19505][Python] AttributeError on Exception...

2017-02-24 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/16845#discussion_r103054615 --- Diff: python/pyspark/util.py --- @@ -0,0 +1,45 @@ +# -*- coding: utf-8 -*- +# +# Licensed to the Apache Software Foundation (ASF) under one o

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103054126 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSinkSuite.scala --- @@ -0,0 +1,413 @@ +/* + * Licensed to the A

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103047334 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceProvider.scala --- @@ -152,6 +157,56 @@ private[kafka010] class K

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103053743 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSinkSuite.scala --- @@ -0,0 +1,413 @@ +/* + * Licensed to the A

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103049620 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriteTask.scala --- @@ -0,0 +1,119 @@ +/* + * Licensed to the A

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103049550 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriteTask.scala --- @@ -0,0 +1,119 @@ +/* + * Licensed to the A

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103050112 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriteTask.scala --- @@ -0,0 +1,119 @@ +/* + * Licensed to the A

[GitHub] spark pull request #16845: [SPARK-19505][Python] AttributeError on Exception...

2017-02-24 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/16845#discussion_r101346487 --- Diff: python/pyspark/broadcast.py --- @@ -82,7 +83,8 @@ def dump(self, value, f): except pickle.PickleError: raise

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103044313 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSink.scala --- @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103049353 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriteTask.scala --- @@ -0,0 +1,119 @@ +/* + * Licensed to the A

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103053544 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceProvider.scala --- @@ -152,6 +157,56 @@ private[kafka010] class K

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103045768 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceProvider.scala --- @@ -377,10 +432,11 @@ private[kafka010] class

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103050085 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriteTask.scala --- @@ -0,0 +1,119 @@ +/* + * Licensed to the A

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103052166 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriter.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apac

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103053368 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriter.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apac

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103048542 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriteTask.scala --- @@ -0,0 +1,119 @@ +/* + * Licensed to the A

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103054866 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSinkSuite.scala --- @@ -0,0 +1,413 @@ +/* + * Licensed to the A

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103044379 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSink.scala --- @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103053285 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriter.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apac

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103054142 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSinkSuite.scala --- @@ -0,0 +1,413 @@ +/* + * Licensed to the A

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103054319 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSinkSuite.scala --- @@ -0,0 +1,413 @@ +/* + * Licensed to the A

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103053352 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriter.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apac

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103048422 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriteTask.scala --- @@ -0,0 +1,119 @@ +/* + * Licensed to the A

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103050143 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriteTask.scala --- @@ -0,0 +1,119 @@ +/* + * Licensed to the A

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103047462 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceProvider.scala --- @@ -152,6 +157,56 @@ private[kafka010] class K

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-24 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r103046697 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceProvider.scala --- @@ -152,6 +157,56 @@ private[kafka010] class K

[GitHub] spark pull request #17059: Removed unnecessary castings and refactored check...

2017-02-24 Thread datumbox
GitHub user datumbox opened a pull request: https://github.com/apache/spark/pull/17059 Removed unnecessary castings and refactored checked casts in ALS. ## What changes were proposed in this pull request? The original ALS was performing unnecessary casting to the user and it

[GitHub] spark pull request #15821: [SPARK-13534][PySpark] Using Apache Arrow to incr...

2017-02-24 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/15821#discussion_r103052418 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1610,8 +1623,14 @@ def toPandas(self): 02 Alice 15Bob "

[GitHub] spark pull request #15821: [SPARK-13534][PySpark] Using Apache Arrow to incr...

2017-02-24 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/15821#discussion_r103052508 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1597,11 +1607,14 @@ def toDF(self, *cols): return DataFrame(jdf, self.sql_ctx)

[GitHub] spark pull request #15821: [SPARK-13534][PySpark] Using Apache Arrow to incr...

2017-02-24 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/15821#discussion_r103047241 --- Diff: python/pyspark/sql/dataframe.py --- @@ -391,6 +392,15 @@ def collect(self): return list(_load_from_socket(port, BatchedSerializer(Pic

[GitHub] spark pull request #15821: [SPARK-13534][PySpark] Using Apache Arrow to incr...

2017-02-24 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/15821#discussion_r103047178 --- Diff: python/pyspark/serializers.py --- @@ -182,6 +182,23 @@ def loads(self, obj): raise NotImplementedError +class ArrowSer

[GitHub] spark pull request #15821: [SPARK-13534][PySpark] Using Apache Arrow to incr...

2017-02-24 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/15821#discussion_r103052601 --- Diff: python/pyspark/sql/dataframe.py --- @@ -391,6 +392,15 @@ def collect(self): return list(_load_from_socket(port, BatchedSerializer(Pic

[GitHub] spark issue #11211: [SPARK-13330][PYSPARK] PYTHONHASHSEED is not propgated t...

2017-02-24 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/11211 thanks @zjffdu , merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16845: [SPARK-19505][Python] AttributeError on Exception.messag...

2017-02-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16845 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16845: [SPARK-19505][Python] AttributeError on Exception.messag...

2017-02-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16845 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73446/ Test PASSed. ---

[GitHub] spark pull request #11211: [SPARK-13330][PYSPARK] PYTHONHASHSEED is not prop...

2017-02-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/11211 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #16845: [SPARK-19505][Python] AttributeError on Exception.messag...

2017-02-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16845 **[Test build #73446 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73446/testReport)** for PR 16845 at commit [`7b8ace4`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #14963: [SPARK-16992][PYSPARK] Virtualenv for Pylint and pep8 in...

2017-02-24 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14963 That's a reasonable question, I'll try and look at this some next week but if you've got a chance to look at/address the current comments on it that would be great. --- If your project is set up f

[GitHub] spark issue #10307: [SPARK-12334][SQL][PYSPARK] Support read from multiple i...

2017-02-24 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/10307 Gentle ping @zjffdu :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes s

[GitHub] spark issue #17012: [SPARK-19677][SS] Renaming a file atop an existing one s...

2017-02-24 Thread hejix
Github user hejix commented on the issue: https://github.com/apache/spark/pull/17012 @zsxwing Is it possible for you to give some approximate timeframe for when this might be accepted? A week or month? Since we are blocked in our spark streaming adoption (this HDFS error happens not o

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-24 Thread budde
Github user budde commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r103051801 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -226,6 +258,41 @@ private[hive] class HiveMetastoreCatalog(spark

[GitHub] spark issue #16965: [Spark-18450][ML] Scala API Change for LSH AND-amplifica...

2017-02-24 Thread Yunni
Github user Yunni commented on the issue: https://github.com/apache/spark/pull/16965 @merlintang (1) `hashDistance` is only used for multi-probe NN Search. The term `numHashTables`, `numHashFunctions` is very hard to interpret in OR-AND cases. (2) For similarity join, we

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-24 Thread budde
Github user budde commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r103051158 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -226,6 +258,41 @@ private[hive] class HiveMetastoreCatalog(spark

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-24 Thread budde
Github user budde commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r103050652 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -510,8 +510,13 @@ private[spark] class HiveExternalCatalog(conf:

[GitHub] spark issue #17027: [SPARK-19650] Commands should not trigger a Spark job

2017-02-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17027 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fea

[GitHub] spark pull request #17027: [SPARK-19650] Commands should not trigger a Spark...

2017-02-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17027#discussion_r103050359 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala --- @@ -125,8 +125,6 @@ class QueryExecution(val sparkSession: Spark

[GitHub] spark issue #16927: [SPARK-19571][R] Fix SparkR test break on Windows via Ap...

2017-02-24 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/16927 I'm worrying about this now: have my attempts to fix the messages gone horribly wrong. Admittedly, it was sitting in a Budapest airport with a post-ApacheCon hangover, but @afs was giving real

[GitHub] spark pull request #17027: [SPARK-19650] Commands should not trigger a Spark...

2017-02-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17027#discussion_r103050183 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -175,19 +175,14 @@ class Dataset[T] private[sql]( } @transi

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-24 Thread budde
Github user budde commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r103050192 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -226,6 +258,41 @@ private[hive] class HiveMetastoreCatalog(spark

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-24 Thread budde
Github user budde commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r103050080 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -161,22 +164,51 @@ private[hive] class HiveMetastoreCatalog(spa

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r103050028 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -226,6 +258,41 @@ private[hive] class HiveMetastoreCatalog(s

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r103049915 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -226,6 +258,41 @@ private[hive] class HiveMetastoreCatalog(s

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-24 Thread budde
Github user budde commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r103049838 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -296,6 +296,25 @@ object SQLConf { .longConf .creat

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r103049597 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -161,22 +164,51 @@ private[hive] class HiveMetastoreCatalog

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-24 Thread budde
Github user budde commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r103049381 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -296,6 +296,25 @@ object SQLConf { .longConf .creat

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-24 Thread budde
Github user budde commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r103049359 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/internal/SQLConfSuite.scala --- @@ -21,6 +21,7 @@ import org.apache.hadoop.fs.Path import or

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r103049318 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -296,6 +296,25 @@ object SQLConf { .longConf .c

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-02-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16929 Sure, let me take a look and try. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark issue #14830: [SPARK-16992][PYSPARK][DOCS] import sort and autopep8 on...

2017-02-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14830 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73445/ Test PASSed. ---

[GitHub] spark issue #14830: [SPARK-16992][PYSPARK][DOCS] import sort and autopep8 on...

2017-02-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14830 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14830: [SPARK-16992][PYSPARK][DOCS] import sort and autopep8 on...

2017-02-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14830 **[Test build #73445 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73445/testReport)** for PR 14830 at commit [`582c822`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #16845: [SPARK-19505][Python] AttributeError on Exception.messag...

2017-02-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16845 **[Test build #73446 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73446/testReport)** for PR 16845 at commit [`7b8ace4`](https://github.com/apache/spark/commit/7b

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r103048200 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -510,8 +510,13 @@ private[spark] class HiveExternalCatalog(co

[GitHub] spark issue #16845: [SPARK-19505][Python] AttributeError on Exception.messag...

2017-02-24 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16845 Ok in theory I've been added, so lets see Jenkins test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project do

[GitHub] spark issue #14579: [SPARK-16921][PYSPARK] RDD/DataFrame persist()/cache() s...

2017-02-24 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14579 Do you have time to update this @MLnick or maybe would it be OK if someone else made an updated PR based on this? It would be a nice feature to have for 2.2 :) --- If your project is set up for it

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r103047268 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/internal/SQLConfSuite.scala --- @@ -21,6 +21,7 @@ import org.apache.hadoop.fs.Path impor

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r103046982 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -296,6 +296,25 @@ object SQLConf { .longConf .c

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-02-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16944 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73444/ Test PASSed. ---

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-02-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16944 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-02-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16944 **[Test build #73444 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73444/testReport)** for PR 16944 at commit [`e1ca7c8`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #16774: [SPARK-19357][ML] Adding parallel model evaluation in ML...

2017-02-24 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/16774 Hi @thunterdb , thanks for the review and all of the details you provided! I agree that a configurable execution service would be needed for running under a shared environment instead of simply

<    1   2   3   4   5   6   >