Github user ijuma commented on the issue:
https://github.com/apache/spark/pull/23099
Amazingly fast update, nice one. :)
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user ijuma commented on the issue:
https://github.com/apache/spark/pull/21955
Looks similar to https://issues.apache.org/jira/browse/KAFKA-7193, are we
using the same ZK ip in the server and client
Github user ijuma commented on the issue:
https://github.com/apache/spark/pull/21955
@srowen I think you're pretty close, I added a few comments.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user ijuma commented on a diff in the pull request:
https://github.com/apache/spark/pull/21955#discussion_r207318249
--- Diff:
external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/KafkaRDDSuite.scala
---
@@ -72,33 +72,39 @@ class KafkaRDDSuite extends
Github user ijuma commented on a diff in the pull request:
https://github.com/apache/spark/pull/21955#discussion_r207317252
--- Diff:
external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/KafkaRDDSuite.scala
---
@@ -72,31 +72,37 @@ class KafkaRDDSuite extends
Github user ijuma commented on a diff in the pull request:
https://github.com/apache/spark/pull/21955#discussion_r207316897
--- Diff:
external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/KafkaRDDSuite.scala
---
@@ -72,31 +72,37 @@ class KafkaRDDSuite extends
Github user ijuma commented on the issue:
https://github.com/apache/spark/pull/21955
Looks like some test code is using internal Kafka classes that have changed
or have been removed:
> [error]
/home/jenkins/workspace/SparkPullRequestBuilder/external/kafka-0-10/src/test/sc
Github user ijuma commented on the issue:
https://github.com/apache/spark/pull/21488
@wangyum, can you please file a Kafka JIRA with details of what the test is
doing (even if the failure is transient)? From the stacktrace, it looks like a
potential broker issue (assuming
Github user ijuma commented on the issue:
https://github.com/apache/spark/pull/21488
Anyway, overall I think you should definitely make this change. Spark users
are currently penalised heavily when running on clusters with the message
format introduced in 0.11.0, which has important
Github user ijuma commented on the issue:
https://github.com/apache/spark/pull/21488
@zsxwing `org.apache.kafka.clients.admin.AdminClient` is in the clients
jar. It's not relevant for this PR, I am just mentioning it so if Spark decides
to migrate to that eventually. I see code using
Github user ijuma commented on the issue:
https://github.com/apache/spark/pull/21488
Yeah, the Java client libraries have been evolved in a compatible manner
for the most part since 0.10.0. The set of broker versions supported by 0.10.0
and 2.0.0 is exactly the same
Github user ijuma commented on the issue:
https://github.com/apache/spark/pull/21488
@srowen I meant that the library is compatible, so if you just change the
version in the pom, it's fine. If you changed the code to use some methods in
AdminClient, then you'd have to be more careful
Github user ijuma commented on the issue:
https://github.com/apache/spark/pull/21488
Yes, Kafka clients 0.10.2 and higher support brokers from 0.10.0 and higher
if the protocols being used are available in that version. Since we only
changed test code in this PR, upgrading to 2.0.0
Github user ijuma commented on the issue:
https://github.com/apache/spark/pull/21488
1.1.1 has been released, maybe we can upgrade to that.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user ijuma commented on a diff in the pull request:
https://github.com/apache/spark/pull/21488#discussion_r203256766
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetReader.scala
---
@@ -115,7 +116,7 @@ private[kafka010] class
Github user ijuma commented on the issue:
https://github.com/apache/spark/pull/21488
Any luck getting to the bottom of the issue? It would be great to include
this in the next version of Spark?
---
-
To unsubscribe
Github user ijuma commented on a diff in the pull request:
https://github.com/apache/spark/pull/21488#discussion_r192961847
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaTestUtils.scala
---
@@ -203,7 +215,13 @@ class KafkaTestUtils
Github user ijuma commented on a diff in the pull request:
https://github.com/apache/spark/pull/21488#discussion_r192602632
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaTestUtils.scala
---
@@ -96,10 +101,13 @@ class KafkaTestUtils
Github user ijuma commented on a diff in the pull request:
https://github.com/apache/spark/pull/21488#discussion_r192601219
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaTestUtils.scala
---
@@ -96,10 +101,13 @@ class KafkaTestUtils
Github user ijuma commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-190883372
That's right @markgrover, the current approach used by Kafka preserves
compatibility for users, but makes it a bit complicated for libraries/systems
that want to support
Github user ijuma commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-190818071
"When Kafka went from 0.8 to 0.9, they did not leave the old high and low
level consumer APIs around just because there are users that use them."
Kafka
Github user ijuma closed the pull request at:
https://github.com/apache/spark/pull/7569
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user ijuma commented on the pull request:
https://github.com/apache/spark/pull/7569#issuecomment-125282944
OK, closing.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user ijuma commented on the pull request:
https://github.com/apache/spark/pull/7569#issuecomment-124123399
@JoshRosen, fair enough. Would you like to propose a PR and I can close
this one? :) This doesn't really affect me and I don't have free cycles at this
point
Github user ijuma commented on the pull request:
https://github.com/apache/spark/pull/7569#issuecomment-123446563
What would be the desired behaviour in that case? The existing one?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user ijuma commented on the pull request:
https://github.com/apache/spark/pull/7569#issuecomment-123431612
@JoshRosen Great. :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
GitHub user ijuma opened a pull request:
https://github.com/apache/spark/pull/7569
[MINOR][BUILD] Restore to previous branch instead of detached commit
The right command to get the branch name is `git rev-parse --abbrev-ref
HEAD`
instead of `git rev-parse HEAD`. The latter
Github user ijuma commented on the pull request:
https://github.com/apache/spark/pull/7569#issuecomment-12397
@srowen, some output for context:
```
~/s/spark â¯â¯â¯ git rev-parse --abbrev-ref HEAD
merge-script-fix-restore-to-branch
28 matches
Mail list logo
