[jira] [Resolved] (SPARK-20894) Error while checkpointing to HDFS

2017-08-07 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover resolved SPARK-20894. - Resolution: Fixed Fix Version/s: 2.3.0 > Error while checkpointing to HDFS >

[jira] [Commented] (SPARK-19720) Redact sensitive information from SparkSubmit console output

2017-07-28 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105427#comment-16105427 ] Mark Grover commented on SPARK-19720: - I wasn't planning on. One could argue the case that this could

[jira] [Updated] (SPARK-19526) Spark should raise an exception when it tries to read a Hive view but it doesn't have read access on the corresponding table(s)

2017-07-25 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated SPARK-19526: Summary: Spark should raise an exception when it tries to read a Hive view but it doesn't have

[jira] [Created] (SPARK-21525) ReceiverSupervisorImpl seems to ignore the error code when writing to the WAL

2017-07-24 Thread Mark Grover (JIRA)
Mark Grover created SPARK-21525: --- Summary: ReceiverSupervisorImpl seems to ignore the error code when writing to the WAL Key: SPARK-21525 URL: https://issues.apache.org/jira/browse/SPARK-21525 Project:

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2017-06-24 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16062196#comment-16062196 ] Mark Grover commented on SPARK-18016: - Thanks for working on this [~aeskilson]. [~cloud_fan], [~rxin]

[jira] [Created] (SPARK-20756) yarn-shuffle jar has references to unshaded guava and contains scala classes

2017-05-15 Thread Mark Grover (JIRA)
Mark Grover created SPARK-20756: --- Summary: yarn-shuffle jar has references to unshaded guava and contains scala classes Key: SPARK-20756 URL: https://issues.apache.org/jira/browse/SPARK-20756 Project:

[jira] [Resolved] (SPARK-20033) spark sql can not use hive permanent function

2017-05-11 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover resolved SPARK-20033. - Resolution: Not A Problem Marking this JIRA as resolved, accordingly. > spark sql can not use

[jira] [Commented] (SPARK-20033) spark sql can not use hive permanent function

2017-05-11 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006870#comment-16006870 ] Mark Grover commented on SPARK-20033: - Reading the PR, it seems like this is not an issue. The

[jira] [Updated] (SPARK-20514) Upgrade Jetty to 9.3.11.v20160721

2017-04-27 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated SPARK-20514: Summary: Upgrade Jetty to 9.3.11.v20160721 (was: Upgrade Jetty to 9.3.13.v20161014) > Upgrade

[jira] [Updated] (SPARK-20514) Upgrade Jetty to 9.3.13.v20161014

2017-04-27 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated SPARK-20514: Description: Currently, we are using Jetty version 9.2.16.v20160414. However, Hadoop 3, uses

[jira] [Created] (SPARK-20514) Upgrade Jetty to 9.3.13.v20161014

2017-04-27 Thread Mark Grover (JIRA)
Mark Grover created SPARK-20514: --- Summary: Upgrade Jetty to 9.3.13.v20161014 Key: SPARK-20514 URL: https://issues.apache.org/jira/browse/SPARK-20514 Project: Spark Issue Type: Bug

[jira] [Issue Comment Deleted] (SPARK-20435) More thorough redaction of sensitive information from logs/UI, more unit tests

2017-04-26 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated SPARK-20435: Comment: was deleted (was: Thanks Marcelo! ) > More thorough redaction of sensitive information

[jira] [Updated] (SPARK-20435) More thorough redaction of sensitive information from logs/UI, more unit tests

2017-04-26 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated SPARK-20435: Thanks Marcelo! > More thorough redaction of sensitive information from logs/UI, more unit tests >

[jira] [Updated] (SPARK-20435) More thorough redaction of sensitive information from logs/UI, more unit tests

2017-04-26 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated SPARK-20435: Thanks Marcelo! > More thorough redaction of sensitive information from logs/UI, more unit tests >

[jira] [Commented] (SPARK-20435) More thorough redaction of sensitive information from logs/UI, more unit tests

2017-04-24 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15981730#comment-15981730 ] Mark Grover commented on SPARK-20435: - bq. I'm not saying redacting from logs is useless, but I'm

[jira] [Commented] (SPARK-20435) More thorough redaction of sensitive information from logs/UI, more unit tests

2017-04-21 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15979741#comment-15979741 ] Mark Grover commented on SPARK-20435: - {quote} If someone is typing passwords in the process's

[jira] [Created] (SPARK-20435) More thorough redaction of sensitive information from logs/UI, more unit tests

2017-04-21 Thread Mark Grover (JIRA)
Mark Grover created SPARK-20435: --- Summary: More thorough redaction of sensitive information from logs/UI, more unit tests Key: SPARK-20435 URL: https://issues.apache.org/jira/browse/SPARK-20435

[jira] [Commented] (SPARK-20327) Add CLI support for YARN-3926

2017-04-13 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968149#comment-15968149 ] Mark Grover commented on SPARK-20327: - Daniel, we don't assign JIRAs in Spark. Folks issue a PR and

[jira] [Resolved] (SPARK-20066) Add explicit SecurityManager(SparkConf) constructor for backwards compatibility with Java

2017-03-23 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover resolved SPARK-20066. - Resolution: Won't Fix > Add explicit SecurityManager(SparkConf) constructor for backwards >

[jira] [Commented] (SPARK-20066) Add explicit SecurityManager(SparkConf) constructor for backwards compatibility with Java

2017-03-22 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15937658#comment-15937658 ] Mark Grover commented on SPARK-20066: - I have attached some simple test code here:

[jira] [Created] (SPARK-20066) Add explicit SecurityManager(SparkConf) constructor for backwards compatibility with Java

2017-03-22 Thread Mark Grover (JIRA)
Mark Grover created SPARK-20066: --- Summary: Add explicit SecurityManager(SparkConf) constructor for backwards compatibility with Java Key: SPARK-20066 URL: https://issues.apache.org/jira/browse/SPARK-20066

[jira] [Commented] (SPARK-19734) OneHotEncoder __init__ uses dropLast but doc strings all say includeFirst

2017-03-01 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890825#comment-15890825 ] Mark Grover commented on SPARK-19734: - Don't mean to step on any toes but since there wasn't any

[jira] [Created] (SPARK-19720) Redact sensitive information from SparkSubmit console output

2017-02-23 Thread Mark Grover (JIRA)
Mark Grover created SPARK-19720: --- Summary: Redact sensitive information from SparkSubmit console output Key: SPARK-19720 URL: https://issues.apache.org/jira/browse/SPARK-19720 Project: Spark

[jira] [Updated] (SPARK-18120) QueryExecutionListener method doesnt' get executed for DataFrameWriter methods

2016-12-09 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated SPARK-18120: Description: QueryExecutionListener is a class that has methods named onSuccess() and onFailure()

[jira] [Comment Edited] (SPARK-18535) Redact sensitive information from Spark logs and UI

2016-11-21 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15685225#comment-15685225 ] Mark Grover edited comment on SPARK-18535 at 11/22/16 12:36 AM: I just

[jira] [Commented] (SPARK-18535) Redact sensitive information from Spark logs and UI

2016-11-21 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15685225#comment-15685225 ] Mark Grover commented on SPARK-18535: - I just issued a PR for this, that adds a new customizable

[jira] [Updated] (SPARK-18535) Redact sensitive information from Spark logs and UI

2016-11-21 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated SPARK-18535: Attachment: redacted.png > Redact sensitive information from Spark logs and UI >

[jira] [Created] (SPARK-18535) Redact sensitive information from Spark logs and UI

2016-11-21 Thread Mark Grover (JIRA)
Mark Grover created SPARK-18535: --- Summary: Redact sensitive information from Spark logs and UI Key: SPARK-18535 URL: https://issues.apache.org/jira/browse/SPARK-18535 Project: Spark Issue

[jira] [Commented] (SPARK-17850) HadoopRDD should not swallow EOFException

2016-11-21 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15684685#comment-15684685 ] Mark Grover commented on SPARK-17850: - Hi [~zsxwing] and [~srowen], the JIRA fix version seems to

[jira] [Updated] (SPARK-18093) Fix default value test in SQLConfSuite to work regardless of warehouse dir's existence

2016-10-25 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated SPARK-18093: Description: At least on my mac (with JDK 1.7.0_67), {{default value of WAREHOUSE_PATH}} in

[jira] [Commented] (SPARK-18093) Fix default value test in SQLConfSuite to work regardless of warehouse dir's existence

2016-10-25 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15605356#comment-15605356 ] Mark Grover commented on SPARK-18093: - Yeah, I thought so too - but it failed on two different

[jira] [Created] (SPARK-18093) Fix default value test in SQLConfSuite to work regardless of warehouse dir's existence

2016-10-25 Thread Mark Grover (JIRA)
Mark Grover created SPARK-18093: --- Summary: Fix default value test in SQLConfSuite to work regardless of warehouse dir's existence Key: SPARK-18093 URL: https://issues.apache.org/jira/browse/SPARK-18093

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.10 Consumer API

2016-06-14 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330675#comment-15330675 ] Mark Grover commented on SPARK-12177: - bq. I can rename it to spark-streaming-kafka-0-10 to match the

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.10 Consumer API

2016-06-14 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330500#comment-15330500 ] Mark Grover commented on SPARK-12177: - bq. It's worth mentioning that authentication is also

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.10 Consumer API

2016-06-13 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328699#comment-15328699 ] Mark Grover commented on SPARK-12177: - Hi Ismael and Cody, My personal opinion was to hold off

[jira] [Updated] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-05-10 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated SPARK-12177: Target Version/s: (was: 2.0.0) Removing the target version of 2.0.0. Holler if you disagree. >

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-05-06 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15274910#comment-15274910 ] Mark Grover commented on SPARK-12177: - I spent some time earlier today looking at the latest Kafka

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-05-03 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15268696#comment-15268696 ] Mark Grover commented on SPARK-12177: - bq. 1. Rename existing Kafka connector to include 0.8 version

[jira] [Resolved] (SPARK-13252) Bump up Kafka to 0.9.0.0

2016-04-19 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover resolved SPARK-13252. - Resolution: Won't Fix Marking this as won't fix, and taking the focus back on the discussion in

[jira] [Created] (SPARK-14731) Revert SPARK-12130 to make 2.0 shuffle service compatible with 1.x

2016-04-19 Thread Mark Grover (JIRA)
Mark Grover created SPARK-14731: --- Summary: Revert SPARK-12130 to make 2.0 shuffle service compatible with 1.x Key: SPARK-14731 URL: https://issues.apache.org/jira/browse/SPARK-14731 Project: Spark

[jira] [Commented] (SPARK-14711) Examples jar not a part of distribution

2016-04-18 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15246333#comment-15246333 ] Mark Grover commented on SPARK-14711: - Posting here, mostly for search indexing:-) The error without

[jira] [Commented] (SPARK-14711) Examples jar not a part of distribution

2016-04-18 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15246263#comment-15246263 ] Mark Grover commented on SPARK-14711: - One solution is to move the spark-examples.jar in the jars

[jira] [Created] (SPARK-14711) Examples jar not a part of distribution

2016-04-18 Thread Mark Grover (JIRA)
Mark Grover created SPARK-14711: --- Summary: Examples jar not a part of distribution Key: SPARK-14711 URL: https://issues.apache.org/jira/browse/SPARK-14711 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-14601) Minor doc/usage changes related to removal of Spark assembly

2016-04-13 Thread Mark Grover (JIRA)
Mark Grover created SPARK-14601: --- Summary: Minor doc/usage changes related to removal of Spark assembly Key: SPARK-14601 URL: https://issues.apache.org/jira/browse/SPARK-14601 Project: Spark

[jira] [Commented] (SPARK-14477) Allow custom mirrors for downloading artifacts in build/mvn

2016-04-08 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232939#comment-15232939 ] Mark Grover commented on SPARK-14477: - Thanks Marcelo for committing! Much appreciated! Ah, I didn't

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-04-08 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232885#comment-15232885 ] Mark Grover commented on SPARK-12177: - Thanks Cody. I agree about the separate subproject and I will

[jira] [Created] (SPARK-14477) Allow custom mirrors for downloading artifacts in build/mvn

2016-04-07 Thread Mark Grover (JIRA)
Mark Grover created SPARK-14477: --- Summary: Allow custom mirrors for downloading artifacts in build/mvn Key: SPARK-14477 URL: https://issues.apache.org/jira/browse/SPARK-14477 Project: Spark

[jira] [Comment Edited] (SPARK-13670) spark-class doesn't bubble up error from launcher command

2016-03-22 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207146#comment-15207146 ] Mark Grover edited comment on SPARK-13670 at 3/22/16 7:53 PM: -- Apologies for

[jira] [Commented] (SPARK-13670) spark-class doesn't bubble up error from launcher command

2016-03-22 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207146#comment-15207146 ] Mark Grover commented on SPARK-13670: - Apologies for the delay in responding. This works for me in

[jira] [Commented] (SPARK-13877) Consider removing Kafka modules from Spark / Spark Streaming

2016-03-19 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200188#comment-15200188 ] Mark Grover commented on SPARK-13877: - Yeah, that totally makes sense. I agree that it's a big change

[jira] [Commented] (SPARK-13877) Consider removing Kafka modules from Spark / Spark Streaming

2016-03-18 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200079#comment-15200079 ] Mark Grover commented on SPARK-13877: - I am guessing this needs to be done before Spark 2.0 code

[jira] [Commented] (SPARK-13877) Consider removing Kafka modules from Spark / Spark Streaming

2016-03-15 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15195873#comment-15195873 ] Mark Grover commented on SPARK-13877: - I am in support of taking the kafka integration out as well.

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-03-08 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15185461#comment-15185461 ] Mark Grover commented on SPARK-12177: - For a) I think it's a larger discussion, that is relevant to

[jira] [Commented] (SPARK-13670) spark-class doesn't bubble up error from launcher command

2016-03-04 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15180663#comment-15180663 ] Mark Grover commented on SPARK-13670: - I have a mac, I can do some more testing. > spark-class

[jira] [Commented] (SPARK-13670) spark-class doesn't bubble up error from launcher command

2016-03-04 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15180516#comment-15180516 ] Mark Grover commented on SPARK-13670: - And, it's guaranteed that Main would never return that output?

[jira] [Commented] (SPARK-13670) spark-class doesn't bubble up error from launcher command

2016-03-04 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15180463#comment-15180463 ] Mark Grover commented on SPARK-13670: - Thanks for looking at this. What do you think of #2 - writing

[jira] [Commented] (SPARK-13670) spark-class doesn't bubble up error from launcher command

2016-03-03 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15179491#comment-15179491 ] Mark Grover commented on SPARK-13670: - cc [~vanzin] as fyi. I ran into this when I was running

[jira] [Updated] (SPARK-13670) spark-class doesn't bubble up error from launcher command

2016-03-03 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated SPARK-13670: Description: There's a particular snippet in spark-class

[jira] [Created] (SPARK-13670) spark-class doesn't bubble up error from launcher command

2016-03-03 Thread Mark Grover (JIRA)
Mark Grover created SPARK-13670: --- Summary: spark-class doesn't bubble up error from launcher command Key: SPARK-13670 URL: https://issues.apache.org/jira/browse/SPARK-13670 Project: Spark

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-03-02 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177315#comment-15177315 ] Mark Grover commented on SPARK-12177: - One more thing as a potential con for Proposal 1: There are

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-03-02 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176225#comment-15176225 ] Mark Grover commented on SPARK-12177: - Let me clarify what I was saying: There are 2 axes here - one

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-03-02 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175999#comment-15175999 ] Mark Grover commented on SPARK-12177: - I think the core of the question is a much broader Spark

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-03-01 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15174856#comment-15174856 ] Mark Grover commented on SPARK-12177: - Hi [~tdas] and [~rxin], can you help us with your opinion on

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-03-01 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15174149#comment-15174149 ] Mark Grover commented on SPARK-12177: - Thanks Cody, I appreciate your thoughts. I have been keeping

[jira] [Updated] (SPARK-13252) Bump up Kafka to 0.9.0.0

2016-02-09 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated SPARK-13252: Summary: Bump up Kafka to 0.9.0.0 (was: [STREAMING] Bump up Kafka to 0.9.0.0) > Bump up Kafka to

[jira] [Created] (SPARK-13251) Bump up Kafka to 0.9.0.0

2016-02-09 Thread Mark Grover (JIRA)
Mark Grover created SPARK-13251: --- Summary: Bump up Kafka to 0.9.0.0 Key: SPARK-13251 URL: https://issues.apache.org/jira/browse/SPARK-13251 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-13252) [STREAMING] Bump up Kafka to 0.9.0.0

2016-02-09 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated SPARK-13252: Summary: [STREAMING] Bump up Kafka to 0.9.0.0 (was: Bump up Kafka to 0.9.0.0) > [STREAMING] Bump

[jira] [Created] (SPARK-13252) Bump up Kafka to 0.9.0.0

2016-02-09 Thread Mark Grover (JIRA)
Mark Grover created SPARK-13252: --- Summary: Bump up Kafka to 0.9.0.0 Key: SPARK-13252 URL: https://issues.apache.org/jira/browse/SPARK-13252 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-13252) Bump up Kafka to 0.9.0.0

2016-02-09 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15140465#comment-15140465 ] Mark Grover commented on SPARK-13252: - Yeah, this is a very small subset of the patch for

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-02-08 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15137771#comment-15137771 ] Mark Grover commented on SPARK-12177: - Hi Rama, This particular PR adds support for the new API.

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-02-02 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15128465#comment-15128465 ] Mark Grover commented on SPARK-12177: - Thanks Cody, will do! > Update KafkaDStreams to new Kafka 0.9

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-01-27 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120113#comment-15120113 ] Mark Grover commented on SPARK-12177: - I have issued a new PR

[jira] [Commented] (SPARK-11796) Docker JDBC integration tests fail in Maven build due to dependency issue

2016-01-24 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15114383#comment-15114383 ] Mark Grover commented on SPARK-11796: - [~blbradley] I put instructions

[jira] [Commented] (SPARK-11796) Docker JDBC integration tests fail in Maven build due to dependency issue

2016-01-24 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15114590#comment-15114590 ] Mark Grover commented on SPARK-11796: - Awesome, thanks for sharing. > Docker JDBC integration tests

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-01-22 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15112785#comment-15112785 ] Mark Grover commented on SPARK-12177: - Hi Mario, Thanks for checking. I was still hoping to do

[jira] [Created] (SPARK-12960) Some examples are missing support for python2

2016-01-21 Thread Mark Grover (JIRA)
Mark Grover created SPARK-12960: --- Summary: Some examples are missing support for python2 Key: SPARK-12960 URL: https://issues.apache.org/jira/browse/SPARK-12960 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-01-20 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110082#comment-15110082 ] Mark Grover commented on SPARK-12177: - Hi Mario, I may have misunderstood some parts of your previous

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-01-19 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15107267#comment-15107267 ] Mark Grover commented on SPARK-12177: - Thanks Mario! bq. We should also have a

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-01-19 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15107669#comment-15107669 ] Mark Grover commented on SPARK-12177: - Posting an update. Took out Broker.scala, the example picking

[jira] [Commented] (SPARK-12426) Docker JDBC integration tests are failing again

2016-01-12 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094445#comment-15094445 ] Mark Grover commented on SPARK-12426: - Thanks Sean, if you could add this, that'd be great. h2.

[jira] [Resolved] (SPARK-12426) Docker JDBC integration tests are failing again

2016-01-12 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover resolved SPARK-12426. - Resolution: Fixed Thanks Sean! Marking this as resolved since all the necessary information is

[jira] [Comment Edited] (SPARK-12426) Docker JDBC integration tests are failing again

2016-01-11 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15083482#comment-15083482 ] Mark Grover edited comment on SPARK-12426 at 1/11/16 8:56 PM: -- Sean and

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-01-11 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15092455#comment-15092455 ] Mark Grover commented on SPARK-12177: - Thanks Nikita. And, I will be issuing PR's to your

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-01-09 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090717#comment-15090717 ] Mark Grover commented on SPARK-12177: - #1 Sounds great, thanks! #2 Yeah, that's the only way I can

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-01-08 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15089750#comment-15089750 ] Mark Grover commented on SPARK-12177: - Thanks for working on this, Nikita. I'd like to help out. Here

[jira] [Commented] (SPARK-12426) Docker JDBC integration tests are failing again

2016-01-08 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090179#comment-15090179 ] Mark Grover commented on SPARK-12426: - [~sowen]/[~joshrosen] Just a reminder about this, I'd

[jira] [Commented] (SPARK-12426) Docker JDBC integration tests are failing again

2016-01-05 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15083482#comment-15083482 ] Mark Grover commented on SPARK-12426: - Sean and Josh, I got to the bottom of this. This is because

[jira] [Created] (SPARK-12426) Docker JDBC integration tests are failing again

2015-12-18 Thread Mark Grover (JIRA)
Mark Grover created SPARK-12426: --- Summary: Docker JDBC integration tests are failing again Key: SPARK-12426 URL: https://issues.apache.org/jira/browse/SPARK-12426 Project: Spark Issue Type:

[jira] [Commented] (SPARK-11796) Docker JDBC integration tests fail in Maven build due to dependency issue

2015-12-10 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15051937#comment-15051937 ] Mark Grover commented on SPARK-11796: - Excellent, thank you! > Docker JDBC integration tests fail in

[jira] [Commented] (SPARK-11796) Docker JDBC integration tests fail in Maven build due to dependency issue

2015-12-10 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15051931#comment-15051931 ] Mark Grover commented on SPARK-11796: - Hey [~joshrosen], just checking if you have removed the

[jira] [Created] (SPARK-12257) Non partitioned insert into a partitioned Hive table doesn't fail

2015-12-09 Thread Mark Grover (JIRA)
Mark Grover created SPARK-12257: --- Summary: Non partitioned insert into a partitioned Hive table doesn't fail Key: SPARK-12257 URL: https://issues.apache.org/jira/browse/SPARK-12257 Project: Spark

[jira] [Commented] (SPARK-9896) Parquet Schema Assertion

2015-11-17 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15009920#comment-15009920 ] Mark Grover commented on SPARK-9896: Posting here in case someone runs into this as well. Based on my

[jira] [Created] (SPARK-11794) Improve error message when HDFS/S3 access is misconfigured when using Parquet

2015-11-17 Thread Mark Grover (JIRA)
Mark Grover created SPARK-11794: --- Summary: Improve error message when HDFS/S3 access is misconfigured when using Parquet Key: SPARK-11794 URL: https://issues.apache.org/jira/browse/SPARK-11794 Project:

[jira] [Commented] (SPARK-11249) [Launcher] Launcher library fails is app resource is not added

2015-11-03 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14987482#comment-14987482 ] Mark Grover commented on SPARK-11249: - Thanks for filing this, Hari. I am a little torn about this

[jira] [Resolved] (SPARK-10965) Optimize filesEqualRecursive

2015-10-16 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover resolved SPARK-10965. - Resolution: Won't Fix Thanks Sean. Marking this as Won't Fix since I don't think this is super

[jira] [Commented] (SPARK-10965) Optimize filesEqualRecursive

2015-10-07 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947234#comment-14947234 ] Mark Grover commented on SPARK-10965: - Thanks Sean. I haven't really decided on the approach yet but

[jira] [Created] (SPARK-10965) Optimize filesEqualRecursive

2015-10-06 Thread Mark Grover (JIRA)
Mark Grover created SPARK-10965: --- Summary: Optimize filesEqualRecursive Key: SPARK-10965 URL: https://issues.apache.org/jira/browse/SPARK-10965 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-10965) Optimize filesEqualRecursive

2015-10-06 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14946070#comment-14946070 ] Mark Grover commented on SPARK-10965: - I would love to work on this. Can someone please assign this

[jira] [Commented] (SPARK-9790) [YARN] Expose in WebUI if NodeManager is the reason why executors were killed.

2015-09-10 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739502#comment-14739502 ] Mark Grover commented on SPARK-9790: I was waiting on SPARK-8167 to get committed. That just committed

[jira] [Updated] (SPARK-9790) [YARN] Expose in WebUI if NodeManager is the reason why executors were killed.

2015-08-10 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated SPARK-9790: --- Attachment: error_showing_in_UI.png Attaching an image of what the error message in the UI would now

  1   2   >