[jira] [Updated] (SPARK-47556) [K8] Spark App ID collision resulting in deleting wrong resources

2024-03-26 Thread Sundeep K (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sundeep K updated SPARK-47556: -- Description: h3. Issue: We noticed that sometimes K8s executor pods go in a crash loop. Reason being

[jira] [Resolved] (SPARK-47556) [K8] Spark App ID collision resulting in deleting wrong resources

2024-03-25 Thread Sundeep K (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sundeep K resolved SPARK-47556. --- Fix Version/s: 3.3.0 Resolution: Fixed

[jira] [Commented] (SPARK-47556) [K8] Spark App ID collision resulting in deleting wrong resources

2024-03-25 Thread Sundeep K (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830793#comment-17830793 ] Sundeep K commented on SPARK-47556: --- This is actually fixed in 3.3 and above

[jira] (SPARK-47556) [K8] Spark App ID collision resulting in deleting wrong resources

2024-03-25 Thread Sundeep K (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47556 ] Sundeep K deleted comment on SPARK-47556: --- was (Author: JIRAUSER304761): This seems to be fix in 3.2 and above https://github.com/Affirm/spark/commit/fe94bf07f9acec302e7d8becd7e576c777337331

[jira] [Commented] (SPARK-47556) [K8] Spark App ID collision resulting in deleting wrong resources

2024-03-25 Thread Sundeep K (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830792#comment-17830792 ] Sundeep K commented on SPARK-47556: --- This seems to be fix in 3.2 and above

[jira] [Updated] (SPARK-47556) [K8] Spark App ID collision resulting in deleting wrong resources

2024-03-25 Thread Sundeep K (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sundeep K updated SPARK-47556: -- Affects Version/s: 3.1 (was: 3.3.2) (was: 3.5.1)

[jira] [Created] (SPARK-47556) [K8] Spark App ID collision resulting in deleting wrong resources

2024-03-25 Thread Sundeep K (Jira)
Sundeep K created SPARK-47556: - Summary: [K8] Spark App ID collision resulting in deleting wrong resources Key: SPARK-47556 URL: https://issues.apache.org/jira/browse/SPARK-47556 Project: Spark

[jira] [Commented] (SPARK-44556) Reuse `OrcTail` when enable vectorizedReader

2023-12-01 Thread Mike K (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17791965#comment-17791965 ] Mike K commented on SPARK-44556: User 'cxzl25' has created a pull request for this issue:

[jira] [Commented] (SPARK-44924) Add configurations for FileStreamSource cached files

2023-10-07 Thread Mike K (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772845#comment-17772845 ] Mike K commented on SPARK-44924: User 'ragnarok56' has created a pull request for this issue:

[jira] [Commented] (SPARK-44303) Assign names to the error class _LEGACY_ERROR_TEMP_[2320-2324]

2023-07-05 Thread Mike K (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17740380#comment-17740380 ] Mike K commented on SPARK-44303: User 'beliefer' has created a pull request for this issue:

[jira] [Commented] (SPARK-44181) Upgrade Apache Kafka to 3.4.1

2023-06-25 Thread Mike K (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736914#comment-17736914 ] Mike K commented on SPARK-44181: User 'bjornjorgensen' has created a pull request for this issue:

[jira] [Commented] (SPARK-43279) Cleanup unused members from `SparkHadoopUtil`

2023-05-04 Thread Mike K (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719377#comment-17719377 ] Mike K commented on SPARK-43279: User 'LuciferYang' has created a pull request for this issue:

[jira] [Commented] (SPARK-43202) Replace reflection w/ direct calling for YARN Resource API

2023-04-20 Thread Mike K (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17714707#comment-17714707 ] Mike K commented on SPARK-43202: User 'pan3793' has created a pull request for this issue:

[jira] [Commented] (SPARK-41210) Window based executor failure tracking mechanism

2023-04-13 Thread Mike K (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17712175#comment-17712175 ] Mike K commented on SPARK-41210: User 'pan3793' has created a pull request for this issue:

[jira] [Commented] (SPARK-42382) Upgrade `cyclonedx-maven-plugin` to 2.7.6

2023-04-10 Thread Mike K (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17710355#comment-17710355 ] Mike K commented on SPARK-42382: User 'dongjoon-hyun' has created a pull request for this issue:

[jira] [Commented] (SPARK-42903) Avoid documenting None as as a return value in docstring

2023-03-23 Thread Mike K (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17703950#comment-17703950 ] Mike K commented on SPARK-42903: User 'xinrong-meng' has created a pull request for this issue:

[jira] [Commented] (SPARK-34964) Initial job has not accepted any resources

2021-04-06 Thread Nanyin K (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17315368#comment-17315368 ] Nanyin K commented on SPARK-34964: -- Full trace log attached.   could someone tell me why this warning

[jira] [Updated] (SPARK-34964) Initial job has not accepted any resources

2021-04-06 Thread Nanyin K (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanyin K updated SPARK-34964: - Attachment: full-trace-log.txt > Initial job has not accepted any resources >

[jira] [Created] (SPARK-34964) Initial job has not accepted any resources

2021-04-06 Thread Nanyin K (Jira)
Nanyin K created SPARK-34964: Summary: Initial job has not accepted any resources Key: SPARK-34964 URL: https://issues.apache.org/jira/browse/SPARK-34964 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-30745) Spark streaming, kafka broker error, "Failed to get records for spark-executor- .... after polling for 512"

2020-02-05 Thread Harneet K (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harneet K updated SPARK-30745: -- Description: We have a spark streaming application reading data from Kafka. Data size: 15 Million

[jira] [Created] (SPARK-30745) Spark streaming, kafka broker error, "Failed to get records for spark-executor- .... after polling for 512"

2020-02-05 Thread Harneet K (Jira)
Harneet K created SPARK-30745: - Summary: Spark streaming, kafka broker error, "Failed to get records for spark-executor- after polling for 512" Key: SPARK-30745 URL:

[jira] [Comment Edited] (SPARK-27239) Processing Compressed HDFS files with spark failing with error: "java.lang.IllegalArgumentException: requirement failed: length (-1) cannot be negative" from spar

2019-03-26 Thread Rajesh Kumar K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801449#comment-16801449 ] Rajesh Kumar K edited comment on SPARK-27239 at 3/26/19 7:13 AM: -

[jira] [Commented] (SPARK-27239) Processing Compressed HDFS files with spark failing with error: "java.lang.IllegalArgumentException: requirement failed: length (-1) cannot be negative" from spark 2.2

2019-03-26 Thread Rajesh Kumar K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801449#comment-16801449 ] Rajesh Kumar K commented on SPARK-27239: [~hyukjin.kwon] you resolved my Jira instead of the

[jira] [Updated] (SPARK-27239) Processing Compressed HDFS files with spark failing with error: "java.lang.IllegalArgumentException: requirement failed: length (-1) cannot be negative" from spark 2.2.X

2019-03-26 Thread Rajesh Kumar K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Kumar K updated SPARK-27239: --- Priority: Major (was: Blocker) > Processing Compressed HDFS files with spark failing with

[jira] [Reopened] (SPARK-27239) Processing Compressed HDFS files with spark failing with error: "java.lang.IllegalArgumentException: requirement failed: length (-1) cannot be negative" from spark 2.2.

2019-03-26 Thread Rajesh Kumar K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Kumar K reopened SPARK-27239: Reopening this Jira, since Jira : SPARK-27259 is duplicate and it exactly cloned from this

[jira] [Updated] (SPARK-27239) Processing Compressed HDFS files with spark failing with error: "java.lang.IllegalArgumentException: requirement failed: length (-1) cannot be negative" from spark 2.2.X

2019-03-22 Thread Rajesh Kumar K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Kumar K updated SPARK-27239: --- Description:   >From spark 2.2.x versions, when spark job processing any compressed HDFS

[jira] [Updated] (SPARK-27239) Processing Compressed HDFS files with spark failing with error: "java.lang.IllegalArgumentException: requirement failed: length (-1) cannot be negative" from spark 2.2.X

2019-03-22 Thread Rajesh Kumar K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Kumar K updated SPARK-27239: --- Description:   >From spark 2.2.x versions, when spark job processing any compressed HDFS

[jira] [Updated] (SPARK-27239) Processing Compressed HDFS files with spark failing with error: "java.lang.IllegalArgumentException: requirement failed: length (-1) cannot be negative" from spark 2.2.X

2019-03-22 Thread Rajesh Kumar K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Kumar K updated SPARK-27239: --- Description:   >From spark 2.2.x versions, when spark job processing any compressed HDFS

[jira] [Updated] (SPARK-27239) Processing Compressed HDFS files with spark failing with error: "java.lang.IllegalArgumentException: requirement failed: length (-1) cannot be negative" from spark 2.2.X

2019-03-22 Thread Rajesh Kumar K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Kumar K updated SPARK-27239: --- Summary: Processing Compressed HDFS files with spark failing with error:

[jira] [Created] (SPARK-27239) Compressed HDFS files processing with spark failing from version Spark 2.2.x with error: "java.lang.IllegalArgumentException: requirement failed: length (-1) cannot be n

2019-03-22 Thread Rajesh Kumar K (JIRA)
Rajesh Kumar K created SPARK-27239: -- Summary: Compressed HDFS files processing with spark failing from version Spark 2.2.x with error: "java.lang.IllegalArgumentException: requirement failed: length (-1) cannot be negative"

[jira] [Created] (SPARK-26737) Executor/Task STDERR & STDOUT log urls are not correct in Yarn deployment mode

2019-01-25 Thread Devaraj K (JIRA)
Devaraj K created SPARK-26737: - Summary: Executor/Task STDERR & STDOUT log urls are not correct in Yarn deployment mode Key: SPARK-26737 URL: https://issues.apache.org/jira/browse/SPARK-26737 Project:

[jira] [Created] (SPARK-26650) Yarn Client throws 'ClassNotFoundException: org.apache.hadoop.hbase.HBaseConfiguration'

2019-01-17 Thread Devaraj K (JIRA)
Devaraj K created SPARK-26650: - Summary: Yarn Client throws 'ClassNotFoundException: org.apache.hadoop.hbase.HBaseConfiguration' Key: SPARK-26650 URL: https://issues.apache.org/jira/browse/SPARK-26650

[jira] [Commented] (SPARK-24787) Events being dropped at an alarming rate due to hsync being slow for eventLogging

2018-10-16 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16652687#comment-16652687 ] Devaraj K commented on SPARK-24787: --- It seems here the overhead is coming due the force call

[jira] [Created] (SPARK-25683) Make AsyncEventQueue.lastReportTimestamp inital value as the currentTime instead of 0

2018-10-08 Thread Devaraj K (JIRA)
Devaraj K created SPARK-25683: - Summary: Make AsyncEventQueue.lastReportTimestamp inital value as the currentTime instead of 0 Key: SPARK-25683 URL: https://issues.apache.org/jira/browse/SPARK-25683

[jira] [Resolved] (SPARK-25645) Add provision to disable EventLoggingListener default flush/hsync/hflush for all events

2018-10-05 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K resolved SPARK-25645. --- Resolution: Duplicate > Add provision to disable EventLoggingListener default flush/hsync/hflush

[jira] [Commented] (SPARK-25645) Add provision to disable EventLoggingListener default flush/hsync/hflush for all events

2018-10-04 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639079#comment-16639079 ] Devaraj K commented on SPARK-25645: --- {code:java|title=with hflush(no hsync)|borderStyle=solid}

[jira] [Commented] (SPARK-25645) Add provision to disable EventLoggingListener default flush/hsync/hflush for all events

2018-10-04 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639046#comment-16639046 ] Devaraj K commented on SPARK-25645: --- Thanks [~vanzin] for the jira pointer, I haven't tried just with

[jira] [Created] (SPARK-25645) Add provision to disable EventLoggingListener default flush/hsync/hflush for all events

2018-10-04 Thread Devaraj K (JIRA)
Devaraj K created SPARK-25645: - Summary: Add provision to disable EventLoggingListener default flush/hsync/hflush for all events Key: SPARK-25645 URL: https://issues.apache.org/jira/browse/SPARK-25645

[jira] [Commented] (SPARK-25645) Add provision to disable EventLoggingListener default flush/hsync/hflush for all events

2018-10-04 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639005#comment-16639005 ] Devaraj K commented on SPARK-25645: --- {code:java|title=Present Behavior(flushLogger=true for some

[jira] [Created] (SPARK-25637) SparkException: Could not find CoarseGrainedScheduler occurs during the application stop

2018-10-03 Thread Devaraj K (JIRA)
Devaraj K created SPARK-25637: - Summary: SparkException: Could not find CoarseGrainedScheduler occurs during the application stop Key: SPARK-25637 URL: https://issues.apache.org/jira/browse/SPARK-25637

[jira] [Created] (SPARK-25636) spark-submit swallows the failure reason when there is an error connecting to master

2018-10-03 Thread Devaraj K (JIRA)
Devaraj K created SPARK-25636: - Summary: spark-submit swallows the failure reason when there is an error connecting to master Key: SPARK-25636 URL: https://issues.apache.org/jira/browse/SPARK-25636

[jira] [Commented] (SPARK-25246) When the spark.eventLog.compress is enabled, the Application is not showing in the History server UI ('incomplete application' page), initially.

2018-09-14 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16615398#comment-16615398 ] Devaraj K commented on SPARK-25246: --- I think it is not a problem, the behavior might be based on the

[jira] [Created] (SPARK-25009) Standalone Cluster mode application submit is not working

2018-08-02 Thread Devaraj K (JIRA)
Devaraj K created SPARK-25009: - Summary: Standalone Cluster mode application submit is not working Key: SPARK-25009 URL: https://issues.apache.org/jira/browse/SPARK-25009 Project: Spark Issue

[jira] [Updated] (SPARK-24540) Support for multiple delimiter in Spark CSV read

2018-06-13 Thread Ashwin K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashwin K updated SPARK-24540: - Description: Currently, the delimiter option Spark 2.0 to read and split CSV files/data only support a

[jira] [Updated] (SPARK-24540) Support for multiple delimiter in Spark CSV read

2018-06-12 Thread Ashwin K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashwin K updated SPARK-24540: - Description: Currently, the delimiter option Spark 2.0 to read and split CSV files/data only support a

[jira] [Created] (SPARK-24540) Support for multiple delimiter in Spark CSV read

2018-06-12 Thread Ashwin K (JIRA)
Ashwin K created SPARK-24540: Summary: Support for multiple delimiter in Spark CSV read Key: SPARK-24540 URL: https://issues.apache.org/jira/browse/SPARK-24540 Project: Spark Issue Type:

[jira] [Created] (SPARK-24354) Adding support for quoteMode in Spark's build in CSV DataFrameWriter

2018-05-22 Thread Umesh K (JIRA)
Umesh K created SPARK-24354: --- Summary: Adding support for quoteMode in Spark's build in CSV DataFrameWriter Key: SPARK-24354 URL: https://issues.apache.org/jira/browse/SPARK-24354 Project: Spark

[jira] [Comment Edited] (SPARK-13638) Support for saving with a quote mode

2018-05-22 Thread Umesh K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484255#comment-16484255 ] Umesh K edited comment on SPARK-13638 at 5/22/18 4:36 PM: -- [~rxin] Just want to

[jira] [Commented] (SPARK-13638) Support for saving with a quote mode

2018-05-22 Thread Umesh K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484255#comment-16484255 ] Umesh K commented on SPARK-13638: - Just want to confirm are we ever going to have quoteMode or we always

[jira] [Created] (SPARK-24129) Add option to pass --build-arg's to docker-image-tool.sh

2018-04-30 Thread Devaraj K (JIRA)
Devaraj K created SPARK-24129: - Summary: Add option to pass --build-arg's to docker-image-tool.sh Key: SPARK-24129 URL: https://issues.apache.org/jira/browse/SPARK-24129 Project: Spark Issue

[jira] [Created] (SPARK-24003) Add support to provide spark.executor.extraJavaOptions in terms of App Id and/or Executor Id's

2018-04-17 Thread Devaraj K (JIRA)
Devaraj K created SPARK-24003: - Summary: Add support to provide spark.executor.extraJavaOptions in terms of App Id and/or Executor Id's Key: SPARK-24003 URL: https://issues.apache.org/jira/browse/SPARK-24003

[jira] [Commented] (SPARK-22567) spark.mesos.executor.memoryOverhead equivalent for the Driver when running on Mesos

2018-02-26 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377370#comment-16377370 ] Devaraj K commented on SPARK-22567: --- Dup of SPARK-17928. [~michaelmoss], can you check the PR

[jira] [Commented] (SPARK-22404) Provide an option to use unmanaged AM in yarn-client mode

2018-01-02 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16308802#comment-16308802 ] Devaraj K commented on SPARK-22404: --- Thanks [~irashid] for the comment. bq. can you provide a little

[jira] [Updated] (SPARK-22860) Spark workers log ssl passwords passed to the executors

2017-12-21 Thread Felix K. (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix K. updated SPARK-22860: - Description: The workers log the spark.ssl.keyStorePassword and spark.ssl.trustStorePassword passed by

[jira] [Updated] (SPARK-22860) Spark workers log ssl passwords passed to the executors

2017-12-21 Thread Felix K. (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix K. updated SPARK-22860: - Description: The workers log the spark.ssl.keyStorePassword and spark.ssl.trustStorePassword passed by

[jira] [Updated] (SPARK-22860) Spark workers log ssl passwords passed to the executors

2017-12-21 Thread Felix K. (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix K. updated SPARK-22860: - Description: The workers log the spark.ssl.keyStorePassword and spark.ssl.trustStorePassword passed by

[jira] [Created] (SPARK-22860) Spark workers log ssl passwords passed to the executors

2017-12-21 Thread Felix K. (JIRA)
Felix K. created SPARK-22860: Summary: Spark workers log ssl passwords passed to the executors Key: SPARK-22860 URL: https://issues.apache.org/jira/browse/SPARK-22860 Project: Spark Issue Type:

[jira] [Commented] (SPARK-14228) Lost executor of RPC disassociated, and occurs exception: Could not find CoarseGrainedScheduler or it has been stopped

2017-12-11 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286282#comment-16286282 ] Devaraj K commented on SPARK-14228: --- [~KaiXinXIaoLei], Thanks for checking this. Is the issue you are

[jira] [Updated] (SPARK-22519) Remove unnecessary stagingDirPath null check in ApplicationMaster.cleanupStagingDir()

2017-11-14 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated SPARK-22519: -- Summary: Remove unnecessary stagingDirPath null check in ApplicationMaster.cleanupStagingDir() (was:

[jira] [Updated] (SPARK-22519) Remove unnecessary stagingDirPath null check in ApplicationMaster.cleanupStagingDir()

2017-11-14 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated SPARK-22519: -- Priority: Trivial (was: Minor) > Remove unnecessary stagingDirPath null check in >

[jira] [Commented] (SPARK-22519) ApplicationMaster.cleanupStagingDir() throws NPE when SPARK_YARN_STAGING_DIR env var is not available

2017-11-14 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251925#comment-16251925 ] Devaraj K commented on SPARK-22519: --- It is not an usual case, I have seen this NPE while working

[jira] [Created] (SPARK-22519) ApplicationMaster.cleanupStagingDir() throws NPE when SPARK_YARN_STAGING_DIR env var is not available

2017-11-14 Thread Devaraj K (JIRA)
Devaraj K created SPARK-22519: - Summary: ApplicationMaster.cleanupStagingDir() throws NPE when SPARK_YARN_STAGING_DIR env var is not available Key: SPARK-22519 URL: https://issues.apache.org/jira/browse/SPARK-22519

[jira] [Created] (SPARK-22404) Provide an option to use unmanaged AM in yarn-client mode

2017-10-30 Thread Devaraj K (JIRA)
Devaraj K created SPARK-22404: - Summary: Provide an option to use unmanaged AM in yarn-client mode Key: SPARK-22404 URL: https://issues.apache.org/jira/browse/SPARK-22404 Project: Spark Issue

[jira] [Commented] (SPARK-22404) Provide an option to use unmanaged AM in yarn-client mode

2017-10-30 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226015#comment-16226015 ] Devaraj K commented on SPARK-22404: --- I am working on this, will update this jira with the proposal PR.

[jira] [Created] (SPARK-22172) Worker hangs when the external shuffle service port is already in use

2017-09-29 Thread Devaraj K (JIRA)
Devaraj K created SPARK-22172: - Summary: Worker hangs when the external shuffle service port is already in use Key: SPARK-22172 URL: https://issues.apache.org/jira/browse/SPARK-22172 Project: Spark

[jira] [Commented] (SPARK-19417) spark.files.overwrite is ignored

2017-09-22 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16177253#comment-16177253 ] Devaraj K commented on SPARK-19417: --- Thanks [~ckanich] for the test case.

[jira] [Commented] (SPARK-18648) spark-shell --jars option does not add jars to classpath on windows

2017-08-10 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121817#comment-16121817 ] Devaraj K commented on SPARK-18648: --- [~FlamingMike], It has fixed as part of SPARK-21339, can you check

[jira] [Commented] (SPARK-15142) Spark Mesos dispatcher becomes unusable when the Mesos master restarts

2017-07-25 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100079#comment-16100079 ] Devaraj K commented on SPARK-15142: --- bq. That means there is no way to detect the new master while the

[jira] [Resolved] (SPARK-15142) Spark Mesos dispatcher becomes unusable when the Mesos master restarts

2017-07-24 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K resolved SPARK-15142. --- Resolution: Duplicate > Spark Mesos dispatcher becomes unusable when the Mesos master restarts >

[jira] [Commented] (SPARK-15142) Spark Mesos dispatcher becomes unusable when the Mesos master restarts

2017-07-24 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099535#comment-16099535 ] Devaraj K commented on SPARK-15142: --- [~skonto] Thanks for showing interest on this. I have already

[jira] [Updated] (SPARK-21146) Master/Worker should handle and shutdown when any thread gets UncaughtException

2017-06-30 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated SPARK-21146: -- Summary: Master/Worker should handle and shutdown when any thread gets UncaughtException (was: Worker

[jira] [Closed] (SPARK-21148) Set SparkUncaughtExceptionHandler to the Master

2017-06-29 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K closed SPARK-21148. - Resolution: Duplicate > Set SparkUncaughtExceptionHandler to the Master >

[jira] [Created] (SPARK-21170) Utils.tryWithSafeFinallyAndFailureCallbacks throws IllegalArgumentException: Self-suppression not permitted

2017-06-21 Thread Devaraj K (JIRA)
Devaraj K created SPARK-21170: - Summary: Utils.tryWithSafeFinallyAndFailureCallbacks throws IllegalArgumentException: Self-suppression not permitted Key: SPARK-21170 URL:

[jira] [Created] (SPARK-21148) Set SparkUncaughtExceptionHandler to the Master

2017-06-19 Thread Devaraj K (JIRA)
Devaraj K created SPARK-21148: - Summary: Set SparkUncaughtExceptionHandler to the Master Key: SPARK-21148 URL: https://issues.apache.org/jira/browse/SPARK-21148 Project: Spark Issue Type:

[jira] [Created] (SPARK-21146) Worker should handle and shutdown when any thread gets UncaughtException

2017-06-19 Thread Devaraj K (JIRA)
Devaraj K created SPARK-21146: - Summary: Worker should handle and shutdown when any thread gets UncaughtException Key: SPARK-21146 URL: https://issues.apache.org/jira/browse/SPARK-21146 Project: Spark

[jira] [Commented] (SPARK-20415) SPARK job hangs while writing DataFrame to HDFS

2017-04-21 Thread P K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15978342#comment-15978342 ] P K commented on SPARK-20415: - Today I confirmed that the problem independently of whether writing in parquet

[jira] [Updated] (SPARK-20415) SPARK job hangs while writing DataFrame to HDFS

2017-04-20 Thread P K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] P K updated SPARK-20415: Environment: EMR 5.4.0 (was: Latest EMR) > SPARK job hangs while writing DataFrame to HDFS >

[jira] [Updated] (SPARK-20415) SPARK job hangs while writing DataFrame to HDFS

2017-04-20 Thread P K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] P K updated SPARK-20415: Summary: SPARK job hangs while writing DataFrame to HDFS (was: SPARK job hangs while writing table to HDFS) >

[jira] [Created] (SPARK-20415) SPARK job hangs while writing table to HDFS

2017-04-20 Thread P K (JIRA)
P K created SPARK-20415: --- Summary: SPARK job hangs while writing table to HDFS Key: SPARK-20415 URL: https://issues.apache.org/jira/browse/SPARK-20415 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-15665) spark-submit --kill and --status are not working

2017-03-25 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15942141#comment-15942141 ] Devaraj K commented on SPARK-15665: --- [~samuel-soubeyran], This issue has been resolved, Please create

[jira] [Created] (SPARK-19689) Job Details page doesn't show 'Tasks: Succeeded/Total' progress bar text properly

2017-02-21 Thread Devaraj K (JIRA)
Devaraj K created SPARK-19689: - Summary: Job Details page doesn't show 'Tasks: Succeeded/Total' progress bar text properly Key: SPARK-19689 URL: https://issues.apache.org/jira/browse/SPARK-19689 Project:

[jira] [Updated] (SPARK-19689) Job Details page doesn't show 'Tasks: Succeeded/Total' progress bar text properly

2017-02-21 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated SPARK-19689: -- Attachment: Tasks Progress bar - Job Details Page.png > Job Details page doesn't show 'Tasks:

[jira] [Updated] (SPARK-19354) Killed tasks are getting marked as FAILED

2017-01-26 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated SPARK-19354: -- Description: When we enable speculation, we can see there are multiple attempts running for the same

[jira] [Updated] (SPARK-19377) Killed tasks should have the status as KILLED

2017-01-26 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated SPARK-19377: -- Description: |143|10 |0 |SUCCESS|NODE_LOCAL |6 / x.xx.x.x stdout stderr

[jira] [Created] (SPARK-19377) Killed tasks should have the status as KILLED

2017-01-26 Thread Devaraj K (JIRA)
Devaraj K created SPARK-19377: - Summary: Killed tasks should have the status as KILLED Key: SPARK-19377 URL: https://issues.apache.org/jira/browse/SPARK-19377 Project: Spark Issue Type:

[jira] [Commented] (SPARK-19354) Killed tasks are getting marked as FAILED

2017-01-25 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15838396#comment-15838396 ] Devaraj K commented on SPARK-19354: --- bq. The question is, why the error during shutdown? The shutdown

[jira] [Commented] (SPARK-19354) Killed tasks are getting marked as FAILED

2017-01-24 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837346#comment-15837346 ] Devaraj K commented on SPARK-19354: --- Thanks [~uncleGen] for the comment. Here the error occurred during

[jira] [Created] (SPARK-19354) Killed tasks are getting marked as FAILED

2017-01-24 Thread Devaraj K (JIRA)
Devaraj K created SPARK-19354: - Summary: Killed tasks are getting marked as FAILED Key: SPARK-19354 URL: https://issues.apache.org/jira/browse/SPARK-19354 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-19059) Unable to retrieve data from a parquet table whose name starts with underscore

2017-01-03 Thread Remon K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15796394#comment-15796394 ] Remon K commented on SPARK-19059: - My apologies, yes, I had my SPARK_HOME set to a different path/version

[jira] [Commented] (SPARK-19059) Unable to retrieve data from a parquet table whose name starts with underscore

2017-01-03 Thread Remon K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15796213#comment-15796213 ] Remon K commented on SPARK-19059: - I was not able to reproduce it {code} spark-2.1.0-bin-hadoop2.7$

[jira] [Commented] (SPARK-15359) Mesos dispatcher should handle DRIVER_ABORTED status from mesosDriver.run()

2016-12-29 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786878#comment-15786878 ] Devaraj K commented on SPARK-15359: --- Thanks [~yu2003w] for verifying this PR, I forgot to mention that

[jira] [Commented] (SPARK-15359) Mesos dispatcher should handle DRIVER_ABORTED status from mesosDriver.run()

2016-12-28 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15783821#comment-15783821 ] Devaraj K commented on SPARK-15359: --- [~yu2003w], seems you are also facing the same issue which I

[jira] [Created] (SPARK-18784) Managed memory leak - spark-2.0.2

2016-12-08 Thread Appu K (JIRA)
Appu K created SPARK-18784: -- Summary: Managed memory leak - spark-2.0.2 Key: SPARK-18784 URL: https://issues.apache.org/jira/browse/SPARK-18784 Project: Spark Issue Type: Bug Affects Versions:

[jira] [Created] (SPARK-18524) Cannot create dataframe on jdbc data source from spark 2.0.2

2016-11-21 Thread Som K (JIRA)
Som K created SPARK-18524: - Summary: Cannot create dataframe on jdbc data source from spark 2.0.2 Key: SPARK-18524 URL: https://issues.apache.org/jira/browse/SPARK-18524 Project: Spark Issue Type:

[jira] [Commented] (SPARK-16845) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB

2016-10-12 Thread K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569265#comment-15569265 ] K commented on SPARK-16845: --- Code and data are also here as well.

[jira] [Commented] (SPARK-16845) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB

2016-10-11 Thread K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15566741#comment-15566741 ] K commented on SPARK-16845: --- We manually wrote parts that were throwing errors (StringIndexer and

[jira] [Commented] (SPARK-17688) xception in thread "main" java.lang.NoSuchMethodError: javax.servlet.ServletContext.getInitParameter(Ljava/lang/String;)Ljava/lang/String;

2016-09-28 Thread Kishore K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529569#comment-15529569 ] Kishore K commented on SPARK-17688: --- Sean Owen - Thank you, you were correct about API conflicting. I

[jira] [Commented] (SPARK-17688) xception in thread "main" java.lang.NoSuchMethodError: javax.servlet.ServletContext.getInitParameter(Ljava/lang/String;)Ljava/lang/String;

2016-09-28 Thread Kishore K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529245#comment-15529245 ] Kishore K commented on SPARK-17688: --- Sean Owen - Thanks for responding. Below is my application

[jira] [Updated] (SPARK-17688) xception in thread "main" java.lang.NoSuchMethodError: javax.servlet.ServletContext.getInitParameter(Ljava/lang/String;)Ljava/lang/String;

2016-09-27 Thread Kishore K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kishore K updated SPARK-17688: -- Description: SparkConf conf = new

[jira] [Created] (SPARK-17688) Spark Config issue

2016-09-27 Thread Kishore K (JIRA)
Kishore K created SPARK-17688: - Summary: Spark Config issue Key: SPARK-17688 URL: https://issues.apache.org/jira/browse/SPARK-17688 Project: Spark Issue Type: Question Components: Java

  1   2   >