[jira] [Commented] (SPARK-26961) Found Java-level deadlock in Spark Driver

2021-01-26 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272272#comment-17272272 ] Ajith S commented on SPARK-26961: - Scala issue : https://github.com/scala/bug/issues/114

[jira] [Created] (SPARK-30621) Dynamic Pruning thread propagates the localProperties to task

2020-01-23 Thread Ajith S (Jira)
Ajith S created SPARK-30621: --- Summary: Dynamic Pruning thread propagates the localProperties to task Key: SPARK-30621 URL: https://issues.apache.org/jira/browse/SPARK-30621 Project: Spark Issue Ty

[jira] [Commented] (SPARK-30556) Copy sparkContext.localproperties to child thread inSubqueryExec.executionContext

2020-01-23 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021937#comment-17021937 ] Ajith S commented on SPARK-30556: - Yes, it exist in lower version like 2.3.x too > Copy

[jira] [Commented] (SPARK-30556) Copy sparkContext.localproperties to child thread inSubqueryExec.executionContext

2020-01-23 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021936#comment-17021936 ] Ajith S commented on SPARK-30556: - Raised backport PR for branch 2.4 [https://github.com

[jira] [Updated] (SPARK-30360) Avoid Redact classpath entries in History Server UI

2020-01-22 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-30360: Description: Currently SPARK history server display the classpath entries in the Environment tab with clas

[jira] [Updated] (SPARK-30360) Avoid Redact classpath entries in History Server UI

2020-01-22 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-30360: Summary: Avoid Redact classpath entries in History Server UI (was: Redact classpath entries in Spark UI)

[jira] [Updated] (SPARK-22590) Broadcast thread propagates the localProperties to task

2020-01-17 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-22590: Affects Version/s: 3.0.0 2.4.4 > Broadcast thread propagates the localProperties to

[jira] [Created] (SPARK-30556) SubqueryExec passes local properties to SubqueryExec.executionContext

2020-01-17 Thread Ajith S (Jira)
Ajith S created SPARK-30556: --- Summary: SubqueryExec passes local properties to SubqueryExec.executionContext Key: SPARK-30556 URL: https://issues.apache.org/jira/browse/SPARK-30556 Project: Spark

[jira] [Updated] (SPARK-22590) Broadcast thread propagates the localProperties to task

2020-01-17 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-22590: Summary: Broadcast thread propagates the localProperties to task (was: SparkContext's local properties mi

[jira] [Updated] (SPARK-22590) SparkContext's local properties missing from TaskContext properties

2020-01-17 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-22590: Description: Local properties set via sparkContext are not available as TaskContext properties when execu

[jira] [Reopened] (SPARK-22590) SparkContext's local properties missing from TaskContext properties

2020-01-17 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S reopened SPARK-22590: - Adding Fix > SparkContext's local properties missing from TaskContext properties >

[jira] [Reopened] (SPARK-23626) DAGScheduler blocked due to JobSubmitted event

2020-01-16 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S reopened SPARK-23626: - Old PR was closed due to inactivity. Reopening with a new PR and hence to conclude > DAGScheduler blocked

[jira] [Comment Edited] (SPARK-30517) Support SHOW TABLES EXTENDED

2020-01-15 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17015706#comment-17015706 ] Ajith S edited comment on SPARK-30517 at 1/15/20 8:01 AM: -- [~sr

[jira] [Commented] (SPARK-30517) Support SHOW TABLES EXTENDED

2020-01-15 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17015706#comment-17015706 ] Ajith S commented on SPARK-30517: - [~srowen] [~dongjoon] [~vanzin] Please let me know ab

[jira] [Created] (SPARK-30517) Support SHOW TABLES EXTENDED

2020-01-14 Thread Ajith S (Jira)
Ajith S created SPARK-30517: --- Summary: Support SHOW TABLES EXTENDED Key: SPARK-30517 URL: https://issues.apache.org/jira/browse/SPARK-30517 Project: Spark Issue Type: New Feature Componen

[jira] [Comment Edited] (SPARK-30484) Job History Storage Tab does not display RDD Table

2020-01-11 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17013656#comment-17013656 ] Ajith S edited comment on SPARK-30484 at 1/12/20 6:36 AM: -- This

[jira] [Commented] (SPARK-30484) Job History Storage Tab does not display RDD Table

2020-01-11 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17013656#comment-17013656 ] Ajith S commented on SPARK-30484: - As SparkListenerBlockUpdated is filtered by default f

[jira] [Commented] (SPARK-30488) Deadlock between block-manager-slave-async-thread-pool and spark context cleaner

2020-01-11 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17013651#comment-17013651 ] Ajith S commented on SPARK-30488: - As from log, i see `sbt` classes  in the deadlock thr

[jira] [Commented] (SPARK-30440) Flaky test: org.apache.spark.scheduler.TaskSetManagerSuite.reset

2020-01-06 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17009433#comment-17009433 ] Ajith S commented on SPARK-30440: - Found a race between reviveOffers in org.apache.spar

[jira] [Created] (SPARK-30406) OneForOneStreamManager use AtomicLong

2020-01-01 Thread Ajith S (Jira)
Ajith S created SPARK-30406: --- Summary: OneForOneStreamManager use AtomicLong Key: SPARK-30406 URL: https://issues.apache.org/jira/browse/SPARK-30406 Project: Spark Issue Type: Bug Compone

[jira] [Created] (SPARK-30405) ArrayKeyIndexType should use Arrays.hashCode

2020-01-01 Thread Ajith S (Jira)
Ajith S created SPARK-30405: --- Summary: ArrayKeyIndexType should use Arrays.hashCode Key: SPARK-30405 URL: https://issues.apache.org/jira/browse/SPARK-30405 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-30382) start-thriftserver throws ClassNotFoundException

2019-12-29 Thread Ajith S (Jira)
Ajith S created SPARK-30382: --- Summary: start-thriftserver throws ClassNotFoundException Key: SPARK-30382 URL: https://issues.apache.org/jira/browse/SPARK-30382 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-25061) Spark SQL Thrift Server fails to not pick up hiveconf passing parameter

2019-12-29 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17004736#comment-17004736 ] Ajith S commented on SPARK-25061: - I could reproduce this and as per documentation, htt

[jira] [Commented] (SPARK-25030) SparkSubmit.doSubmit will not return result if the mainClass submitted creates a Timer()

2019-12-26 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17003932#comment-17003932 ] Ajith S commented on SPARK-25030: - [~jiangxb1987] is there a reproduce case for this.? i

[jira] [Created] (SPARK-30361) Monitoring URL do not redact information about environment

2019-12-26 Thread Ajith S (Jira)
Ajith S created SPARK-30361: --- Summary: Monitoring URL do not redact information about environment Key: SPARK-30361 URL: https://issues.apache.org/jira/browse/SPARK-30361 Project: Spark Issue Type:

[jira] [Created] (SPARK-30360) Redact classpath entries in Spark UI

2019-12-26 Thread Ajith S (Jira)
Ajith S created SPARK-30360: --- Summary: Redact classpath entries in Spark UI Key: SPARK-30360 URL: https://issues.apache.org/jira/browse/SPARK-30360 Project: Spark Issue Type: Bug Componen

[jira] [Commented] (SPARK-27719) Set maxDisplayLogSize for spark history server

2019-11-29 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16985021#comment-16985021 ] Ajith S commented on SPARK-27719: - Currently our production also encounter this issue, I

[jira] [Commented] (SPARK-29174) LOCAL is not supported in INSERT OVERWRITE DIRECTORY to data source

2019-09-19 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16933203#comment-16933203 ] Ajith S commented on SPARK-29174: - Had checked with original author for this https://git

[jira] [Resolved] (SPARK-28848) insert overwrite local directory stored as parquet does not creates snappy.parquet data file at local directory path

2019-08-22 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S resolved SPARK-28848. - Resolution: Duplicate Will be fixed as part of SPARK-28659 > insert overwrite local directory stored a

[jira] [Commented] (SPARK-28848) insert overwrite local directory stored as parquet does not creates snappy.parquet data file at local directory path

2019-08-21 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913027#comment-16913027 ] Ajith S commented on SPARK-28848: - Thanks for reporting. Will look into this. On initial

[jira] [Commented] (SPARK-28726) Spark with DynamicAllocation always got connect rest by peers

2019-08-14 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907112#comment-16907112 ] Ajith S commented on SPARK-28726: - As i see, this is driver trying to clean up RDDs, bro

[jira] [Commented] (SPARK-28696) create database _; allowing in Spark but Hive throws Parse Exception

2019-08-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16906881#comment-16906881 ] Ajith S commented on SPARK-28696: - [~hyukjin.kwon] okay, should we disallow tables names

[jira] [Commented] (SPARK-28696) create database _; allowing in Spark but Hive throws Parse Exception

2019-08-12 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905196#comment-16905196 ] Ajith S commented on SPARK-28696: - I think we should disallow if a identifier starts wit

[jira] [Comment Edited] (SPARK-28697) select * from _; throws InvalidInputException and says path does not exists at HDFS side

2019-08-12 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905144#comment-16905144 ] Ajith S edited comment on SPARK-28697 at 8/12/19 1:28 PM: -- !sc

[jira] [Comment Edited] (SPARK-28697) select * from _; throws InvalidInputException and says path does not exists at HDFS side

2019-08-12 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905144#comment-16905144 ] Ajith S edited comment on SPARK-28697 at 8/12/19 12:44 PM: --- !

[jira] [Commented] (SPARK-28697) select * from _; throws InvalidInputException and says path does not exists at HDFS side

2019-08-12 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905144#comment-16905144 ] Ajith S commented on SPARK-28697: - !screenshot-1.png! Here due to org.apache.hadoop.

[jira] [Updated] (SPARK-28697) select * from _; throws InvalidInputException and says path does not exists at HDFS side

2019-08-12 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-28697: Attachment: screenshot-1.png > select * from _; throws InvalidInputException and says path does not exist

[jira] [Comment Edited] (SPARK-28697) select * from _; throws InvalidInputException and says path does not exists at HDFS side

2019-08-12 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905138#comment-16905138 ] Ajith S edited comment on SPARK-28697 at 8/12/19 12:35 PM: --- Fo

[jira] [Comment Edited] (SPARK-28697) select * from _; throws InvalidInputException and says path does not exists at HDFS side

2019-08-12 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905138#comment-16905138 ] Ajith S edited comment on SPARK-28697 at 8/12/19 12:35 PM: --- Fo

[jira] [Commented] (SPARK-28697) select * from _; throws InvalidInputException and says path does not exists at HDFS side

2019-08-12 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905138#comment-16905138 ] Ajith S commented on SPARK-28697: - Found this even in single node, local filesystem case

[jira] [Commented] (SPARK-28696) create database _; allowing in Spark but Hive throws Parse Exception

2019-08-12 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905127#comment-16905127 ] Ajith S commented on SPARK-28696: - Thanks for reporting. I can see this due to https://

[jira] [Created] (SPARK-28676) Avoid Excessive logging from ContextCleaner

2019-08-09 Thread Ajith S (JIRA)
Ajith S created SPARK-28676: --- Summary: Avoid Excessive logging from ContextCleaner Key: SPARK-28676 URL: https://issues.apache.org/jira/browse/SPARK-28676 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-23626) DAGScheduler blocked due to JobSubmitted event

2019-05-31 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-23626: Affects Version/s: 2.4.3 > DAGScheduler blocked due to JobSubmitted event > -

[jira] [Updated] (SPARK-23626) DAGScheduler blocked due to JobSubmitted event

2019-05-31 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-23626: Summary: DAGScheduler blocked due to JobSubmitted event (was: Spark DAGScheduler scheduling performance

[jira] [Updated] (SPARK-23626) Spark DAGScheduler scheduling performance hindered on JobSubmitted Event

2019-05-31 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-23626: Labels: (was: bulk-closed) > Spark DAGScheduler scheduling performance hindered on JobSubmitted Event >

[jira] [Updated] (SPARK-27907) HiveUDAF with 0 rows throw NPE

2019-05-31 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27907: Description: When query returns zero rows, the HiveUDAFFunction throws NPE CASE 1: create table abc(a int

[jira] [Updated] (SPARK-27907) HiveUDAF with 0 rows throw NPE

2019-05-31 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27907: Summary: HiveUDAF with 0 rows throw NPE (was: HiveUDAF with 0 rows throw NPE when try to serialize) > Hi

[jira] [Created] (SPARK-27907) HiveUDAF with 0 rows throw NPE when try to serialize

2019-05-31 Thread Ajith S (JIRA)
Ajith S created SPARK-27907: --- Summary: HiveUDAF with 0 rows throw NPE when try to serialize Key: SPARK-27907 URL: https://issues.apache.org/jira/browse/SPARK-27907 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-23626) Spark DAGScheduler scheduling performance hindered on JobSubmitted Event

2019-05-21 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-23626: Affects Version/s: 3.0.0 > Spark DAGScheduler scheduling performance hindered on JobSubmitted Event >

[jira] [Updated] (SPARK-23626) Spark DAGScheduler scheduling performance hindered on JobSubmitted Event

2019-05-21 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-23626: Affects Version/s: 2.3.3 > Spark DAGScheduler scheduling performance hindered on JobSubmitted Event >

[jira] [Reopened] (SPARK-23626) Spark DAGScheduler scheduling performance hindered on JobSubmitted Event

2019-05-21 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S reopened SPARK-23626: - Resolution in progress > Spark DAGScheduler scheduling performance hindered on JobSubmitted Event > ---

[jira] [Commented] (SPARK-27264) spark sql released all executor but the job is not done

2019-03-24 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16800323#comment-16800323 ] Ajith S commented on SPARK-27264: - Can you add a snippet to reproduce this.? > spark sq

[jira] [Commented] (SPARK-27219) Misleading exceptions in transport code's SASL fallback path

2019-03-20 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16797435#comment-16797435 ] Ajith S commented on SPARK-27219: - So do we just log a simple warn with one line message

[jira] [Commented] (SPARK-27220) Remove Yarn specific leftover from CoarseGrainedSchedulerBackend

2019-03-20 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16797428#comment-16797428 ] Ajith S commented on SPARK-27220: - # About making *currentExecutorIdCounter* datatype co

[jira] [Commented] (SPARK-27194) Job failures when task attempts do not clean up spark-staging parquet files

2019-03-20 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16797241#comment-16797241 ] Ajith S commented on SPARK-27194: - Hi [~dongjoon] , i have some analysis here : [https:

[jira] [Comment Edited] (SPARK-27194) Job failures when task attempts do not clean up spark-staging parquet files

2019-03-20 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16797216#comment-16797216 ] Ajith S edited comment on SPARK-27194 at 3/20/19 2:38 PM: -- [~do

[jira] [Commented] (SPARK-27194) Job failures when task attempts do not clean up spark-staging parquet files

2019-03-20 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16797216#comment-16797216 ] Ajith S commented on SPARK-27194: - [~dongjoon] Yes i tried with spark 2.3.3, and the iss

[jira] [Created] (SPARK-27200) History Environment tab must sort Configurations/Properties by default

2019-03-19 Thread Ajith S (JIRA)
Ajith S created SPARK-27200: --- Summary: History Environment tab must sort Configurations/Properties by default Key: SPARK-27200 URL: https://issues.apache.org/jira/browse/SPARK-27200 Project: Spark

[jira] [Commented] (SPARK-27198) Heartbeat interval mismatch in driver and executor

2019-03-18 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16795666#comment-16795666 ] Ajith S commented on SPARK-27198: - will be working on this > Heartbeat interval mismatc

[jira] [Created] (SPARK-27198) Heartbeat interval mismatch in driver and executor

2019-03-18 Thread Ajith S (JIRA)
Ajith S created SPARK-27198: --- Summary: Heartbeat interval mismatch in driver and executor Key: SPARK-27198 URL: https://issues.apache.org/jira/browse/SPARK-27198 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-27194) Job failures when task attempts do not clean up spark-staging parquet files

2019-03-18 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16795575#comment-16795575 ] Ajith S commented on SPARK-27194: - Currently looks like from logs the file name for task

[jira] [Commented] (SPARK-26961) Found Java-level deadlock in Spark Driver

2019-03-17 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16794551#comment-16794551 ] Ajith S commented on SPARK-26961: - [~srowen] ok, will raise a PR for this. Thanks > Fou

[jira] [Commented] (SPARK-27142) Provide REST API for SQL level information

2019-03-17 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16794519#comment-16794519 ] Ajith S commented on SPARK-27142: - [~srowen] any inputs about proposal.? > Provide REST

[jira] [Commented] (SPARK-27164) RDD.countApprox on empty RDDs schedules jobs which never complete

2019-03-14 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16793222#comment-16793222 ] Ajith S commented on SPARK-27164: - i will be working on this > RDD.countApprox on empty

[jira] [Commented] (SPARK-27122) YARN test failures in Java 9+

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792355#comment-16792355 ] Ajith S commented on SPARK-27122: - ping [~srowen] [~dongjoon] [~Gengliang.Wang] > YARN

[jira] [Comment Edited] (SPARK-27122) YARN test failures in Java 9+

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792318#comment-16792318 ] Ajith S edited comment on SPARK-27122 at 3/14/19 4:15 AM: -- The

[jira] [Comment Edited] (SPARK-27122) YARN test failures in Java 9+

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792318#comment-16792318 ] Ajith S edited comment on SPARK-27122 at 3/14/19 4:14 AM: -- The

[jira] [Comment Edited] (SPARK-27122) YARN test failures in Java 9+

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792318#comment-16792318 ] Ajith S edited comment on SPARK-27122 at 3/14/19 4:07 AM: -- The

[jira] [Comment Edited] (SPARK-27122) YARN test failures in Java 9+

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792318#comment-16792318 ] Ajith S edited comment on SPARK-27122 at 3/14/19 4:06 AM: -- The

[jira] [Updated] (SPARK-27122) YARN test failures in Java 9+

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27122: Attachment: image-2019-03-14-09-35-23-046.png > YARN test failures in Java 9+ > --

[jira] [Commented] (SPARK-27122) YARN test failures in Java 9+

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792318#comment-16792318 ] Ajith S commented on SPARK-27122: - The problem seems to be shading of jetty package. Wh

[jira] [Updated] (SPARK-27122) YARN test failures in Java 9+

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27122: Attachment: image-2019-03-14-09-34-20-592.png > YARN test failures in Java 9+ > --

[jira] [Commented] (SPARK-27122) YARN test failures in Java 9+

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792299#comment-16792299 ] Ajith S commented on SPARK-27122: - I can reproduce this issue even in Java8. I would lik

[jira] [Comment Edited] (SPARK-26961) Found Java-level deadlock in Spark Driver

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791751#comment-16791751 ] Ajith S edited comment on SPARK-26961 at 3/13/19 2:37 PM: -- 1) Y

[jira] [Updated] (SPARK-27142) Provide REST API for SQL level information

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27142: Description: Currently for Monitoring Spark application SQL information is not available from REST but on

[jira] [Comment Edited] (SPARK-26961) Found Java-level deadlock in Spark Driver

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791751#comment-16791751 ] Ajith S edited comment on SPARK-26961 at 3/13/19 2:32 PM: -- 1) Y

[jira] [Commented] (SPARK-26961) Found Java-level deadlock in Spark Driver

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791751#comment-16791751 ] Ajith S commented on SPARK-26961: - Yes, the registerAsParallelCapable will return true,

[jira] [Updated] (SPARK-26961) Found Java-level deadlock in Spark Driver

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-26961: Attachment: image-2019-03-13-19-53-52-390.png > Found Java-level deadlock in Spark Driver > --

[jira] [Updated] (SPARK-26961) Found Java-level deadlock in Spark Driver

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-26961: Attachment: (was: image-2019-03-13-19-51-38-708.png) > Found Java-level deadlock in Spark Driver > ---

[jira] [Updated] (SPARK-26961) Found Java-level deadlock in Spark Driver

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-26961: Attachment: image-2019-03-13-19-51-38-708.png > Found Java-level deadlock in Spark Driver > --

[jira] [Updated] (SPARK-27142) Provide REST API for SQL level information

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27142: Attachment: image-2019-03-13-19-29-26-896.png > Provide REST API for SQL level information > -

[jira] [Commented] (SPARK-27142) Provide REST API for SQL level information

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791728#comment-16791728 ] Ajith S commented on SPARK-27142: - Ok apologies for being abstract about this requiremen

[jira] [Updated] (SPARK-27142) Provide REST API for SQL level information

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27142: Attachment: (was: image-2019-03-13-19-19-27-831.png) > Provide REST API for SQL level information > --

[jira] [Updated] (SPARK-27142) Provide REST API for SQL level information

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27142: Attachment: (was: image-2019-03-13-19-19-24-951.png) > Provide REST API for SQL level information > --

[jira] [Updated] (SPARK-27142) Provide REST API for SQL level information

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27142: Attachment: image-2019-03-13-19-19-24-951.png > Provide REST API for SQL level information > -

[jira] [Updated] (SPARK-27142) Provide REST API for SQL level information

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27142: Attachment: image-2019-03-13-19-19-27-831.png > Provide REST API for SQL level information > -

[jira] [Commented] (SPARK-26961) Found Java-level deadlock in Spark Driver

2019-03-13 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791712#comment-16791712 ] Ajith S commented on SPARK-26961: - [~srowen] That too will not work Here is my custom c

[jira] [Commented] (SPARK-27143) Provide REST API for JDBC/ODBC level information

2019-03-12 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791272#comment-16791272 ] Ajith S commented on SPARK-27143: - ping [~srowen] [~cloud_fan] [~dongjoon]  Please sugg

[jira] [Commented] (SPARK-27143) Provide REST API for JDBC/ODBC level information

2019-03-12 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791270#comment-16791270 ] Ajith S commented on SPARK-27143: - I will be working on this > Provide REST API for JDB

[jira] [Created] (SPARK-27143) Provide REST API for JDBC/ODBC level information

2019-03-12 Thread Ajith S (JIRA)
Ajith S created SPARK-27143: --- Summary: Provide REST API for JDBC/ODBC level information Key: SPARK-27143 URL: https://issues.apache.org/jira/browse/SPARK-27143 Project: Spark Issue Type: New Featur

[jira] [Commented] (SPARK-27142) Provide REST API for SQL level information

2019-03-12 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791269#comment-16791269 ] Ajith S commented on SPARK-27142: - I will be working on this > Provide REST API for SQL

[jira] [Created] (SPARK-27142) Provide REST API for SQL level information

2019-03-12 Thread Ajith S (JIRA)
Ajith S created SPARK-27142: --- Summary: Provide REST API for SQL level information Key: SPARK-27142 URL: https://issues.apache.org/jira/browse/SPARK-27142 Project: Spark Issue Type: New Feature

[jira] [Comment Edited] (SPARK-26961) Found Java-level deadlock in Spark Driver

2019-03-11 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790208#comment-16790208 ] Ajith S edited comment on SPARK-26961 at 3/12/19 6:45 AM: -- [~sr

[jira] [Comment Edited] (SPARK-26961) Found Java-level deadlock in Spark Driver

2019-03-11 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790208#comment-16790208 ] Ajith S edited comment on SPARK-26961 at 3/12/19 6:43 AM: -- [~sr

[jira] [Commented] (SPARK-27011) reset command fails after cache table

2019-03-11 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790220#comment-16790220 ] Ajith S commented on SPARK-27011: - @ [~cloud_fan] As [https://github.com/apache/spark/pu

[jira] [Commented] (SPARK-26961) Found Java-level deadlock in Spark Driver

2019-03-11 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790208#comment-16790208 ] Ajith S commented on SPARK-26961: - [~srowen] Yes. I too have same opinion of fixing it v

[jira] [Commented] (SPARK-27114) SQL Tab shows duplicate executions for some commands

2019-03-11 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790136#comment-16790136 ] Ajith S commented on SPARK-27114: - [~srowen] as *LocalRelation* is eagerly evaluated hen

[jira] [Updated] (SPARK-26152) Flaky test: BroadcastSuite

2019-03-11 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-26152: Attachment: Screenshot from 2019-03-11 17-03-40.png > Flaky test: BroadcastSuite > ---

[jira] [Comment Edited] (SPARK-26152) Flaky test: BroadcastSuite

2019-03-11 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16789462#comment-16789462 ] Ajith S edited comment on SPARK-26152 at 3/11/19 12:07 PM: --- I

[jira] [Comment Edited] (SPARK-26152) Flaky test: BroadcastSuite

2019-03-11 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16789462#comment-16789462 ] Ajith S edited comment on SPARK-26152 at 3/11/19 12:07 PM: --- I

  1   2   >