[jira] [Commented] (SPARK-23722) Add support for inserting empty map or array to table

2018-03-19 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405181#comment-16405181 ] Sital Kedia commented on SPARK-23722: - Actually this is a duplicate of SPARK-21281 an

[jira] [Resolved] (SPARK-23722) Add support for inserting empty map or array to table

2018-03-19 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia resolved SPARK-23722. - Resolution: Duplicate > Add support for inserting empty map or array to table > -

[jira] [Created] (SPARK-23722) Add support for inserting empty map or array to table

2018-03-16 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-23722: --- Summary: Add support for inserting empty map or array to table Key: SPARK-23722 URL: https://issues.apache.org/jira/browse/SPARK-23722 Project: Spark Issue Typ

[jira] [Commented] (SPARK-18134) SQL: MapType in Group BY and Joins not working

2018-03-11 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394571#comment-16394571 ] Sital Kedia commented on SPARK-18134: - [~hvanhovell]- What is the state of this JIRA?

[jira] [Commented] (SPARK-23502) Support async init of spark context during spark-shell startup

2018-02-28 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16380942#comment-16380942 ] Sital Kedia commented on SPARK-23502: - >> what happens when you operate on {{sc}} bef

[jira] [Commented] (SPARK-23502) Support async init of spark context during spark-shell startup

2018-02-23 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16375051#comment-16375051 ] Sital Kedia commented on SPARK-23502: - I realized that we are printing the web url li

[jira] [Created] (SPARK-23502) Support async init of spark context during spark-shell startup

2018-02-23 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-23502: --- Summary: Support async init of spark context during spark-shell startup Key: SPARK-23502 URL: https://issues.apache.org/jira/browse/SPARK-23502 Project: Spark

[jira] [Commented] (SPARK-23310) Perf regression introduced by SPARK-21113

2018-02-02 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16351097#comment-16351097 ] Sital Kedia commented on SPARK-23310: - https://github.com/apache/spark/pull/20492 >

[jira] [Commented] (SPARK-23310) Perf regression introduced by SPARK-21113

2018-02-02 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350875#comment-16350875 ] Sital Kedia commented on SPARK-23310: - [~yhuai] - Sorry about introducing the regress

[jira] [Created] (SPARK-22827) Avoid throwing OutOfMemoryError in case of exception in spill

2017-12-18 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-22827: --- Summary: Avoid throwing OutOfMemoryError in case of exception in spill Key: SPARK-22827 URL: https://issues.apache.org/jira/browse/SPARK-22827 Project: Spark

[jira] [Created] (SPARK-22312) Spark job stuck with no executor due to bug in Executor Allocation Manager

2017-10-18 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-22312: --- Summary: Spark job stuck with no executor due to bug in Executor Allocation Manager Key: SPARK-22312 URL: https://issues.apache.org/jira/browse/SPARK-22312 Project: Spa

[jira] [Commented] (SPARK-21867) Support async spilling in UnsafeShuffleWriter

2017-09-12 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16163705#comment-16163705 ] Sital Kedia commented on SPARK-21867: - [~rxin] - You are right, it is very tricky to

[jira] [Commented] (SPARK-21867) Support async spilling in UnsafeShuffleWriter

2017-08-29 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16146060#comment-16146060 ] Sital Kedia commented on SPARK-21867: - cc - [~rxin], [~joshrosen], [~sameer] - What d

[jira] [Created] (SPARK-21867) Support async spilling in UnsafeShuffleWriter

2017-08-29 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-21867: --- Summary: Support async spilling in UnsafeShuffleWriter Key: SPARK-21867 URL: https://issues.apache.org/jira/browse/SPARK-21867 Project: Spark Issue Type: Impro

[jira] [Created] (SPARK-21834) Incorrect executor request in case of dynamic allocation

2017-08-24 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-21834: --- Summary: Incorrect executor request in case of dynamic allocation Key: SPARK-21834 URL: https://issues.apache.org/jira/browse/SPARK-21834 Project: Spark Issue

[jira] [Closed] (SPARK-21833) CoarseGrainedSchedulerBackend leaks executors in case of dynamic allocation

2017-08-24 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia closed SPARK-21833. --- Resolution: Duplicate Duplicate of SPARK-20540 > CoarseGrainedSchedulerBackend leaks executors in ca

[jira] [Updated] (SPARK-21833) CoarseGrainedSchedulerBackend leaks executors in case of dynamic allocation

2017-08-24 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-21833: Description: We have seen this issue in coarse grained scheduler that in case of dynamic executor

[jira] [Commented] (SPARK-21833) CoarseGrainedSchedulerBackend leaks executors in case of dynamic allocation

2017-08-24 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16140533#comment-16140533 ] Sital Kedia commented on SPARK-21833: - Actually, SPARK-20540 already addressed this i

[jira] [Updated] (SPARK-21833) CoarseGrainedSchedulerBackend leaks executors in case of dynamic allocation

2017-08-24 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-21833: Description: We have seen this issue in coarse grained scheduler that in case of dynamic executor

[jira] [Updated] (SPARK-21833) CoarseGrainedSchedulerBackend leaks executors in case of dynamic allocation

2017-08-24 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-21833: Description: We have seen this issue in coarse grained scheduler that in case of dynamic executor

[jira] [Created] (SPARK-21833) CoarseGrainedSchedulerBackend leaks executors in case of dynamic allocation

2017-08-24 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-21833: --- Summary: CoarseGrainedSchedulerBackend leaks executors in case of dynamic allocation Key: SPARK-21833 URL: https://issues.apache.org/jira/browse/SPARK-21833 Project: Sp

[jira] [Commented] (SPARK-19112) add codec for ZStandard

2017-08-01 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16109784#comment-16109784 ] Sital Kedia commented on SPARK-19112: - [~sowen], [~tgraves] - Using zstd compression

[jira] [Commented] (SPARK-21444) Fetch failure due to node reboot causes job failure

2017-07-17 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16090800#comment-16090800 ] Sital Kedia commented on SPARK-21444: - Any idea how to fix this issue? > Fetch failu

[jira] [Commented] (SPARK-21444) Fetch failure due to node reboot causes job failure

2017-07-17 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16090799#comment-16090799 ] Sital Kedia commented on SPARK-21444: - cc - [~joshrosen] > Fetch failure due to node

[jira] [Created] (SPARK-21444) Fetch failure due to node reboot causes job failure

2017-07-17 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-21444: --- Summary: Fetch failure due to node reboot causes job failure Key: SPARK-21444 URL: https://issues.apache.org/jira/browse/SPARK-21444 Project: Spark Issue Type:

[jira] [Created] (SPARK-21113) Support for read ahead input stream to amortize disk IO cost in the Spill reader

2017-06-15 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-21113: --- Summary: Support for read ahead input stream to amortize disk IO cost in the Spill reader Key: SPARK-21113 URL: https://issues.apache.org/jira/browse/SPARK-21113 Projec

[jira] [Commented] (SPARK-18838) High latency of event processing for large jobs

2017-06-12 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16046957#comment-16046957 ] Sital Kedia commented on SPARK-18838: - [~joshrosen] - The PR for my change to multi-t

[jira] [Commented] (SPARK-20178) Improve Scheduler fetch failures

2017-05-30 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030284#comment-16030284 ] Sital Kedia commented on SPARK-20178: - https://github.com/apache/spark/pull/18150 >

[jira] [Commented] (SPARK-20178) Improve Scheduler fetch failures

2017-05-26 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16027036#comment-16027036 ] Sital Kedia commented on SPARK-20178: - >> So to get the robustness for now I'm fine w

[jira] [Commented] (SPARK-18838) High latency of event processing for large jobs

2017-05-18 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016217#comment-16016217 ] Sital Kedia commented on SPARK-18838: - [~joshrosen] - >> Alternatively, we could use

[jira] [Created] (SPARK-20640) Make rpc timeout and retry for shuffle registration configurable

2017-05-08 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-20640: --- Summary: Make rpc timeout and retry for shuffle registration configurable Key: SPARK-20640 URL: https://issues.apache.org/jira/browse/SPARK-20640 Project: Spark

[jira] [Commented] (SPARK-20178) Improve Scheduler fetch failures

2017-04-05 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15957341#comment-15957341 ] Sital Kedia commented on SPARK-20178: - [~tgraves] Thanks for creating the JIRA and dr

[jira] [Closed] (SPARK-20163) Kill all running tasks in a stage in case of fetch failure

2017-03-31 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia closed SPARK-20163. --- Resolution: Duplicate > Kill all running tasks in a stage in case of fetch failure >

[jira] [Commented] (SPARK-20163) Kill all running tasks in a stage in case of fetch failure

2017-03-31 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15951738#comment-15951738 ] Sital Kedia commented on SPARK-20163: - Thanks [~imranr], closing this as this is dupl

[jira] [Created] (SPARK-20163) Kill all running tasks in a stage in case of fetch failure

2017-03-30 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-20163: --- Summary: Kill all running tasks in a stage in case of fetch failure Key: SPARK-20163 URL: https://issues.apache.org/jira/browse/SPARK-20163 Project: Spark Issu

[jira] [Commented] (SPARK-3577) Add task metric to report spill time

2017-03-29 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947669#comment-15947669 ] Sital Kedia commented on SPARK-3577: I am making a change to report correct spill data

[jira] [Created] (SPARK-20091) DagScheduler should allow running concurrent attempts of a stage in case of multiple fetch failure

2017-03-24 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-20091: --- Summary: DagScheduler should allow running concurrent attempts of a stage in case of multiple fetch failure Key: SPARK-20091 URL: https://issues.apache.org/jira/browse/SPARK-20091

[jira] [Created] (SPARK-20074) Make buffer size in unsafe external sorter configurable

2017-03-23 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-20074: --- Summary: Make buffer size in unsafe external sorter configurable Key: SPARK-20074 URL: https://issues.apache.org/jira/browse/SPARK-20074 Project: Spark Issue T

[jira] [Updated] (SPARK-20014) Optimize mergeSpillsWithFileStream method

2017-03-18 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-20014: Description: When the individual partition size in a spill is small, mergeSpillsWithTransferTo met

[jira] [Created] (SPARK-20014) Optimize mergeSpillsWithFileStream method

2017-03-18 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-20014: --- Summary: Optimize mergeSpillsWithFileStream method Key: SPARK-20014 URL: https://issues.apache.org/jira/browse/SPARK-20014 Project: Spark Issue Type: Improveme

[jira] [Commented] (SPARK-19837) Fetch failure throws a SparkException in SparkHiveWriter

2017-03-06 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897961#comment-15897961 ] Sital Kedia commented on SPARK-19837: - `SparkHiveDynamicPartitionWriterContainer` has

[jira] [Closed] (SPARK-19837) Fetch failure throws a SparkException in SparkHiveWriter

2017-03-06 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia closed SPARK-19837. --- Resolution: Fixed > Fetch failure throws a SparkException in SparkHiveWriter > --

[jira] [Created] (SPARK-19837) Fetch failure throws a SparkException in SparkHiveWriter

2017-03-06 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-19837: --- Summary: Fetch failure throws a SparkException in SparkHiveWriter Key: SPARK-19837 URL: https://issues.apache.org/jira/browse/SPARK-19837 Project: Spark Issue

[jira] [Created] (SPARK-19803) Flaky BlockManagerProactiveReplicationSuite tests

2017-03-02 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-19803: --- Summary: Flaky BlockManagerProactiveReplicationSuite tests Key: SPARK-19803 URL: https://issues.apache.org/jira/browse/SPARK-19803 Project: Spark Issue Type: B

[jira] [Created] (SPARK-19753) Remove all shuffle files on a host in case of slave lost of fetch failure

2017-02-27 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-19753: --- Summary: Remove all shuffle files on a host in case of slave lost of fetch failure Key: SPARK-19753 URL: https://issues.apache.org/jira/browse/SPARK-19753 Project: Spar

[jira] [Commented] (SPARK-18838) High latency of event processing for large jobs

2016-12-15 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15753535#comment-15753535 ] Sital Kedia commented on SPARK-18838: - cc - [~kayousterhout] > High latency of event

[jira] [Updated] (SPARK-18838) High latency of event processing for large jobs

2016-12-15 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-18838: Description: Currently we are observing the issue of very high event processing delay in driver's

[jira] [Commented] (SPARK-18838) High latency of event processing for large jobs

2016-12-14 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15750217#comment-15750217 ] Sital Kedia commented on SPARK-18838: - [~zsxwing] - Its not only the ExecutorAllocati

[jira] [Updated] (SPARK-18838) High latency of event processing for large jobs

2016-12-12 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-18838: Description: Currently we are observing the issue of very high event processing delay in driver's

[jira] [Commented] (SPARK-18838) High latency of event processing for large jobs

2016-12-12 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15744086#comment-15744086 ] Sital Kedia commented on SPARK-18838: - [~rxin], [~zsxwing] - Any thoughts on this?

[jira] [Updated] (SPARK-18838) High latency of event processing for large jobs

2016-12-12 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-18838: Description: Currently we are observing the issue of very high event processing delay in driver's

[jira] [Created] (SPARK-18838) High latency of event processing for large jobs

2016-12-12 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-18838: --- Summary: High latency of event processing for large jobs Key: SPARK-18838 URL: https://issues.apache.org/jira/browse/SPARK-18838 Project: Spark Issue Type: Imp

[jira] [Commented] (SPARK-13510) Shuffle may throw FetchFailedException: Direct buffer memory

2016-11-16 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671415#comment-15671415 ] Sital Kedia commented on SPARK-13510: - [~shenhong] - We are seeing the same issue on

[jira] [Updated] (SPARK-16827) Stop reporting spill metrics as shuffle metrics

2016-10-12 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-16827: Summary: Stop reporting spill metrics as shuffle metrics (was: Query with Join produces excessive

[jira] [Updated] (SPARK-17839) Use Nio's directbuffer instead of BufferedInputStream in order to avoid additional copy from os buffer cache to user buffer

2016-10-10 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-17839: Summary: Use Nio's directbuffer instead of BufferedInputStream in order to avoid additional copy fr

[jira] [Created] (SPARK-17839) UnsafeSorterSpillReader should use Nio's directbuffer to read the spill files in order to avoid additional copy

2016-10-09 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-17839: --- Summary: UnsafeSorterSpillReader should use Nio's directbuffer to read the spill files in order to avoid additional copy Key: SPARK-17839 URL: https://issues.apache.org/jira/browse/

[jira] [Updated] (SPARK-17509) When wrapping catalyst datatype to Hive data type avoid pattern matching

2016-09-12 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-17509: Affects Version/s: 2.0.0 Description: Profiling a job, we saw that patten matching in wrap

[jira] [Created] (SPARK-17509) When wrapping catalyst datatype to Hive data type avoid pattern matching

2016-09-12 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-17509: --- Summary: When wrapping catalyst datatype to Hive data type avoid pattern matching Key: SPARK-17509 URL: https://issues.apache.org/jira/browse/SPARK-17509 Project: Spark

[jira] [Commented] (SPARK-16922) Query with Broadcast Hash join fails due to executor OOM in Spark 2.0

2016-09-06 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15469092#comment-15469092 ] Sital Kedia commented on SPARK-16922: - There is no noticable performance gain I obser

[jira] [Commented] (SPARK-16922) Query with Broadcast Hash join fails due to executor OOM in Spark 2.0

2016-09-06 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15468339#comment-15468339 ] Sital Kedia commented on SPARK-16922: - [~davies] - Thanks for looking into this. I t

[jira] [Commented] (SPARK-16922) Query with Broadcast Hash join fails due to executor OOM in Spark 2.0

2016-09-01 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15456744#comment-15456744 ] Sital Kedia commented on SPARK-16922: - Thanks for the fix [~davies]. I will test this

[jira] [Closed] (SPARK-17164) Query with colon in the table name fails to parse in 2.0

2016-08-22 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia closed SPARK-17164. --- Resolution: Won't Fix > Query with colon in the table name fails to parse in 2.0 > --

[jira] [Commented] (SPARK-17164) Query with colon in the table name fails to parse in 2.0

2016-08-22 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15431031#comment-15431031 ] Sital Kedia commented on SPARK-17164: - Thanks [~rxin], [~hvanhovell], that makes sens

[jira] [Commented] (SPARK-17164) Query with colon in the table name fails to parse in 2.0

2016-08-19 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429193#comment-15429193 ] Sital Kedia commented on SPARK-17164: - cc - [~hvanhovell], [~rxin] > Query with colo

[jira] [Created] (SPARK-17164) Query with colon in the table name fails to parse in 2.0

2016-08-19 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-17164: --- Summary: Query with colon in the table name fails to parse in 2.0 Key: SPARK-17164 URL: https://issues.apache.org/jira/browse/SPARK-17164 Project: Spark Issue

[jira] [Commented] (SPARK-16922) Query with Broadcast Hash join fails due to executor OOM in Spark 2.0

2016-08-18 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427429#comment-15427429 ] Sital Kedia commented on SPARK-16922: - Kryo > Query with Broadcast Hash join fails

[jira] [Commented] (SPARK-16922) Query with Broadcast Hash join fails due to executor OOM in Spark 2.0

2016-08-18 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427259#comment-15427259 ] Sital Kedia commented on SPARK-16922: - >> Could you also try to disable the dense mod

[jira] [Commented] (SPARK-16922) Query with Broadcast Hash join fails due to executor OOM in Spark 2.0

2016-08-18 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427250#comment-15427250 ] Sital Kedia commented on SPARK-16922: - The failure is deterministic, we are reproduci

[jira] [Updated] (SPARK-17113) Job failure due to Executor OOM in offheap mode

2016-08-17 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-17113: Summary: Job failure due to Executor OOM in offheap mode (was: Job failure due to Executor OOM) >

[jira] [Commented] (SPARK-17113) Job failure due to Executor OOM

2016-08-17 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425216#comment-15425216 ] Sital Kedia commented on SPARK-17113: - cc - [~davies] > Job failure due to Executor

[jira] [Created] (SPARK-17113) Job failure due to Executor OOM

2016-08-17 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-17113: --- Summary: Job failure due to Executor OOM Key: SPARK-17113 URL: https://issues.apache.org/jira/browse/SPARK-17113 Project: Spark Issue Type: Bug Affects Ver

[jira] [Commented] (SPARK-16922) Query with Broadcast Hash join fails due to executor OOM in Spark 2.0

2016-08-15 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421524#comment-15421524 ] Sital Kedia commented on SPARK-16922: - Yes, I have the above mentioned PR as well.

[jira] [Commented] (SPARK-16922) Query with Broadcast Hash join fails due to executor OOM in Spark 2.0

2016-08-12 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419458#comment-15419458 ] Sital Kedia commented on SPARK-16922: - I am using the fix in https://github.com/apach

[jira] [Updated] (SPARK-16922) Query with Broadcast Hash join fails due to executor OOM in Spark 2.0

2016-08-12 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-16922: Summary: Query with Broadcast Hash join fails due to executor OOM in Spark 2.0 (was: Query failure

[jira] [Comment Edited] (SPARK-16922) Query failure due to executor OOM in Spark 2.0

2016-08-12 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419383#comment-15419383 ] Sital Kedia edited comment on SPARK-16922 at 8/12/16 8:06 PM: -

[jira] [Commented] (SPARK-16922) Query failure due to executor OOM in Spark 2.0

2016-08-12 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419383#comment-15419383 ] Sital Kedia commented on SPARK-16922: - I found that the regression was introduced in

[jira] [Commented] (SPARK-16922) Query failure due to executor OOM in Spark 2.0

2016-08-06 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410703#comment-15410703 ] Sital Kedia commented on SPARK-16922: - Update - The query works fine when Broadcast h

[jira] [Commented] (SPARK-16922) Query failure due to executor OOM in Spark 2.0

2016-08-05 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409878#comment-15409878 ] Sital Kedia commented on SPARK-16922: - PS - Rerunning the query with spark.sql.codege

[jira] [Commented] (SPARK-16922) Query failure due to executor OOM in Spark 2.0

2016-08-05 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409813#comment-15409813 ] Sital Kedia commented on SPARK-16922: - cc - [~rxin] > Query failure due to executor

[jira] [Created] (SPARK-16922) Query failure due to executor OOM in Spark 2.0

2016-08-05 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-16922: --- Summary: Query failure due to executor OOM in Spark 2.0 Key: SPARK-16922 URL: https://issues.apache.org/jira/browse/SPARK-16922 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-16827) Query with Join produces excessive amount of shuffle data

2016-08-01 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15402461#comment-15402461 ] Sital Kedia commented on SPARK-16827: - Performance is worse for this job. But I suspe

[jira] [Commented] (SPARK-16827) Query with Join produces excessive amount of shuffle data

2016-07-31 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401523#comment-15401523 ] Sital Kedia commented on SPARK-16827: - Actually it seems like this is a bug in shuffl

[jira] [Comment Edited] (SPARK-16827) Query with Join produces excessive amount of shuffle data

2016-07-31 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401413#comment-15401413 ] Sital Kedia edited comment on SPARK-16827 at 8/1/16 12:52 AM: -

[jira] [Commented] (SPARK-16827) Query with Join produces excessive amount of shuffle data

2016-07-31 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401413#comment-15401413 ] Sital Kedia commented on SPARK-16827: - That is not the case. There is no broadcast jo

[jira] [Updated] (SPARK-16827) Query with Join produces excessive amount of shuffle data

2016-07-31 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-16827: Description: One of our hive job which looks like this - {code} SELECT userid FROM table1

[jira] [Commented] (SPARK-16827) Query with Join produces excessive amount of shuffle data

2016-07-31 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401405#comment-15401405 ] Sital Kedia commented on SPARK-16827: - [~rxin] - Any idea how to debug this issue? >

[jira] [Updated] (SPARK-16827) Query with Join produces excessive shuffle data

2016-07-31 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-16827: Description: One of our hive job which looks like this - {code} SELECT userid FROM table1

[jira] [Updated] (SPARK-16827) Query with Join produces excessive amount of shuffle data

2016-07-31 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-16827: Summary: Query with Join produces excessive amount of shuffle data (was: Query with Join produces

[jira] [Updated] (SPARK-16827) Query with Join produces excessive shuffle data

2016-07-31 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-16827: Description: One of our hive job which looks like this - {code] SELECT userid FROM table1

[jira] [Created] (SPARK-16827) Query with Join produces excessive shuffle data

2016-07-31 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-16827: --- Summary: Query with Join produces excessive shuffle data Key: SPARK-16827 URL: https://issues.apache.org/jira/browse/SPARK-16827 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-15958) Make initial buffer size for the Sorter configurable

2016-06-14 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-15958: --- Summary: Make initial buffer size for the Sorter configurable Key: SPARK-15958 URL: https://issues.apache.org/jira/browse/SPARK-15958 Project: Spark Issue Type

[jira] [Created] (SPARK-15569) Executors spending significant time in DiskObjectWriter.updateBytesWritten function

2016-05-26 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-15569: --- Summary: Executors spending significant time in DiskObjectWriter.updateBytesWritten function Key: SPARK-15569 URL: https://issues.apache.org/jira/browse/SPARK-15569 Pro

[jira] [Commented] (SPARK-15391) Spark executor OOM during TimSort

2016-05-18 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15289791#comment-15289791 ] Sital Kedia commented on SPARK-15391: - cc- [~davies] - Any idea how to fix this issu

[jira] [Updated] (SPARK-15391) Spark executor OOM during TimSort

2016-05-18 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-15391: Description: While running a query, we are seeing a lot of executor OOM while doing TimSort. Stack

[jira] [Updated] (SPARK-15391) Spark executor OOM during TimSort

2016-05-18 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-15391: Description: While running a query, we are seeing a lot of executor OOM while doing TimSort. Stack

[jira] [Created] (SPARK-15391) Spark executor OOM during TimSort

2016-05-18 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-15391: --- Summary: Spark executor OOM during TimSort Key: SPARK-15391 URL: https://issues.apache.org/jira/browse/SPARK-15391 Project: Spark Issue Type: Bug R

[jira] [Commented] (SPARK-13850) TimSort Comparison method violates its general contract

2016-05-17 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15286906#comment-15286906 ] Sital Kedia commented on SPARK-13850: - I am not 100% sure of the root cause, but I su

[jira] [Commented] (SPARK-13850) TimSort Comparison method violates its general contract

2016-05-13 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15283110#comment-15283110 ] Sital Kedia commented on SPARK-13850: - I have found a workaround for this issue. Plea

[jira] [Updated] (SPARK-15233) Spark task metrics should include hdfs read write latency

2016-05-09 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-15233: Affects Version/s: 1.6.1 Priority: Minor (was: Major) Description: Currently

[jira] [Created] (SPARK-15233) Spark UI should show metrics for hdfs read write latency

2016-05-09 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-15233: --- Summary: Spark UI should show metrics for hdfs read write latency Key: SPARK-15233 URL: https://issues.apache.org/jira/browse/SPARK-15233 Project: Spark Issue

  1   2   >