[jira] [Commented] (SPARK-22757) Init-container in the driver/executor pods for downloading remote dependencies

[ https://issues.apache.org/jira/browse/SPARK-22757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288856#comment-16288856 ] Jiang Xingbo commented on SPARK-22757: -- Is this also targeted to 2.3 release? > Ini

[jira] [Created] (SPARK-22770) When driver stopping, there is problem: org.apache.spark.SparkException: Could not find CoarseGrainedScheduler

KaiXinXIaoLei created SPARK-22770: - Summary: When driver stopping, there is problem: org.apache.spark.SparkException: Could not find CoarseGrainedScheduler Key: SPARK-22770 URL: https://issues.apache.org/jira/brow

[jira] [Resolved] (SPARK-22745) read partition stats from Hive

[ https://issues.apache.org/jira/browse/SPARK-22745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-22745. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19932 [https://githu

[jira] [Assigned] (SPARK-22745) read partition stats from Hive

[ https://issues.apache.org/jira/browse/SPARK-22745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-22745: --- Assignee: Zhenhua Wang > read partition stats from Hive > -- > >

[jira] [Assigned] (SPARK-22574) Wrong request causing Spark Dispatcher going inactive

[ https://issues.apache.org/jira/browse/SPARK-22574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22574: Assignee: Apache Spark > Wrong request causing Spark Dispatcher going inactive > -

[jira] [Assigned] (SPARK-22574) Wrong request causing Spark Dispatcher going inactive

[ https://issues.apache.org/jira/browse/SPARK-22574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22574: Assignee: (was: Apache Spark) > Wrong request causing Spark Dispatcher going inactive

[jira] [Updated] (SPARK-22683) DynamicAllocation wastes resources by allocating containers that will barely be used

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Cuquemelle updated SPARK-22683: -- Summary: DynamicAllocation wastes resources by allocating containers that will barely b

[jira] [Updated] (SPARK-22683) DynamicAllocation wastes resources by allocating containers that will barely be used

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Cuquemelle updated SPARK-22683: -- Description: let's say an executor has spark.executor.cores / spark.task.cpus taskSlots

[jira] [Updated] (SPARK-22598) ExecutorAllocationManager does not requests new executors when executor fail and target has not change

[ https://issues.apache.org/jira/browse/SPARK-22598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lijia Liu updated SPARK-22598: -- Fix Version/s: 2.2.1 duplicate with SPARK22598 > ExecutorAllocationManager does not requests new execu

[jira] [Comment Edited] (SPARK-22598) ExecutorAllocationManager does not requests new executors when executor fail and target has not change

[ https://issues.apache.org/jira/browse/SPARK-22598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288966#comment-16288966 ] Lijia Liu edited comment on SPARK-22598 at 12/13/17 9:34 AM: -

[jira] [Created] (SPARK-22771) SQL concat for binary

Fernando Pereira created SPARK-22771: Summary: SQL concat for binary Key: SPARK-22771 URL: https://issues.apache.org/jira/browse/SPARK-22771 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-22771) SQL concat for binary

[ https://issues.apache.org/jira/browse/SPARK-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fernando Pereira updated SPARK-22771: - Description: spark.sql {{concat}} function automatically casts arguments to StringType a

[jira] [Updated] (SPARK-22749) df.na().fill(0) over an unnamed FIRST() fails the query

[ https://issues.apache.org/jira/browse/SPARK-22749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Efrat updated SPARK-22749: -- Description: When I created a query with FIRST() and didn't renamed the field I queried over - applying .na().f

[jira] [Commented] (SPARK-19357) Parallel Model Evaluation for ML Tuning: Scala

[ https://issues.apache.org/jira/browse/SPARK-19357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288993#comment-16288993 ] Nick Pentreath commented on SPARK-19357: I've thought about this and taken a look

[jira] [Issue Comment Deleted] (SPARK-22598) ExecutorAllocationManager does not requests new executors when executor fail and target has not change

[ https://issues.apache.org/jira/browse/SPARK-22598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lijia Liu updated SPARK-22598: -- Comment: was deleted (was: duplicate with SPARK-22598) > ExecutorAllocationManager does not requests n

[jira] [Commented] (SPARK-22711) _pickle.PicklingError: args[0] from __newobj__ args has the wrong class from cloudpickle.py

[ https://issues.apache.org/jira/browse/SPARK-22711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289028#comment-16289028 ] Hyukjin Kwon commented on SPARK-22711: -- I meant if it worked fine before in lower ve

[jira] [Commented] (SPARK-22683) DynamicAllocation wastes resources by allocating containers that will barely be used

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289046#comment-16289046 ] Julien Cuquemelle commented on SPARK-22683: --- Hi [~xuefuz], Thanks for noticing

[jira] [Created] (SPARK-22772) elt should use splitExpressionsWithCurrentInputs to split expression codes

Liang-Chi Hsieh created SPARK-22772: --- Summary: elt should use splitExpressionsWithCurrentInputs to split expression codes Key: SPARK-22772 URL: https://issues.apache.org/jira/browse/SPARK-22772 Proj

[jira] [Assigned] (SPARK-22772) elt should use splitExpressionsWithCurrentInputs to split expression codes

[ https://issues.apache.org/jira/browse/SPARK-22772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22772: Assignee: Apache Spark > elt should use splitExpressionsWithCurrentInputs to split express

[jira] [Commented] (SPARK-22772) elt should use splitExpressionsWithCurrentInputs to split expression codes

[ https://issues.apache.org/jira/browse/SPARK-22772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289093#comment-16289093 ] Apache Spark commented on SPARK-22772: -- User 'viirya' has created a pull request for

[jira] [Assigned] (SPARK-22772) elt should use splitExpressionsWithCurrentInputs to split expression codes

[ https://issues.apache.org/jira/browse/SPARK-22772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22772: Assignee: (was: Apache Spark) > elt should use splitExpressionsWithCurrentInputs to sp

[jira] [Commented] (SPARK-22683) DynamicAllocation wastes resources by allocating containers that will barely be used

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289101#comment-16289101 ] Julien Cuquemelle commented on SPARK-22683: --- Hi [~srowen], Just a small remar

[jira] [Updated] (SPARK-2926) Add MR-style (merge-sort) SortShuffleReader for sort-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Yuanjian updated SPARK-2926: --- Attachment: Spark Shuffle Test Report on Spark2.x.pdf [~jerryshao] Hi saisai, thanks for your advise,

[jira] [Assigned] (SPARK-22769) When driver stopping, there is problem: RpcEnv already stopped

[ https://issues.apache.org/jira/browse/SPARK-22769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22769: Assignee: Apache Spark > When driver stopping, there is problem: RpcEnv already stopped >

[jira] [Commented] (SPARK-22769) When driver stopping, there is problem: RpcEnv already stopped

[ https://issues.apache.org/jira/browse/SPARK-22769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289213#comment-16289213 ] Apache Spark commented on SPARK-22769: -- User 'KaiXinXiaoLei' has created a pull requ

[jira] [Assigned] (SPARK-22769) When driver stopping, there is problem: RpcEnv already stopped

[ https://issues.apache.org/jira/browse/SPARK-22769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22769: Assignee: (was: Apache Spark) > When driver stopping, there is problem: RpcEnv already

[jira] [Commented] (SPARK-22770) When driver stopping, there is problem: org.apache.spark.SparkException: Could not find CoarseGrainedScheduler

[ https://issues.apache.org/jira/browse/SPARK-22770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289241#comment-16289241 ] Apache Spark commented on SPARK-22770: -- User 'KaiXinXiaoLei' has created a pull requ

[jira] [Assigned] (SPARK-22770) When driver stopping, there is problem: org.apache.spark.SparkException: Could not find CoarseGrainedScheduler

[ https://issues.apache.org/jira/browse/SPARK-22770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22770: Assignee: Apache Spark > When driver stopping, there is problem: org.apache.spark.SparkExc

[jira] [Assigned] (SPARK-22770) When driver stopping, there is problem: org.apache.spark.SparkException: Could not find CoarseGrainedScheduler

[ https://issues.apache.org/jira/browse/SPARK-22770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22770: Assignee: (was: Apache Spark) > When driver stopping, there is problem: org.apache.spa

[jira] [Commented] (SPARK-22574) Wrong request causing Spark Dispatcher going inactive

[ https://issues.apache.org/jira/browse/SPARK-22574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289249#comment-16289249 ] Apache Spark commented on SPARK-22574: -- User 'Gschiavon' has created a pull request

[jira] [Commented] (SPARK-22683) DynamicAllocation wastes resources by allocating containers that will barely be used

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289258#comment-16289258 ] Sean Owen commented on SPARK-22683: --- Yes, that's always true though. No matter what you

[jira] [Resolved] (SPARK-22749) df.na().fill(0) over an unnamed FIRST() fails the query

[ https://issues.apache.org/jira/browse/SPARK-22749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22749. --- Resolution: Invalid > df.na().fill(0) over an unnamed FIRST() fails the query > -

[jira] [Resolved] (SPARK-22770) When driver stopping, there is problem: org.apache.spark.SparkException: Could not find CoarseGrainedScheduler

[ https://issues.apache.org/jira/browse/SPARK-22770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22770. --- Resolution: Duplicate > When driver stopping, there is problem: org.apache.spark.SparkException: > C

[jira] [Updated] (SPARK-22598) ExecutorAllocationManager does not requests new executors when executor fail and target has not change

[ https://issues.apache.org/jira/browse/SPARK-22598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22598: -- Fix Version/s: (was: 2.2.1) This should have been resolved as Duplicate, FYI > ExecutorAllocationM

[jira] [Commented] (SPARK-22770) When driver stopping, there is problem: org.apache.spark.SparkException: Could not find CoarseGrainedScheduler

[ https://issues.apache.org/jira/browse/SPARK-22770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289288#comment-16289288 ] Apache Spark commented on SPARK-22770: -- User 'KaiXinXiaoLei' has created a pull requ

[jira] [Commented] (SPARK-22359) Improve the test coverage of window functions

[ https://issues.apache.org/jira/browse/SPARK-22359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289300#comment-16289300 ] Sandor Murakozi commented on SPARK-22359: - Is there anyone working on these issue

[jira] [Updated] (SPARK-22770) When driver stopping, there is problem: org.apache.spark.SparkException: Could not find CoarseGrainedScheduler

[ https://issues.apache.org/jira/browse/SPARK-22770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-22770: -- Description: I run "spark-sql --master yarn --num-executors 1000 -f createTable.sql". When tas

[jira] [Updated] (SPARK-22770) When driver stopping, there is errors: "Could not find CoarseGrainedScheduler" and "RpcEnv already stopped"

[ https://issues.apache.org/jira/browse/SPARK-22770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-22770: -- Summary: When driver stopping, there is errors: "Could not find CoarseGrainedScheduler" and "Rp

[jira] [Resolved] (SPARK-15282) UDF executed twice when filter on new column created by withColumn and the final value may be not correct

[ https://issues.apache.org/jira/browse/SPARK-15282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15282. --- Resolution: Duplicate > UDF executed twice when filter on new column created by withColumn and the >

[jira] [Updated] (SPARK-22770) When driver stopping, there is errors: "Could not find CoarseGrainedScheduler" and

[ https://issues.apache.org/jira/browse/SPARK-22770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-22770: -- Summary: When driver stopping, there is errors: "Could not find CoarseGrainedScheduler" and (

[jira] [Updated] (SPARK-22769) When driver stopping, there is problem: RpcEnv already stopped

[ https://issues.apache.org/jira/browse/SPARK-22769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-22769: -- Description: I run "spark-sql --master yarn --num-executors 1000 -f createTable.sql". When tas

[jira] [Updated] (SPARK-22769) When driver stopping, there is errors: "Could not find CoarseGrainedScheduler" and "RpcEnv already stopped"

[ https://issues.apache.org/jira/browse/SPARK-22769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-22769: -- Summary: When driver stopping, there is errors: "Could not find CoarseGrainedScheduler" and "Rp

[jira] [Commented] (SPARK-22769) When driver stopping, there is errors: "Could not find CoarseGrainedScheduler" and "RpcEnv already stopped"

[ https://issues.apache.org/jira/browse/SPARK-22769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289310#comment-16289310 ] Apache Spark commented on SPARK-22769: -- User 'KaiXinXiaoLei' has created a pull requ

[jira] [Updated] (SPARK-22744) Add a configuration to show the application submit hostname

[ https://issues.apache.org/jira/browse/SPARK-22744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22744: -- Priority: Minor (was: Major) > Add a configuration to show the application submit hostname > -

[jira] [Commented] (SPARK-22359) Improve the test coverage of window functions

[ https://issues.apache.org/jira/browse/SPARK-22359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289363#comment-16289363 ] Jiang Xingbo commented on SPARK-22359: -- [~smurakozi] Please feel free to PR for this

[jira] [Resolved] (SPARK-22744) Add a configuration to show the application submit hostname

[ https://issues.apache.org/jira/browse/SPARK-22744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22744. --- Resolution: Won't Fix > Add a configuration to show the application submit hostname > ---

[jira] [Commented] (SPARK-22757) Init-container in the driver/executor pods for downloading remote dependencies

[ https://issues.apache.org/jira/browse/SPARK-22757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289378#comment-16289378 ] Yinan Li commented on SPARK-22757: -- Yes, this is also targeting 2.3. > Init-container

[jira] [Commented] (SPARK-22359) Improve the test coverage of window functions

[ https://issues.apache.org/jira/browse/SPARK-22359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289383#comment-16289383 ] Gabor Somogyi commented on SPARK-22359: --- Joining as well. As I see plenty of things

[jira] [Created] (SPARK-22773) Empty arrays are not equal after transformation

Laurent Legrand created SPARK-22773: --- Summary: Empty arrays are not equal after transformation Key: SPARK-22773 URL: https://issues.apache.org/jira/browse/SPARK-22773 Project: Spark Issue T

[jira] [Updated] (SPARK-22773) Empty arrays are not equal after transformation

[ https://issues.apache.org/jira/browse/SPARK-22773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laurent Legrand updated SPARK-22773: Priority: Minor (was: Major) > Empty arrays are not equal after transformation > -

[jira] [Created] (SPARK-22774) Add compilation check for generated code in TPCDSQuerySuite

Kazuaki Ishizaki created SPARK-22774: Summary: Add compilation check for generated code in TPCDSQuerySuite Key: SPARK-22774 URL: https://issues.apache.org/jira/browse/SPARK-22774 Project: Spark

[jira] [Commented] (SPARK-22711) _pickle.PicklingError: args[0] from __newobj__ args has the wrong class from cloudpickle.py

[ https://issues.apache.org/jira/browse/SPARK-22711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289456#comment-16289456 ] Prateek commented on SPARK-22711: - I tried it on 2.2.0 and then 2.2.1 RDD2=Rdd.map(func

[jira] [Comment Edited] (SPARK-22774) Add compilation check for generated code in TPCDSQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-22774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289471#comment-16289471 ] Kazuaki Ishizaki edited comment on SPARK-22774 at 12/13/17 4:24 PM: ---

[jira] [Commented] (SPARK-22774) Add compilation check for generated code in TPCDSQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-22774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289471#comment-16289471 ] Kazuaki Ishizaki commented on SPARK-22774: -- I will commit a PR for this soon. >

[jira] [Commented] (SPARK-22773) Empty arrays are not equal after transformation

[ https://issues.apache.org/jira/browse/SPARK-22773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289491#comment-16289491 ] Marco Gaido commented on SPARK-22773: - It is not a bug, since {res} is not an empty a

[jira] [Commented] (SPARK-22600) Fix 64kb limit for deeply nested expressions under wholestage codegen

[ https://issues.apache.org/jira/browse/SPARK-22600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289492#comment-16289492 ] Apache Spark commented on SPARK-22600: -- User 'maropu' has created a pull request for

[jira] [Commented] (SPARK-22453) Zookeeper configuration for MesosClusterDispatcher is not documented

[ https://issues.apache.org/jira/browse/SPARK-22453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289495#comment-16289495 ] Jais Sebastian commented on SPARK-22453: [~srowen] Can you point to the documenta

[jira] [Created] (SPARK-22775) move dictionary related APIs from ColumnVector to WritableColumnVector

Wenchen Fan created SPARK-22775: --- Summary: move dictionary related APIs from ColumnVector to WritableColumnVector Key: SPARK-22775 URL: https://issues.apache.org/jira/browse/SPARK-22775 Project: Spark

[jira] [Commented] (SPARK-22453) Zookeeper configuration for MesosClusterDispatcher is not documented

[ https://issues.apache.org/jira/browse/SPARK-22453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289498#comment-16289498 ] Sean Owen commented on SPARK-22453: --- https://github.com/apache/spark/tree/master/docs ?

[jira] [Commented] (SPARK-22773) Empty arrays are not equal after transformation

[ https://issues.apache.org/jira/browse/SPARK-22773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289500#comment-16289500 ] Laurent Legrand commented on SPARK-22773: - Indeed. My mistake. > Empty arrays a

[jira] [Assigned] (SPARK-22775) move dictionary related APIs from ColumnVector to WritableColumnVector

[ https://issues.apache.org/jira/browse/SPARK-22775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22775: Assignee: Apache Spark (was: Wenchen Fan) > move dictionary related APIs from ColumnVecto

[jira] [Assigned] (SPARK-22775) move dictionary related APIs from ColumnVector to WritableColumnVector

[ https://issues.apache.org/jira/browse/SPARK-22775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22775: Assignee: Wenchen Fan (was: Apache Spark) > move dictionary related APIs from ColumnVecto

[jira] [Resolved] (SPARK-22773) Empty arrays are not equal after transformation

[ https://issues.apache.org/jira/browse/SPARK-22773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laurent Legrand resolved SPARK-22773. - Resolution: Not A Bug > Empty arrays are not equal after transformation > ---

[jira] [Commented] (SPARK-22775) move dictionary related APIs from ColumnVector to WritableColumnVector

[ https://issues.apache.org/jira/browse/SPARK-22775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289501#comment-16289501 ] Apache Spark commented on SPARK-22775: -- User 'cloud-fan' has created a pull request

[jira] [Commented] (SPARK-22771) SQL concat for binary

[ https://issues.apache.org/jira/browse/SPARK-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289539#comment-16289539 ] Takeshi Yamamuro commented on SPARK-22771: -- fyi: postgresql has the behaiouvr as

[jira] [Commented] (SPARK-22773) Empty arrays are not equal after transformation

[ https://issues.apache.org/jira/browse/SPARK-22773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289547#comment-16289547 ] Apache Spark commented on SPARK-22773: -- User 'kiszk' has created a pull request for

[jira] [Resolved] (SPARK-22767) use ctx.addReferenceObj in InSet and ScalaUDF

[ https://issues.apache.org/jira/browse/SPARK-22767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-22767. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19962 [https://githu

[jira] [Commented] (SPARK-22774) Add compilation check for generated code in TPCDSQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-22774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289576#comment-16289576 ] Apache Spark commented on SPARK-22774: -- User 'kiszk' has created a pull request for

[jira] [Assigned] (SPARK-22774) Add compilation check for generated code in TPCDSQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-22774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22774: Assignee: (was: Apache Spark) > Add compilation check for generated code in TPCDSQuery

[jira] [Assigned] (SPARK-22774) Add compilation check for generated code in TPCDSQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-22774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22774: Assignee: Apache Spark > Add compilation check for generated code in TPCDSQuerySuite > ---

[jira] [Commented] (SPARK-22765) Create a new executor allocation scheme based on that of MR

[ https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289613#comment-16289613 ] Thomas Graves commented on SPARK-22765: --- why doesn't idle timeout very small < 5 se

[jira] [Commented] (SPARK-22683) DynamicAllocation wastes resources by allocating containers that will barely be used

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289614#comment-16289614 ] Thomas Graves commented on SPARK-22683: --- So the issue brought up here seems to be r

[jira] [Commented] (SPARK-22683) DynamicAllocation wastes resources by allocating containers that will barely be used

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289656#comment-16289656 ] Thomas Graves commented on SPARK-22683: --- I am also curious, when you are comparing

[jira] [Commented] (SPARK-22765) Create a new executor allocation scheme based on that of MR

[ https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289676#comment-16289676 ] Xuefu Zhang commented on SPARK-22765: - Hi [~tgraves], Thanks for your input. In our

[jira] [Comment Edited] (SPARK-22765) Create a new executor allocation scheme based on that of MR

[ https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289676#comment-16289676 ] Xuefu Zhang edited comment on SPARK-22765 at 12/13/17 6:30 PM:

[jira] [Updated] (SPARK-18278) SPIP: Support native submission of spark jobs to a kubernetes cluster

[ https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yinan Li updated SPARK-18278: - Component/s: Kubernetes > SPIP: Support native submission of spark jobs to a kubernetes cluster > ---

[jira] [Commented] (SPARK-22683) DynamicAllocation wastes resources by allocating containers that will barely be used

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289689#comment-16289689 ] Xuefu Zhang commented on SPARK-22683: - [~tgraves], I can speak on our use case, where

[jira] [Commented] (SPARK-22765) Create a new executor allocation scheme based on that of MR

[ https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289700#comment-16289700 ] Thomas Graves commented on SPARK-22765: --- ok so its basically they idle timeout duri

[jira] [Commented] (SPARK-22765) Create a new executor allocation scheme based on that of MR

[ https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289726#comment-16289726 ] Xuefu Zhang commented on SPARK-22765: - Yes, we are using Hive on Spark. Our Spark ver

[jira] [Created] (SPARK-22776) Increase default value of spark.sql.codegen.maxFields

Kazuaki Ishizaki created SPARK-22776: Summary: Increase default value of spark.sql.codegen.maxFields Key: SPARK-22776 URL: https://issues.apache.org/jira/browse/SPARK-22776 Project: Spark

[jira] [Resolved] (SPARK-22754) Check spark.executor.heartbeatInterval setting in case of ExecutorLost

[ https://issues.apache.org/jira/browse/SPARK-22754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-22754. Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19942 [https:/

[jira] [Assigned] (SPARK-22754) Check spark.executor.heartbeatInterval setting in case of ExecutorLost

[ https://issues.apache.org/jira/browse/SPARK-22754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-22754: -- Assignee: zhoukang > Check spark.executor.heartbeatInterval setting in case of Executo

[jira] [Commented] (SPARK-22359) Improve the test coverage of window functions

[ https://issues.apache.org/jira/browse/SPARK-22359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289792#comment-16289792 ] Sandor Murakozi commented on SPARK-22359: - I'm glad that you join [~gsomogyi]. D

[jira] [Commented] (SPARK-22765) Create a new executor allocation scheme based on that of MR

[ https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289793#comment-16289793 ] Thomas Graves commented on SPARK-22765: --- ok so before you do anything else I would

[jira] [Created] (SPARK-22777) Docker container built for Kubernetes doesn't allow running entrypoint.sh

Anirudh Ramanathan created SPARK-22777: -- Summary: Docker container built for Kubernetes doesn't allow running entrypoint.sh Key: SPARK-22777 URL: https://issues.apache.org/jira/browse/SPARK-22777

[jira] [Updated] (SPARK-22777) Docker container built for Kubernetes doesn't allow running entrypoint.sh

[ https://issues.apache.org/jira/browse/SPARK-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan updated SPARK-22777: --- [~liyinan926] [~ssuchter] > Docker container built for Kubernetes doesn't allow running

[jira] [Comment Edited] (SPARK-22777) Docker container built for Kubernetes doesn't allow running entrypoint.sh

[ https://issues.apache.org/jira/browse/SPARK-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289795#comment-16289795 ] Anirudh Ramanathan edited comment on SPARK-22777 at 12/13/17 8:02 PM: -

[jira] [Updated] (SPARK-22777) Docker container built for Kubernetes doesn't allow running entrypoint.sh

[ https://issues.apache.org/jira/browse/SPARK-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan updated SPARK-22777: --- Description: Default docker images built thrown an error when trying to run on a clus

[jira] [Updated] (SPARK-22777) Docker container built for Kubernetes doesn't allow running entrypoint.sh

[ https://issues.apache.org/jira/browse/SPARK-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan updated SPARK-22777: --- Description: Default docker images built thrown an error when trying to run on a clus

[jira] [Updated] (SPARK-22777) Docker container built for Kubernetes doesn't allow running entrypoint.sh

[ https://issues.apache.org/jira/browse/SPARK-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan updated SPARK-22777: --- Description: Default docker images built thrown an error when trying to run on a clus

[jira] [Updated] (SPARK-22777) Docker container built for Kubernetes doesn't allow running entrypoint.sh

[ https://issues.apache.org/jira/browse/SPARK-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan updated SPARK-22777: --- Description: Default docker images built thrown an error when trying to run on a clus

[jira] [Comment Edited] (SPARK-22360) Add unit test for Window Specifications

[ https://issues.apache.org/jira/browse/SPARK-22360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289803#comment-16289803 ] Sandor Murakozi edited comment on SPARK-22360 at 12/13/17 8:08 PM:

[jira] [Commented] (SPARK-22360) Add unit test for Window Specifications

[ https://issues.apache.org/jira/browse/SPARK-22360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289803#comment-16289803 ] Sandor Murakozi commented on SPARK-22360: - I just did a quick check of the existi

[jira] [Commented] (SPARK-22647) Docker files for image creation

[ https://issues.apache.org/jira/browse/SPARK-22647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289805#comment-16289805 ] Anirudh Ramanathan commented on SPARK-22647: [~mcheah] [~erikerlandson] PTAL

[jira] [Comment Edited] (SPARK-22360) Add unit test for Window Specifications

[ https://issues.apache.org/jira/browse/SPARK-22360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289803#comment-16289803 ] Sandor Murakozi edited comment on SPARK-22360 at 12/13/17 8:09 PM:

[jira] [Created] (SPARK-22778) Kubernetes scheduler at master failing tests

Anirudh Ramanathan created SPARK-22778: -- Summary: Kubernetes scheduler at master failing tests Key: SPARK-22778 URL: https://issues.apache.org/jira/browse/SPARK-22778 Project: Spark Issu

[jira] [Updated] (SPARK-22778) Kubernetes scheduler at master failing to run applications successfully

[ https://issues.apache.org/jira/browse/SPARK-22778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan updated SPARK-22778: --- Summary: Kubernetes scheduler at master failing to run applications successfully (wa

[jira] [Updated] (SPARK-22778) Kubernetes scheduler at master failing to run applications successfully.

[ https://issues.apache.org/jira/browse/SPARK-22778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan updated SPARK-22778: --- Summary: Kubernetes scheduler at master failing to run applications successfully. (w

[jira] [Commented] (SPARK-22778) Kubernetes scheduler at master failing tests

[ https://issues.apache.org/jira/browse/SPARK-22778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289808#comment-16289808 ] Anirudh Ramanathan commented on SPARK-22778: [~mcheah] [~kimoonkim] [~ifilone

[jira] [Updated] (SPARK-22778) Kubernetes scheduler at master failing to run applications successfully

[ https://issues.apache.org/jira/browse/SPARK-22778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan updated SPARK-22778: --- Description: Building images based on master and deploying Spark PI results in the fo

  1   2   >