[jira] [Commented] (SPARK-22055) Port release scripts

[ https://issues.apache.org/jira/browse/SPARK-22055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486728#comment-16486728 ] Felix Cheung commented on SPARK-22055: -- interesting - I'd definitely be happy to help. do you have

[jira] [Comment Edited] (SPARK-24359) SPIP: ML Pipelines in R

[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486719#comment-16486719 ] Felix Cheung edited comment on SPARK-24359 at 5/23/18 5:21 AM: --- # could you

[jira] [Comment Edited] (SPARK-24359) SPIP: ML Pipelines in R

[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486719#comment-16486719 ] Felix Cheung edited comment on SPARK-24359 at 5/23/18 5:18 AM: --- # could you

[jira] [Commented] (SPARK-24359) SPIP: ML Pipelines in R

[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486719#comment-16486719 ] Felix Cheung commented on SPARK-24359: -- # could you include design doc as google doc - it will be

[jira] [Commented] (SPARK-22366) Support ignoreMissingFiles flag parallel to ignoreCorruptFiles

[ https://issues.apache.org/jira/browse/SPARK-22366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486717#comment-16486717 ] Hyukjin Kwon commented on SPARK-22366: -- Oops, sorry. I mistakenly edited the JIRA. I reverted it

[jira] [Updated] (SPARK-22366) Support ignoreMissingFiles flag parallel to ignoreCorruptFiles

[ https://issues.apache.org/jira/browse/SPARK-22366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-22366: - Description: +underlined text+There's an existing flag "spark.sql.files.ignoreCorruptFiles"

[jira] [Updated] (SPARK-22366) Support ignoreMissingFiles flag parallel to ignoreCorruptFiles

[ https://issues.apache.org/jira/browse/SPARK-22366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-22366: - Description: There's an existing flag "spark.sql.files.ignoreCorruptFiles" that will quietly

[jira] [Closed] (SPARK-24349) obtainDelegationTokens() exits JVM if Driver use JDBC instead of using metastore

[ https://issues.apache.org/jira/browse/SPARK-24349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin closed SPARK-24349. -- > obtainDelegationTokens() exits JVM if Driver use JDBC instead of using > metastore >

[jira] [Resolved] (SPARK-24349) obtainDelegationTokens() exits JVM if Driver use JDBC instead of using metastore

[ https://issues.apache.org/jira/browse/SPARK-24349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin resolved SPARK-24349. Resolution: Not A Problem delegationTokensRequired has been checked in SparkSQLCLIDriver.scala >

[jira] [Assigned] (SPARK-24361) Polish code block manipulation API

[ https://issues.apache.org/jira/browse/SPARK-24361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24361: Assignee: (was: Apache Spark) > Polish code block manipulation API >

[jira] [Assigned] (SPARK-24361) Polish code block manipulation API

[ https://issues.apache.org/jira/browse/SPARK-24361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24361: Assignee: Apache Spark > Polish code block manipulation API >

[jira] [Commented] (SPARK-24361) Polish code block manipulation API

[ https://issues.apache.org/jira/browse/SPARK-24361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486652#comment-16486652 ] Apache Spark commented on SPARK-24361: -- User 'viirya' has created a pull request for this issue:

[jira] [Updated] (SPARK-24342) Large Task prior scheduling to Reduce overall execution time

[ https://issues.apache.org/jira/browse/SPARK-24342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gao updated SPARK-24342: Component/s: (was: Optimizer) Spark Core > Large Task prior scheduling to Reduce overall

[jira] [Created] (SPARK-24361) Polish code block manipulation API

Liang-Chi Hsieh created SPARK-24361: --- Summary: Polish code block manipulation API Key: SPARK-24361 URL: https://issues.apache.org/jira/browse/SPARK-24361 Project: Spark Issue Type:

[jira] [Updated] (SPARK-24339) spark sql can not prune column in transform/map/reduce query

[ https://issues.apache.org/jira/browse/SPARK-24339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xdcjie updated SPARK-24339: --- Affects Version/s: (was: 2.2.1) (was: 2.1.2) (was:

[jira] [Commented] (SPARK-21945) pyspark --py-files doesn't work in yarn client mode

[ https://issues.apache.org/jira/browse/SPARK-21945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486610#comment-16486610 ] Hyukjin Kwon commented on SPARK-21945: -- [~vanzin], I tried spark-submit in client mode and cluster

[jira] [Commented] (SPARK-24358) createDataFrame in Python 3 should be able to infer bytes type as Binary type

[ https://issues.apache.org/jira/browse/SPARK-24358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486594#comment-16486594 ] Joel Croteau commented on SPARK-24358: -- Done. > createDataFrame in Python 3 should be able to infer

[jira] [Updated] (SPARK-24358) createDataFrame in Python 3 should be able to infer bytes type as Binary type

[ https://issues.apache.org/jira/browse/SPARK-24358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Croteau updated SPARK-24358: - Labels: Python3 (was: ) Description: createDataFrame can infer Python 3's bytearray

[jira] [Comment Edited] (SPARK-24358) createDataFrame in Python should be able to infer bytes type as Binary type

[ https://issues.apache.org/jira/browse/SPARK-24358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486584#comment-16486584 ] Hyukjin Kwon edited comment on SPARK-24358 at 5/23/18 1:50 AM: --- Yea, let's

[jira] [Commented] (SPARK-24358) createDataFrame in Python should be able to infer bytes type as Binary type

[ https://issues.apache.org/jira/browse/SPARK-24358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486584#comment-16486584 ] Hyukjin Kwon commented on SPARK-24358: -- Yea, let's note that it's specific to Python 3. >

[jira] [Comment Edited] (SPARK-24358) createDataFrame in Python should be able to infer bytes type as Binary type

[ https://issues.apache.org/jira/browse/SPARK-24358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486581#comment-16486581 ] Joel Croteau edited comment on SPARK-24358 at 5/23/18 1:47 AM: --- No, I mean

[jira] [Commented] (SPARK-24358) createDataFrame in Python should be able to infer bytes type as Binary type

[ https://issues.apache.org/jira/browse/SPARK-24358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486581#comment-16486581 ] Joel Croteau commented on SPARK-24358: -- No, I mean the bytes type in Python 3. This code:

[jira] [Commented] (SPARK-24356) Duplicate strings in File.path managed by FileSegmentManagedBuffer

[ https://issues.apache.org/jira/browse/SPARK-24356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486580#comment-16486580 ] Misha Dmitriev commented on SPARK-24356: I plan to work on this feature. > Duplicate strings in

[jira] [Updated] (SPARK-24349) obtainDelegationTokens() exits JVM if Driver use JDBC instead of using metastore

[ https://issues.apache.org/jira/browse/SPARK-24349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-24349: --- Description: In [SPARK-23639|https://issues.apache.org/jira/browse/SPARK-23639], use --proxy-user

[jira] [Commented] (SPARK-24357) createDataFrame in Python infers large integers as long type and then fails silently when converting them

[ https://issues.apache.org/jira/browse/SPARK-24357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486571#comment-16486571 ] Joel Croteau commented on SPARK-24357: -- Fair enough, here is some code to reproduce it:

[jira] [Commented] (SPARK-24358) createDataFrame in Python should be able to infer bytes type as Binary type

[ https://issues.apache.org/jira/browse/SPARK-24358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486569#comment-16486569 ] Hyukjin Kwon commented on SPARK-24358: -- ? do you mean bytes in Python 2? that's an alias for str,

[jira] [Commented] (SPARK-24324) UserDefinedFunction mixes column labels

[ https://issues.apache.org/jira/browse/SPARK-24324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486565#comment-16486565 ] Hyukjin Kwon commented on SPARK-24324: -- Ah, I meant shorter reproducer should make other guys easier

[jira] [Commented] (SPARK-24339) spark sql can not prune column in transform/map/reduce query

[ https://issues.apache.org/jira/browse/SPARK-24339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486561#comment-16486561 ] Hyukjin Kwon commented on SPARK-24339: -- (Don't set the target versions usually reserved for

[jira] [Updated] (SPARK-24339) spark sql can not prune column in transform/map/reduce query

[ https://issues.apache.org/jira/browse/SPARK-24339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24339: - Target Version/s: (was: 2.1.1, 2.1.2, 2.2.0, 2.2.1) > spark sql can not prune column in

[jira] [Updated] (SPARK-24339) spark sql can not prune column in transform/map/reduce query

[ https://issues.apache.org/jira/browse/SPARK-24339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24339: - Fix Version/s: (was: 2.2.1) (was: 2.1.2) (was:

[jira] [Commented] (SPARK-24354) Adding support for quoteMode in Spark's build in CSV DataFrameWriter

[ https://issues.apache.org/jira/browse/SPARK-24354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486557#comment-16486557 ] Hyukjin Kwon commented on SPARK-24354: -- Nope, the library was changed and it doesn't have such

[jira] [Commented] (SPARK-24357) createDataFrame in Python infers large integers as long type and then fails silently when converting them

[ https://issues.apache.org/jira/browse/SPARK-24357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486554#comment-16486554 ] Hyukjin Kwon commented on SPARK-24357: -- It should have been much more readable if there are some

[jira] [Updated] (SPARK-24342) Large Task prior scheduling to Reduce overall execution time

[ https://issues.apache.org/jira/browse/SPARK-24342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gao updated SPARK-24342: Priority: Major (was: Minor) > Large Task prior scheduling to Reduce overall execution time >

[jira] [Updated] (SPARK-24359) SPIP: ML Pipelines in R

[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hossein Falaki updated SPARK-24359: --- Description: h1. Background and motivation SparkR supports calling MLlib functionality with

[jira] [Updated] (SPARK-24322) Upgrade Apache ORC to 1.4.4

[ https://issues.apache.org/jira/browse/SPARK-24322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24322: -- Description: ORC 1.4.4 (released on May 14th) includes nine fixes. This issue aims to update

[jira] [Assigned] (SPARK-24360) Support Hive 3.0 metastore

[ https://issues.apache.org/jira/browse/SPARK-24360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24360: Assignee: Apache Spark > Support Hive 3.0 metastore > -- > >

[jira] [Commented] (SPARK-24360) Support Hive 3.0 metastore

[ https://issues.apache.org/jira/browse/SPARK-24360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16485646#comment-16485646 ] Apache Spark commented on SPARK-24360: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-24360) Support Hive 3.0 metastore

[ https://issues.apache.org/jira/browse/SPARK-24360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24360: Assignee: (was: Apache Spark) > Support Hive 3.0 metastore >

[jira] [Created] (SPARK-24360) Support Hive 3.0 metastore

Dongjoon Hyun created SPARK-24360: - Summary: Support Hive 3.0 metastore Key: SPARK-24360 URL: https://issues.apache.org/jira/browse/SPARK-24360 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-24359) SPIP: ML Pipelines in R

[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hossein Falaki updated SPARK-24359: --- Description: h1. Background and motivation SparkR supports calling MLlib functionality with

[jira] [Updated] (SPARK-24359) SPIP: ML Pipelines in R

[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hossein Falaki updated SPARK-24359: --- Description: h1. Background and motivation SparkR supports calling MLlib functionality with

[jira] [Updated] (SPARK-24359) SPIP: ML Pipelines in R

[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hossein Falaki updated SPARK-24359: --- Description: h1. Background and motivation SparkR supports calling MLlib functionality with

[jira] [Commented] (SPARK-24313) Collection functions interpreted execution doesn't work with complex types

[ https://issues.apache.org/jira/browse/SPARK-24313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484679#comment-16484679 ] Apache Spark commented on SPARK-24313: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Created] (SPARK-24359) SPIP: ML Pipelines in R

Hossein Falaki created SPARK-24359: -- Summary: SPIP: ML Pipelines in R Key: SPARK-24359 URL: https://issues.apache.org/jira/browse/SPARK-24359 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-24359) SPIP: ML Pipelines in R

[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hossein Falaki updated SPARK-24359: --- Attachment: SparkML_ ML Pipelines in R.pdf > SPIP: ML Pipelines in R >

[jira] [Commented] (SPARK-24355) Improve Spark shuffle server responsiveness to non-ChunkFetch requests

[ https://issues.apache.org/jira/browse/SPARK-24355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484654#comment-16484654 ] Apache Spark commented on SPARK-24355: -- User 'Victsm' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24355) Improve Spark shuffle server responsiveness to non-ChunkFetch requests

[ https://issues.apache.org/jira/browse/SPARK-24355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24355: Assignee: Apache Spark > Improve Spark shuffle server responsiveness to non-ChunkFetch

[jira] [Assigned] (SPARK-24355) Improve Spark shuffle server responsiveness to non-ChunkFetch requests

[ https://issues.apache.org/jira/browse/SPARK-24355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24355: Assignee: (was: Apache Spark) > Improve Spark shuffle server responsiveness to

[jira] [Commented] (SPARK-24355) Improve Spark shuffle server responsiveness to non-ChunkFetch requests

[ https://issues.apache.org/jira/browse/SPARK-24355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484650#comment-16484650 ] Min Shen commented on SPARK-24355: -- [~felixcheung]  [~jinxing6...@126.com] [~cloud_fan] Could you

[jira] [Assigned] (SPARK-24335) Dataset.map schema not applied in some cases

[ https://issues.apache.org/jira/browse/SPARK-24335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24335: Assignee: Apache Spark > Dataset.map schema not applied in some cases >

[jira] [Commented] (SPARK-24335) Dataset.map schema not applied in some cases

[ https://issues.apache.org/jira/browse/SPARK-24335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484649#comment-16484649 ] Apache Spark commented on SPARK-24335: -- User 'Victsm' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24335) Dataset.map schema not applied in some cases

[ https://issues.apache.org/jira/browse/SPARK-24335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24335: Assignee: (was: Apache Spark) > Dataset.map schema not applied in some cases >

[jira] [Created] (SPARK-24358) createDataFrame in Python should be able to infer bytes type as Binary type

Joel Croteau created SPARK-24358: Summary: createDataFrame in Python should be able to infer bytes type as Binary type Key: SPARK-24358 URL: https://issues.apache.org/jira/browse/SPARK-24358 Project:

[jira] [Created] (SPARK-24357) createDataFrame in Python infers large integers as long type and then fails silently when converting them

Joel Croteau created SPARK-24357: Summary: createDataFrame in Python infers large integers as long type and then fails silently when converting them Key: SPARK-24357 URL:

[jira] [Created] (SPARK-24356) Duplicate strings in File.path managed by FileSegmentManagedBuffer

Misha Dmitriev created SPARK-24356: -- Summary: Duplicate strings in File.path managed by FileSegmentManagedBuffer Key: SPARK-24356 URL: https://issues.apache.org/jira/browse/SPARK-24356 Project:

[jira] [Created] (SPARK-24355) Improve Spark shuffle server responsiveness to non-ChunkFetch requests

Min Shen created SPARK-24355: Summary: Improve Spark shuffle server responsiveness to non-ChunkFetch requests Key: SPARK-24355 URL: https://issues.apache.org/jira/browse/SPARK-24355 Project: Spark

[jira] [Assigned] (SPARK-19185) ConcurrentModificationExceptions with CachedKafkaConsumers when Windowing

[ https://issues.apache.org/jira/browse/SPARK-19185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-19185: -- Assignee: Gabor Somogyi > ConcurrentModificationExceptions with CachedKafkaConsumers

[jira] [Resolved] (SPARK-19185) ConcurrentModificationExceptions with CachedKafkaConsumers when Windowing

[ https://issues.apache.org/jira/browse/SPARK-19185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-19185. Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20997

[jira] [Comment Edited] (SPARK-24324) UserDefinedFunction mixes column labels

[ https://issues.apache.org/jira/browse/SPARK-24324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484138#comment-16484138 ] Cristian Consonni edited comment on SPARK-24324 at 5/22/18 8:17 PM:

[jira] [Comment Edited] (SPARK-24324) UserDefinedFunction mixes column labels

[ https://issues.apache.org/jira/browse/SPARK-24324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484138#comment-16484138 ] Cristian Consonni edited comment on SPARK-24324 at 5/22/18 8:16 PM:

[jira] [Comment Edited] (SPARK-22055) Port release scripts

[ https://issues.apache.org/jira/browse/SPARK-22055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483217#comment-16483217 ] Marcelo Vanzin edited comment on SPARK-22055 at 5/22/18 8:11 PM: - IMO it

[jira] [Resolved] (SPARK-24348) scala.MatchError in the "element_at" expression

[ https://issues.apache.org/jira/browse/SPARK-24348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24348. - Resolution: Fixed > scala.MatchError in the "element_at" expression >

[jira] [Assigned] (SPARK-24348) scala.MatchError in the "element_at" expression

[ https://issues.apache.org/jira/browse/SPARK-24348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-24348: --- Assignee: Alex Vayda > scala.MatchError in the "element_at" expression >

[jira] [Comment Edited] (SPARK-23899) Built-in SQL Function Improvement

[ https://issues.apache.org/jira/browse/SPARK-23899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484477#comment-16484477 ] Alex Vayda edited comment on SPARK-23899 at 5/22/18 7:56 PM: - What do you

[jira] [Commented] (SPARK-23899) Built-in SQL Function Improvement

[ https://issues.apache.org/jira/browse/SPARK-23899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484477#comment-16484477 ] Alex Vayda commented on SPARK-23899: What do you guys think about adding another set of convenient

[jira] [Updated] (SPARK-23780) Failed to use googleVis library with new SparkR

[ https://issues.apache.org/jira/browse/SPARK-23780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-23780: --- Fix Version/s: (was: 2.3.2) 2.3.1 > Failed to use googleVis library

[jira] [Updated] (SPARK-24353) Add support for pod affinity/anti-affinity

[ https://issues.apache.org/jira/browse/SPARK-24353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-24353: Description: Spark on K8s allows to place driver/executor pods on specific k8s

[jira] [Created] (SPARK-24354) Adding support for quoteMode in Spark's build in CSV DataFrameWriter

Umesh K created SPARK-24354: --- Summary: Adding support for quoteMode in Spark's build in CSV DataFrameWriter Key: SPARK-24354 URL: https://issues.apache.org/jira/browse/SPARK-24354 Project: Spark

[jira] [Commented] (SPARK-24333) Add fit with validation set to spark.ml GBT: Python API

[ https://issues.apache.org/jira/browse/SPARK-24333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484376#comment-16484376 ] Huaxin Gao commented on SPARK-24333: I will work on this. Thanks! > Add fit with validation set to

[jira] [Updated] (SPARK-24353) Add support for pod affinity/anti-affinity

[ https://issues.apache.org/jira/browse/SPARK-24353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-24353: Description: Spark on K8s allows to place driver/executor pods on specific k8s

[jira] [Assigned] (SPARK-24121) The API for handling expression code generation in expression codegen

[ https://issues.apache.org/jira/browse/SPARK-24121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-24121: --- Assignee: Liang-Chi Hsieh > The API for handling expression code generation in expression

[jira] [Resolved] (SPARK-24121) The API for handling expression code generation in expression codegen

[ https://issues.apache.org/jira/browse/SPARK-24121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24121. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21193

[jira] [Commented] (SPARK-24353) Add support for pod affinity/anti-affinity

[ https://issues.apache.org/jira/browse/SPARK-24353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484346#comment-16484346 ] Stavros Kontopoulos commented on SPARK-24353: - I will create a design doc for this feature.

[jira] [Updated] (SPARK-24353) Add support for pod affinity/anti-affinity

[ https://issues.apache.org/jira/browse/SPARK-24353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-24353: Description: Spark on K8s allows to place driver/executor pods on specific k8s

[jira] [Updated] (SPARK-24353) Add support for pod affinity/anti-affinity

[ https://issues.apache.org/jira/browse/SPARK-24353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-24353: Description: Spark on K8s allows to place driver/executor pods on specific k8s

[jira] [Updated] (SPARK-24257) LongToUnsafeRowMap calculate the new size may be wrong

[ https://issues.apache.org/jira/browse/SPARK-24257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24257: Target Version/s: 2.3.2 > LongToUnsafeRowMap calculate the new size may be wrong >

[jira] [Updated] (SPARK-24257) LongToUnsafeRowMap calculate the new size may be wrong

[ https://issues.apache.org/jira/browse/SPARK-24257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24257: Target Version/s: 2.3.1 (was: 2.3.2) > LongToUnsafeRowMap calculate the new size may be wrong >

[jira] [Updated] (SPARK-24257) LongToUnsafeRowMap calculate the new size may be wrong

[ https://issues.apache.org/jira/browse/SPARK-24257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24257: Labels: correctness (was: ) > LongToUnsafeRowMap calculate the new size may be wrong >

[jira] [Updated] (SPARK-24257) LongToUnsafeRowMap calculate the new size may be wrong

[ https://issues.apache.org/jira/browse/SPARK-24257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24257: Priority: Blocker (was: Minor) > LongToUnsafeRowMap calculate the new size may be wrong >

[jira] [Comment Edited] (SPARK-13638) Support for saving with a quote mode

[ https://issues.apache.org/jira/browse/SPARK-13638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484255#comment-16484255 ] Umesh K edited comment on SPARK-13638 at 5/22/18 4:36 PM: -- [~rxin] Just want to

[jira] [Commented] (SPARK-13638) Support for saving with a quote mode

[ https://issues.apache.org/jira/browse/SPARK-13638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484255#comment-16484255 ] Umesh K commented on SPARK-13638: - Just want to confirm are we ever going to have quoteMode or we always

[jira] [Updated] (SPARK-24353) Add support for pod affinity/anti-affinity

[ https://issues.apache.org/jira/browse/SPARK-24353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-24353: Description: Spark on K8s allows to place driver/executor pods on specific k8s

[jira] [Created] (SPARK-24353) Add support for pod affinity/anti-affinity

Stavros Kontopoulos created SPARK-24353: --- Summary: Add support for pod affinity/anti-affinity Key: SPARK-24353 URL: https://issues.apache.org/jira/browse/SPARK-24353 Project: Spark

[jira] [Created] (SPARK-24352) Flaky test: StandaloneDynamicAllocationSuite

Marcelo Vanzin created SPARK-24352: -- Summary: Flaky test: StandaloneDynamicAllocationSuite Key: SPARK-24352 URL: https://issues.apache.org/jira/browse/SPARK-24352 Project: Spark Issue Type:

[jira] [Commented] (SPARK-24341) Codegen compile error from predicate subquery

[ https://issues.apache.org/jira/browse/SPARK-24341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484225#comment-16484225 ] Xiao Li commented on SPARK-24341: - cc [~dkbiswal] Could you take a look at this too? > Codegen compile

[jira] [Updated] (SPARK-24351) offsetLog/commitLog purge thresholdBatchId should be computed with current committed epoch but not currentBatchId in CP mode

[ https://issues.apache.org/jira/browse/SPARK-24351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huangtengfei updated SPARK-24351: - Description: In structured streaming, there is a conf spark.sql.streaming.minBatchesToRetain

[jira] [Assigned] (SPARK-24351) offsetLog/commitLog purge thresholdBatchId should be computed with current committed epoch but not currentBatchId in CP mode

[ https://issues.apache.org/jira/browse/SPARK-24351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24351: Assignee: (was: Apache Spark) > offsetLog/commitLog purge thresholdBatchId should be

[jira] [Assigned] (SPARK-24351) offsetLog/commitLog purge thresholdBatchId should be computed with current committed epoch but not currentBatchId in CP mode

[ https://issues.apache.org/jira/browse/SPARK-24351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24351: Assignee: Apache Spark > offsetLog/commitLog purge thresholdBatchId should be computed

[jira] [Commented] (SPARK-24350) ClassCastException in "array_position" function

[ https://issues.apache.org/jira/browse/SPARK-24350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484172#comment-16484172 ] Apache Spark commented on SPARK-24350: -- User 'wajda' has created a pull request for this issue:

[jira] [Commented] (SPARK-24351) offsetLog/commitLog purge thresholdBatchId should be computed with current committed epoch but not currentBatchId in CP mode

[ https://issues.apache.org/jira/browse/SPARK-24351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484173#comment-16484173 ] Apache Spark commented on SPARK-24351: -- User 'ivoson' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24350) ClassCastException in "array_position" function

[ https://issues.apache.org/jira/browse/SPARK-24350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24350: Assignee: Apache Spark > ClassCastException in "array_position" function >

[jira] [Assigned] (SPARK-24350) ClassCastException in "array_position" function

[ https://issues.apache.org/jira/browse/SPARK-24350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24350: Assignee: (was: Apache Spark) > ClassCastException in "array_position" function >

[jira] [Created] (SPARK-24351) offsetLog/commitLog purge thresholdBatchId should be computed with current committed epoch but not currentBatchId in CP mode

huangtengfei created SPARK-24351: Summary: offsetLog/commitLog purge thresholdBatchId should be computed with current committed epoch but not currentBatchId in CP mode Key: SPARK-24351 URL:

[jira] [Created] (SPARK-24350) ClassCastException in "array_position" function

Alex Wajda created SPARK-24350: -- Summary: ClassCastException in "array_position" function Key: SPARK-24350 URL: https://issues.apache.org/jira/browse/SPARK-24350 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-22269) Java style checks should be run in Jenkins

[ https://issues.apache.org/jira/browse/SPARK-22269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22269: Assignee: Apache Spark > Java style checks should be run in Jenkins >

[jira] [Commented] (SPARK-22269) Java style checks should be run in Jenkins

[ https://issues.apache.org/jira/browse/SPARK-22269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484151#comment-16484151 ] Apache Spark commented on SPARK-22269: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-22269) Java style checks should be run in Jenkins

[ https://issues.apache.org/jira/browse/SPARK-22269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22269: Assignee: (was: Apache Spark) > Java style checks should be run in Jenkins >

[jira] [Comment Edited] (SPARK-24324) UserDefinedFunction mixes column labels

[ https://issues.apache.org/jira/browse/SPARK-24324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484138#comment-16484138 ] Cristian Consonni edited comment on SPARK-24324 at 5/22/18 3:30 PM:

[jira] [Commented] (SPARK-24324) UserDefinedFunction mixes column labels

[ https://issues.apache.org/jira/browse/SPARK-24324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484138#comment-16484138 ] Cristian Consonni commented on SPARK-24324: --- [~hyukjin.kwon] said: > Can you narrow down the

[jira] [Assigned] (SPARK-24338) Spark SQL fails to create a Hive table when running in a Apache Sentry-secured Environment

[ https://issues.apache.org/jira/browse/SPARK-24338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24338: Assignee: (was: Apache Spark) > Spark SQL fails to create a Hive table when running

  1   2   >