[jira] [Created] (SPARK-26086) Spark streaming max records per batch interval

vijayant soni created SPARK-26086: - Summary: Spark streaming max records per batch interval Key: SPARK-26086 URL: https://issues.apache.org/jira/browse/SPARK-26086 Project: Spark Issue Type:

[jira] [Updated] (SPARK-26086) Spark streaming max records per batch interval

[ https://issues.apache.org/jira/browse/SPARK-26086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vijayant soni updated SPARK-26086: -- Affects Version/s: (was: 2.3.2) 2.3.0 > Spark streaming max records

[jira] [Updated] (SPARK-26086) Spark streaming max records per batch interval

[ https://issues.apache.org/jira/browse/SPARK-26086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vijayant soni updated SPARK-26086: -- Affects Version/s: (was: 2.3.1) 2.3.2 > Spark streaming max records

[jira] [Updated] (SPARK-26086) Spark streaming max records per batch interval

[ https://issues.apache.org/jira/browse/SPARK-26086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vijayant soni updated SPARK-26086: -- Description: We have an Spark Streaming application that reads from Kinesis and writes to Red

[jira] [Updated] (SPARK-26086) Spark streaming max records per batch interval

[ https://issues.apache.org/jira/browse/SPARK-26086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vijayant soni updated SPARK-26086: -- Description: We have an Spark Streaming application that reads from Kinesis and writes to Red

[jira] [Updated] (SPARK-26086) Spark streaming max records per batch interval

[ https://issues.apache.org/jira/browse/SPARK-26086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vijayant soni updated SPARK-26086: -- Description: We have an Spark Streaming application that reads from Kinesis and writes to Red

[jira] [Commented] (SPARK-26041) catalyst cuts out some columns from dataframes: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: Binding attribute

[ https://issues.apache.org/jira/browse/SPARK-26041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16689056#comment-16689056 ] Ruslan Dautkhanov commented on SPARK-26041: --- [~hyukjin.kwon] I didn't request

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16689052#comment-16689052 ] Ruslan Dautkhanov commented on SPARK-26019: --- No, it was the only instance I ha

[jira] [Resolved] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruslan Dautkhanov resolved SPARK-26019. --- Resolution: Cannot Reproduce > pyspark/accumulators.py: "TypeError: object of type '

[jira] [Commented] (SPARK-26078) WHERE .. IN fails to filter rows when used in combination with UNION

[ https://issues.apache.org/jira/browse/SPARK-26078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16689040#comment-16689040 ] Wenchen Fan commented on SPARK-26078: - looks like a bug when we rewrite correlated s

[jira] [Commented] (SPARK-24255) Require Java 8 in SparkR description

[ https://issues.apache.org/jira/browse/SPARK-24255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16689038#comment-16689038 ] Shivaram Venkataraman commented on SPARK-24255: --- This is a great list -- I

[jira] [Commented] (SPARK-20236) Overwrite a partitioned data source table should only overwrite related partitions

[ https://issues.apache.org/jira/browse/SPARK-20236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688995#comment-16688995 ] Wenchen Fan commented on SPARK-20236: - This looks like a bug to me. Can you come up

[jira] [Commented] (SPARK-26026) Published Scaladoc jars missing from Maven Central

[ https://issues.apache.org/jira/browse/SPARK-26026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688997#comment-16688997 ] Sean Owen commented on SPARK-26026: --- Ah, I think it was this: {code} /Users/seanowen/D

[jira] [Commented] (SPARK-26056) java api spark streaming spark-avro ui

[ https://issues.apache.org/jira/browse/SPARK-26056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688978#comment-16688978 ] Hyukjin Kwon commented on SPARK-26056: -- Looks we should fix. > java api spark stre

[jira] [Commented] (SPARK-26056) java api spark streaming spark-avro ui

[ https://issues.apache.org/jira/browse/SPARK-26056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688976#comment-16688976 ] wish commented on SPARK-26056: -- [~hyukjin.kwon] done > java api spark streaming spark-avro

[jira] [Commented] (SPARK-26026) Published Scaladoc jars missing from Maven Central

[ https://issues.apache.org/jira/browse/SPARK-26026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688973#comment-16688973 ] Sean Owen commented on SPARK-26026: --- Hm, I don't recall why I removed that now. It cou

[jira] [Commented] (SPARK-26067) Pandas GROUPED_MAP udf breaks if DF has >255 columns

[ https://issues.apache.org/jira/browse/SPARK-26067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688964#comment-16688964 ] Hyukjin Kwon commented on SPARK-26067: -- I don't think we should fix this. This is s

[jira] [Commented] (SPARK-26064) Unable to fetch jar from remote repo while running spark-submit on kubernetes

[ https://issues.apache.org/jira/browse/SPARK-26064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688961#comment-16688961 ] Hyukjin Kwon commented on SPARK-26064: -- Is it a question or an issue? > Unable to

[jira] [Commented] (SPARK-26027) Unable to build Spark for Scala 2.12 with Maven script provided

[ https://issues.apache.org/jira/browse/SPARK-26027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688968#comment-16688968 ] Hyukjin Kwon commented on SPARK-26027: -- Oh, right. Let me keep this in mind and reo

[jira] [Assigned] (SPARK-26034) Break large mllib/tests.py files into smaller files

[ https://issues.apache.org/jira/browse/SPARK-26034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26034: Assignee: Bryan Cutler (was: Apache Spark) > Break large mllib/tests.py files into small

[jira] [Updated] (SPARK-26056) java api spark streaming spark-avro ui

[ https://issues.apache.org/jira/browse/SPARK-26056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wish updated SPARK-26056: - Attachment: sql.jpg > java api spark streaming spark-avro ui > --- > >

[jira] [Commented] (SPARK-26075) Cannot broadcast the table that is larger than 8GB : Spark 2.3

[ https://issues.apache.org/jira/browse/SPARK-26075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688966#comment-16688966 ] Hyukjin Kwon commented on SPARK-26075: -- Does this happen in Spark 2.4 as well? > C

[jira] [Commented] (SPARK-26077) Reserved SQL words are not escaped by JDBC writer for table name

[ https://issues.apache.org/jira/browse/SPARK-26077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688967#comment-16688967 ] Hyukjin Kwon commented on SPARK-26077: -- cc'ing [~maropu] FYI > Reserved SQL words

[jira] [Resolved] (SPARK-26067) Pandas GROUPED_MAP udf breaks if DF has >255 columns

[ https://issues.apache.org/jira/browse/SPARK-26067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26067. -- Resolution: Not A Problem > Pandas GROUPED_MAP udf breaks if DF has >255 columns > ---

[jira] [Resolved] (SPARK-26063) CatalystDataToAvro gives "UnresolvedException: Invalid call to dataType on unresolved object" when requested for numberedTreeString

[ https://issues.apache.org/jira/browse/SPARK-26063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26063. -- Resolution: Duplicate > CatalystDataToAvro gives "UnresolvedException: Invalid call to dataTyp

[jira] [Commented] (SPARK-26034) Break large mllib/tests.py files into smaller files

[ https://issues.apache.org/jira/browse/SPARK-26034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688962#comment-16688962 ] Apache Spark commented on SPARK-26034: -- User 'BryanCutler' has created a pull reque

[jira] [Commented] (SPARK-26034) Break large mllib/tests.py files into smaller files

[ https://issues.apache.org/jira/browse/SPARK-26034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688963#comment-16688963 ] Apache Spark commented on SPARK-26034: -- User 'BryanCutler' has created a pull reque

[jira] [Assigned] (SPARK-26034) Break large mllib/tests.py files into smaller files

[ https://issues.apache.org/jira/browse/SPARK-26034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26034: Assignee: Apache Spark (was: Bryan Cutler) > Break large mllib/tests.py files into small

[jira] [Comment Edited] (SPARK-26027) Unable to build Spark for Scala 2.12 with Maven script provided

[ https://issues.apache.org/jira/browse/SPARK-26027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688957#comment-16688957 ] Jason Moore edited comment on SPARK-26027 at 11/16/18 3:10 AM: ---

[jira] [Commented] (SPARK-26059) Spark standalone mode, does not correctly record a failed Spark Job.

[ https://issues.apache.org/jira/browse/SPARK-26059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688958#comment-16688958 ] Hyukjin Kwon commented on SPARK-26059: -- Can you also describe reproducer, and outpu

[jira] [Commented] (SPARK-26027) Unable to build Spark for Scala 2.12 with Maven script provided

[ https://issues.apache.org/jira/browse/SPARK-26027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688957#comment-16688957 ] Jason Moore commented on SPARK-26027: - I was originally going to withdraw the ticket

[jira] [Commented] (SPARK-26056) java api spark streaming spark-avro ui

[ https://issues.apache.org/jira/browse/SPARK-26056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688956#comment-16688956 ] Hyukjin Kwon commented on SPARK-26056: -- Can you upload screenshots? > java api spa

[jira] [Commented] (SPARK-26050) Implment withColumnExpr method on DataFrame

[ https://issues.apache.org/jira/browse/SPARK-26050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688955#comment-16688955 ] Hyukjin Kwon commented on SPARK-26050: -- It's easily able to work around. Currently

[jira] [Resolved] (SPARK-26050) Implment withColumnExpr method on DataFrame

[ https://issues.apache.org/jira/browse/SPARK-26050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26050. -- Resolution: Won't Fix > Implment withColumnExpr method on DataFrame >

[jira] [Updated] (SPARK-26048) Flume connector for Spark 2.4 does not exist in Maven repository

[ https://issues.apache.org/jira/browse/SPARK-26048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26048: - Priority: Major (was: Blocker) > Flume connector for Spark 2.4 does not exist in Maven reposito

[jira] [Updated] (SPARK-26048) Flume connector for Spark 2.4 does not exist in Maven repository

[ https://issues.apache.org/jira/browse/SPARK-26048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26048: - Target Version/s: (was: 2.4.1) > Flume connector for Spark 2.4 does not exist in Maven reposit

[jira] [Commented] (SPARK-26048) Flume connector for Spark 2.4 does not exist in Maven repository

[ https://issues.apache.org/jira/browse/SPARK-26048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688951#comment-16688951 ] Hyukjin Kwon commented on SPARK-26048: -- Please avoid to set target versions and Cri

[jira] [Resolved] (SPARK-26031) dataframe can't load correct after saving to local disk in cluster mode

[ https://issues.apache.org/jira/browse/SPARK-26031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26031. -- Resolution: Invalid > dataframe can't load correct after saving to local disk in cluster mode

[jira] [Commented] (SPARK-26047) Py4JNetworkError (on IPV6): An error occurred while trying to connect to the Java server (127.0.0.1

[ https://issues.apache.org/jira/browse/SPARK-26047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688950#comment-16688950 ] Hyukjin Kwon commented on SPARK-26047: -- Thanks for working around for it. Is it Py4

[jira] [Updated] (SPARK-26045) Error in the spark 2.4 release package with the spark-avro_2.11 depdency

[ https://issues.apache.org/jira/browse/SPARK-26045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26045: - Target Version/s: (was: 2.4.0) > Error in the spark 2.4 release package with the spark-avro_2.

[jira] [Resolved] (SPARK-26041) catalyst cuts out some columns from dataframes: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: Binding attribute

[ https://issues.apache.org/jira/browse/SPARK-26041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26041. -- Resolution: Cannot Reproduce > catalyst cuts out some columns from dataframes: > org.apache.s

[jira] [Commented] (SPARK-26041) catalyst cuts out some columns from dataframes: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: Binding attribute

[ https://issues.apache.org/jira/browse/SPARK-26041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688947#comment-16688947 ] Hyukjin Kwon commented on SPARK-26041: -- [~Tagar] no, don't request investigation he

[jira] [Resolved] (SPARK-26040) CSV Row delimiters not consistent between platforms

[ https://issues.apache.org/jira/browse/SPARK-26040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26040. -- Resolution: Duplicate > CSV Row delimiters not consistent between platforms >

[jira] [Commented] (SPARK-26040) CSV Row delimiters not consistent between platforms

[ https://issues.apache.org/jira/browse/SPARK-26040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688943#comment-16688943 ] Hyukjin Kwon commented on SPARK-26040: -- I think this is not an issue when {{multiLi

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688934#comment-16688934 ] Hyukjin Kwon commented on SPARK-26019: -- Are you able to make a simple reproducer? I

[jira] [Resolved] (SPARK-26016) Encoding not working when using a map / mapPartitions call

[ https://issues.apache.org/jira/browse/SPARK-26016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26016. -- Resolution: Invalid > Encoding not working when using a map / mapPartitions call > ---

[jira] [Commented] (SPARK-26039) Reading an empty folder as ORC causes an Analysis Exception

[ https://issues.apache.org/jira/browse/SPARK-26039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688941#comment-16688941 ] Hyukjin Kwon commented on SPARK-26039: -- Does this happen in Spark 2.4 as well? > R

[jira] [Commented] (SPARK-26031) dataframe can't load correct after saving to local disk in cluster mode

[ https://issues.apache.org/jira/browse/SPARK-26031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688939#comment-16688939 ] Hyukjin Kwon commented on SPARK-26031: -- That's because you're using {{file://...}}

[jira] [Commented] (SPARK-26027) Unable to build Spark for Scala 2.12 with Maven script provided

[ https://issues.apache.org/jira/browse/SPARK-26027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688938#comment-16688938 ] Hyukjin Kwon commented on SPARK-26027: -- Is the goal to use Scala 2.12.6? The defaul

[jira] [Resolved] (SPARK-26027) Unable to build Spark for Scala 2.12 with Maven script provided

[ https://issues.apache.org/jira/browse/SPARK-26027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26027. -- Resolution: Not A Problem > Unable to build Spark for Scala 2.12 with Maven script provided >

[jira] [Commented] (SPARK-26026) Published Scaladoc jars missing from Maven Central

[ https://issues.apache.org/jira/browse/SPARK-26026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688937#comment-16688937 ] Hyukjin Kwon commented on SPARK-26026: -- cc [~srowen] > Published Scaladoc jars mis

[jira] [Commented] (SPARK-26016) Encoding not working when using a map / mapPartitions call

[ https://issues.apache.org/jira/browse/SPARK-26016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688932#comment-16688932 ] Hyukjin Kwon commented on SPARK-26016: -- Let's avoid to ask investigation in JIRA. I

[jira] [Commented] (SPARK-25992) Accumulators giving KeyError in pyspark

[ https://issues.apache.org/jira/browse/SPARK-25992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688931#comment-16688931 ] Hyukjin Kwon commented on SPARK-25992: -- Sounds unclear if it's an issue within Spar

[jira] [Updated] (SPARK-26080) Unable to run worker.py on Windows

[ https://issues.apache.org/jira/browse/SPARK-26080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26080: - Target Version/s: 2.4.1, 3.0.0 (was: 3.0.0) > Unable to run worker.py on Windows >

[jira] [Assigned] (SPARK-26080) Unable to run worker.py on Windows

[ https://issues.apache.org/jira/browse/SPARK-26080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26080: Assignee: (was: Apache Spark) > Unable to run worker.py on Windows >

[jira] [Assigned] (SPARK-26080) Unable to run worker.py on Windows

[ https://issues.apache.org/jira/browse/SPARK-26080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26080: Assignee: Apache Spark > Unable to run worker.py on Windows > ---

[jira] [Commented] (SPARK-26080) Unable to run worker.py on Windows

[ https://issues.apache.org/jira/browse/SPARK-26080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688904#comment-16688904 ] Apache Spark commented on SPARK-26080: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-26085) Key attribute of primitive type under typed aggregation should be named as "key" too

[ https://issues.apache.org/jira/browse/SPARK-26085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688900#comment-16688900 ] Apache Spark commented on SPARK-26085: -- User 'viirya' has created a pull request fo

[jira] [Commented] (SPARK-26085) Key attribute of primitive type under typed aggregation should be named as "key" too

[ https://issues.apache.org/jira/browse/SPARK-26085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688902#comment-16688902 ] Apache Spark commented on SPARK-26085: -- User 'viirya' has created a pull request fo

[jira] [Assigned] (SPARK-26085) Key attribute of primitive type under typed aggregation should be named as "key" too

[ https://issues.apache.org/jira/browse/SPARK-26085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26085: Assignee: Apache Spark > Key attribute of primitive type under typed aggregation should b

[jira] [Assigned] (SPARK-26085) Key attribute of primitive type under typed aggregation should be named as "key" too

[ https://issues.apache.org/jira/browse/SPARK-26085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26085: Assignee: (was: Apache Spark) > Key attribute of primitive type under typed aggregati

[jira] [Created] (SPARK-26085) Key attribute of primitive type under typed aggregation should be named as "key" too

Liang-Chi Hsieh created SPARK-26085: --- Summary: Key attribute of primitive type under typed aggregation should be named as "key" too Key: SPARK-26085 URL: https://issues.apache.org/jira/browse/SPARK-26085

[jira] [Updated] (SPARK-26080) Unable to run worker.py on Windows

[ https://issues.apache.org/jira/browse/SPARK-26080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26080: - Priority: Blocker (was: Major) > Unable to run worker.py on Windows > -

[jira] [Updated] (SPARK-26080) Unable to run worker.py on Windows

[ https://issues.apache.org/jira/browse/SPARK-26080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26080: - Target Version/s: 3.0.0 > Unable to run worker.py on Windows > -

[jira] [Commented] (SPARK-26080) Unable to run worker.py on Windows

[ https://issues.apache.org/jira/browse/SPARK-26080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1665#comment-1665 ] Hyukjin Kwon commented on SPARK-26080: -- We should fix this. > Unable to run worker

[jira] [Commented] (SPARK-25957) Skip building spark-r docker image if spark distribution does not have R support

[ https://issues.apache.org/jira/browse/SPARK-25957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688869#comment-16688869 ] Nagaram Prasad Addepally commented on SPARK-25957: -- Posted PR [https://

[jira] [Commented] (SPARK-25957) Skip building spark-r docker image if spark distribution does not have R support

[ https://issues.apache.org/jira/browse/SPARK-25957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688871#comment-16688871 ] Apache Spark commented on SPARK-25957: -- User 'ramaddepally' has created a pull requ

[jira] [Commented] (SPARK-25957) Skip building spark-r docker image if spark distribution does not have R support

[ https://issues.apache.org/jira/browse/SPARK-25957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688870#comment-16688870 ] Apache Spark commented on SPARK-25957: -- User 'ramaddepally' has created a pull requ

[jira] [Assigned] (SPARK-25957) Skip building spark-r docker image if spark distribution does not have R support

[ https://issues.apache.org/jira/browse/SPARK-25957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25957: Assignee: Apache Spark > Skip building spark-r docker image if spark distribution does no

[jira] [Assigned] (SPARK-25957) Skip building spark-r docker image if spark distribution does not have R support

[ https://issues.apache.org/jira/browse/SPARK-25957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25957: Assignee: (was: Apache Spark) > Skip building spark-r docker image if spark distribut

[jira] [Updated] (SPARK-25883) Override method `prettyName` in `from_avro`/`to_avro`

[ https://issues.apache.org/jira/browse/SPARK-25883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25883: - Fix Version/s: 2.4.1 > Override method `prettyName` in `from_avro`/`to_avro` > -

[jira] [Resolved] (SPARK-26035) Break large streaming/tests.py files into smaller files

[ https://issues.apache.org/jira/browse/SPARK-26035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26035. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23034 [https://gi

[jira] [Assigned] (SPARK-26035) Break large streaming/tests.py files into smaller files

[ https://issues.apache.org/jira/browse/SPARK-26035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-26035: Assignee: Hyukjin Kwon > Break large streaming/tests.py files into smaller files > --

[jira] [Created] (SPARK-26084) AggregateExpression.references fails on unresolved expression trees

Simeon Simeonov created SPARK-26084: --- Summary: AggregateExpression.references fails on unresolved expression trees Key: SPARK-26084 URL: https://issues.apache.org/jira/browse/SPARK-26084 Project: Sp

[jira] [Commented] (SPARK-26084) AggregateExpression.references fails on unresolved expression trees

[ https://issues.apache.org/jira/browse/SPARK-26084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688760#comment-16688760 ] Simeon Simeonov commented on SPARK-26084: - /cc [~maropu] [~hvanhovell] who worke

[jira] [Commented] (SPARK-26083) Pyspark command is not working properly with default Docker Image build

[ https://issues.apache.org/jira/browse/SPARK-26083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688743#comment-16688743 ] Apache Spark commented on SPARK-26083: -- User 'AzureQ' has created a pull request fo

[jira] [Commented] (SPARK-26083) Pyspark command is not working properly with default Docker Image build

[ https://issues.apache.org/jira/browse/SPARK-26083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688741#comment-16688741 ] Apache Spark commented on SPARK-26083: -- User 'AzureQ' has created a pull request fo

[jira] [Assigned] (SPARK-26083) Pyspark command is not working properly with default Docker Image build

[ https://issues.apache.org/jira/browse/SPARK-26083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26083: Assignee: Apache Spark > Pyspark command is not working properly with default Docker Imag

[jira] [Assigned] (SPARK-26083) Pyspark command is not working properly with default Docker Image build

[ https://issues.apache.org/jira/browse/SPARK-26083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26083: Assignee: (was: Apache Spark) > Pyspark command is not working properly with default

[jira] [Updated] (SPARK-26083) Pyspark command is not working properly with default Docker Image build

[ https://issues.apache.org/jira/browse/SPARK-26083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Shao updated SPARK-26083: Description: When I try to run {code:java} ./bin/pyspark{code} in a pod in Kubernetes(image built without

[jira] [Updated] (SPARK-26083) Pyspark command is not working properly with default Docker Image build

[ https://issues.apache.org/jira/browse/SPARK-26083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Shao updated SPARK-26083: Description: When I try to run {code:java} ./bin/pyspark{code} in a pod in Kubernetes(image built without

[jira] [Created] (SPARK-26083) Pyspark command is not working properly with default Docker Image build

Qi Shao created SPARK-26083: --- Summary: Pyspark command is not working properly with default Docker Image build Key: SPARK-26083 URL: https://issues.apache.org/jira/browse/SPARK-26083 Project: Spark

[jira] [Updated] (SPARK-25035) Replicating disk-stored blocks should avoid memory mapping

[ https://issues.apache.org/jira/browse/SPARK-25035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-25035: - Labels: memory-analysis (was: ) > Replicating disk-stored blocks should avoid memory mapping >

[jira] [Updated] (SPARK-26082) Misnaming of spark.mesos.fetch(er)Cache.enable in MesosClusterScheduler

[ https://issues.apache.org/jira/browse/SPARK-26082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Loncaric updated SPARK-26082: Description: Currently in [docs|https://spark.apache.org/docs/latest/running-on-mesos.html

[jira] [Updated] (SPARK-26082) Misnaming of spark.mesos.fetch(er)Cache.enable in MesosClusterScheduler

[ https://issues.apache.org/jira/browse/SPARK-26082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Loncaric updated SPARK-26082: Description: Currently in [docs|https://spark.apache.org/docs/latest/running-on-mesos.html

[jira] [Updated] (SPARK-26082) Misnaming of spark.mesos.fetch(er)Cache.enable in MesosClusterScheduler

[ https://issues.apache.org/jira/browse/SPARK-26082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Loncaric updated SPARK-26082: Description: Currently in [docs|https://spark.apache.org/docs/latest/running-on-mesos.html

[jira] [Created] (SPARK-26082) Misnaming of spark.mesos.fetch(er)Cache.enable in MesosClusterScheduler

Martin Loncaric created SPARK-26082: --- Summary: Misnaming of spark.mesos.fetch(er)Cache.enable in MesosClusterScheduler Key: SPARK-26082 URL: https://issues.apache.org/jira/browse/SPARK-26082 Project

[jira] [Assigned] (SPARK-26081) Do not write empty files by text datasources

[ https://issues.apache.org/jira/browse/SPARK-26081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26081: Assignee: (was: Apache Spark) > Do not write empty files by text datasources > --

[jira] [Assigned] (SPARK-26081) Do not write empty files by text datasources

[ https://issues.apache.org/jira/browse/SPARK-26081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26081: Assignee: Apache Spark > Do not write empty files by text datasources > -

[jira] [Commented] (SPARK-26081) Do not write empty files by text datasources

[ https://issues.apache.org/jira/browse/SPARK-26081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688664#comment-16688664 ] Apache Spark commented on SPARK-26081: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-23128) A new approach to do adaptive execution in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-23128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688653#comment-16688653 ] Apache Spark commented on SPARK-23128: -- User 'justinuang' has created a pull reques

[jira] [Commented] (SPARK-23128) A new approach to do adaptive execution in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-23128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688654#comment-16688654 ] Apache Spark commented on SPARK-23128: -- User 'justinuang' has created a pull reques

[jira] [Created] (SPARK-26081) Do not write empty files by text datasources

Maxim Gekk created SPARK-26081: -- Summary: Do not write empty files by text datasources Key: SPARK-26081 URL: https://issues.apache.org/jira/browse/SPARK-26081 Project: Spark Issue Type: Improvem

[jira] [Created] (SPARK-26080) Unable to run worker.py on Windows

Hayden Jeune created SPARK-26080: Summary: Unable to run worker.py on Windows Key: SPARK-26080 URL: https://issues.apache.org/jira/browse/SPARK-26080 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-26079) Flaky test: StreamingQueryListenersConfSuite

[ https://issues.apache.org/jira/browse/SPARK-26079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26079: Assignee: Apache Spark > Flaky test: StreamingQueryListenersConfSuite > -

[jira] [Comment Edited] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688564#comment-16688564 ] Stavros Kontopoulos edited comment on SPARK-14220 at 11/15/18 7:45 PM: ---

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688564#comment-16688564 ] Stavros Kontopoulos commented on SPARK-14220: - That one was fixed here: [htt

[jira] [Assigned] (SPARK-26079) Flaky test: StreamingQueryListenersConfSuite

[ https://issues.apache.org/jira/browse/SPARK-26079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26079: Assignee: (was: Apache Spark) > Flaky test: StreamingQueryListenersConfSuite > --

[jira] [Commented] (SPARK-26079) Flaky test: StreamingQueryListenersConfSuite

[ https://issues.apache.org/jira/browse/SPARK-26079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688561#comment-16688561 ] Apache Spark commented on SPARK-26079: -- User 'vanzin' has created a pull request fo

[jira] [Created] (SPARK-26079) Flaky test: StreamingQueryListenersConfSuite

Marcelo Vanzin created SPARK-26079: -- Summary: Flaky test: StreamingQueryListenersConfSuite Key: SPARK-26079 URL: https://issues.apache.org/jira/browse/SPARK-26079 Project: Spark Issue Type:

  1   2   >