[jira] [Comment Edited] (SPARK-27939) Defining a schema with VectorUDT

2019-06-04 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855966#comment-16855966 ] Bryan Cutler edited comment on SPARK-27939 at 6/4/19 6:11 PM:

[jira] [Commented] (SPARK-27939) Defining a schema with VectorUDT

2019-06-04 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855969#comment-16855969 ] Bryan Cutler commented on SPARK-27939: -- Another problem with Python {{Row}} class

[jira] [Comment Edited] (SPARK-27939) Defining a schema with VectorUDT

2019-06-04 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855969#comment-16855969 ] Bryan Cutler edited comment on SPARK-27939 at 6/4/19 6:13 PM:

[jira] [Resolved] (SPARK-27939) Defining a schema with VectorUDT

2019-06-04 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-27939. -- Resolution: Not A Problem > Defining a schema with VectorUDT > ---

[jira] [Updated] (SPARK-27992) PySpark socket server should sync with JVM connection thread future

2019-06-10 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-27992: - Description: Both SPARK-27805 and SPARK-27548 identified an issue that errors in a Spark job ar

[jira] [Updated] (SPARK-27992) PySpark socket server should sync with JVM connection thread future

2019-06-10 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-27992: - Environment: (was: Both SPARK-27805 and SPARK-27548 identified an issue that errors in a Spa

[jira] [Created] (SPARK-27992) PySpark socket server should sync with JVM connection thread future

2019-06-10 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-27992: Summary: PySpark socket server should sync with JVM connection thread future Key: SPARK-27992 URL: https://issues.apache.org/jira/browse/SPARK-27992 Project: Spark

[jira] [Updated] (SPARK-27992) PySpark socket server should sync with JVM connection thread future

2019-06-10 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-27992: - Affects Version/s: (was: 2.4.3) 3.0.0 > PySpark socket server should

[jira] [Created] (SPARK-28041) Increase the minimum pandas version to 0.23.2

2019-06-13 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-28041: Summary: Increase the minimum pandas version to 0.23.2 Key: SPARK-28041 URL: https://issues.apache.org/jira/browse/SPARK-28041 Project: Spark Issue Type: Imp

[jira] [Commented] (SPARK-28041) Increase the minimum pandas version to 0.23.2

2019-06-13 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16863570#comment-16863570 ] Bryan Cutler commented on SPARK-28041: -- Yes, definitely. I made a quick PR, but we

[jira] [Created] (SPARK-28128) Pandas Grouped UDFs should skip over empty partitions

2019-06-20 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-28128: Summary: Pandas Grouped UDFs should skip over empty partitions Key: SPARK-28128 URL: https://issues.apache.org/jira/browse/SPARK-28128 Project: Spark Issue T

[jira] [Resolved] (SPARK-28131) Update document type conversion between Python data and SQL types in normal UDFs (Python 3.7)

2019-06-21 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-28131. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24929 [https://gi

[jira] [Assigned] (SPARK-28131) Update document type conversion between Python data and SQL types in normal UDFs (Python 3.7)

2019-06-21 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-28131: Assignee: Hyukjin Kwon > Update document type conversion between Python data and SQL type

[jira] [Assigned] (SPARK-28132) Update document type conversion for Pandas UDFs (pyarrow 0.13.0, pandas 0.24.2, Python 3.7)

2019-06-21 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-28132: Assignee: Hyukjin Kwon > Update document type conversion for Pandas UDFs (pyarrow 0.13.0,

[jira] [Resolved] (SPARK-28132) Update document type conversion for Pandas UDFs (pyarrow 0.13.0, pandas 0.24.2, Python 3.7)

2019-06-21 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-28132. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24930 [https://gi

[jira] [Updated] (SPARK-27992) PySpark socket server should sync with JVM connection thread future

2019-06-24 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-27992: - Description: Both SPARK-27805 and SPARK-27548 identified an issue that errors in a Spark job ar

[jira] [Resolved] (SPARK-28003) spark.createDataFrame with Arrow doesn't work with pandas.NaT

2019-06-24 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-28003. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24844 [https://gi

[jira] [Assigned] (SPARK-28003) spark.createDataFrame with Arrow doesn't work with pandas.NaT

2019-06-24 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-28003: Assignee: Li Jin > spark.createDataFrame with Arrow doesn't work with pandas.NaT > -

[jira] [Assigned] (SPARK-27992) PySpark socket server should sync with JVM connection thread future

2019-06-26 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-27992: Assignee: Bryan Cutler > PySpark socket server should sync with JVM connection thread fut

[jira] [Resolved] (SPARK-27992) PySpark socket server should sync with JVM connection thread future

2019-06-26 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-27992. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24834 [https://gi

[jira] [Updated] (SPARK-28269) Pandas Grouped Map UDF can get deadlocked

2019-07-16 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-28269: - Summary: Pandas Grouped Map UDF can get deadlocked (was: ArrowStreamPandasSerializer get stack)

[jira] [Commented] (SPARK-28269) Pandas Grouped Map UDF can get deadlocked

2019-07-16 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16886400#comment-16886400 ] Bryan Cutler commented on SPARK-28269: -- cc [~icexelloss] > Pandas Grouped Map UDF

[jira] [Commented] (SPARK-28264) Revisiting Python / pandas UDF

2019-07-25 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16893209#comment-16893209 ] Bryan Cutler commented on SPARK-28264: -- It's great to be taking another look at thi

[jira] [Commented] (SPARK-28502) Error with struct conversion while using pandas_udf

2019-07-26 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16894025#comment-16894025 ] Bryan Cutler commented on SPARK-28502: -- I'm not sure, but I don't think you can use

[jira] [Resolved] (SPARK-32686) Un-deprecate inferring DataFrame schema from list of dictionaries

2020-08-24 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-32686. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29510 [https://gi

[jira] [Assigned] (SPARK-32686) Un-deprecate inferring DataFrame schema from list of dictionaries

2020-08-24 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-32686: Assignee: Nicholas Chammas > Un-deprecate inferring DataFrame schema from list of diction

[jira] [Commented] (SPARK-32312) Upgrade Apache Arrow to 1.0.0

2020-09-03 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17190553#comment-17190553 ] Bryan Cutler commented on SPARK-32312: -- Sorry for the delay, I was holding off for

[jira] [Commented] (SPARK-24554) Add MapType Support for Arrow in PySpark

2020-10-01 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205719#comment-17205719 ] Bryan Cutler commented on SPARK-24554: -- I started working on this, but ran into an

[jira] [Created] (SPARK-33073) Improve error handling on Pandas to Arrow conversion failures

2020-10-05 Thread Bryan Cutler (Jira)
Bryan Cutler created SPARK-33073: Summary: Improve error handling on Pandas to Arrow conversion failures Key: SPARK-33073 URL: https://issues.apache.org/jira/browse/SPARK-33073 Project: Spark

[jira] [Updated] (SPARK-33073) Improve error handling on Pandas to Arrow conversion failures

2020-10-05 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-33073: - Description: Currently, when converting from Pandas to Arrow for Pandas UDF return values or fr

[jira] [Commented] (SPARK-33189) Support PyArrow 2.0.0+

2020-10-20 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17217779#comment-17217779 ] Bryan Cutler commented on SPARK-33189: -- There is an env var we can set that will us

[jira] [Commented] (SPARK-33213) Upgrade Apache Arrow to 2.0.0

2020-10-23 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17219840#comment-17219840 ] Bryan Cutler commented on SPARK-33213: -- Just a couple notes: The library and forma

[jira] [Commented] (SPARK-33279) Spark 3.0 failure due to lack of Arrow dependency

2020-11-01 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17224420#comment-17224420 ] Bryan Cutler commented on SPARK-33279: -- [~fan_li_ya] we should change the Arrow-Spa

[jira] [Comment Edited] (SPARK-33279) Spark 3.0 failure due to lack of Arrow dependency

2020-11-01 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17224420#comment-17224420 ] Bryan Cutler edited comment on SPARK-33279 at 11/2/20, 5:21 AM: --

[jira] [Commented] (SPARK-16832) CrossValidator and TrainValidationSplit are not random without seed

2016-08-01 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15403067#comment-15403067 ] Bryan Cutler commented on SPARK-16832: -- The default seed value is a constant, this i

[jira] [Commented] (SPARK-16832) CrossValidator and TrainValidationSplit are not random without seed

2016-08-04 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408190#comment-15408190 ] Bryan Cutler commented on SPARK-16832: -- Yeah, I'm not sure of the reason myself, but

[jira] [Updated] (SPARK-16260) ML Example Improvements and Cleanup

2016-08-05 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-16260: - Summary: ML Example Improvements and Cleanup (was: PySpark ML Example Improvements and Cleanup)

[jira] [Updated] (SPARK-16260) ML Example Improvements and Cleanup

2016-08-05 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-16260: - Description: This parent task is to track a few possible improvements and cleanup for PySpark ML

[jira] [Reopened] (SPARK-15702) Update document programming-guide accumulator section

2016-08-05 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reopened SPARK-15702: -- I'm reopening this because I think the current programming guide accumulator section is confusing.

[jira] [Created] (SPARK-16932) Programming-guide Accumulator section should be more clear w.r.t new API

2016-08-05 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-16932: Summary: Programming-guide Accumulator section should be more clear w.r.t new API Key: SPARK-16932 URL: https://issues.apache.org/jira/browse/SPARK-16932 Project: Spa

[jira] [Closed] (SPARK-15702) Update document programming-guide accumulator section

2016-08-05 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler closed SPARK-15702. Resolution: Fixed > Update document programming-guide accumulator section > ---

[jira] [Updated] (SPARK-15018) PySpark ML Pipeline fails when no stages set

2016-08-19 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-15018: - Priority: Minor (was: Major) > PySpark ML Pipeline fails when no stages set > --

[jira] [Updated] (SPARK-15018) PySpark ML Pipeline fails when no stages set

2016-08-19 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-15018: - Issue Type: Improvement (was: Bug) > PySpark ML Pipeline fails when no stages set >

[jira] [Updated] (SPARK-15018) PySpark ML Pipeline raises unclear error when no stages set

2016-08-19 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-15018: - Summary: PySpark ML Pipeline raises unclear error when no stages set (was: PySpark ML Pipeline f

[jira] [Updated] (SPARK-15018) PySpark ML Pipeline raises unclear error when no stages set

2016-08-19 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-15018: - Description: When fitting a PySpark Pipeline with no stages, it should work as an identity trans

[jira] [Updated] (SPARK-15018) PySpark ML Pipeline raises unclear error when no stages set

2016-08-19 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-15018: - Description: When fitting a PySpark Pipeline with no stages, it should work as an identity trans

[jira] [Resolved] (SPARK-16197) Cleanup PySpark status api and example

2016-08-19 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-16197. -- Resolution: Won't Fix This minor change is would be better addressed during a QA audit > Clean

[jira] [Created] (SPARK-17161) Add PySpark-ML JavaWrapper convienience function to create py4j JavaArrays

2016-08-19 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-17161: Summary: Add PySpark-ML JavaWrapper convienience function to create py4j JavaArrays Key: SPARK-17161 URL: https://issues.apache.org/jira/browse/SPARK-17161 Project: S

[jira] [Updated] (SPARK-17161) Add PySpark-ML JavaWrapper convenience function to create py4j JavaArrays

2016-08-19 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-17161: - Summary: Add PySpark-ML JavaWrapper convenience function to create py4j JavaArrays (was: Add PyS

[jira] [Commented] (SPARK-17387) Creating SparkContext() from python without spark-submit ignores user conf

2016-09-08 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475242#comment-15475242 ] Bryan Cutler commented on SPARK-17387: -- [~vanzin] you said if you use PySpark you co

[jira] [Commented] (SPARK-31299) Pyspark.ml.clustering illegalArgumentException with dataframe created from rows

2020-04-01 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073027#comment-17073027 ] Bryan Cutler commented on SPARK-31299: -- It looks like you are using {{DenseVector}}

[jira] [Resolved] (SPARK-31299) Pyspark.ml.clustering illegalArgumentException with dataframe created from rows

2020-04-01 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-31299. -- Resolution: Not A Problem > Pyspark.ml.clustering illegalArgumentException with dataframe crea

[jira] [Updated] (SPARK-31299) Pyspark.ml.clustering illegalArgumentException with dataframe created from rows

2020-04-01 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-31299: - Description: I hope this is the right place and way to report a bug in (at least) the PySpark A

[jira] [Resolved] (SPARK-31306) rand() function documentation suggests an inclusive upper bound of 1.0

2020-04-10 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-31306. -- Resolution: Fixed Issue resolved by pull request 28071 https://github.com/apache/spark/pull/28

[jira] [Assigned] (SPARK-31306) rand() function documentation suggests an inclusive upper bound of 1.0

2020-04-10 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-31306: Assignee: Bryan Cutler > rand() function documentation suggests an inclusive upper bound

[jira] [Assigned] (SPARK-31306) rand() function documentation suggests an inclusive upper bound of 1.0

2020-04-10 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-31306: Assignee: (was: Bryan Cutler) > rand() function documentation suggests an inclusive u

[jira] [Assigned] (SPARK-31306) rand() function documentation suggests an inclusive upper bound of 1.0

2020-04-13 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-31306: Assignee: Ben > rand() function documentation suggests an inclusive upper bound of 1.0 >

[jira] [Commented] (SPARK-31629) "py4j.protocol.Py4JJavaError: An error occurred while calling o90.save" in pyspark 2.3.1

2020-05-05 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100154#comment-17100154 ] Bryan Cutler commented on SPARK-31629: -- [~appleyuchi] are you able to try out a mor

[jira] [Commented] (SPARK-31704) PandasUDFType.GROUPED_AGG with Java 11

2020-05-13 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17106500#comment-17106500 ] Bryan Cutler commented on SPARK-31704: -- This is due to a Netty API that Arrow uses

[jira] [Assigned] (SPARK-25351) Handle Pandas category type when converting from Python with Arrow

2020-05-27 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-25351: Assignee: Jalpan Randeri > Handle Pandas category type when converting from Python with A

[jira] [Resolved] (SPARK-25351) Handle Pandas category type when converting from Python with Arrow

2020-05-27 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-25351. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 26585 [https://gi

[jira] [Resolved] (SPARK-31915) Resolve the grouping column properly per the case sensitivity in grouped and cogrouped pandas UDFs

2020-06-10 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-31915. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28777 [https://gi

[jira] [Assigned] (SPARK-31915) Resolve the grouping column properly per the case sensitivity in grouped and cogrouped pandas UDFs

2020-06-10 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-31915: Assignee: Hyukjin Kwon > Resolve the grouping column properly per the case sensitivity in

[jira] [Created] (SPARK-31964) Avoid Pandas import for CategoricalDtype with Arrow conversion

2020-06-10 Thread Bryan Cutler (Jira)
Bryan Cutler created SPARK-31964: Summary: Avoid Pandas import for CategoricalDtype with Arrow conversion Key: SPARK-31964 URL: https://issues.apache.org/jira/browse/SPARK-31964 Project: Spark

[jira] [Commented] (SPARK-27463) SPIP: Support Dataframe Cogroup via Pandas UDFs

2019-05-17 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16842335#comment-16842335 ] Bryan Cutler commented on SPARK-27463: -- [~d80tb7] I think you could remove the SPIP

[jira] [Resolved] (SPARK-27712) createDataFrame() reorders row

2019-05-17 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-27712. -- Resolution: Duplicate > createDataFrame() reorders row > -- > >

[jira] [Commented] (SPARK-6931) python: struct.pack('!q', value) in write_long(value, stream) in serializers.py require int(but doesn't raise exceptions in common cases)

2015-04-16 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499162#comment-14499162 ] Bryan Cutler commented on SPARK-6931: - I just checked and it looks like some int() cas

[jira] [Commented] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-17 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500180#comment-14500180 ] Bryan Cutler commented on SPARK-6980: - I'll take this on if you don't mind. I've been

[jira] [Commented] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-21 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14505493#comment-14505493 ] Bryan Cutler commented on SPARK-6980: - Hi [~pwendell], I already have been working on

[jira] [Commented] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-21 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14505768#comment-14505768 ] Bryan Cutler commented on SPARK-6980: - Sure, I'd be happy to work together :D I haven

[jira] [Commented] (SPARK-7127) Broadcast spark.ml tree ensemble models for predict

2015-04-24 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511526#comment-14511526 ] Bryan Cutler commented on SPARK-7127: - Hey [~josephkb], I'd love to work on this to st

[jira] [Commented] (SPARK-7127) Broadcast spark.ml tree ensemble models for predict

2015-04-24 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511844#comment-14511844 ] Bryan Cutler commented on SPARK-7127: - Sounds good, thank you :D > Broadcast spark.ml

[jira] [Commented] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-24 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512069#comment-14512069 ] Bryan Cutler commented on SPARK-6980: - I'm working out of trunk. Changing the ActorWo

[jira] [Updated] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-24 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-6980: Attachment: Spark-6980-Test.scala Modified ActorWordCount example to produce akka timeout > Akka ti

[jira] [Issue Comment Deleted] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-24 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-6980: Comment: was deleted (was: Modified ActorWordCount example to produce akka timeout) > Akka timeout

[jira] [Commented] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-24 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512209#comment-14512209 ] Bryan Cutler commented on SPARK-6980: - Thanks for the clarification [~imranr], that ma

[jira] [Comment Edited] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-26 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512069#comment-14512069 ] Bryan Cutler edited comment on SPARK-6980 at 4/26/15 9:57 PM: --

[jira] [Commented] (SPARK-7127) Broadcast spark.ml tree ensemble models for predict

2015-04-26 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14513290#comment-14513290 ] Bryan Cutler commented on SPARK-7127: - I saw that the blocking issue was resolved and

[jira] [Comment Edited] (SPARK-7127) Broadcast spark.ml tree ensemble models for predict

2015-04-26 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14513290#comment-14513290 ] Bryan Cutler edited comment on SPARK-7127 at 4/26/15 10:59 PM: -

[jira] [Updated] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-28 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-6980: Attachment: (was: Spark-6980-Test.scala) > Akka timeout exceptions indicate which conf controls

[jira] [Updated] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-28 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-6980: Attachment: Spark-6980-Test.scala > Akka timeout exceptions indicate which conf controls them >

[jira] [Commented] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-28 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14516607#comment-14516607 ] Bryan Cutler commented on SPARK-6980: - Hey [~imranr] and [~harshg], I just added a pul

[jira] [Created] (SPARK-7236) AkkaUtils askWithReply sleeps indefinitely when a timeout exception is thrown

2015-04-29 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-7236: --- Summary: AkkaUtils askWithReply sleeps indefinitely when a timeout exception is thrown Key: SPARK-7236 URL: https://issues.apache.org/jira/browse/SPARK-7236 Project: Sp

[jira] [Updated] (SPARK-7236) AkkaUtils askWithReply sleeps indefinitely when a timeout exception is thrown

2015-04-29 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-7236: Attachment: SparkLongSleepAfterTimeout.scala Attaching some code to reproduce this issue. > AkkaUti

[jira] [Commented] (SPARK-7236) AkkaUtils askWithReply sleeps indefinitely when a timeout exception is thrown

2015-04-29 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14519862#comment-14519862 ] Bryan Cutler commented on SPARK-7236: - According to git blame, it looks like the defau

[jira] [Comment Edited] (SPARK-7236) AkkaUtils askWithReply sleeps indefinitely when a timeout exception is thrown

2015-04-29 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14519862#comment-14519862 ] Bryan Cutler edited comment on SPARK-7236 at 4/29/15 6:38 PM: --

[jira] [Updated] (SPARK-7236) AkkaUtils askWithReply sleeps indefinitely when a timeout exception is thrown

2015-04-29 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-7236: Description: When {{AkkaUtils.askWithReply}} gets a TimeoutException, the default parameters {{maxA

[jira] [Commented] (SPARK-7127) Broadcast spark.ml tree ensemble models for predict

2015-04-30 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14522832#comment-14522832 ] Bryan Cutler commented on SPARK-7127: - I've been looking through the new ensemble API

[jira] [Commented] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-05-03 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14525927#comment-14525927 ] Bryan Cutler commented on SPARK-6980: - I added another commit to the PR and some basic

[jira] [Commented] (SPARK-7127) Broadcast spark.ml tree ensemble models for predict

2015-05-03 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526322#comment-14526322 ] Bryan Cutler commented on SPARK-7127: - I have a couple questions [~josephkb] to make s

[jira] [Commented] (SPARK-7127) Broadcast spark.ml tree ensemble models for predict

2015-05-04 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14528028#comment-14528028 ] Bryan Cutler commented on SPARK-7127: - Sure, I'll keep at it :D, thanks for the tips!

[jira] [Commented] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-05-06 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531498#comment-14531498 ] Bryan Cutler commented on SPARK-6980: - [~harshg], sorry I have not seen those problems

[jira] [Created] (SPARK-7522) ML Examples option for dataFormat should not be enclosed in angle brackets

2015-05-10 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-7522: --- Summary: ML Examples option for dataFormat should not be enclosed in angle brackets Key: SPARK-7522 URL: https://issues.apache.org/jira/browse/SPARK-7522 Project: Spark

[jira] [Commented] (SPARK-7522) ML Examples option for dataFormat should not be enclosed in angle brackets

2015-05-11 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14537674#comment-14537674 ] Bryan Cutler commented on SPARK-7522: - pinging [~josephkb] to verify > ML Examples op

[jira] [Commented] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-05-12 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14541072#comment-14541072 ] Bryan Cutler commented on SPARK-6980: - [~harshg] I'm close to pushing the changes from

[jira] [Commented] (SPARK-7127) Broadcast spark.ml tree ensemble models for predict

2015-05-12 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14541326#comment-14541326 ] Bryan Cutler commented on SPARK-7127: - Hi [~josephkb], I've been working with to inco

[jira] [Resolved] (SPARK-38098) Add support for ArrayType of nested StructType to arrow-based conversion

2022-09-22 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-38098. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 35391 [https://gi

[jira] [Assigned] (SPARK-38098) Add support for ArrayType of nested StructType to arrow-based conversion

2022-09-22 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-38098: Assignee: Luca Canali > Add support for ArrayType of nested StructType to arrow-based con

[jira] [Commented] (SPARK-10560) Make StreamingLogisticRegressionWithSGD Python API equals with Scala one

2015-09-30 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939067#comment-14939067 ] Bryan Cutler commented on SPARK-10560: -- Hi [~yanboliang], I'd be happy to do this un

[jira] [Commented] (SPARK-10560) Make StreamingLogisticRegressionWithSGD Python API equals with Scala one

2015-10-01 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14940050#comment-14940050 ] Bryan Cutler commented on SPARK-10560: -- Great, thanks! I will work on this > Make

<    2   3   4   5   6   7   8   >