[
https://issues.apache.org/jira/browse/SPARK-22947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496142#comment-17496142
]
Li Jin commented on SPARK-22947:
For those who are interested, I will no longer on work on this SPIP.
[
https://issues.apache.org/jira/browse/SPARK-33057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215468#comment-17215468
]
Li Jin commented on SPARK-33057:
I agree this is an improvement rather than a bug.
Although, I am not
[
https://issues.apache.org/jira/browse/SPARK-33057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-33057:
---
Description:
Current, trying to use filter with a window operations will fail:
{code:java}
df =
Li Jin created SPARK-33057:
--
Summary: Cannot use filter with window operations
Key: SPARK-33057
URL: https://issues.apache.org/jira/browse/SPARK-33057
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-28482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896132#comment-16896132
]
Li Jin edited comment on SPARK-28482 at 7/30/19 1:28 PM:
-
Hi [~jiangyu1211],
[
https://issues.apache.org/jira/browse/SPARK-28482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896132#comment-16896132
]
Li Jin edited comment on SPARK-28482 at 7/30/19 1:28 PM:
-
Hi [~jiangyu1211],
[
https://issues.apache.org/jira/browse/SPARK-28482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896132#comment-16896132
]
Li Jin commented on SPARK-28482:
Hi [~jiangyu1211], thank you for the bug report.
>From the
[
https://issues.apache.org/jira/browse/SPARK-28502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894134#comment-16894134
]
Li Jin commented on SPARK-28502:
Hmm.. I think this has sth to do with timezone, can you try setting the
[
https://issues.apache.org/jira/browse/SPARK-28422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-28422:
---
Description:
{code:java}
@pandas_udf('double', PandasUDFType.GROUPED_AGG)
def max_udf(v):
return
[
https://issues.apache.org/jira/browse/SPARK-28422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-28422:
---
Summary: GROUPED_AGG pandas_udf doesn't with spark.sql() without group by
clause (was: GROUPED_AGG
[
https://issues.apache.org/jira/browse/SPARK-28422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-28422:
---
Description:
{code:java}
@pandas_udf('double', PandasUDFType.GROUPED_AGG)
def max_udf(v):
return
Li Jin created SPARK-28422:
--
Summary: GROUPED_AGG pandas_udf doesn't with spark.sql without
group by clause
Key: SPARK-28422
URL: https://issues.apache.org/jira/browse/SPARK-28422
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-28006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863186#comment-16863186
]
Li Jin edited comment on SPARK-28006 at 6/13/19 3:36 PM:
-
Hi [~viirya] good
[
https://issues.apache.org/jira/browse/SPARK-28006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863186#comment-16863186
]
Li Jin commented on SPARK-28006:
Hi [~viirya] good questions:
>> Can we use pandas agg udfs as window
[
https://issues.apache.org/jira/browse/SPARK-27463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863177#comment-16863177
]
Li Jin commented on SPARK-27463:
Yeah I think the exact spelling of the API can go either way. I think
[
https://issues.apache.org/jira/browse/SPARK-28006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-28006:
---
Description:
Currently, pandas_udf supports "grouped aggregate" type that can be used with
unbounded and
[
https://issues.apache.org/jira/browse/SPARK-28006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862505#comment-16862505
]
Li Jin commented on SPARK-28006:
Thanks [~hyukjin.kwon] for the comments! I updated the description to
[
https://issues.apache.org/jira/browse/SPARK-28006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-28006:
---
Description:
Currently, pandas_udf supports "grouped aggregate" type that can be used with
unbounded and
[
https://issues.apache.org/jira/browse/SPARK-28006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-28006:
---
Description:
Currently, pandas_udf supports "grouped aggregate" type that can be used with
unbounded and
[
https://issues.apache.org/jira/browse/SPARK-27463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862287#comment-16862287
]
Li Jin commented on SPARK-27463:
I think one way to design this API to mimic the existing dataset
[
https://issues.apache.org/jira/browse/SPARK-27463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862225#comment-16862225
]
Li Jin commented on SPARK-27463:
For cogroup, I don't think there is analogous API in pandas. There is
[
https://issues.apache.org/jira/browse/SPARK-28006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861559#comment-16861559
]
Li Jin commented on SPARK-28006:
cc [~hyukjin.kwon] [~LI,Xiao] [~ueshin] [~bryanc]
I think code wise
[
https://issues.apache.org/jira/browse/SPARK-28006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-28006:
---
Description:
Currently, pandas_udf supports "grouped aggregate" type that can be used with
unbounded and
Li Jin created SPARK-28006:
--
Summary: User-defined grouped transform pandas_udf for window
operations
Key: SPARK-28006
URL: https://issues.apache.org/jira/browse/SPARK-28006
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-28003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-28003:
---
Affects Version/s: (was: 2.3.2)
2.3.3
> spark.createDataFrame with Arrow doesn't
Li Jin created SPARK-28003:
--
Summary: spark.createDataFrame with Arrow doesn't work with
pandas.NaT
Key: SPARK-28003
URL: https://issues.apache.org/jira/browse/SPARK-28003
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-28003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-28003:
---
Affects Version/s: (was: 2.4.0)
2.3.2
2.4.3
>
[
https://issues.apache.org/jira/browse/SPARK-27538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16851185#comment-16851185
]
Li Jin commented on SPARK-27538:
[~hyukjin.kwon] I saw you closed this. I wonder if this should be a sub
[
https://issues.apache.org/jira/browse/SPARK-26410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726824#comment-16726824
]
Li Jin commented on SPARK-26410:
One thing we want think about is whether or not to mix different size
[
https://issues.apache.org/jira/browse/SPARK-26410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726820#comment-16726820
]
Li Jin commented on SPARK-26410:
Thanks for the explanation. I think it makes sense to have batch size
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725957#comment-16725957
]
Li Jin commented on SPARK-26412:
So this is similar to the mapPartitions API in Scala but instead of
[
https://issues.apache.org/jira/browse/SPARK-26410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725948#comment-16725948
]
Li Jin commented on SPARK-26410:
I am curious why would user want to configure maxRecordsPerBatch? As
Li Jin created SPARK-26364:
--
Summary: Clean up import statements in pandas udf tests
Key: SPARK-26364
URL: https://issues.apache.org/jira/browse/SPARK-26364
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-26328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin resolved SPARK-26328.
Resolution: Not A Problem
> Use GenerateOrdering for group key comparison in WindowExec
>
Li Jin created SPARK-26328:
--
Summary: Use GenerateOrdering for group key comparison in
WindowExec
Key: SPARK-26328
URL: https://issues.apache.org/jira/browse/SPARK-26328
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-25640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-25640:
---
Description:
Currently, grouped aggregate and window aggregate uses different EvalType,
however, they map
Li Jin created SPARK-25640:
--
Summary: Clarify/Improve EvalType for grouped aggregate and window
aggregate
Key: SPARK-25640
URL: https://issues.apache.org/jira/browse/SPARK-25640
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-25213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594973#comment-16594973
]
Li Jin commented on SPARK-25213:
This is resolved by https://github.com/apache/spark/pull/22104
>
[
https://issues.apache.org/jira/browse/SPARK-25216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-25216:
---
Description:
The current error message is often confusing to a new Spark user that a column
containing
[
https://issues.apache.org/jira/browse/SPARK-25216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-25216:
---
Description:
The current error message is often confusing to a new Spark user that a column
containing
Li Jin created SPARK-25216:
--
Summary: Provide better error message when a column contains dot
and needs backticks quote
Key: SPARK-25216
URL: https://issues.apache.org/jira/browse/SPARK-25216
Project: Spark
Li Jin created SPARK-25213:
--
Summary: DataSourceV2 doesn't seem to produce unsafe rows
Key: SPARK-25213
URL: https://issues.apache.org/jira/browse/SPARK-25213
Project: Spark
Issue Type: Task
[
https://issues.apache.org/jira/browse/SPARK-24561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16579965#comment-16579965
]
Li Jin commented on SPARK-24561:
I am looking into this. Early investigation:
[
https://issues.apache.org/jira/browse/SPARK-24721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-24721:
---
Component/s: SQL
> Failed to use PythonUDF with literal inputs in filter with data sources
>
[
https://issues.apache.org/jira/browse/SPARK-24721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-24721:
---
Issue Type: Bug (was: Sub-task)
Parent: (was: SPARK-22216)
> Failed to use PythonUDF with
[
https://issues.apache.org/jira/browse/SPARK-24721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16579956#comment-16579956
]
Li Jin edited comment on SPARK-24721 at 8/14/18 3:26 PM:
-
Updated Jira title to
[
https://issues.apache.org/jira/browse/SPARK-24721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16579956#comment-16579956
]
Li Jin commented on SPARK-24721:
Updates Jira title to reflect the actual issue
> Failed to use
[
https://issues.apache.org/jira/browse/SPARK-24721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-24721:
---
Summary: Failed to use PythonUDF with literal inputs in filter with data
sources (was: Failed to call
[
https://issues.apache.org/jira/browse/SPARK-24721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560337#comment-16560337
]
Li Jin edited comment on SPARK-24721 at 7/27/18 9:18 PM:
-
I think the issue is
[
https://issues.apache.org/jira/browse/SPARK-24721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560337#comment-16560337
]
Li Jin commented on SPARK-24721:
I think the issue is the UDF is being pushed down to the
[
https://issues.apache.org/jira/browse/SPARK-24721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560283#comment-16560283
]
Li Jin commented on SPARK-24721:
{code:java}
from pyspark.sql.functions import udf, lit, col
[
https://issues.apache.org/jira/browse/SPARK-24624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-24624:
---
Issue Type: Sub-task (was: Improvement)
Parent: SPARK-22216
> Can not mix vectorized and
[
https://issues.apache.org/jira/browse/SPARK-24721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-24721:
---
Issue Type: Sub-task (was: Improvement)
Parent: SPARK-22216
> Failed to call PythonUDF whose input
[
https://issues.apache.org/jira/browse/SPARK-24721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544656#comment-16544656
]
Li Jin commented on SPARK-24721:
I am currently traveling but will try to take a look when I get back
>
[
https://issues.apache.org/jira/browse/SPARK-24796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-24796:
---
Issue Type: Sub-task (was: Improvement)
Parent: SPARK-22216
> Support GROUPED_AGG_PANDAS_UDF in
[
https://issues.apache.org/jira/browse/SPARK-24796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544655#comment-16544655
]
Li Jin commented on SPARK-24796:
Sorry I am traveling now but I will try to take a look when I get back
[
https://issues.apache.org/jira/browse/SPARK-24760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16537662#comment-16537662
]
Li Jin commented on SPARK-24760:
I think the issue here is that the output schema for the UDF is not
[
https://issues.apache.org/jira/browse/SPARK-24721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16530462#comment-16530462
]
Li Jin commented on SPARK-24721:
Yep I can take a look
> Failed to call PythonUDF whose input is the
[
https://issues.apache.org/jira/browse/SPARK-24624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519590#comment-16519590
]
Li Jin commented on SPARK-24624:
I can take a look at this one
> Can not mix vectorized and
[
https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16515879#comment-16515879
]
Li Jin edited comment on SPARK-24578 at 6/18/18 3:24 PM:
-
cc @gatorsmile
[
https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16515879#comment-16515879
]
Li Jin commented on SPARK-24578:
cc @gatorsmile
We found this when switching from 2.2.1 to 2.3.0 in one
[
https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-24578:
---
Component/s: (was: Input/Output)
Spark Core
> Reading remote cache block behavior
[
https://issues.apache.org/jira/browse/SPARK-24563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16512844#comment-16512844
]
Li Jin commented on SPARK-24563:
Will submit a PR soon
> Allow running PySpark shell without Hive
>
[
https://issues.apache.org/jira/browse/SPARK-24563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-24563:
---
Description:
A previous commit:
Li Jin created SPARK-24563:
--
Summary: Allow running PySpark shell without Hive
Key: SPARK-24563
URL: https://issues.apache.org/jira/browse/SPARK-24563
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-22239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-22239:
---
Description:
Window function is another place we can benefit from vectored udf and add
another useful
Li Jin created SPARK-24561:
--
Summary: User-defined window functions with pandas udf (bounded
window)
Key: SPARK-24561
URL: https://issues.apache.org/jira/browse/SPARK-24561
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-22239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-22239:
---
Summary: User-defined window functions with pandas udf (unbounded window)
(was: User-defined window
[
https://issues.apache.org/jira/browse/SPARK-22239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16511495#comment-16511495
]
Li Jin commented on SPARK-22239:
[~hyukjin.kwon] I actually don't think this Jira is done. The PR only
Li Jin created SPARK-24521:
--
Summary: Fix ineffective test in CachedTableSuite
Key: SPARK-24521
URL: https://issues.apache.org/jira/browse/SPARK-24521
Project: Spark
Issue Type: Test
[
https://issues.apache.org/jira/browse/SPARK-24258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506399#comment-16506399
]
Li Jin commented on SPARK-24258:
I ran into [~mengxr] and chatted about this. Seems a good first step is
[
https://issues.apache.org/jira/browse/SPARK-24373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16495695#comment-16495695
]
Li Jin commented on SPARK-24373:
[~smilegator] Thank you for the suggestion.
> "df.cache() df.count()"
[
https://issues.apache.org/jira/browse/SPARK-22947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16493815#comment-16493815
]
Li Jin edited comment on SPARK-22947 at 5/29/18 4:34 PM:
-
Hi [~TomaszGaweda]
[
https://issues.apache.org/jira/browse/SPARK-22947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16493815#comment-16493815
]
Li Jin edited comment on SPARK-22947 at 5/29/18 4:34 PM:
-
Hi [~TomaszGaweda]
[
https://issues.apache.org/jira/browse/SPARK-22947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16493815#comment-16493815
]
Li Jin commented on SPARK-22947:
Hi [~TomaszGaweda] thanks for your interest! Yes I am willing to work
[
https://issues.apache.org/jira/browse/SPARK-22947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449935#comment-16449935
]
Li Jin edited comment on SPARK-22947 at 5/29/18 4:33 PM:
-
I came across this
[
https://issues.apache.org/jira/browse/SPARK-24373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16491309#comment-16491309
]
Li Jin commented on SPARK-24373:
[~smilegator] do you mean that add AnalysisBarrier to
[
https://issues.apache.org/jira/browse/SPARK-24324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16491125#comment-16491125
]
Li Jin commented on SPARK-24324:
Moved under Spark-22216 for better ticket organization.
> Pandas
[
https://issues.apache.org/jira/browse/SPARK-24324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-24324:
---
Issue Type: Sub-task (was: Bug)
Parent: SPARK-22216
> Pandas Grouped Map UserDefinedFunction mixes
[
https://issues.apache.org/jira/browse/SPARK-24373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16491086#comment-16491086
]
Li Jin commented on SPARK-24373:
We use groupby() and pivot()
> "df.cache() df.count()" no longer
[
https://issues.apache.org/jira/browse/SPARK-24373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489060#comment-16489060
]
Li Jin edited comment on SPARK-24373 at 5/24/18 9:00 PM:
-
This is a reproduce:
[
https://issues.apache.org/jira/browse/SPARK-24373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489060#comment-16489060
]
Li Jin edited comment on SPARK-24373 at 5/24/18 9:00 PM:
-
This is a reproduce:
[
https://issues.apache.org/jira/browse/SPARK-24373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489060#comment-16489060
]
Li Jin edited comment on SPARK-24373 at 5/24/18 8:51 PM:
-
This is a reproduce:
[
https://issues.apache.org/jira/browse/SPARK-24324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489759#comment-16489759
]
Li Jin commented on SPARK-24324:
This is a dup of https://issues.apache.org/jira/browse/SPARK-23929, I am
[
https://issues.apache.org/jira/browse/SPARK-24373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489060#comment-16489060
]
Li Jin commented on SPARK-24373:
This is a reproduce in unit test:
{code:java}
test("cache and count") {
[
https://issues.apache.org/jira/browse/SPARK-24373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-24373:
---
Summary: "df.cache() df.count()" no longer eagerly caches data (was: Spark
Dataset groupby.agg/count
[
https://issues.apache.org/jira/browse/SPARK-24373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16488109#comment-16488109
]
Li Jin edited comment on SPARK-24373 at 5/23/18 11:18 PM:
--
We found after
[
https://issues.apache.org/jira/browse/SPARK-24373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-24373:
---
Component/s: (was: Input/Output)
SQL
> Spark Dataset groupby.agg/count doesn't respect
[
https://issues.apache.org/jira/browse/SPARK-24373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16488109#comment-16488109
]
Li Jin commented on SPARK-24373:
I think this might be a regression from 2.2
Any one uses "df.cache()
[
https://issues.apache.org/jira/browse/SPARK-24334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484086#comment-16484086
]
Li Jin commented on SPARK-24334:
[~pi3ni0] did it happen for you when your UDF throws exception?
> Race
[
https://issues.apache.org/jira/browse/SPARK-24334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483004#comment-16483004
]
Li Jin commented on SPARK-24334:
I have done some investigation and will submit a PR soon.
> Race
Li Jin created SPARK-24334:
--
Summary: Race condition in ArrowPythonRunner causes unclean
shutdown of Arrow memory allocator
Key: SPARK-24334
URL: https://issues.apache.org/jira/browse/SPARK-24334
Project:
[
https://issues.apache.org/jira/browse/SPARK-22239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452977#comment-16452977
]
Li Jin commented on SPARK-22239:
[~hvanhovell], I have done a bit further research of UDF over rolling
[
https://issues.apache.org/jira/browse/SPARK-23929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452355#comment-16452355
]
Li Jin commented on SPARK-23929:
[~tr3w] does using OrderedDict help in your case?
> pandas_udf schema
[
https://issues.apache.org/jira/browse/SPARK-22947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449935#comment-16449935
]
Li Jin edited comment on SPARK-22947 at 4/24/18 2:16 PM:
-
I came across this blog
[
https://issues.apache.org/jira/browse/SPARK-22947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449935#comment-16449935
]
Li Jin commented on SPARK-22947:
I came across this blog today:
[
https://issues.apache.org/jira/browse/SPARK-24019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-24019:
---
Component/s: (was: Spark Core)
SQL
> AnalysisException for Window function expression
[
https://issues.apache.org/jira/browse/SPARK-23929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438935#comment-16438935
]
Li Jin edited comment on SPARK-23929 at 4/16/18 2:40 AM:
-
I agree with
[
https://issues.apache.org/jira/browse/SPARK-23929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438935#comment-16438935
]
Li Jin commented on SPARK-23929:
I agree with [~hyukjin.kwon]. Seems like there is not a strong enough
[
https://issues.apache.org/jira/browse/SPARK-23030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438001#comment-16438001
]
Li Jin commented on SPARK-23030:
Hey [~bryanc], did you by an chance have some process on this? I guess
1 - 100 of 220 matches
Mail list logo
