[
https://issues.apache.org/jira/browse/ARROW-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17661912#comment-17661912
]
Rok Mihevc commented on ARROW-4890:
---
This issue has been migrated to [issue
[
https://issues.apache.org/jira/browse/ARROW-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17307553#comment-17307553
]
Micah Kornfield commented on ARROW-4890:
This should be fixed as of Spark 3.1 which upgrades the
[
https://issues.apache.org/jira/browse/ARROW-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17251705#comment-17251705
]
Dmitry Kravchuk commented on ARROW-4890:
[~fan_li_ya] okay.
[
https://issues.apache.org/jira/browse/ARROW-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17251693#comment-17251693
]
Liya Fan commented on ARROW-4890:
-
I think some discussions are needed.
> [Python] Spark+Arrow Grouped
[
https://issues.apache.org/jira/browse/ARROW-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17251687#comment-17251687
]
Dmitry Kravchuk commented on ARROW-4890:
[~fan_li_ya] do I need to create new issue for all this
[
https://issues.apache.org/jira/browse/ARROW-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17251653#comment-17251653
]
Liya Fan commented on ARROW-4890:
-
{{ARROW_PRE_0_15_IPC_FORMAT}} controls the {{write_legacy_ipc_format}}
[
https://issues.apache.org/jira/browse/ARROW-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17251632#comment-17251632
]
Dmitry Kravchuk commented on ARROW-4890:
Finally I've tried pandas_udf with 1.72 gb dataset,
[
https://issues.apache.org/jira/browse/ARROW-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17251627#comment-17251627
]
Dmitry Kravchuk commented on ARROW-4890:
I found out that this error relates to pyarrow version
[
https://issues.apache.org/jira/browse/ARROW-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17251575#comment-17251575
]
Dmitry Kravchuk commented on ARROW-4890:
[~fan_li_ya] [~emkornfield] thanks for your replies!
[
https://issues.apache.org/jira/browse/ARROW-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17251544#comment-17251544
]
Liya Fan commented on ARROW-4890:
-
[~emkornfield] You are right. According to the above stack trace, the
[
https://issues.apache.org/jira/browse/ARROW-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17251537#comment-17251537
]
Micah Kornfield commented on ARROW-4890:
[~fan_li_ya] I believe the 32-bit integer is for the
[
https://issues.apache.org/jira/browse/ARROW-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17251535#comment-17251535
]
Liya Fan commented on ARROW-4890:
-
[~dishka_krauch] Thanks for the detailed information.
It seems the
[
https://issues.apache.org/jira/browse/ARROW-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17251061#comment-17251061
]
Dmitry Kravchuk commented on ARROW-4890:
[~fan_li_ya] okay, here we go.
Spark version - 2.4.4
[
https://issues.apache.org/jira/browse/ARROW-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250862#comment-17250862
]
Liya Fan commented on ARROW-4890:
-
[~dishka_krauch] The problem occured during IPC from python to Java?
I
[
https://issues.apache.org/jira/browse/ARROW-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250833#comment-17250833
]
Dmitry Kravchuk commented on ARROW-4890:
[~fan_li_ya] I've just used code in the description of
[
https://issues.apache.org/jira/browse/ARROW-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250749#comment-17250749
]
Liya Fan commented on ARROW-4890:
-
[~dishka_krauch] It is difficult to find the cause without providing
[
https://issues.apache.org/jira/browse/ARROW-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250393#comment-17250393
]
Dmitry Kravchuk commented on ARROW-4890:
I've tryed pyarrow 2.0.0 on spark 2.4.4 and still got
[
https://issues.apache.org/jira/browse/ARROW-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17156903#comment-17156903
]
Ruslan Dautkhanov commented on ARROW-4890:
--
Thanks for the details [~emkornfi...@gmail.com]!
[
https://issues.apache.org/jira/browse/ARROW-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17156205#comment-17156205
]
Micah Kornfield commented on ARROW-4890:
[~Tagar] this doesn't automatically resolve the JIRA.
[
https://issues.apache.org/jira/browse/ARROW-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155244#comment-17155244
]
Ruslan Dautkhanov commented on ARROW-4890:
--
[~fan_li_ya] looks like all the follow up jiras you
20 matches
Mail list logo
