[
https://issues.apache.org/jira/browse/SPARK-33184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17217001#comment-17217001
]
colin fang commented on SPARK-33184:
I notice there is a quotation mark before `Proj
[
https://issues.apache.org/jira/browse/SPARK-33184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
colin fang updated SPARK-33184:
---
Issue Type: Bug (was: Improvement)
> spark doesn't read data source column if it is used as an inde
[
https://issues.apache.org/jira/browse/SPARK-33184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
colin fang updated SPARK-33184:
---
Summary: spark doesn't read data source column if it is used as an index to
an array under a struct
[
https://issues.apache.org/jira/browse/SPARK-33184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
colin fang updated SPARK-33184:
---
Description:
{code:python}
df = spark.createDataFrame([[1, [[1, 2,
schema='x:int,y:struct>')
df
[
https://issues.apache.org/jira/browse/SPARK-33184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
colin fang updated SPARK-33184:
---
Description:
{code:python}
df = spark.createDataFrame([[1, [[1, 2,
schema='x:int,y:struct>')
df
colin fang created SPARK-33184:
--
Summary: spark doesn't read data source column if it is needed as
an index to an array in a nested struct
Key: SPARK-33184
URL: https://issues.apache.org/jira/browse/SPARK-33184
colin fang created SPARK-28148:
--
Summary: repartition after join is not optimized away
Key: SPARK-28148
URL: https://issues.apache.org/jira/browse/SPARK-28148
Project: Spark
Issue Type: Improvem
[
https://issues.apache.org/jira/browse/SPARK-27759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
colin fang updated SPARK-27759:
---
Description:
{code:java}
pd_df = pd.DataFrame({'x': np.random.rand(11, 3, 5).tolist()})
df = spark.c
colin fang created SPARK-27759:
--
Summary: Do not auto cast array to np.array in vectorized
udf
Key: SPARK-27759
URL: https://issues.apache.org/jira/browse/SPARK-27759
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-17859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830504#comment-16830504
]
colin fang commented on SPARK-17859:
The above case works for me in v2.4
{code:java}
colin fang created SPARK-27559:
--
Summary: Nullable in a given schema is not respected when reading
from parquet
Key: SPARK-27559
URL: https://issues.apache.org/jira/browse/SPARK-27559
Project: Spark
colin fang created SPARK-27217:
--
Summary: Nested schema pruning doesn't work for aggregation e.g.
`sum`.
Key: SPARK-27217
URL: https://issues.apache.org/jira/browse/SPARK-27217
Project: Spark
I
12 matches
Mail list logo