[jira] [Updated] (SPARK-48492) batch-read parquet files written by streaming returns non-nullable fields in schema

2024-06-03 Thread Julien Peloton (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-48492: --- Issue Type: Bug (was: New Feature) > batch-read parquet files written by streaming returns

[jira] [Updated] (SPARK-48492) batch-read parquet files written by streaming returns non-nullable fields in schema

2024-05-31 Thread Julien Peloton (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-48492: --- Description: Hello, In the documentation, it is stated that > When reading Parquet files,

[jira] [Updated] (SPARK-48492) batch-read parquet files written by streaming returns non-nullable fields in schema

2024-05-31 Thread Julien Peloton (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-48492: --- Description: Hello, In the documentation, it is stated that > When reading Parquet files,

[jira] [Updated] (SPARK-48492) batch-read parquet files written by streaming returns non-nullable fields in schema

2024-05-31 Thread Julien Peloton (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-48492: --- Description: Hello, In the documentation, it is stated that > When reading Parquet files,

[jira] [Updated] (SPARK-48492) batch-read parquet files written by streaming returns non-nullable fields in schema

2024-05-31 Thread Julien Peloton (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-48492: --- Description: Hello, In the documentation, it is stated that > When reading Parquet files,

[jira] [Updated] (SPARK-48492) batch-read parquet files written by streaming returns non-nullable fields in schema

2024-05-31 Thread Julien Peloton (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-48492: --- Description: Hello, In the documentation, it is stated that > When reading Parquet files,

[jira] [Updated] (SPARK-48492) batch-read parquet files written by streaming returns non-nullable fields in schema

2024-05-31 Thread Julien Peloton (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-48492: --- Description: Hello, In the documentation, it is stated that > When reading Parquet files,

[jira] [Updated] (SPARK-48492) batch-read parquet files written by streaming returns non-nullable fields in schema

2024-05-31 Thread Julien Peloton (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-48492: --- Description: Hello, In the documentation, it is stated that > When reading Parquet files,

[jira] [Created] (SPARK-48492) batch-read parquet files written by streaming returns non-nullable fields in schema

2024-05-31 Thread Julien Peloton (Jira)
Julien Peloton created SPARK-48492: -- Summary: batch-read parquet files written by streaming returns non-nullable fields in schema Key: SPARK-48492 URL: https://issues.apache.org/jira/browse/SPARK-48492

[jira] [Updated] (SPARK-38435) Pandas UDF with type hints crashes at import

2022-03-07 Thread Julien Peloton (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-38435: --- Description: h2. Old style pandas UDF let's consider a pandas UDF defined in the old

[jira] [Updated] (SPARK-38435) Pandas UDF with type hints crashes at import

2022-03-07 Thread Julien Peloton (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-38435: --- Description: h2. Old style pandas UDF let's consider a pandas UDF defined in the old

[jira] [Updated] (SPARK-38435) Pandas UDF with type hints crashes at import

2022-03-07 Thread Julien Peloton (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-38435: --- Description: h2. Old style pandas UDF let's consider a pandas UDF defined in the old

[jira] [Updated] (SPARK-38435) Pandas UDF with type hints crashes at import

2022-03-07 Thread Julien Peloton (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-38435: --- Description: h2. Old style pandas UDF let's consider a pandas UDF defined in the old

[jira] [Updated] (SPARK-38435) Pandas UDF with type hints crashes at import

2022-03-07 Thread Julien Peloton (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-38435: --- Description: h2. Old style pandas UDF let's consider a pandas UDF defined in the old

[jira] [Updated] (SPARK-38435) Pandas UDF with type hints crashes at import

2022-03-07 Thread Julien Peloton (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-38435: --- Description: h2. Old style pandas UDF let's consider a pandas UDF defined in the old

[jira] [Updated] (SPARK-38435) Pandas UDF with type hints crashes at import

2022-03-07 Thread Julien Peloton (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-38435: --- Description: h2. Old style pandas UDF let's consider a pandas UDF defined in the old

[jira] [Created] (SPARK-38435) Pandas UDF with type hints crashes at import

2022-03-07 Thread Julien Peloton (Jira)
Julien Peloton created SPARK-38435: -- Summary: Pandas UDF with type hints crashes at import Key: SPARK-38435 URL: https://issues.apache.org/jira/browse/SPARK-38435 Project: Spark Issue Type:

[jira] [Commented] (SPARK-29367) pandas udf not working with latest pyarrow release (0.15.0)

2019-10-07 Thread Julien Peloton (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16946266#comment-16946266 ] Julien Peloton commented on SPARK-29367: Thanks, it works like a charm! Julien > pandas udf

[jira] [Created] (SPARK-29367) pandas udf not working with latest pyarrow release (0.15.0)

2019-10-07 Thread Julien Peloton (Jira)
Julien Peloton created SPARK-29367: -- Summary: pandas udf not working with latest pyarrow release (0.15.0) Key: SPARK-29367 URL: https://issues.apache.org/jira/browse/SPARK-29367 Project: Spark

[jira] [Commented] (SPARK-26024) Dataset API: repartitionByRange(...) has inconsistent behaviour

2018-11-13 Thread Julien Peloton (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685638#comment-16685638 ] Julien Peloton commented on SPARK-26024: [~cloud_fan] I would not advocate to increase the size

[jira] [Commented] (SPARK-26024) Dataset API: repartitionByRange(...) has inconsistent behaviour

2018-11-13 Thread Julien Peloton (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685181#comment-16685181 ] Julien Peloton commented on SPARK-26024: Hi Marco and thanks for your quick reply. You are

[jira] [Updated] (SPARK-26024) Dataset API: repartitionByRange(...) has inconsistent behaviour

2018-11-12 Thread Julien Peloton (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-26024: --- Affects Version/s: 2.3.0 2.3.1 > Dataset API:

[jira] [Created] (SPARK-26024) Dataset API: repartitionByRange(...) has inconsistent behaviour

2018-11-12 Thread Julien Peloton (JIRA)
Julien Peloton created SPARK-26024: -- Summary: Dataset API: repartitionByRange(...) has inconsistent behaviour Key: SPARK-26024 URL: https://issues.apache.org/jira/browse/SPARK-26024 Project: Spark