[jira] [Updated] (HUDI-4858) HoodieSparkSqlWriter can't work concurrently
[
https://issues.apache.org/jira/browse/HUDI-4858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
cadl updated HUDI-4858:
---
Description:
When I am doing dataframe.write.format("org.apache.hudi") with ForkJoinPool, I
got a NPE at
[HoodieSparkSqlWriter.scala#L609|https://github.com/apache/hudi/blob/a3921a845f608f2018ec3237f9d7421696b7b05d/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala#L609],
the tableExists is ture, but tableConfig is null. The path of the table is
empty, actually. Variable `tableExists` is overwritten by other write request.
We may need to remove the variable tableExists from object class properties
was:
When I am doing dataframe.write.format("org.apache.hudi") with ForkJoinPool, I
got a NPE at
[HoodieSparkSqlWriter.scala#L609|https://github.com/apache/hudi/blob/a3921a845f608f2018ec3237f9d7421696b7b05d/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala#L609],
the tableExists is ture, but tableConfig is null. The path of the table is
empty, actually. Variable `tableExists` is overwritten by other write request.
We might remove the variable tableExists from object class properties
> HoodieSparkSqlWriter can't work concurrently
>
>
> Key: HUDI-4858
> URL: https://issues.apache.org/jira/browse/HUDI-4858
> Project: Apache Hudi
> Issue Type: Bug
>Reporter: cadl
>Priority: Major
>
> When I am doing dataframe.write.format("org.apache.hudi") with ForkJoinPool,
> I got a NPE at
> [HoodieSparkSqlWriter.scala#L609|https://github.com/apache/hudi/blob/a3921a845f608f2018ec3237f9d7421696b7b05d/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala#L609],
> the tableExists is ture, but tableConfig is null. The path of the table is
> empty, actually. Variable `tableExists` is overwritten by other write request.
>
> We may need to remove the variable tableExists from object class properties
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HUDI-4858) HoodieSparkSqlWriter can't work concurrently
cadl created HUDI-4858:
--
Summary: HoodieSparkSqlWriter can't work concurrently
Key: HUDI-4858
URL: https://issues.apache.org/jira/browse/HUDI-4858
Project: Apache Hudi
Issue Type: Bug
Reporter: cadl
When I am doing dataframe.write.format("org.apache.hudi") with ForkJoinPool, I
got a NPE at
[HoodieSparkSqlWriter.scala#L609|https://github.com/apache/hudi/blob/a3921a845f608f2018ec3237f9d7421696b7b05d/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala#L609],
the tableExists is ture, but tableConfig is null. The path of the table is
empty, actually. Variable `tableExists` is overwritten by other write request.
We might remove the variable tableExists from object class properties
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HUDI-1139) Add support for JuiceFS
[ https://issues.apache.org/jira/browse/HUDI-1139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cadl updated HUDI-1139: --- Status: New (was: Open) > Add support for JuiceFS > --- > > Key: HUDI-1139 > URL: https://issues.apache.org/jira/browse/HUDI-1139 > Project: Apache Hudi > Issue Type: New Feature >Reporter: cadl >Priority: Minor > Fix For: 0.6.0 > > > JuiceFS is a POSIX-compatible shared filesystem base on object storage > service. It also provides Hadoop fs SDK, we can access juicefs with `jfs://` > scheme. > > JuiceFS: [https://juicefs.com|https://juicefs.com/] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HUDI-1139) Add support for JuiceFS
[ https://issues.apache.org/jira/browse/HUDI-1139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cadl updated HUDI-1139: --- Status: Open (was: New) > Add support for JuiceFS > --- > > Key: HUDI-1139 > URL: https://issues.apache.org/jira/browse/HUDI-1139 > Project: Apache Hudi > Issue Type: New Feature >Reporter: cadl >Priority: Minor > Fix For: 0.6.0 > > > JuiceFS is a POSIX-compatible shared filesystem base on object storage > service. It also provides Hadoop fs SDK, we can access juicefs with `jfs://` > scheme. > > JuiceFS: [https://juicefs.com|https://juicefs.com/] -- This message was sent by Atlassian Jira (v8.3.4#803005)
