Thanks for opening the ticket. I've asked a committer who knows the
streaming sink well to take a look at the ticket.
On Fri, Apr 24, 2020 at 6:47 AM Lu Niu wrote:
> Hi, Robert
>
> BTW, I did some field study and I think it's possible to support streaming
> sink using presto s3 filesystem. I
Hi, Robert
BTW, I did some field study and I think it's possible to support streaming
sink using presto s3 filesystem. I think that would help user to use presto
s3 fs in all access to s3. I created this jira ticket
https://issues.apache.org/jira/browse/FLINK-17364 . what do you think?
Best
Lu
Cool, thanks!
On Tue, Apr 21, 2020 at 4:51 AM Robert Metzger wrote:
> I'm not aware of anything. I think the presto s3 file system is generally
> the recommended S3 FS implementation.
>
> On Mon, Apr 13, 2020 at 11:46 PM Lu Niu wrote:
>
>> Thank you both. Given the debug overhead, I might just
I'm not aware of anything. I think the presto s3 file system is generally
the recommended S3 FS implementation.
On Mon, Apr 13, 2020 at 11:46 PM Lu Niu wrote:
> Thank you both. Given the debug overhead, I might just try out presto s3
> file system then. Besides that presto s3 file system
Thank you both. Given the debug overhead, I might just try out presto s3
file system then. Besides that presto s3 file system doesn't support
streaming sink, is there anything else I need to keep in mind? Thanks!
Best
Lu
On Thu, Apr 9, 2020 at 12:29 AM Robert Metzger wrote:
> Hey,
> Others
Hey,
Others have experienced this as well, yes:
https://lists.apache.org/thread.html/5cfb48b36e2aa2b91b2102398ddf561877c28fdbabfdb59313965f0a%40%3Cuser.flink.apache.org%3EDiskErrorException
I have also notified the Hadoop project about this issue:
https://issues.apache.org/jira/browse/HADOOP-15915
Hi LU
I'm not familiar with S3 file system, maybe others in Flink community can
help you in this case, or maybe you can also reach out to s3
teams/community for help.
Best,
Congxian
Lu Niu 于2020年4月8日周三 上午11:05写道：
> Hi, Congxiao
>
> Thanks for replying. yeah, I also found those references.
Hi, Congxiao
Thanks for replying. yeah, I also found those references. However, as I
mentioned in original post, there is enough capacity in all disk. Also,
when I switch to presto file system, the problem goes away. Wondering
whether others encounter similar issue.
Best
Lu
On Tue, Apr 7, 2020
Hi
>From the stack, seems the problem is that "org.apache.flink.fs.shaded.
hadoop3.org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not
find any valid local directory for s3ablock-0001-", and I googled the
exception, found there is some relative page[1], could you please make sure
