[GitHub] [incubator-uniffle] jerqi commented on issue #297: [Bug] Potenial data lost when local storage meets high-watermark

jerqi commented on issue #297: URL: https://github.com/apache/incubator-uniffle/issues/297#issuecomment-1303045963 How do we determine that disk is broken? I think the disk can't be read or written. So I don't think this is a bug. So we should let the application fail fast. But I alrea

[GitHub] [incubator-uniffle] jerqi commented on issue #263: [Improvement] Introduce statestore of shuffle-server for upgrading quickly

jerqi commented on issue #263: URL: https://github.com/apache/incubator-uniffle/issues/263#issuecomment-1302873080 I hope the behavior can be managed by our admin api, you can go ahead with your plan, but please make compatible design with admin api. -- This is an automated message from

[GitHub] [incubator-uniffle] zuston commented on issue #263: [Improvement] Introduce statestore of shuffle-server for upgrading quickly

zuston commented on issue #263: URL: https://github.com/apache/incubator-uniffle/issues/263#issuecomment-1302855824 I think I will take this issue that dumping all state when shuffle-server is triggered by external call. If you don't have any idea, I will go ahead and propose a design. @je

[GitHub] [incubator-uniffle] zuston commented on issue #297: [Bug] Possible data lost when local storage meets high-watermark

zuston commented on issue #297: URL: https://github.com/apache/incubator-uniffle/issues/297#issuecomment-1302102975 > Maybe we should merge data in disk0 and disk1? It is OK but firstly we should know the partial data in other storage. Right? This should be considered in detail.

[GitHub] [incubator-uniffle] xianjingfeng commented on issue #297: [Bug] Possible data lost when local storage meets high-watermark

xianjingfeng commented on issue #297: URL: https://github.com/apache/incubator-uniffle/issues/297#issuecomment-1302092752 Maybe we should merge data in disk0 and disk1? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [incubator-uniffle] zuston commented on issue #297: [Bug] Possible data lost when local storage meets high-watermark

zuston commented on issue #297: URL: https://github.com/apache/incubator-uniffle/issues/297#issuecomment-1302076638 It's better to introduce dynamic storage selection mechanism to solve this @jerqi PTAL, I just read the code to find this problem, If I'm wrong, please tell me. Thanks

[GitHub] [incubator-uniffle] zuston opened a new issue, #297: [Bug] Possible data lost when local storage meets high-watermark

zuston opened a new issue, #297: URL: https://github.com/apache/incubator-uniffle/issues/297 ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) ### Search before asking - [X] I have searche

[GitHub] [incubator-uniffle] zuston commented on issue #137: [Improvement][AQE] Sort MapId before the data are flushed

zuston commented on issue #137: URL: https://github.com/apache/incubator-uniffle/issues/137#issuecomment-1301958156 #293 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [incubator-uniffle] jerqi commented on issue #196: Flaky test ShuffleFlushManagerOnKerberizedHdfsTest

jerqi commented on issue #196: URL: https://github.com/apache/incubator-uniffle/issues/196#issuecomment-1301941797 @zuston Kerberos test seems unstable, I will reopen this issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu