[GitHub] [incubator-uniffle] jerqi commented on issue #198: [Problem] Inconsistent blocks when reading shuffle data

2023-07-20 Thread via GitHub
jerqi commented on issue #198: URL: https://github.com/apache/incubator-uniffle/issues/198#issuecomment-1643699868 > > From the dashboard metrics, it looks good. Please attach pictures when this error happens and then ping me. Thanks. > > @zuston hi, when huge table join huge table,

[GitHub] [incubator-uniffle] jerqi commented on issue #198: [Problem] Inconsistent blocks when reading shuffle data

2022-10-24 Thread GitBox
jerqi commented on issue #198: URL: https://github.com/apache/incubator-uniffle/issues/198#issuecomment-1289892378 > we also met this error, but in fact it's rpc timeout, when we in compose client mode, the one time read exception will be catch, then try to read other storage , and if the

[GitHub] [incubator-uniffle] jerqi commented on issue #198: [Problem] Inconsistent blocks when reading shuffle data

2022-10-21 Thread GitBox
jerqi commented on issue #198: URL: https://github.com/apache/incubator-uniffle/issues/198#issuecomment-1286657466 > we also met this error, but in fact it's rpc timeout, when we in compose client mode, the one time read exception will be catch, then try to read other storage , and if the

[GitHub] [incubator-uniffle] jerqi commented on issue #198: [Problem] Inconsistent blocks when reading shuffle data

2022-09-13 Thread GitBox
jerqi commented on issue #198: URL: https://github.com/apache/incubator-uniffle/issues/198#issuecomment-1246287622 > Follow up this problem. > > I found the Grpc client sometimes will throw DEADLINE exception like as follows > > ``` > org.apache.uniffle.common.exception.Rss

[GitHub] [incubator-uniffle] jerqi commented on issue #198: [Problem] Inconsistent blocks when reading shuffle data

2022-09-12 Thread GitBox
jerqi commented on issue #198: URL: https://github.com/apache/incubator-uniffle/issues/198#issuecomment-1244806643 > Can you help me understand #issue_63? @jerqi `When we use the storageType MEMORY_HDFS, MEMORY_LOCALFILE_HDFS, after we read data from memory, the data may be flushed to HDFS

[GitHub] [incubator-uniffle] jerqi commented on issue #198: [Problem] Inconsistent blocks when reading shuffle data

2022-09-06 Thread GitBox
jerqi commented on issue #198: URL: https://github.com/apache/incubator-uniffle/issues/198#issuecomment-1237804431 > > We should fail fast when encountering BufferUnderflowException instead of ignoring. > > What I am curious about is whether this is caused by the fact that the data

[GitHub] [incubator-uniffle] jerqi commented on issue #198: [Problem] Inconsistent blocks when reading shuffle data

2022-09-06 Thread GitBox
jerqi commented on issue #198: URL: https://github.com/apache/incubator-uniffle/issues/198#issuecomment-1237801489 For MEMORY_HDFS or MEMORY_LOCALFILE, it's normal to read incomplete index data, so we choose to ignore instead of failing fast. -- This is an automated message from

[GitHub] [incubator-uniffle] jerqi commented on issue #198: [Problem] Inconsistent blocks when reading shuffle data

2022-09-06 Thread GitBox
jerqi commented on issue #198: URL: https://github.com/apache/incubator-uniffle/issues/198#issuecomment-1237792809 > transIndexDataToSegments It seems that we read incomplete index file. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [incubator-uniffle] jerqi commented on issue #198: [Problem] Inconsistent blocks when reading shuffle data

2022-09-05 Thread GitBox
jerqi commented on issue #198: URL: https://github.com/apache/incubator-uniffle/issues/198#issuecomment-123780 It may be a bug of our rss. I ever try to fix the similar problems by https://github.com/apache/incubator-uniffle/pull/40 and https://github.com/Tencent/Firestorm/pull/92. Cou