nastra commented on issue #13178:
URL: https://github.com/apache/iceberg/issues/13178#issuecomment-2943588428
> On the contrary the implementation of Java parallel streams uses a
globally shared ForkJoinPool and does not allow you to provide your own pool.
Fork/join pools implement work-stealing, where any thread might steal a task
from a different thread's queue when blocked waiting for a subtask to complete.
This might not seem like an issue at first glance, but if you use .parallel()
for short tasks extensively throughout your codebase and later on you add one
piece of code that uses .parallel() for long (e.g. I/O) tasks, the other parts
of your codebase that use .parallel(), and that you'd expect to have consistent
performance, might experience performance degradation for no apparent reason.
The reason is work stealing.
You can suppress this warning if you are certain that all your code will
always only use .parallel() for short tasks, but even then, you have no real
control over the level of parallelism, so you're still better off using
MoreStreams (linked above)
You can find more info here: https://stackoverflow.com/a/54581148/7182570
(see
https://github.com/palantir/gradle-baseline#baseline-error-prone-checks)
@talatuyarer can you please double-check whether we should really be using
parallel streams there? It would be good to either exclude the warning or
remove parallel usage there (I'd prefer to remove it).
@CuteChuanChuan there's also now the below ones that would be good to fix:
```
> Task :iceberg-flink:iceberg-flink-2.0:compileJava
/Users/eduard.tudenhoefner/Development/workspace/iceberg/flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicWriter.java:202:
warning: [MixedMutabilityReturnType] This method returns both mutable and
immutable collections or maps from different paths. This may be confusing for
users of the method.
private static List<Integer> getEqualityFields(Table table, List<Integer>
equalityFieldIds) {
^
(see https://errorprone.info/bugpattern/MixedMutabilityReturnType)
Did you mean 'private static ImmutableList<Integer>
getEqualityFields(Table table, List<Integer> equalityFieldIds) {'?
/Users/eduard.tudenhoefner/Development/workspace/iceberg/flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicRecordInternalSerializer.java:234:
warning: [ObjectsHashCodePrimitive] Objects.hashCode(Object o) should not be
passed a primitive value
return Objects.hashCode(writeSchemaAndSpec);
^
(see https://errorprone.info/bugpattern/ObjectsHashCodePrimitive)
Did you mean 'return Boolean.hashCode(writeSchemaAndSpec);'?
```
> ImmutableEnumChecker in Timestamps#47: [Link to
code](https://github.com/apache/iceberg/blob/a4b2a0dab092821d4843749b8abc30208622e164/api/src/main/java/org/apache/iceberg/transforms/Timestamps.java#L47)
Context: The SerializableFunction field isn't annotated with Immutable. I
noticed that [Dates
enum](https://github.com/apache/iceberg/blob/main/api/src/main/java/org/apache/iceberg/transforms/Dates.java)
uses a static nested class with Immutable annotation.
Question: Should I follow the same pattern as Dates - creating an Immutable
static nested class?
@CuteChuanChuan I'm not seeing that one locally. Could you please paste the
full errorprone warning msg here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
