gemini-code-assist[bot] commented on code in PR #38783:
URL: https://github.com/apache/beam/pull/38783#discussion_r3347415253
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java:
##########
@@ -379,12 +383,32 @@ public WriteResult expandUntriggered(
PCollection<KV<DestinationT, StorageApiWritePayload>>
successfulConvertedRows =
convertMessagesResult.get(successfulConvertedRowsTag);
- if (numShards > 0) {
+ if (numShards > 0 && input.isBounded() == PCollection.IsBounded.UNBOUNDED)
{
successfulConvertedRows =
successfulConvertedRows.apply(
"ResdistibuteNumShards",
Redistribute.<KV<DestinationT,
StorageApiWritePayload>>arbitrarily()
.withNumBuckets(numShards));
+ } else if (numShards > 0 && input.isBounded() ==
PCollection.IsBounded.BOUNDED) {
+ successfulConvertedRows =
+ successfulConvertedRows
+ .apply(
+ "Add shard",
+ WithKeys.of(
+ (SerializableFunction<KV<DestinationT,
StorageApiWritePayload>, Integer>)
+ kv ->
+ Math.abs(
+ Hashing.murmur3_32_fixed()
+ .hashString(
+ dynamicDestinations
+ .getTable(kv.getKey())
+ .getShortTableUrn(),
+ StandardCharsets.UTF_8)
+ .asInt()
+ ^
ThreadLocalRandom.current().nextInt(numShards))
+ % numShards))
Review Comment:
Using `Math.abs(...) % numShards` can result in a negative shard number if
the hashed value XORed with the random integer is `Integer.MIN_VALUE`. This is
because `Math.abs(Integer.MIN_VALUE)` is still `Integer.MIN_VALUE`, and taking
the modulo of a negative number in Java yields a negative (or zero) result. To
guarantee a non-negative shard index, use `Math.floorMod` instead.
```suggestion
Math.floorMod(
Hashing.murmur3_32_fixed()
.hashString(
dynamicDestinations
.getTable(kv.getKey())
.getShortTableUrn(),
StandardCharsets.UTF_8)
.asInt()
^
ThreadLocalRandom.current().nextInt(numShards),
numShards)))
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
