nzw921rx commented on issue #13223:
URL: https://github.com/apache/seatunnel/issues/13223#issuecomment-6796338096

   @SEPURI-SAI-KRISHNA Here is a simpler implementation sketch that reuses the 
existing `sinkPrimaryKeys` cache. No `TableRouting` object or separate mode 
cache is needed.
   
   The capability is checked during initialization. After validation, a present 
routing key means `KEYED`; an empty routing key means explicitly declared 
`RANDOM` when multiple replicas are enabled.
   
   ## 1. Capability declaration
   
   Add the enum:
   
   ```java
   public enum MultiTableSinkReplicaMode {
       UNSUPPORTED,
       KEYED,
       RANDOM
   }
   ```
   
   Extend the existing interface, keeping `primaryKey()` unchanged:
   
   ```java
   public interface SupportMultiTableSinkWriter<T> extends 
SupportResourceShare<T> {
   
       default MultiTableSinkReplicaMode replicaMode() {
           return MultiTableSinkReplicaMode.UNSUPPORTED;
       }
   
       default Optional<Integer> primaryKey() {
           return Optional.empty();
       }
   }
   ```
   
   The default must not infer support from the presence of a key.
   
   For the ordinary `JdbcSinkWriter`, explicitly preserve the keyed path:
   
   ```java
   @Override
   public MultiTableSinkReplicaMode replicaMode() {
       return MultiTableSinkReplicaMode.KEYED;
   }
   ```
   
   If JDBC declares `KEYED` but has no key, reject multiple replicas during 
initialization. Single-replica configurations remain supported.
   
   Leave `JdbcExactlyOnceSinkWriter` at the default until its own migration.
   
   ## 2. Initialization validation in MultiTableSinkWriter
   
   Keep the existing cache and add the initial single-table ID:
   
   ```java
   private final ConcurrentMap<String, Optional<Integer>> sinkPrimaryKeys =
           new ConcurrentHashMap<>();
   
   private final String singleTableId;
   ```
   
   Validate the declaration before adding routing metadata:
   
   ```java
   private void registerReplicaRouting(
           SinkIdentifier identifier,
           SupportMultiTableSinkWriter<?> writer,
           int replicaNum) {
       String tableId = identifier.getTableIdentifier();
       MultiTableSinkReplicaMode mode = writer.replicaMode();
       Optional<Integer> primaryKey = writer.primaryKey();
       int keyIndex = primaryKey.orElse(-1);
   
       boolean supportsReplicas =
               mode == MultiTableSinkReplicaMode.RANDOM
                       || (mode == MultiTableSinkReplicaMode.KEYED && keyIndex 
>= 0);
   
       if (replicaNum > 1 && !supportsReplicas) {
           throw new IllegalArgumentException(
                   String.format(
                           "Writer %s for table %s cannot use "
                                   + "multi_table_sink_replica=%d "
                                   + "(mode=%s, keyIndex=%d). "
                                   + "Use multi_table_sink_replica=1 "
                                   + "or configure a supported routing mode.",
                           writer.getClass().getName(),
                           tableId,
                           replicaNum,
                           mode,
                           keyIndex));
       }
   
       Optional<Integer> routingKey =
               mode == MultiTableSinkReplicaMode.KEYED
                       ? primaryKey
                       : Optional.empty();
   
       Optional<Integer> previous =
               sinkPrimaryKeys.putIfAbsent(tableId, routingKey);
   
       if (previous != null && !previous.equals(routingKey)) {
           throw new IllegalArgumentException(
                   "Inconsistent replica routing for table " + tableId);
       }
   }
   ```
   
   For multiple replicas, only these validated cases can reach the cache:
   
   - `KEYED` stores its valid key index.
   - Explicit `RANDOM` stores `Optional.empty()`.
   - `UNSUPPORTED` is rejected before registration.
   
   Comparing the cached `Optional` values therefore checks both keyed-index 
consistency and the distinction between keyed and random routing.
   
   In the final `MultiTableSinkWriter` constructor, after assigning 
`sinkWriters` and before creating the executor or shared resources:
   
   ```java
   for (Map.Entry<SinkIdentifier, SinkWriter<SeaTunnelRow, ?, ?>> entry :
           sinkWriters.entrySet()) {
       registerReplicaRouting(
               entry.getKey(),
               (SupportMultiTableSinkWriter<?>) entry.getValue(),
               queueSize);
   }
   
   singleTableId =
           sinkPrimaryKeys.size() == 1
                   ? sinkPrimaryKeys.keySet().iterator().next()
                   : null;
   ```
   
   Both existing `createWriter()` and `restoreWriter()` paths construct 
`MultiTableSinkWriter`, so they share this capability validation.
   
   Keep resource injection in its current place. Remove the old cache 
population from `initResourceManager()`:
   
   ```java
   // Remove this old assignment:
   // sinkPrimaryKeys.put(entry.getKey().getTableIdentifier(), 
sink.primaryKey());
   ```
   
   Otherwise, it could overwrite the validated routing key, particularly when a 
writer declares `RANDOM` but also provides `primaryKey()`.
   
   Ensure already-created writers and initialized resources are closed if 
initialization fails. Keep the existing creation and restoration structure.
   
   ## 3. Update the write-routing logic
   
   Keep the existing event filtering, worker bootstrap and asynchronous error 
checks. Replace the table lookup, quarantine check and routing section with:
   
   ```java
   String tableId = element.getTableId();
   if (tableId == null) {
       tableId = singleTableId;
   }
   
   if (failurePolicy.continueOtherTables() && isTableFailed(tableId)) {
       log.debug("Skip row for quarantined table {}", tableId);
       return;
   }
   
   Optional<Integer> primaryKey =
           tableId == null ? null : sinkPrimaryKeys.get(tableId);
   
   if (primaryKey == null) {
       if (failurePolicy.continueOtherTables()) {
           handleTableFailure(
                   tableId,
                   MultiTableFailurePhase.RUNTIME_WRITE,
                   new IllegalStateException(
                           "No active sink writer found for table " + tableId));
           return;
       }
   
       throw new RuntimeException(
               "multi table sink can not write table: " + tableId);
   }
   
   int index;
   if (blockingQueues.size() == 1) {
       index = 0;
   } else if (primaryKey.isPresent()) {
       Object key = element.getField(primaryKey.get());
       index =
               key == null
                       ? 0
                       : HashUtils.bucketIndex(
                               key.hashCode(), blockingQueues.size());
   } else {
       // With multiple replicas, initialization permits an empty routing key
       // only for a writer that explicitly declares RANDOM.
       index = random.nextInt(blockingQueues.size());
   }
   
   offerRowElement(index, element);
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to