DanielLeens commented on issue #13318:
URL: https://github.com/apache/seatunnel/issues/13318#issuecomment-6983607036

   I reproduced the close-chain failure with a focused Java 11 probe against 
unmodified production methods at `89f91a89af15f33d97a942d8b65f4fc1b0c80bf5`.
   
   Steps:
   1. Create a real `SinkFlowLifeCycle` fixture with its completion future 
initialized but its writer still `null`, representing the pre-restore condition.
   2. Put it first in a concrete task's `allCycles`, followed by a lifecycle 
whose `close()` increments a counter.
   3. Invoke the actual inherited `SeaTunnelTask.close()`.
   
   Observed: `NullPointerException` originates at `SinkFlowLifeCycle.close` 
(line 279) through `SeaTunnelTask.close` (line 487), and the subsequent 
lifecycle's close counter is **0**.
   
   The fixture uses `Unsafe.allocateInstance` to avoid unrelated runtime 
constructors and explicitly initializes the fields required by this path. The 
concrete task is `TransformSeaTunnelTask`, which inherits the tested base 
implementation. This is a **method-level pre-restore cleanup probe**, not a 
live cancellation race or a measured resource leak.
   
   The executable probe below also exercises #13319. Its #13315 section is 
deliberately labeled **PARTIAL**: bounded WAITING_RESTORE polling alone does 
not prove lost error reporting or an indefinite master hang.
   
   ### How I ran the focused probe
   
   Baseline: `89f91a89af15f33d97a942d8b65f4fc1b0c80bf5`, macOS ARM64, JDK 
11.0.27. The production source is unmodified. These are **in-process 
fault-injection probes**, not end-to-end cluster/failover tests; passing the 
probe means its asserted failure signature was observed.
   
   Save the Java source below outside the checkout. From a clean checkout of 
that commit:
   
   ```bash
   export JAVA_HOME=/path/to/jdk-11
   export PATH="$JAVA_HOME/bin:$PATH"
   mvn -B -pl seatunnel-engine/seatunnel-engine-server -am \
     -DskipTests -DskipUT=true -DskipIT=true -Dskip.ui=true \
     -Dskip.spotless=true -Dmaven.gitcommitid.skip=true \
     -Dmaven.javadoc.skip=true -Drat.skip=true package
   mvn -X -Dstyle.color=never -pl seatunnel-engine/seatunnel-engine-server -am \
     -DskipTests -DskipUT=true -DskipIT=true -Dskip.ui=true \
     -Dskip.spotless=true -Dmaven.gitcommitid.skip=true \
     -Dmaven.javadoc.skip=true -Drat.skip=true test-compile > classpath.log 2>&1
   CP=$(sed -n 's/^.*(f) testPath = \[\(.*\)\]$/\1/p' classpath.log | tail -1 | 
sed 's/, /:/g')
   # The last reactor module is engine-server; CP uses these freshly compiled 
classes.
   ```
   
   This builds the required reactor modules, not the full project's test suite. 
No Docker or live database is needed. I also compared the tested revision with 
`dev` at `94adef596685ee37e506c96a52baa6ae13de5da3`; the production classes 
exercised here are unchanged between them. That comparison is not a separate 
runtime test of the newer commit.
   
   ```bash
   mkdir -p probe-classes
   javac -proc:none -cp "$CP" -d probe-classes TaskIssueProbes.java
   java -cp "probe-classes:$CP" TaskIssueProbes
   ```
   
   <details>
   <summary>Executable Java probe: TaskIssueProbes.java</summary>
   
   ```java
   /*
    * Licensed to the Apache Software Foundation (ASF) under one or more
    * contributor license agreements.  See the NOTICE file distributed with
    * this work for additional information regarding copyright ownership.
    * The ASF licenses this file to You under the Apache License, Version 2.0
    * (the "License"); you may not use this file except in compliance with
    * the License.  You may obtain a copy of the License at
    *
    *    http://www.apache.org/licenses/LICENSE-2.0
    *
    * Unless required by applicable law or agreed to in writing, software
    * distributed under the License is distributed on an "AS IS" BASIS,
    * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    * See the License for the specific language governing permissions and
    * limitations under the License.
    */
   
   import java.lang.reflect.*;
   import java.util.*;
   import java.util.concurrent.*;
   import java.util.concurrent.atomic.*;
   import sun.misc.Unsafe;
   
   /** Isolated fault-injection probes: invokes real engine methods, not a 
reimplementation. */
   public class TaskIssueProbes {
    static final String S="org.apache.seatunnel.engine.server.";
    static Unsafe u;
    static {try {Field 
f=Unsafe.class.getDeclaredField("theUnsafe");f.setAccessible(true);u=(Unsafe)f.get(null);}catch(Exception
 e){throw new RuntimeException(e);}}
    static Class<?> c(String s)throws Exception{return Class.forName(s);}
    static Object alloc(String s)throws Exception{return 
u.allocateInstance(c(s));}
    static Field field(Object o,String n)throws Exception{for(Class<?> 
k=o.getClass();k!=null;k=k.getSuperclass())try{Field 
f=k.getDeclaredField(n);f.setAccessible(true);return 
f;}catch(NoSuchFieldException e){}throw new NoSuchFieldException(n);}
    static void set(Object o,String n,Object v)throws 
Exception{field(o,n).set(o,v);}
    static Object get(Object o,String n)throws Exception{return 
field(o,n).get(o);}
    static Object future()throws Exception{return 
c("org.apache.seatunnel.engine.common.utils.concurrent.CompletableFuture").getConstructor().newInstance();}
    static Object call(Object o,String n,Class<?>[] ts,Object...v)throws 
Throwable{Method m=null;for(Class<?> 
k=o.getClass();k!=null;k=k.getSuperclass())try{m=k.getDeclaredMethod(n,ts);break;}catch(NoSuchMethodException
 e){}m.setAccessible(true);try{return 
m.invoke(o,v);}catch(InvocationTargetException e){throw e.getCause();}}
    static Object proxy(String n,InvocationHandler h)throws Exception{return 
Proxy.newProxyInstance(TaskIssueProbes.class.getClassLoader(),new 
Class[]{c(n)},h);}
    static void check(boolean x,String s){if(!x)throw new AssertionError(s);}
    static void sinkClose()throws Throwable{
     Object 
sink=alloc(S+"task.flow.SinkFlowLifeCycle");set(sink,"completableFuture",future());
     Object 
task=alloc(S+"task.TransformSeaTunnelTask");set(task,"restoreComplete",future());AtomicInteger
 later=new AtomicInteger();
     Object 
tail=proxy(S+"task.flow.FlowLifeCycle",(p,m,a)->{if(m.getName().equals("close"))later.incrementAndGet();return
 null;});
     set(task,"allCycles",Arrays.asList(sink,tail));Throwable failure=null;
     try{call(task,"close",new Class[0]);}catch(Throwable 
t){failure=t;t.printStackTrace(System.out);}
     check(failure instanceof NullPointerException,"Expected null-writer 
NPE");check(later.get()==0,"Later lifecycle unexpectedly closed");
     
check(Arrays.stream(failure.getStackTrace()).anyMatch(x->x.getClassName().endsWith("SinkFlowLifeCycle")&&x.getMethodName().equals("close")),"Wrong
 NPE origin");
     System.out.println("13318 REPRODUCED unit scope: null writer close NPE; 
subsequent lifecycle close count=0");
    }
    static void restoreWaiting()throws Throwable{
     Object task=alloc(S+"task.TransformSeaTunnelTask");Object 
f=future();set(task,"restoreComplete",f);set(task,"allCycles",Collections.emptyList());
     Class 
state=c(S+"task.statemachine.SeaTunnelTaskState");set(task,"currState",Enum.valueOf(state,"WAITING_RESTORE"));
     try{call(task,"restoreState",new 
Class[]{List.class},Arrays.asList((Object)null));throw new AssertionError("No 
injected restore failure");}catch(NullPointerException 
expected){System.out.println("Injected malformed restored state throws: 
"+expected);}
     long start=System.nanoTime();for(int 
i=0;i<20;i++)call(task,"stateProcess",new Class[0]);
     check(!((java.util.concurrent.CompletableFuture)f).isDone(),"restore 
future 
completed");check(get(task,"currState").toString().equals("WAITING_RESTORE"),"state
 changed");
     System.out.println("13315 PARTIAL task-state probe: 20 polls, 
elapsed_ms="+(System.nanoTime()-start)/1000000+", state=WAITING_RESTORE, 
restoreComplete=false. Does not simulate lost RPC or prove indefinite master 
hang.");
    }
    static void shareInit()throws Throwable{
     Object 
svc=alloc(S+"TaskExecutionService");set(svc,"threadShareTaskQueue",new 
LinkedBlockingDeque<>());
     
set(svc,"logger",proxy("com.hazelcast.logging.ILogger",(p,m,a)->m.getReturnType()==boolean.class?false:null));
     ExecutorService 
executor=Executors.newSingleThreadExecutor();set(svc,"executorService",executor);
     set(svc,"taskAsyncFunctionFuture",new 
ConcurrentHashMap<>());set(svc,"timerFlushFutures",new ConcurrentHashMap<>());
     AtomicInteger secondInit=new AtomicInteger();RuntimeException marker=new 
RuntimeException("injected-init-failure");
     Object 
first=proxy(S+"execution.Task",(p,m,a)->{if(m.getName().equals("init"))throw 
marker;if(m.getName().equals("getTaskID"))return 1L;return null;});
     Object 
second=proxy(S+"execution.Task",(p,m,a)->{if(m.getName().equals("init"))secondInit.incrementAndGet();if(m.getName().equals("getTaskID"))return
 2L;return null;});
     List tasks=Arrays.asList(first,second);Object 
loc=c(S+"execution.TaskGroupLocation").getConstructor(long.class,int.class,long.class).newInstance(1L,1,1L);
     Object 
group=proxy(S+"execution.TaskGroup",(p,m,a)->m.getName().equals("getTasks")?tasks:m.getName().equals("getTaskGroupLocation")?loc:null);
     Object 
context=alloc(S+"execution.TaskGroupContext");set(context,"taskGroup",group);
     Object 
tracker=alloc(S+"TaskExecutionService$TaskGroupExecutionTracker");set(tracker,"this$0",svc);set(tracker,"taskGroup",group);set(tracker,"context",context);set(tracker,"completionLatch",new
 AtomicInteger(2));set(tracker,"executionException",new 
AtomicReference<Throwable>());set(tracker,"isCancel",new 
AtomicBoolean(false));set(tracker,"blockingFutures",Collections.emptyList());set(tracker,"currRunningTaskFuture",new
 ConcurrentHashMap<>());Object f=future();set(tracker,"future",f);
     Throwable failure=null;try{call(svc,"submitThreadShareTask",new 
Class[]{tracker.getClass(),List.class},tracker,tasks);}catch(Throwable 
t){failure=t;t.printStackTrace(System.out);}finally{executor.shutdownNow();executor.awaitTermination(2,TimeUnit.SECONDS);}
     check(failure instanceof NullPointerException,"Expected add(null) 
NPE");check(Arrays.stream(failure.getStackTrace()).anyMatch(x->x.getClassName().equals("java.util.concurrent.LinkedBlockingDeque")),"Wrong
 NPE 
origin");check(((AtomicInteger)get(tracker,"completionLatch")).get()==1,"Expected
 remaining latch=1");check(secondInit.get()==0,"Second task init 
ran");check(!((java.util.concurrent.CompletableFuture)f).isDone(),"Tracker 
unexpectedly completed");
     System.out.println("13319 REPRODUCED submit unit scope: add(null) NPE, 
latch=1, second init=0, tracker future incomplete. Real deploy/redeploy and map 
leak not exercised.");
    }
    public static void main(String[] args)throws 
Throwable{System.out.println("Java 
"+System.getProperty("java.version"));sinkClose();restoreWaiting();shareInit();}
   }
   
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to