Thanks Hang and Gyula for the quick response! That makes sense, I can resolve manually for now.
On Thu, May 4, 2023 at 8:37 PM Hang Ruan <ruanhang1...@gmail.com> wrote: > Hi, Nathan, > > I think the previous discussion[1] is helpful for you. > > Best, > Hang > > [1] https://lists.apache.org/thread/cgwsr6f1l3202ktwcvtyxtdsgj2vyms1 > > Nathan Moderwell <nathan.moderw...@robinhood.com> 于2023年5月5日周五 03:54写道： > >> Hi, >> >> I'm running the sql-runner example in the flink-kuberenetes-operator repo >> and hitting an error. I've tried building the image from different releases >> of the repo and that does not help. The pods run (status stays at Running) >> and it is able to build the execution graph successfully, however I see the >> error below (the important part seems to be `The file STDOUT does not exist >> on the TaskExecutor`). I've also tried building the python example that has >> a similar sql query and I get the same error. I'm running this example on >> minikube, building everything as-is from the OSS flink-kubernetes-operator >> repo. Any idea why I'm getting this error? >> >> Full Error Logs: >> >> 2023-05-04 19:36:29,983 INFO >> org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: >> orders[1] -> Sink: print_table[2] (1/1) >> (fa0a897ca85acd1d2b7719bb2c929d2f_cbc357ccb763df2852fee8c4fc7d55f2_0_0) >> switched from SCHEDULED to DEPLOYING. >> 2023-05-04 19:36:30,092 INFO >> org.apache.flink.runtime.executiongraph.ExecutionGraph [] - >> Deploying Source: orders[1] -> Sink: print_table[2] (1/1) (attempt #0) with >> attempt id >> fa0a897ca85acd1d2b7719bb2c929d2f_cbc357ccb763df2852fee8c4fc7d55f2_0_0 and >> vertex id cbc357ccb763df2852fee8c4fc7d55f2_0 to >> python-example-taskmanager-1-1 @ 10.244.0.40 (dataPort=39441) with >> allocation id fe50815410ace91f428c4aee34bce7ab >> 2023-05-04 19:36:30,388 INFO >> org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: >> orders[1] -> Sink: print_table[2] (1/1) >> (fa0a897ca85acd1d2b7719bb2c929d2f_cbc357ccb763df2852fee8c4fc7d55f2_0_0) >> switched from DEPLOYING to INITIALIZING. >> 2023-05-04 19:36:30,496 INFO >> org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: >> orders[1] -> Sink: print_table[2] (1/1) >> (fa0a897ca85acd1d2b7719bb2c929d2f_cbc357ccb763df2852fee8c4fc7d55f2_0_0) >> switched from INITIALIZING to RUNNING. >> 2023-05-04 19:39:23,567 ERROR >> org.apache.flink.runtime.rest.handler.taskmanager.TaskManagerStdoutFileHandler >> [] - Failed to transfer file from TaskExecutor >> python-example-taskmanager-1-1. >> java.util.concurrent.CompletionException: >> org.apache.flink.util.FlinkException: The file STDOUT does not exist on the >> TaskExecutor. >> at >> java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331) >> ~[?:?] >> at >> java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:346) >> ~[?:?] >> at >> java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:704) >> ~[?:?] >> at >> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) >> ~[?:?] >> at >> java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) >> ~[?:?] >> at >> org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$1(AkkaInvocationHandler.java:261) >> ~[?:?] >> at >> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) >> ~[?:?] >> at >> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) >> ~[?:?] >> at >> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) >> ~[?:?] >> at >> java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) >> ~[?:?] >> at >> org.apache.flink.util.concurrent.FutureUtils.doForward(FutureUtils.java:1275) >> ~[flink-dist-1.16.1.jar:1.16.1] >> at >> org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.lambda$null$1(ClassLoadingUtils.java:93) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at >> org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:68) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at >> org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.lambda$guardCompletionWithContextClassLoader$2(ClassLoadingUtils.java:92) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at >> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) >> ~[?:?] >> at >> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) >> ~[?:?] >> at >> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) >> ~[?:?] >> at >> java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) >> ~[?:?] >> at >> org.apache.flink.runtime.concurrent.akka.AkkaFutureUtils$1.onComplete(AkkaFutureUtils.java:45) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at akka.dispatch.OnComplete.internal(Future.scala:299) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at akka.dispatch.OnComplete.internal(Future.scala:297) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at akka.dispatch.japi$CallbackBridge.apply(Future.scala:224) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at akka.dispatch.japi$CallbackBridge.apply(Future.scala:221) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at >> org.apache.flink.runtime.concurrent.akka.AkkaFutureUtils$DirectExecutionContext.execute(AkkaFutureUtils.java:65) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at >> scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:68) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at >> scala.concurrent.impl.Promise$DefaultPromise.$anonfun$tryComplete$1(Promise.scala:284) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at >> scala.concurrent.impl.Promise$DefaultPromise.$anonfun$tryComplete$1$adapted(Promise.scala:284) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at >> scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:284) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:621) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at >> akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:25) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at >> akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:23) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at scala.concurrent.Future.$anonfun$andThen$1(Future.scala:532) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at >> scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at >> akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at >> akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100) >> ~[flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at >> scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12) >> [flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at >> scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:81) >> [flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at >> akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100) >> [flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) >> [flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at >> akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) >> [flink-rpc-akka_02bdeb39-d7bb-42a6-9466-c8eb2b8bfca2.jar:1.16.1] >> at >> java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) [?:?] >> at >> java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) >> [?:?] >> at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) >> [?:?] >> at >> java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) [?:?] >> at >> java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183) >> [?:?] >> Caused by: org.apache.flink.util.FlinkException: The file STDOUT does not >> exist on the TaskExecutor. >> at >> org.apache.flink.runtime.taskexecutor.TaskExecutor.lambda$requestFileUploadByFilePath$24(TaskExecutor.java:2154) >> ~[flink-dist-1.16.1.jar:1.16.1] >> at >> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700) >> ~[?:?] >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) >> ~[?:?] >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) >> ~[?:?] >> at java.lang.Thread.run(Thread.java:829) ~[?:?] >> 2023-05-04 19:39:23,576 ERROR >> org.apache.flink.runtime.rest.handler.taskmanager.TaskManagerStdoutFileHandler >> [] - Unhandled exception. >> org.apache.flink.util.FlinkException: The file STDOUT does not exist on >> the TaskExecutor. >> at >> org.apache.flink.runtime.taskexecutor.TaskExecutor.lambda$requestFileUploadByFilePath$24(TaskExecutor.java:2154) >> ~[flink-dist-1.16.1.jar:1.16.1] >> at >> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700) >> ~[?:?] >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) >> ~[?:?] >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) >> ~[?:?] >> at java.lang.Thread.run(Thread.java:829) ~[?:?] >> >> >> -- <http://www.robinhood.com/> Nathan Moderwell Senior Machine Learning Engineer Menlo Park, CA Don't copy, share, or use this email without permission. If you received it by accident, please let us know and then delete it right away.
