Hi Arvid Heise,
Thank you for your reply.
Yes,my connection to the JM is bad !!!
Best wishes,forideal
At 2020-11-04 15:32:38, "Arvid Heise" <[email protected]> wrote:
A jar upload shouldn't take minutes. There are two possibilities that likely
co-occured:
- your jar is much bigger than needed. Did you make sure that you don't put
Flink into the fatjar? That's counterproductive on many levels. Please check
the jar size.
- your connection to the JM is bad. Where is your Flink cluster running? If
connection is superslow, it's often better to build the jar on the respective
server and submit on command line or through REST.
On Wed, Nov 4, 2020 at 7:15 AM forideal <[email protected]> wrote:
Hello my friend：
My line of code runs very slowly. What are the possibilities?
code：
CompletableFuture<JobGraph> jarUploadFuture =
jobGraphFuture.thenCombine(blobServerPortFuture, (jobGraph, blobServerPort) -> {
final InetSocketAddress address = new
InetSocketAddress(gateway.getHostname(), blobServerPort);
try {
ClientUtils.extractAndUploadJobGraphFiles(jobGraph, () -> new
BlobClient(address, configuration));
} catch (FlinkException e) {
throw new CompletionException(e);
}
return jobGraph;
});
code
link：https://github.com/apache/flink/blob/release-1.10.0/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/handlers/JarRunHandler.java#L93
Through jstack, we can also find that the thread
Flink-DispatcherRestEndpoint-thread-1 has been running for 5 minutes.
"Flink-DispatcherRestEndpoint-thread-1" #82 daemon prio=5 os_prio=0
tid=0x00007f590c03c800 nid=0x179 runnable [0x00007f5a34165000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
at
org.apache.flink.runtime.blob.BlobOutputStream.write(BlobOutputStream.java:88)
at org.apache.flink.util.IOUtils.copyBytes(IOUtils.java:65)
at org.apache.flink.runtime.blob.BlobClient.putInputStream(BlobClient.java:368)
at org.apache.flink.runtime.blob.BlobClient.uploadFile(BlobClient.java:428)
at
org.apache.flink.runtime.client.ClientUtils.uploadUserJars(ClientUtils.java:102)
at
org.apache.flink.runtime.client.ClientUtils.uploadAndSetUserJars(ClientUtils.java:95)
at
org.apache.flink.runtime.client.ClientUtils.uploadJobGraphFiles(ClientUtils.java:77)
at
org.apache.flink.runtime.client.ClientUtils.extractAndUploadJobGraphFiles(ClientUtils.java:57)
at
org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$handleRequest$1(RedJarRunHandler.java:92)
at
org.apache.flink.runtime.webmonitor.handlers.JarRunHandler$$Lambda$770/941040494.apply(Unknown
Source)
at java.util.concurrent.CompletableFuture.biApply(CompletableFuture.java:1105)
at
java.util.concurrent.CompletableFuture$BiApply.tryFire(CompletableFuture.java:1070)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at
java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1595)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Thank you very much for your reply, forideal
--
Arvid Heise| Senior Java Developer
Follow us @VervericaData
--
Join Flink Forward - The Apache Flink Conference
Stream Processing | Event Driven | Real Time
--
Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni)
Cheng
