[ https://issues.apache.org/jira/browse/KUDU-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Abhishek Chennaka resolved KUDU-3489. ------------------------------------- Fix Version/s: 1.17.0 Resolution: Fixed > Subprocess communication with large messages > -------------------------------------------- > > Key: KUDU-3489 > URL: https://issues.apache.org/jira/browse/KUDU-3489 > Project: Kudu > Issue Type: Bug > Reporter: Abhishek Chennaka > Priority: Major > Fix For: 1.17.0 > > > With KUDU-3450 we started supporting messages of size up to 8MB by default to > be transmitted between Kudu master and subprocess server. Given the the > maximum pipe buffer size is 1MB in linux machines [1] we see flaky failures > if the size of the message is more than 1MB between Kudu master and the > subprocess server. > [1] [https://man7.org/linux/man-pages/man7/pipe.7.html] (pipe-max-size) > Example failure: > {code:java} > Exception in thread "main" java.util.concurrent.CompletionException: > org.apache.kudu.subprocess.KuduSubprocessException: Unable to read the > protobuf message > at > java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273) > at > java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280) > at > java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1629) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.kudu.subprocess.KuduSubprocessException: Unable to read > the protobuf message > at org.apache.kudu.subprocess.MessageReader.run(MessageReader.java:77) > at > java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626) > ... 3 more > Caused by: java.io.IOException: unable to receive message, expected (2814368) > bytes but read (1777660) bytes > at org.apache.kudu.subprocess.MessageIO.doRead(MessageIO.java:101) > at org.apache.kudu.subprocess.MessageIO.readBytes(MessageIO.java:81) > at org.apache.kudu.subprocess.MessageReader.run(MessageReader.java:68) > ... 4 more > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)