[ https://issues.apache.org/jira/browse/STORM-3396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ethan Li updated STORM-3396: ---------------------------- Fix Version/s: (was: 2.0.1) 2.1.0 > uploading dependency jars too slow when StormSubmitter and Nimbus located in > different IDC > ------------------------------------------------------------------------------------------ > > Key: STORM-3396 > URL: https://issues.apache.org/jira/browse/STORM-3396 > Project: Apache Storm > Issue Type: Improvement > Components: storm-core > Affects Versions: 2.0.0, 1.2.2 > Reporter: StaticMian > Assignee: StaticMian > Priority: Minor > Labels: pull-request-available > Fix For: 2.1.0 > > Time Spent: 1h > Remaining Estimate: 0h > > when storm client and server is locating in different IDC(one is in > Beijing, while another in Shanghai), uploading dependency jars may take a > very long long time(in my case, 31minutes!)... > when I digged into this, I found that in DependencyUploader, method > "uploadDependencyToBlobStore" using JDK NIO's Files.copy to upload local jars > to remote Blob server. In Files.copy(Path, OutputStream), the buffer size is > 8k by default, given that latency between Beijing and Shanghai is about 20ms, > a dependency fat jar of 360M finally cost me 'a lunch time' to finish > uploading!!! > -- This message was sent by Atlassian JIRA (v7.6.14#76016)