[
https://issues.apache.org/jira/browse/STORM-3396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated STORM-3396:
----------------------------------
Labels: pull-request-available (was: )
> uploading dependency jars too slow when StormSubmitter and Nimbus located in
> different IDC
> ------------------------------------------------------------------------------------------
>
> Key: STORM-3396
> URL: https://issues.apache.org/jira/browse/STORM-3396
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-core
> Affects Versions: 2.0.0, 1.2.2
> Reporter: StaticMian
> Assignee: StaticMian
> Priority: Minor
> Labels: pull-request-available
>
> when storm client and server is locating in different IDC(one is in
> Beijing, while another in Shanghai), uploading dependency jars may take a
> very long long time(in my case, 31minutes!)...
> when I digged into this, I found that in DependencyUploader, method
> "uploadDependencyToBlobStore" using JDK NIO's Files.copy to upload local jars
> to remote Blob server. In Files.copy(Path, OutputStream), the buffer size is
> 8k by default, given that latency between Beijing and Shanghai is about 20ms,
> a dependency fat jar of 360M finally cost me 'a lunch time' to finish
> uploading!!!
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
