[jira] [Commented] (IVY-1197) OutOfMemoryError during ivy:publish

2022-10-19 Thread Maarten Coene (Jira)


[ 
https://issues.apache.org/jira/browse/IVY-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17620615#comment-17620615
 ] 

Maarten Coene commented on IVY-1197:


[~andrew norman] : did you configure Ivy to use the Apache HTTP client 
libraries?

> OutOfMemoryError during ivy:publish
> ---
>
> Key: IVY-1197
> URL: https://issues.apache.org/jira/browse/IVY-1197
> Project: Ivy
>  Issue Type: Bug
>  Components: Core
>Affects Versions: 2.0
>Reporter: Michael Rumpf
>Priority: Major
> Attachments: ASF.LICENSE.NOT.GRANTED--clipboard.txt, ivylarge.zip, 
> org.apache.ivy.util.url.HttpClientHandler.patch
>
>
> When publishing a large file, an OutOfMemoryError occurs.
> {code}
> [ivy:publish] published ppg to 
> 
> BUILD FAILED
> /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:152:
>  The following error occurred while executing this line:
> /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:277:
>  java.lang.OutOfMemoryError: Java heap space
>   at java.util.Arrays.copyOf(Arrays.java:2786)
>   at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
>   at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
>   at org.apache.ivy.util.FileUtil.copy(FileUtil.java:168)
>   at 
> org.apache.ivy.util.url.BasicURLHandler.upload(BasicURLHandler.java:200)
>   at 
> org.apache.ivy.util.url.URLHandlerDispatcher.upload(URLHandlerDispatcher.java:82)
>   at org.apache.ivy.util.FileUtil.copy(FileUtil.java:140)
>   at 
> org.apache.ivy.plugins.repository.url.URLRepository.put(URLRepository.java:85)
>   at 
> org.apache.ivy.plugins.repository.AbstractRepository.put(AbstractRepository.java:130)
>   at 
> org.apache.ivy.plugins.resolver.RepositoryResolver.put(RepositoryResolver.java:219)
>   at 
> org.apache.ivy.plugins.resolver.RepositoryResolver.publish(RepositoryResolver.java:209)
>   at 
> org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:282)
>   at 
> org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:261)
>   at 
> org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:170)
>   at org.apache.ivy.Ivy.publish(Ivy.java:600)
>   at org.apache.ivy.ant.IvyPublish.doExecute(IvyPublish.java:299)
>   at org.apache.ivy.ant.IvyTask.execute(IvyTask.java:277)
>   at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
>   at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>   at java.lang.reflect.Method.invoke(Method.java:597)
>   at 
> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
>   at org.apache.tools.ant.Task.perform(Task.java:348)
>   at org.apache.tools.ant.Target.execute(Target.java:390)
>   at org.apache.tools.ant.Target.performTasks(Target.java:411)
>   at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
>   at 
> org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
>   at org.apache.tools.ant.Project.executeTargets(Project.java:1249)
>   at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442)
>   at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105)
>   at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> Total time: 14 minutes 24 seconds
> Finished: FAILURE
> {code}
> The size of the file that is being uploaded is: 687712714, so around 
> 650-700MB.
> The publish task is part of a Hudson Ant build where the artefacts are 
> published to an Artifactory repository at the end.
> I have given the Job 1300MB for the max heap size.
> It seems as if the whole file is loaded into memory for the upload.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IVY-1197) OutOfMemoryError during ivy:publish

2022-10-19 Thread Andrew Norman (Jira)


[ 
https://issues.apache.org/jira/browse/IVY-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17620549#comment-17620549
 ] 

Andrew Norman commented on IVY-1197:


the file size in my usecase just barely exceeds the 512m size and used to work 
when it was less than that size so I think that the 512M is a hard limit of the 
jenkins ivy plugin

> OutOfMemoryError during ivy:publish
> ---
>
> Key: IVY-1197
> URL: https://issues.apache.org/jira/browse/IVY-1197
> Project: Ivy
>  Issue Type: Bug
>  Components: Core
>Affects Versions: 2.0
>Reporter: Michael Rumpf
>Priority: Major
> Attachments: ASF.LICENSE.NOT.GRANTED--clipboard.txt, ivylarge.zip, 
> org.apache.ivy.util.url.HttpClientHandler.patch
>
>
> When publishing a large file, an OutOfMemoryError occurs.
> {code}
> [ivy:publish] published ppg to 
> 
> BUILD FAILED
> /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:152:
>  The following error occurred while executing this line:
> /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:277:
>  java.lang.OutOfMemoryError: Java heap space
>   at java.util.Arrays.copyOf(Arrays.java:2786)
>   at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
>   at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
>   at org.apache.ivy.util.FileUtil.copy(FileUtil.java:168)
>   at 
> org.apache.ivy.util.url.BasicURLHandler.upload(BasicURLHandler.java:200)
>   at 
> org.apache.ivy.util.url.URLHandlerDispatcher.upload(URLHandlerDispatcher.java:82)
>   at org.apache.ivy.util.FileUtil.copy(FileUtil.java:140)
>   at 
> org.apache.ivy.plugins.repository.url.URLRepository.put(URLRepository.java:85)
>   at 
> org.apache.ivy.plugins.repository.AbstractRepository.put(AbstractRepository.java:130)
>   at 
> org.apache.ivy.plugins.resolver.RepositoryResolver.put(RepositoryResolver.java:219)
>   at 
> org.apache.ivy.plugins.resolver.RepositoryResolver.publish(RepositoryResolver.java:209)
>   at 
> org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:282)
>   at 
> org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:261)
>   at 
> org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:170)
>   at org.apache.ivy.Ivy.publish(Ivy.java:600)
>   at org.apache.ivy.ant.IvyPublish.doExecute(IvyPublish.java:299)
>   at org.apache.ivy.ant.IvyTask.execute(IvyTask.java:277)
>   at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
>   at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>   at java.lang.reflect.Method.invoke(Method.java:597)
>   at 
> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
>   at org.apache.tools.ant.Task.perform(Task.java:348)
>   at org.apache.tools.ant.Target.execute(Target.java:390)
>   at org.apache.tools.ant.Target.performTasks(Target.java:411)
>   at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
>   at 
> org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
>   at org.apache.tools.ant.Project.executeTargets(Project.java:1249)
>   at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442)
>   at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105)
>   at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> Total time: 14 minutes 24 seconds
> Finished: FAILURE
> {code}
> The size of the file that is being uploaded is: 687712714, so around 
> 650-700MB.
> The publish task is part of a Hudson Ant build where the artefacts are 
> published to an Artifactory repository at the end.
> I have given the Job 1300MB for the max heap size.
> It seems as if the whole file is loaded into memory for the upload.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IVY-1197) OutOfMemoryError during ivy:publish

2022-10-19 Thread Andrew Norman (Jira)


[ 
https://issues.apache.org/jira/browse/IVY-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17620530#comment-17620530
 ] 

Andrew Norman commented on IVY-1197:


any chance this gets fixed soon? My use case is this out of memory failing when 
running an "sbt publish" of a large zip file >512M on jenkins

> OutOfMemoryError during ivy:publish
> ---
>
> Key: IVY-1197
> URL: https://issues.apache.org/jira/browse/IVY-1197
> Project: Ivy
>  Issue Type: Bug
>  Components: Core
>Affects Versions: 2.0
>Reporter: Michael Rumpf
>Priority: Major
> Attachments: ASF.LICENSE.NOT.GRANTED--clipboard.txt, ivylarge.zip, 
> org.apache.ivy.util.url.HttpClientHandler.patch
>
>
> When publishing a large file, an OutOfMemoryError occurs.
> {code}
> [ivy:publish] published ppg to 
> 
> BUILD FAILED
> /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:152:
>  The following error occurred while executing this line:
> /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:277:
>  java.lang.OutOfMemoryError: Java heap space
>   at java.util.Arrays.copyOf(Arrays.java:2786)
>   at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
>   at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
>   at org.apache.ivy.util.FileUtil.copy(FileUtil.java:168)
>   at 
> org.apache.ivy.util.url.BasicURLHandler.upload(BasicURLHandler.java:200)
>   at 
> org.apache.ivy.util.url.URLHandlerDispatcher.upload(URLHandlerDispatcher.java:82)
>   at org.apache.ivy.util.FileUtil.copy(FileUtil.java:140)
>   at 
> org.apache.ivy.plugins.repository.url.URLRepository.put(URLRepository.java:85)
>   at 
> org.apache.ivy.plugins.repository.AbstractRepository.put(AbstractRepository.java:130)
>   at 
> org.apache.ivy.plugins.resolver.RepositoryResolver.put(RepositoryResolver.java:219)
>   at 
> org.apache.ivy.plugins.resolver.RepositoryResolver.publish(RepositoryResolver.java:209)
>   at 
> org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:282)
>   at 
> org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:261)
>   at 
> org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:170)
>   at org.apache.ivy.Ivy.publish(Ivy.java:600)
>   at org.apache.ivy.ant.IvyPublish.doExecute(IvyPublish.java:299)
>   at org.apache.ivy.ant.IvyTask.execute(IvyTask.java:277)
>   at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
>   at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>   at java.lang.reflect.Method.invoke(Method.java:597)
>   at 
> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
>   at org.apache.tools.ant.Task.perform(Task.java:348)
>   at org.apache.tools.ant.Target.execute(Target.java:390)
>   at org.apache.tools.ant.Target.performTasks(Target.java:411)
>   at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
>   at 
> org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
>   at org.apache.tools.ant.Project.executeTargets(Project.java:1249)
>   at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442)
>   at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105)
>   at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> Total time: 14 minutes 24 seconds
> Finished: FAILURE
> {code}
> The size of the file that is being uploaded is: 687712714, so around 
> 650-700MB.
> The publish task is part of a Hudson Ant build where the artefacts are 
> published to an Artifactory repository at the end.
> I have given the Job 1300MB for the max heap size.
> It seems as if the whole file is loaded into memory for the upload.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IVY-1197) OutOfMemoryError during ivy:publish

2019-10-06 Thread Ekkarat Wareesing (Jira)


[ 
https://issues.apache.org/jira/browse/IVY-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16945264#comment-16945264
 ] 

Ekkarat Wareesing commented on IVY-1197:


http://ekkara...@gmail.com

> OutOfMemoryError during ivy:publish
> ---
>
> Key: IVY-1197
> URL: https://issues.apache.org/jira/browse/IVY-1197
> Project: Ivy
>  Issue Type: Bug
>  Components: Core
>Affects Versions: 2.0
>Reporter: Michael Rumpf
>Priority: Major
> Attachments: ASF.LICENSE.NOT.GRANTED--clipboard.txt, ivylarge.zip, 
> org.apache.ivy.util.url.HttpClientHandler.patch
>
>
> When publishing a large file, an OutOfMemoryError occurs.
> {code}
> [ivy:publish] published ppg to 
> 
> BUILD FAILED
> /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:152:
>  The following error occurred while executing this line:
> /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:277:
>  java.lang.OutOfMemoryError: Java heap space
>   at java.util.Arrays.copyOf(Arrays.java:2786)
>   at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
>   at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
>   at org.apache.ivy.util.FileUtil.copy(FileUtil.java:168)
>   at 
> org.apache.ivy.util.url.BasicURLHandler.upload(BasicURLHandler.java:200)
>   at 
> org.apache.ivy.util.url.URLHandlerDispatcher.upload(URLHandlerDispatcher.java:82)
>   at org.apache.ivy.util.FileUtil.copy(FileUtil.java:140)
>   at 
> org.apache.ivy.plugins.repository.url.URLRepository.put(URLRepository.java:85)
>   at 
> org.apache.ivy.plugins.repository.AbstractRepository.put(AbstractRepository.java:130)
>   at 
> org.apache.ivy.plugins.resolver.RepositoryResolver.put(RepositoryResolver.java:219)
>   at 
> org.apache.ivy.plugins.resolver.RepositoryResolver.publish(RepositoryResolver.java:209)
>   at 
> org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:282)
>   at 
> org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:261)
>   at 
> org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:170)
>   at org.apache.ivy.Ivy.publish(Ivy.java:600)
>   at org.apache.ivy.ant.IvyPublish.doExecute(IvyPublish.java:299)
>   at org.apache.ivy.ant.IvyTask.execute(IvyTask.java:277)
>   at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
>   at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>   at java.lang.reflect.Method.invoke(Method.java:597)
>   at 
> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
>   at org.apache.tools.ant.Task.perform(Task.java:348)
>   at org.apache.tools.ant.Target.execute(Target.java:390)
>   at org.apache.tools.ant.Target.performTasks(Target.java:411)
>   at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
>   at 
> org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
>   at org.apache.tools.ant.Project.executeTargets(Project.java:1249)
>   at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442)
>   at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105)
>   at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> Total time: 14 minutes 24 seconds
> Finished: FAILURE
> {code}
> The size of the file that is being uploaded is: 687712714, so around 
> 650-700MB.
> The publish task is part of a Hudson Ant build where the artefacts are 
> published to an Artifactory repository at the end.
> I have given the Job 1300MB for the max heap size.
> It seems as if the whole file is loaded into memory for the upload.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IVY-1197) OutOfMemoryError during ivy:publish

2018-08-02 Thread jaikiran pai (JIRA)


[ 
https://issues.apache.org/jira/browse/IVY-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567757#comment-16567757
 ] 

jaikiran pai commented on IVY-1197:
---

Charles, this should work fine with 2.5.0-rc1 where we use 4.x of Apache HTTP 
client library. You will have to include that in your classpath so that Ivy 
picks that up instead of the JDK implementation.


> OutOfMemoryError during ivy:publish
> ---
>
> Key: IVY-1197
> URL: https://issues.apache.org/jira/browse/IVY-1197
> Project: Ivy
>  Issue Type: Bug
>  Components: Core
>Affects Versions: 2.0
>Reporter: Michael Rumpf
>Priority: Major
> Attachments: ASF.LICENSE.NOT.GRANTED--clipboard.txt, ivylarge.zip, 
> org.apache.ivy.util.url.HttpClientHandler.patch
>
>
> When publishing a large file, an OutOfMemoryError occurs.
> {code}
> [ivy:publish] published ppg to 
> 
> BUILD FAILED
> /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:152:
>  The following error occurred while executing this line:
> /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:277:
>  java.lang.OutOfMemoryError: Java heap space
>   at java.util.Arrays.copyOf(Arrays.java:2786)
>   at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
>   at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
>   at org.apache.ivy.util.FileUtil.copy(FileUtil.java:168)
>   at 
> org.apache.ivy.util.url.BasicURLHandler.upload(BasicURLHandler.java:200)
>   at 
> org.apache.ivy.util.url.URLHandlerDispatcher.upload(URLHandlerDispatcher.java:82)
>   at org.apache.ivy.util.FileUtil.copy(FileUtil.java:140)
>   at 
> org.apache.ivy.plugins.repository.url.URLRepository.put(URLRepository.java:85)
>   at 
> org.apache.ivy.plugins.repository.AbstractRepository.put(AbstractRepository.java:130)
>   at 
> org.apache.ivy.plugins.resolver.RepositoryResolver.put(RepositoryResolver.java:219)
>   at 
> org.apache.ivy.plugins.resolver.RepositoryResolver.publish(RepositoryResolver.java:209)
>   at 
> org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:282)
>   at 
> org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:261)
>   at 
> org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:170)
>   at org.apache.ivy.Ivy.publish(Ivy.java:600)
>   at org.apache.ivy.ant.IvyPublish.doExecute(IvyPublish.java:299)
>   at org.apache.ivy.ant.IvyTask.execute(IvyTask.java:277)
>   at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
>   at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>   at java.lang.reflect.Method.invoke(Method.java:597)
>   at 
> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
>   at org.apache.tools.ant.Task.perform(Task.java:348)
>   at org.apache.tools.ant.Target.execute(Target.java:390)
>   at org.apache.tools.ant.Target.performTasks(Target.java:411)
>   at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
>   at 
> org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
>   at org.apache.tools.ant.Project.executeTargets(Project.java:1249)
>   at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442)
>   at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105)
>   at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> Total time: 14 minutes 24 seconds
> Finished: FAILURE
> {code}
> The size of the file that is being uploaded is: 687712714, so around 
> 650-700MB.
> The publish task is part of a Hudson Ant build where the artefacts are 
> published to an Artifactory repository at the end.
> I have given the Job 1300MB for the max heap size.
> It seems as if the whole file is loaded into memory for the upload.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (IVY-1197) OutOfMemoryError during ivy:publish

2018-08-02 Thread Charles Breingan (JIRA)


[ 
https://issues.apache.org/jira/browse/IVY-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567620#comment-16567620
 ] 

Charles Breingan commented on IVY-1197:
---

Is this still an issue? Has JDK or Apache resolved this yet? 

> OutOfMemoryError during ivy:publish
> ---
>
> Key: IVY-1197
> URL: https://issues.apache.org/jira/browse/IVY-1197
> Project: Ivy
>  Issue Type: Bug
>  Components: Core
>Affects Versions: 2.0
>Reporter: Michael Rumpf
>Priority: Major
> Attachments: ASF.LICENSE.NOT.GRANTED--clipboard.txt, ivylarge.zip, 
> org.apache.ivy.util.url.HttpClientHandler.patch
>
>
> When publishing a large file, an OutOfMemoryError occurs.
> {code}
> [ivy:publish] published ppg to 
> 
> BUILD FAILED
> /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:152:
>  The following error occurred while executing this line:
> /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:277:
>  java.lang.OutOfMemoryError: Java heap space
>   at java.util.Arrays.copyOf(Arrays.java:2786)
>   at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
>   at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
>   at org.apache.ivy.util.FileUtil.copy(FileUtil.java:168)
>   at 
> org.apache.ivy.util.url.BasicURLHandler.upload(BasicURLHandler.java:200)
>   at 
> org.apache.ivy.util.url.URLHandlerDispatcher.upload(URLHandlerDispatcher.java:82)
>   at org.apache.ivy.util.FileUtil.copy(FileUtil.java:140)
>   at 
> org.apache.ivy.plugins.repository.url.URLRepository.put(URLRepository.java:85)
>   at 
> org.apache.ivy.plugins.repository.AbstractRepository.put(AbstractRepository.java:130)
>   at 
> org.apache.ivy.plugins.resolver.RepositoryResolver.put(RepositoryResolver.java:219)
>   at 
> org.apache.ivy.plugins.resolver.RepositoryResolver.publish(RepositoryResolver.java:209)
>   at 
> org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:282)
>   at 
> org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:261)
>   at 
> org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:170)
>   at org.apache.ivy.Ivy.publish(Ivy.java:600)
>   at org.apache.ivy.ant.IvyPublish.doExecute(IvyPublish.java:299)
>   at org.apache.ivy.ant.IvyTask.execute(IvyTask.java:277)
>   at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
>   at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>   at java.lang.reflect.Method.invoke(Method.java:597)
>   at 
> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
>   at org.apache.tools.ant.Task.perform(Task.java:348)
>   at org.apache.tools.ant.Target.execute(Target.java:390)
>   at org.apache.tools.ant.Target.performTasks(Target.java:411)
>   at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
>   at 
> org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
>   at org.apache.tools.ant.Project.executeTargets(Project.java:1249)
>   at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442)
>   at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105)
>   at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> Total time: 14 minutes 24 seconds
> Finished: FAILURE
> {code}
> The size of the file that is being uploaded is: 687712714, so around 
> 650-700MB.
> The publish task is part of a Hudson Ant build where the artefacts are 
> published to an Artifactory repository at the end.
> I have given the Job 1300MB for the max heap size.
> It seems as if the whole file is loaded into memory for the upload.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (IVY-1197) OutOfMemoryError during ivy:publish

2015-04-12 Thread Antoine Levy-Lambert (JIRA)

[ 
https://issues.apache.org/jira/browse/IVY-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14491847#comment-14491847
 ] 

Antoine Levy-Lambert commented on IVY-1197:
---

So now there is a common understanding that :

- when running ivy without commons-httpclient + commons-codec + 
commons-logging, ivy is not able to upload files of more than 1 GB of size to 
HTTP destinations requiring authentication. This is due to limitations of the 
way the HttpUrlConnection is implemented in the JDK

- the workaround is to always use ivy with commons-httpclient, commons-codec 
and commons-logging when the use case will involve uploading large files to 
sites requiring authentication

- the change to do in the deliverables of ivy is to add this information to the 
documentation

- trying to change 
[choose-distrib.html|http://ant.apache.org/ivy/choose-distrib.html] currently 
held by an issue with the script to generate the site - or my making an 
unsupported change in the source html ?

 OutOfMemoryError during ivy:publish
 ---

 Key: IVY-1197
 URL: https://issues.apache.org/jira/browse/IVY-1197
 Project: Ivy
  Issue Type: Bug
  Components: Core
Affects Versions: 2.0
Reporter: Michael Rumpf
 Attachments: ASF.LICENSE.NOT.GRANTED--clipboard.txt, ivylarge.zip, 
 org.apache.ivy.util.url.HttpClientHandler.patch


 When publishing a large file, an OutOfMemoryError occurs.
 {code}
 [ivy:publish] published ppg to 
 
 BUILD FAILED
 /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:152:
  The following error occurred while executing this line:
 /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:277:
  java.lang.OutOfMemoryError: Java heap space
   at java.util.Arrays.copyOf(Arrays.java:2786)
   at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
   at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
   at org.apache.ivy.util.FileUtil.copy(FileUtil.java:168)
   at 
 org.apache.ivy.util.url.BasicURLHandler.upload(BasicURLHandler.java:200)
   at 
 org.apache.ivy.util.url.URLHandlerDispatcher.upload(URLHandlerDispatcher.java:82)
   at org.apache.ivy.util.FileUtil.copy(FileUtil.java:140)
   at 
 org.apache.ivy.plugins.repository.url.URLRepository.put(URLRepository.java:85)
   at 
 org.apache.ivy.plugins.repository.AbstractRepository.put(AbstractRepository.java:130)
   at 
 org.apache.ivy.plugins.resolver.RepositoryResolver.put(RepositoryResolver.java:219)
   at 
 org.apache.ivy.plugins.resolver.RepositoryResolver.publish(RepositoryResolver.java:209)
   at 
 org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:282)
   at 
 org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:261)
   at 
 org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:170)
   at org.apache.ivy.Ivy.publish(Ivy.java:600)
   at org.apache.ivy.ant.IvyPublish.doExecute(IvyPublish.java:299)
   at org.apache.ivy.ant.IvyTask.execute(IvyTask.java:277)
   at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
   at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
   at org.apache.tools.ant.Task.perform(Task.java:348)
   at org.apache.tools.ant.Target.execute(Target.java:390)
   at org.apache.tools.ant.Target.performTasks(Target.java:411)
   at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
   at 
 org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
   at org.apache.tools.ant.Project.executeTargets(Project.java:1249)
   at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442)
   at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105)
   at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 Total time: 14 minutes 24 seconds
 Finished: FAILURE
 {code}
 The size of the file that is being uploaded is: 687712714, so around 
 650-700MB.
 The publish task is part of a Hudson Ant build where the artefacts are 
 published to an Artifactory repository at the end.
 I have given the Job 1300MB for the max heap size.
 It seems as if the whole file is loaded into memory for the upload.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (IVY-1197) OutOfMemoryError during ivy:publish

2015-04-08 Thread Antoine Levy-Lambert (JIRA)

[ 
https://issues.apache.org/jira/browse/IVY-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14486535#comment-14486535
 ] 

Antoine Levy-Lambert commented on IVY-1197:
---

I will work on this as soon as I can. Will write an email about this on the ivy 
dev mailing list.

 OutOfMemoryError during ivy:publish
 ---

 Key: IVY-1197
 URL: https://issues.apache.org/jira/browse/IVY-1197
 Project: Ivy
  Issue Type: Bug
  Components: Core
Affects Versions: 2.0
Reporter: Michael Rumpf
 Attachments: ASF.LICENSE.NOT.GRANTED--clipboard.txt, ivylarge.zip, 
 org.apache.ivy.util.url.HttpClientHandler.patch


 When publishing a large file, an OutOfMemoryError occurs.
 {code}
 [ivy:publish] published ppg to 
 
 BUILD FAILED
 /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:152:
  The following error occurred while executing this line:
 /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:277:
  java.lang.OutOfMemoryError: Java heap space
   at java.util.Arrays.copyOf(Arrays.java:2786)
   at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
   at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
   at org.apache.ivy.util.FileUtil.copy(FileUtil.java:168)
   at 
 org.apache.ivy.util.url.BasicURLHandler.upload(BasicURLHandler.java:200)
   at 
 org.apache.ivy.util.url.URLHandlerDispatcher.upload(URLHandlerDispatcher.java:82)
   at org.apache.ivy.util.FileUtil.copy(FileUtil.java:140)
   at 
 org.apache.ivy.plugins.repository.url.URLRepository.put(URLRepository.java:85)
   at 
 org.apache.ivy.plugins.repository.AbstractRepository.put(AbstractRepository.java:130)
   at 
 org.apache.ivy.plugins.resolver.RepositoryResolver.put(RepositoryResolver.java:219)
   at 
 org.apache.ivy.plugins.resolver.RepositoryResolver.publish(RepositoryResolver.java:209)
   at 
 org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:282)
   at 
 org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:261)
   at 
 org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:170)
   at org.apache.ivy.Ivy.publish(Ivy.java:600)
   at org.apache.ivy.ant.IvyPublish.doExecute(IvyPublish.java:299)
   at org.apache.ivy.ant.IvyTask.execute(IvyTask.java:277)
   at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
   at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
   at org.apache.tools.ant.Task.perform(Task.java:348)
   at org.apache.tools.ant.Target.execute(Target.java:390)
   at org.apache.tools.ant.Target.performTasks(Target.java:411)
   at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
   at 
 org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
   at org.apache.tools.ant.Project.executeTargets(Project.java:1249)
   at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442)
   at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105)
   at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 Total time: 14 minutes 24 seconds
 Finished: FAILURE
 {code}
 The size of the file that is being uploaded is: 687712714, so around 
 650-700MB.
 The publish task is part of a Hudson Ant build where the artefacts are 
 published to an Artifactory repository at the end.
 I have given the Job 1300MB for the max heap size.
 It seems as if the whole file is loaded into memory for the upload.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (IVY-1197) OutOfMemoryError during ivy:publish

2015-04-08 Thread Loren Kratzke (JIRA)

[ 
https://issues.apache.org/jira/browse/IVY-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485435#comment-14485435
 ] 

Loren Kratzke commented on IVY-1197:


I created the project and attached to the Jira issue. You were right about my 
patch, it does nothing. I confused the behavior during the OOME with the 
behavior around detecting if httpclient is available. But I did learn a lot 
from the sample project. 

Note that when httpclient is available then the OOME is avoided. Without 
httpclient, I do not have a fix but the project will reproduce the issue 
reliably. Here is the section of readme_debug.txt that pinpoints the issue.

{quote}
6) Behavior

Assuming that you have connected and are waiting somehwere in 
BasicURLHandler.upload() method, 
the problem is simple to illustrate. 

 - An HttpURLConnection is obtained from the URL object.
 - An OutputStream is obtained from the HttpURLConnection. 
 - The OutputStream is implemented by a sun.net.www.http.PosterOutputStream.
 - Writing to the OutputStream (in FileUtil) writes to a ByteArrayOutputStream.
 - The backing array for the ByteArrayOutputStream grows until all content has 
been written.

That last point causes multiple problems for a 1GB+ file. The first is that 
depending upon
your Java implementation you will run up against a hard limit on the maximum 
array size
that can be allocated. 

Another problem is that the backing array starts small and then doubles as 
needed while
bytes are being written. So if you imagine that it is at 500MB and needs to 
grow, then a
1GB array will be allocated and the 500MB array will be copied into it. So for 
a short 
period of time you are using 1.5GB of RAM to hold 500MB of data.

Between the array resizing and the hard limit on the array size, Ivy artifacts 
in the 1GB range
are doomed unless httpclient is used. More on that in a bit.

There are methods available on the HttpURLConnection object to support 
streaming the bytes 
instead of buffering them, but then authentication is not performed. 

See javadoc for HttpURLConnection.setFixedLengthStreamingMode(long l)

Indeed, if I call the above method then I fail to authenticate. I do not have a 
work around 
for streaming and preserving auth using HttpURLConnection.
{quote}


 OutOfMemoryError during ivy:publish
 ---

 Key: IVY-1197
 URL: https://issues.apache.org/jira/browse/IVY-1197
 Project: Ivy
  Issue Type: Bug
  Components: Core
Affects Versions: 2.0
Reporter: Michael Rumpf
 Attachments: ASF.LICENSE.NOT.GRANTED--clipboard.txt, ivylarge.zip, 
 org.apache.ivy.util.url.HttpClientHandler.patch


 When publishing a large file, an OutOfMemoryError occurs.
 {code}
 [ivy:publish] published ppg to 
 
 BUILD FAILED
 /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:152:
  The following error occurred while executing this line:
 /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:277:
  java.lang.OutOfMemoryError: Java heap space
   at java.util.Arrays.copyOf(Arrays.java:2786)
   at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
   at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
   at org.apache.ivy.util.FileUtil.copy(FileUtil.java:168)
   at 
 org.apache.ivy.util.url.BasicURLHandler.upload(BasicURLHandler.java:200)
   at 
 org.apache.ivy.util.url.URLHandlerDispatcher.upload(URLHandlerDispatcher.java:82)
   at org.apache.ivy.util.FileUtil.copy(FileUtil.java:140)
   at 
 org.apache.ivy.plugins.repository.url.URLRepository.put(URLRepository.java:85)
   at 
 org.apache.ivy.plugins.repository.AbstractRepository.put(AbstractRepository.java:130)
   at 
 org.apache.ivy.plugins.resolver.RepositoryResolver.put(RepositoryResolver.java:219)
   at 
 org.apache.ivy.plugins.resolver.RepositoryResolver.publish(RepositoryResolver.java:209)
   at 
 org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:282)
   at 
 org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:261)
   at 
 org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:170)
   at org.apache.ivy.Ivy.publish(Ivy.java:600)
   at org.apache.ivy.ant.IvyPublish.doExecute(IvyPublish.java:299)
   at org.apache.ivy.ant.IvyTask.execute(IvyTask.java:277)
   at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
   at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 

[jira] [Commented] (IVY-1197) OutOfMemoryError during ivy:publish

2015-04-07 Thread Loren Kratzke (JIRA)

[ 
https://issues.apache.org/jira/browse/IVY-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14483468#comment-14483468
 ] 

Loren Kratzke commented on IVY-1197:


I would be happy to provide you with a project that will reproduce the issue. I 
can and will do that. 

Generally speaking from a high level, the utility classes are calling 
convenience methods and writing to streams that ultimately buffer the data 
being written. There is buffering, then more buffering, and even more buffering 
until you have multiple copies of the entire content of the stream stored in 
over sized buffers (because they double in size when they fill up). Oddly, the 
twist is that the JVM hits a limit no matter how much RAM you allocate. Once 
the buffers total more than about ~1GB (which is what happens with a 100-200MB 
upload) the JVM refuses to allocate more buffer space (even is you jack up the 
RAM to 20GB, no cigar). Honestly, there is no benefit in buffering any of this 
data to begin with, it is just a side effect of using high level copy methods. 
There is no memory ballooning at all when the content is written directly to 
the network.

I will provide a test project and note the break points where you can debug and 
watch the process walk all the way down the isle to an OOME. I will have this 
for you asap.

 OutOfMemoryError during ivy:publish
 ---

 Key: IVY-1197
 URL: https://issues.apache.org/jira/browse/IVY-1197
 Project: Ivy
  Issue Type: Bug
  Components: Core
Affects Versions: 2.0
Reporter: Michael Rumpf
 Attachments: ASF.LICENSE.NOT.GRANTED--clipboard.txt, 
 org.apache.ivy.util.url.HttpClientHandler.patch


 When publishing a large file, an OutOfMemoryError occurs.
 {code}
 [ivy:publish] published ppg to 
 
 BUILD FAILED
 /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:152:
  The following error occurred while executing this line:
 /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:277:
  java.lang.OutOfMemoryError: Java heap space
   at java.util.Arrays.copyOf(Arrays.java:2786)
   at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
   at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
   at org.apache.ivy.util.FileUtil.copy(FileUtil.java:168)
   at 
 org.apache.ivy.util.url.BasicURLHandler.upload(BasicURLHandler.java:200)
   at 
 org.apache.ivy.util.url.URLHandlerDispatcher.upload(URLHandlerDispatcher.java:82)
   at org.apache.ivy.util.FileUtil.copy(FileUtil.java:140)
   at 
 org.apache.ivy.plugins.repository.url.URLRepository.put(URLRepository.java:85)
   at 
 org.apache.ivy.plugins.repository.AbstractRepository.put(AbstractRepository.java:130)
   at 
 org.apache.ivy.plugins.resolver.RepositoryResolver.put(RepositoryResolver.java:219)
   at 
 org.apache.ivy.plugins.resolver.RepositoryResolver.publish(RepositoryResolver.java:209)
   at 
 org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:282)
   at 
 org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:261)
   at 
 org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:170)
   at org.apache.ivy.Ivy.publish(Ivy.java:600)
   at org.apache.ivy.ant.IvyPublish.doExecute(IvyPublish.java:299)
   at org.apache.ivy.ant.IvyTask.execute(IvyTask.java:277)
   at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
   at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
   at org.apache.tools.ant.Task.perform(Task.java:348)
   at org.apache.tools.ant.Target.execute(Target.java:390)
   at org.apache.tools.ant.Target.performTasks(Target.java:411)
   at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
   at 
 org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
   at org.apache.tools.ant.Project.executeTargets(Project.java:1249)
   at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442)
   at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105)
   at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 Total time: 14 minutes 24 seconds
 Finished: FAILURE
 {code}
 The size of the file that is being uploaded is: 687712714, so around 
 650-700MB.
 The publish task is part of a Hudson Ant build where the 

[jira] [Commented] (IVY-1197) OutOfMemoryError during ivy:publish

2015-03-29 Thread Antoine Levy-Lambert (JIRA)

[ 
https://issues.apache.org/jira/browse/IVY-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14386098#comment-14386098
 ] 

Antoine Levy-Lambert commented on IVY-1197:
---

I have taken a look at the patch. The only difference I see with ivy's 
FileUtil#copy is that FileUtil#copy is calling dest.flush() . Is that what is 
causing the Out of Memory Exception ?
Also I would like to have a setup to reproduce the out of memory exception so 
that we can be confident about the existence of the problem and of the fix.


 OutOfMemoryError during ivy:publish
 ---

 Key: IVY-1197
 URL: https://issues.apache.org/jira/browse/IVY-1197
 Project: Ivy
  Issue Type: Bug
  Components: Core
Affects Versions: 2.0
Reporter: Michael Rumpf
 Attachments: ASF.LICENSE.NOT.GRANTED--clipboard.txt, 
 org.apache.ivy.util.url.HttpClientHandler.patch


 When publishing a large file, an OutOfMemoryError occurs.
 {code}
 [ivy:publish] published ppg to 
 
 BUILD FAILED
 /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:152:
  The following error occurred while executing this line:
 /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:277:
  java.lang.OutOfMemoryError: Java heap space
   at java.util.Arrays.copyOf(Arrays.java:2786)
   at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
   at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
   at org.apache.ivy.util.FileUtil.copy(FileUtil.java:168)
   at 
 org.apache.ivy.util.url.BasicURLHandler.upload(BasicURLHandler.java:200)
   at 
 org.apache.ivy.util.url.URLHandlerDispatcher.upload(URLHandlerDispatcher.java:82)
   at org.apache.ivy.util.FileUtil.copy(FileUtil.java:140)
   at 
 org.apache.ivy.plugins.repository.url.URLRepository.put(URLRepository.java:85)
   at 
 org.apache.ivy.plugins.repository.AbstractRepository.put(AbstractRepository.java:130)
   at 
 org.apache.ivy.plugins.resolver.RepositoryResolver.put(RepositoryResolver.java:219)
   at 
 org.apache.ivy.plugins.resolver.RepositoryResolver.publish(RepositoryResolver.java:209)
   at 
 org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:282)
   at 
 org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:261)
   at 
 org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:170)
   at org.apache.ivy.Ivy.publish(Ivy.java:600)
   at org.apache.ivy.ant.IvyPublish.doExecute(IvyPublish.java:299)
   at org.apache.ivy.ant.IvyTask.execute(IvyTask.java:277)
   at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
   at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
   at org.apache.tools.ant.Task.perform(Task.java:348)
   at org.apache.tools.ant.Target.execute(Target.java:390)
   at org.apache.tools.ant.Target.performTasks(Target.java:411)
   at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
   at 
 org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
   at org.apache.tools.ant.Project.executeTargets(Project.java:1249)
   at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442)
   at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105)
   at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 Total time: 14 minutes 24 seconds
 Finished: FAILURE
 {code}
 The size of the file that is being uploaded is: 687712714, so around 
 650-700MB.
 The publish task is part of a Hudson Ant build where the artefacts are 
 published to an Artifactory repository at the end.
 I have given the Job 1300MB for the max heap size.
 It seems as if the whole file is loaded into memory for the upload.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)