[
https://issues.apache.org/jira/browse/MADLIB-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Orhan Kislal closed MADLIB-1509.
--------------------------------
Fix Version/s: v1.21.0
(was: v1.19.0)
Resolution: Cannot Reproduce
> Memory Shortage when serializing the model
> ------------------------------------------
>
> Key: MADLIB-1509
> URL: https://issues.apache.org/jira/browse/MADLIB-1509
> Project: Apache MADlib
> Issue Type: Bug
> Components: Deep Learning
> Reporter: Xinyi Zhang
> Priority: Major
> Fix For: v1.21.0
>
>
> When I train a model whose size is about 400MB, the execution time for
> serialize_nd_weights can be very slow. Specifically, the first-time execution
> of serialize_nd_weights lasts about {color:#FF0000}2 seconds{color}, which is
> reasonable. However, afterward, its execution time becomes about
> {color:#FF0000}190 seconds{color}.
> I think the memory shortage causes the long execution time for
> serialize_nd_weights, since the time can be reduced with a more
> memory-efficient implementation for serializing (from 190 seconds to 70
> seconds).
> However, the instance I run Madlib has 290G memory available. I think that
> Madlib might set the wrong memory limit for its task. Is there any way to
> configure the memory limit for Madlib tasks?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
