Xinyi Zhang created MADLIB-1509:
-----------------------------------
Summary: Memory Shortage when serializing the model
Key: MADLIB-1509
URL: https://issues.apache.org/jira/browse/MADLIB-1509
Project: Apache MADlib
Issue Type: Bug
Components: Deep Learning
Reporter: Xinyi Zhang
Fix For: v1.19.0
When I train a model whose size is about 400MB, the execution time for
serialize_nd_weights can be very slow. Specifically, the first-time execution
of serialize_nd_weights lasts about {color:#FF0000}2 seconds{color}, which is
reasonable. However, afterward, its execution time becomes about
{color:#FF0000}190 seconds{color}.
I think the memory shortage causes the long execution time for
serialize_nd_weights, since the time can be reduced with a more
memory-efficient implementation for serializing (from 190 seconds to 70
seconds).
However, the instance I run Madlib has 290G memory available. I think that
Madlib might set the wrong memory limit for its task. Is there any way to
configure the memory limit for Madlib tasks?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
