[GitHub] [incubator-mxnet] leezu commented on issue #18501: It costs over 16GB memory to compile indexing_op.o

2020-06-12 Thread GitBox
leezu commented on issue #18501: URL: https://github.com/apache/incubator-mxnet/issues/18501#issuecomment-643385571 Single process build of MXNet master with gcc7 gives the following results: ``` Child user: 5288.372 s Child sys : 188.645 s Child wall: 5481.062 s Child h

[GitHub] [incubator-mxnet] leezu commented on issue #18501: It costs over 16GB memory to compile indexing_op.o

2020-06-11 Thread GitBox
leezu commented on issue #18501: URL: https://github.com/apache/incubator-mxnet/issues/18501#issuecomment-643016517 Thanks @wkcn. I'll report the same with gcc7. You are using gcc10 right? This is an automated message from t

[GitHub] [incubator-mxnet] leezu commented on issue #18501: It costs over 16GB memory to compile indexing_op.o

2020-06-11 Thread GitBox
leezu commented on issue #18501: URL: https://github.com/apache/incubator-mxnet/issues/18501#issuecomment-643014864 Doing a single-threaded build (`ninja -j1`) just costs around 2 GB memory at maximum. Child user: 4167.479 s Child sys : 159.497 s Child wall: 4327.964 s Chi

[GitHub] [incubator-mxnet] leezu commented on issue #18501: It costs over 16GB memory to compile indexing_op.o

2020-06-11 Thread GitBox
leezu commented on issue #18501: URL: https://github.com/apache/incubator-mxnet/issues/18501#issuecomment-642980960 Based on my preliminary test, the memory issue is present in 1.7.x @ciyongch I measure the overall memory consumption during compilation using linux control group feature

[GitHub] [incubator-mxnet] leezu commented on issue #18501: It costs over 16GB memory to compile indexing_op.o

2020-06-11 Thread GitBox
leezu commented on issue #18501: URL: https://github.com/apache/incubator-mxnet/issues/18501#issuecomment-642785535 I disagree. Official MXNet releases are source releases. At this point in time, there exist 0 compliant binary releases. It's very important that we don't introduce regres

[GitHub] [incubator-mxnet] leezu commented on issue #18501: It costs over 16GB memory to compile indexing_op.o

2020-06-09 Thread GitBox
leezu commented on issue #18501: URL: https://github.com/apache/incubator-mxnet/issues/18501#issuecomment-641669164 There seem to be some more issues. In certain build configuration with llvm 7, many of the numpy object files blow up ``` 875M build/CMakeFiles/mxnet.dir/src/op

[GitHub] [incubator-mxnet] leezu commented on issue #18501: It costs over 16GB memory to compile indexing_op.o

2020-06-09 Thread GitBox
leezu commented on issue #18501: URL: https://github.com/apache/incubator-mxnet/issues/18501#issuecomment-641655747 There are more problems with the numpy ops: ``` 875M build/CMakeFiles/mxnet.dir/src/operator/tensor/broadcast_reduce_norm_value.cc.o 918M build/CMakeFiles

[GitHub] [incubator-mxnet] leezu commented on issue #18501: It costs over 16GB memory to compile indexing_op.o

2020-06-09 Thread GitBox
leezu commented on issue #18501: URL: https://github.com/apache/incubator-mxnet/issues/18501#issuecomment-640730423 I think we should consider this a release-critical bug. @woreom @wkcn did you try if this affects the 1.7 / 1.x branches as well? cc: @ciyongch ---

[GitHub] [incubator-mxnet] leezu commented on issue #18501: It costs over 16GB memory to compile indexing_op.o

2020-06-07 Thread GitBox
leezu commented on issue #18501: URL: https://github.com/apache/incubator-mxnet/issues/18501#issuecomment-640286646 Fixing this would be a welcome improvement. Did you investigate if the high memory consumption is consistent among gcc and clang, as well as still present on gcc 9 (or 10) /