samskalicky opened a new issue #19256:
URL: https://github.com/apache/incubator-mxnet/issues/19256
## Description
Theres a difference in the flow between `get_backend_symbol` and
`optimize_for` when a single backend has multiple subgraph properties that
results in the indexed graph not being updated between subgraph properties.
### Error Message
```
Traceback (most recent call last):
File "test.py", line 22, in <module>
sym_block.optimize_for(mx.nd.zeros((64, 4, 10, 10)),
backend='MKLDNN_QUANTIZE')
File "/home/ubuntu/v1.7.x/python/mxnet/gluon/block.py", line 1089, in
optimize_for
self._build_cache(x, *args)
File "/home/ubuntu/v1.7.x/python/mxnet/gluon/block.py", line 979, in
_build_cache
out = out.optimize_for(self._backend, arg_array, aux_array, ctx,
**self._backend_opts)
File "/home/ubuntu/v1.7.x/python/mxnet/symbol/symbol.py", line 1531, in
optimize_for
ctypes.byref(new_aux_names)))
File "/home/ubuntu/v1.7.x/python/mxnet/base.py", line 246, in check_call
raise get_last_ffi_error()
mxnet.base.MXNetError: Traceback (most recent call last):
File "src/operator/subgraph/build_subgraph.cc", line 80
MXNetError: Check failed: input_nid < simple_nodes->size() (6 vs. 6) :
```
## To Reproduce
```
from mxnet.util import use_np
from mxnet.gluon import nn, HybridBlock
import mxnet as mx
import numpy as np
attr = {'sg_mkldnn_conv_bn_0' : {'with_bn': 'true'}}
data = mx.symbol.Variable('data', shape=(64, 4, 10, 10), dtype='float32')
data2 = mx.symbol.Variable('data2', shape=(64, 64, 10, 10), dtype='float32')
weight1 = mx.symbol.Variable('conv1_weight', dtype='float32')
weight2 = mx.symbol.Variable('conv2_weight', dtype='float32',
shape=(64,64,1,1))
conv1 = mx.symbol.Convolution(data=data, weight=weight1, name='conv1',
num_filter=64,
kernel=(1, 1), stride=(1, 1), no_bias=True)
bn1 = mx.symbol.BatchNorm(data=conv1, name="bn1")
conv2 = mx.symbol.Convolution(data=bn1, weight=weight2, name='conv2',
num_filter=64,
kernel=(1, 1), stride=(1, 1), no_bias=True)
bn2 = mx.symbol.BatchNorm(data=conv2, name="bn2")
sum = bn2 + data2
inputs = mx.sym.var('data', dtype='float32')
sym_block = mx.gluon.SymbolBlock(sum, [inputs])
for k, v in sym_block.collect_params().items():
v.initialize()
mm = sym_block(mx.nd.zeros((64, 4, 10, 10)))
sym_block.optimize_for(mx.nd.zeros((64, 4, 10, 10)),
backend='MKLDNN_QUANTIZE')
```
### Steps to reproduce
1. clone v1.7.x branch and build from source
2. run code above
Some debugging finds that the indexed graph inside of an nnvm::Graph object
is only built once:
https://github.com/apache/incubator-tvm/blob/0535fd1df1313471ee789b121ecdde1e39520f8f/nnvm/src/core/graph.cc#L31-L36
So after subsequent subgraph properties the graph is not updated, causing
the error in this issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@mxnet.apache.org
For additional commands, e-mail: issues-h...@mxnet.apache.org
