stephenrawls opened a new issue #13969: Hybridization Feature Request (better dynamic shape inference) URL: https://github.com/apache/incubator-mxnet/issues/13969 Hi, I'm starting to look into making certain NLP-style models and post-processing steps HybridBlocks, and I ran into a few pain points I wanted to bring up. I basically have the K-best Viterbi algorithm implemented as a HybridBlock, however it has a very fundamental limitation: when I create it, various parameters like the value of "k", the batch size, and the output feature space are locked-in and cannot be changed. This is because inside of my algorithm, I need to create some very precisely shaped data, e.g. here are a few of the things I'm currently doing: ``` F.broadcast_add(alphas.expand_dims(3), transitions).reshape(self.batch_size, -1, self.ntags) or F.broadcast_add( last_alpha, transitions.slice( begin=(0,self.end_tag), end=(self.ntags,self.end_tag+1) ).reshape(self.ntags) ).reshape(self.batch_size, -1) ``` As far as I can tell, the only way I could do a dynamic shape in the first example is if I already had an input array shaped like I wanted. And then I could do: `reshape_like(foo)`. However my problem is I don't have such an array, so I'm forced to use class member variables that are static/fixed in the network after the first forward call. Even though I could describe the shapes I want dynamically based on a combination of the shapes of all my inputs (e.g. axis 1 of this input and axis 2 times axis 3 of that input, etc), I don't believe this is currently supported. It would be nice if I had access to the shapes of my input arguments, and various operators in my HybridBlock could take them as arguments and dynamically infer their shape based on some combination of the shapes of the input data. It would also be nice to be able to create some arrays dependent on the shapes of my input arguments. There is some ability to do this, e.g. with the functions: `ones_like(), zeros_like()`, but as far as I can tell, there is no way for me to do the following in a HybridBlock: ``` def hybrid_forward(F, x, valid_lengths, transitions, k): # Initialize alphas alphas = F.full(shape=(x.shape[0], k, transitions.shape[1]), val=-1e20) alphas[:, 0, -1] = 0 # Throughout rest of algorithm make use of this value that was dynamically created # based on the input shapes passed in ``` The first line I can't do because the same reason as the earlier examples. But I would additionally like more flexibility in initializing arrays / creating constant arrays that respond dynamically to the input shape that is passed in. Overall I had a really easy time implementing the HybridBlock with the use of foreach and various broadcast operations, however due to this lack of dynamic shaping I don't think we will be able to use it as a HybridBlock since several things still must be static as mentioned above. Best, Stephen
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
