During compilation, is it possible to evaluate a scalar sub-DAG of scalar operations in which all leaf nodes are literals to allow for replacement with a literal? For example, in our `nn` library, our convolution and pooling layers have to pass around the spatial dimensions (height and width) of the images that are stretched out into rows of the input/output matrices. These output dimensions are computed within the forward functions of the above layers as small scalar equations. From a mathematical standpoint, these sizes can be determined at compile time, and it is nice to have these size equations in DML (v.s. hiding them inside the engine within built-in functions). However, we do not currently evaluate these expressions during compilation, and thus we are left with unknown sizes even during recompilation. This naturally leads to max memory estimates and thus often leads to unnecessary distributed runtime ops rather than simple CP ones.
Thoughts? -Mike -- Mike Dusenberry GitHub: github.com/dusenberrymw LinkedIn: linkedin.com/in/mikedusenberry Sent from my iPhone.
