[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r370393532 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,147 @@ +CustomOp Example and Tutorial += + +## Introduction + +Adding new operators in MXNet requires understanding of MXNet backend operator registration and recompiling of MXNet with all its dependencies. Users can use the old Python custom operator to add new operators, but it is slow, complicated and has poor adoption rate. So our approach for adding custom operators is to enable dynamic loading of C++ custom operators compiled in external libraries at runtime. + +Custom operators (CustomOp) enable users to write new operators without compiling against all of MXNet header files and dependencies. When a library containing custom operators is loaded dynamically, the operators found in the library will be re-registered in MXNet so that users can call those operators natively just like other built-in operators. + +## Getting Started + +### Have MXNet Ready + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operators by running some examples provided in the **example/extensions/lib_custom_op** directory. Start with a common linear algebra operator like `gemm` (Generalized Matrix Multiplication). Go to `lib_custom_op` directory and follow these steps: + +1. Run `make gemm_lib`. The Makefile will generate a dynamic library **libgemm_lib.so** compiled from `gemm_lib.cc`. This is the library you are going to load that contains everything for the custom gemm operator. +2. Run `python test_gemm.py`. It’ll first load the above .so library, find the operators, register them in the MXNet backend, print "Found x operators", then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has a source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file `include/mxnet/lib_api.h` from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invokes the operator using both NDArray and Symbol APIs, and prints outputs of the forward and backward passes. The outputs should be the same as the regular MXNet `gemm` operator. + +## Writing Custom Operator Library: + +For building a library containing your own custom operator, compose a C++ source file like `myop_lib.cc`, include `lib_api.h` header file, and write your custom operator implementation with those essential functions: +- `initialize` - Library Initialization Function +- `REGISTER_OP` - Operator Registration Marco +- `parseAttrs` - Attribute Parser +- `inferType` - Type Inference +- `inferShape` - Shape Inference +- `forward` - Forward Computation (can be replace with `createOpState`, see below for details) + +Then compile it to `libmyop_lib.so` dynamic library using the following command + +g++ -shared -fPIC -std=c++11 myop_lib.cc -o libmyop_lib.so -I ../../../include/mxnet + +Finally you can write a python script to load the library and run your custom operator Review comment: ```suggestion Finally, you can write a Python script to load the library and run your custom operator: ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r370392906 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,147 @@ +CustomOp Example and Tutorial += + +## Introduction + +Adding new operators in MXNet requires understanding of MXNet backend operator registration and recompiling of MXNet with all its dependencies. Users can use the old Python custom operator to add new operators, but it is slow, complicated and has poor adoption rate. So our approach for adding custom operators is to enable dynamic loading of C++ custom operators compiled in external libraries at runtime. + +Custom operators (CustomOp) enable users to write new operators without compiling against all of MXNet header files and dependencies. When a library containing custom operators is loaded dynamically, the operators found in the library will be re-registered in MXNet so that users can call those operators natively just like other built-in operators. + +## Getting Started + +### Have MXNet Ready + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operators by running some examples provided in the **example/extensions/lib_custom_op** directory. Start with a common linear algebra operator like `gemm` (Generalized Matrix Multiplication). Go to `lib_custom_op` directory and follow these steps: + +1. Run `make gemm_lib`. The Makefile will generate a dynamic library **libgemm_lib.so** compiled from `gemm_lib.cc`. This is the library you are going to load that contains everything for the custom gemm operator. +2. Run `python test_gemm.py`. It’ll first load the above .so library, find the operators, register them in the MXNet backend, print "Found x operators", then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has a source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file `include/mxnet/lib_api.h` from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invokes the operator using both NDArray and Symbol APIs, and prints outputs of the forward and backward passes. The outputs should be the same as the regular MXNet `gemm` operator. + +## Writing Custom Operator Library: + +For building a library containing your own custom operator, compose a C++ source file like `myop_lib.cc`, include `lib_api.h` header file, and write your custom operator implementation with those essential functions: +- `initialize` - Library Initialization Function +- `REGISTER_OP` - Operator Registration Marco +- `parseAttrs` - Attribute Parser +- `inferType` - Type Inference +- `inferShape` - Shape Inference +- `forward` - Forward Computation (can be replace with `createOpState`, see below for details) + +Then compile it to `libmyop_lib.so` dynamic library using the following command + Review comment: Surround this with code block `bash` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r370392145 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,147 @@ +CustomOp Example and Tutorial += + +## Introduction + +Adding new operators in MXNet requires understanding of MXNet backend operator registration and recompiling of MXNet with all its dependencies. Users can use the old Python custom operator to add new operators, but it is slow, complicated and has poor adoption rate. So our approach for adding custom operators is to enable dynamic loading of C++ custom operators compiled in external libraries at runtime. + +Custom operators (CustomOp) enable users to write new operators without compiling against all of MXNet header files and dependencies. When a library containing custom operators is loaded dynamically, the operators found in the library will be re-registered in MXNet so that users can call those operators natively just like other built-in operators. + +## Getting Started + +### Have MXNet Ready + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operators by running some examples provided in the **example/extensions/lib_custom_op** directory. Start with a common linear algebra operator like `gemm` (Generalized Matrix Multiplication). Go to `lib_custom_op` directory and follow these steps: + +1. Run `make gemm_lib`. The Makefile will generate a dynamic library **libgemm_lib.so** compiled from `gemm_lib.cc`. This is the library you are going to load that contains everything for the custom gemm operator. +2. Run `python test_gemm.py`. It’ll first load the above .so library, find the operators, register them in the MXNet backend, print "Found x operators", then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has a source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file `include/mxnet/lib_api.h` from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invokes the operator using both NDArray and Symbol APIs, and prints outputs of the forward and backward passes. The outputs should be the same as the regular MXNet `gemm` operator. + +## Writing Custom Operator Library: + +For building a library containing your own custom operator, compose a C++ source file like `myop_lib.cc`, include `lib_api.h` header file, and write your custom operator implementation with those essential functions: Review comment: ```suggestion For building a library containing your own custom operator, compose a C++ source file like `myop_lib.cc`, include `lib_api.h` header file, and write your custom operator implementation with these essential functions: ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r370392731 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,147 @@ +CustomOp Example and Tutorial += + +## Introduction + +Adding new operators in MXNet requires understanding of MXNet backend operator registration and recompiling of MXNet with all its dependencies. Users can use the old Python custom operator to add new operators, but it is slow, complicated and has poor adoption rate. So our approach for adding custom operators is to enable dynamic loading of C++ custom operators compiled in external libraries at runtime. + +Custom operators (CustomOp) enable users to write new operators without compiling against all of MXNet header files and dependencies. When a library containing custom operators is loaded dynamically, the operators found in the library will be re-registered in MXNet so that users can call those operators natively just like other built-in operators. + +## Getting Started + +### Have MXNet Ready + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operators by running some examples provided in the **example/extensions/lib_custom_op** directory. Start with a common linear algebra operator like `gemm` (Generalized Matrix Multiplication). Go to `lib_custom_op` directory and follow these steps: + +1. Run `make gemm_lib`. The Makefile will generate a dynamic library **libgemm_lib.so** compiled from `gemm_lib.cc`. This is the library you are going to load that contains everything for the custom gemm operator. +2. Run `python test_gemm.py`. It’ll first load the above .so library, find the operators, register them in the MXNet backend, print "Found x operators", then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has a source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file `include/mxnet/lib_api.h` from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invokes the operator using both NDArray and Symbol APIs, and prints outputs of the forward and backward passes. The outputs should be the same as the regular MXNet `gemm` operator. + +## Writing Custom Operator Library: + +For building a library containing your own custom operator, compose a C++ source file like `myop_lib.cc`, include `lib_api.h` header file, and write your custom operator implementation with those essential functions: +- `initialize` - Library Initialization Function +- `REGISTER_OP` - Operator Registration Marco +- `parseAttrs` - Attribute Parser +- `inferType` - Type Inference +- `inferShape` - Shape Inference +- `forward` - Forward Computation (can be replace with `createOpState`, see below for details) + +Then compile it to `libmyop_lib.so` dynamic library using the following command Review comment: ```suggestion Then compile it to `libmyop_lib.so` dynamic library using the following command: ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r370393630 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,147 @@ +CustomOp Example and Tutorial += + +## Introduction + +Adding new operators in MXNet requires understanding of MXNet backend operator registration and recompiling of MXNet with all its dependencies. Users can use the old Python custom operator to add new operators, but it is slow, complicated and has poor adoption rate. So our approach for adding custom operators is to enable dynamic loading of C++ custom operators compiled in external libraries at runtime. + +Custom operators (CustomOp) enable users to write new operators without compiling against all of MXNet header files and dependencies. When a library containing custom operators is loaded dynamically, the operators found in the library will be re-registered in MXNet so that users can call those operators natively just like other built-in operators. + +## Getting Started + +### Have MXNet Ready + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operators by running some examples provided in the **example/extensions/lib_custom_op** directory. Start with a common linear algebra operator like `gemm` (Generalized Matrix Multiplication). Go to `lib_custom_op` directory and follow these steps: + +1. Run `make gemm_lib`. The Makefile will generate a dynamic library **libgemm_lib.so** compiled from `gemm_lib.cc`. This is the library you are going to load that contains everything for the custom gemm operator. +2. Run `python test_gemm.py`. It’ll first load the above .so library, find the operators, register them in the MXNet backend, print "Found x operators", then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has a source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file `include/mxnet/lib_api.h` from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invokes the operator using both NDArray and Symbol APIs, and prints outputs of the forward and backward passes. The outputs should be the same as the regular MXNet `gemm` operator. + +## Writing Custom Operator Library: + +For building a library containing your own custom operator, compose a C++ source file like `myop_lib.cc`, include `lib_api.h` header file, and write your custom operator implementation with those essential functions: +- `initialize` - Library Initialization Function +- `REGISTER_OP` - Operator Registration Marco +- `parseAttrs` - Attribute Parser +- `inferType` - Type Inference +- `inferShape` - Shape Inference +- `forward` - Forward Computation (can be replace with `createOpState`, see below for details) + +Then compile it to `libmyop_lib.so` dynamic library using the following command + +g++ -shared -fPIC -std=c++11 myop_lib.cc -o libmyop_lib.so -I ../../../include/mxnet + +Finally you can write a python script to load the library and run your custom operator + Review comment: Surround with code block `python` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r369906264 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: + +1. run `make gemm_lib`, the Makefile will generate a dynamic library **libgemm_lib.so** compiled from gemm_lib.cc. This is the library you are going to load that contains everything of the custom gemm operator. +2. run `python test_gemm.py`, and it’ll first load the above .so library, find operators, register them in the MXNet backend, print "Found x operators"; then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file **include/mxnet/lib_api.h** from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invoke the operator using both ndarray and symbol API, and print outputs of forward and backward pass. The outputs should be the same as the regular MXNet gemm operator. + +## Writing Custom Operators: + +### Regular Custom Operator: + +There are several basic building blocks for making a (stateless) custom operator: + +* [parseAttrs](./gemm_lib.cc#L118) - Attribute Parser: Review comment: Here's an example that could work for you: https://build-me-the-docs-please.readthedocs.io/en/latest/Using_Sphinx/ShowingCodeExamplesInSphinx.html#literalinclude-directive This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r369833473 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + Review comment: yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r36650 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: + +1. run `make gemm_lib`, the Makefile will generate a dynamic library **libgemm_lib.so** compiled from gemm_lib.cc. This is the library you are going to load that contains everything of the custom gemm operator. +2. run `python test_gemm.py`, and it’ll first load the above .so library, find operators, register them in the MXNet backend, print "Found x operators"; then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file **include/mxnet/lib_api.h** from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invoke the operator using both ndarray and symbol API, and print outputs of forward and backward pass. The outputs should be the same as the regular MXNet gemm operator. + +## Writing Custom Operators: + +### Regular Custom Operator: + +There are several basic building blocks for making a (stateless) custom operator: + +* [parseAttrs](./gemm_lib.cc#L118) - Attribute Parser: +* This function specifies number of input and output tensors for the custom operator; also this is where a custom operator can validate the attributes (ie. options) specified by the user. + +MXReturnValue parseAttrs( +std::map attrs, +int* num_in, +int* num_out) + + +* [inferType](./gemm_lib.cc#L124) - Type Inference: +* This function specifies how custom operator infers output data types using input data types. + +MXReturnValue inferType( +std::map attrs, +std::vector , +std::vector ) + +* [inferShape](./gemm_lib.cc#L143) - Shape Inference: +* This function specifies how custom operator infers output tensor shape using input shape. + +MXReturnValue inferShape( +std::map attrs, +std::vector> , +std::vector> ) + +* [forward](./gemm_lib.cc#L56) - Forward function: +* This function specifies the computation of forward pass of the operator. + +MXReturnValue forward( +std::map attrs, +std::vector inputs, +std::vector outputs, +OpResource res) + +* [REGISTER_OP(my_op_name) Macro](./gemm_lib.cc#L169): +* This macro registers custom operator to all MXNet APIs by its name, and you need to call setters to bind the above functions to the registered operator. + +REGISTER_OP(my_op_name) +.setForward(forward) +.setParseAttrs(parseAttrs) +.setInferType(inferType) +.setInferShape(inferShape); + +Also there are some optional functions you can specify: + +* [backward](./gemm_lib.cc#L90) - Backward Gradient function: +* This function specifies the computation of backward pass of the operator. + +MXReturnValue backward( +std::map attrs, +std::vector inputs, +std::vector outputs, +OpResource res) + +* [mutateInputs](./gemm_lib.cc#L214) - Specify mutable input: +* This function allows you to mark some inputs to be mutable inputs, useful when using aux parameters for BatchNorm-like operators. + +MXReturnValue mutateInputs( +std::map attrs, +std::vector _indices) + +Let’s take a closer look at those registry functions: + +* **parseAttrs**: This function takes 3 arguments. 1st argument is an input, which is the attributes passed all the way from Python code. When user calls `mx.nd.my_op_name(s,t,keyword=1)`, the keyword is passed to the attributes as an entry of the map. 2nd
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366112551 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: + +1. run `make gemm_lib`, the Makefile will generate a dynamic library **libgemm_lib.so** compiled from gemm_lib.cc. This is the library you are going to load that contains everything of the custom gemm operator. +2. run `python test_gemm.py`, and it’ll first load the above .so library, find operators, register them in the MXNet backend, print "Found x operators"; then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file **include/mxnet/lib_api.h** from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invoke the operator using both ndarray and symbol API, and print outputs of forward and backward pass. The outputs should be the same as the regular MXNet gemm operator. + +## Writing Custom Operators: + +### Regular Custom Operator: + +There are several basic building blocks for making a (stateless) custom operator: + +* [parseAttrs](./gemm_lib.cc#L118) - Attribute Parser: +* This function specifies number of input and output tensors for the custom operator; also this is where a custom operator can validate the attributes (ie. options) specified by the user. + +MXReturnValue parseAttrs( +std::map attrs, +int* num_in, +int* num_out) + + +* [inferType](./gemm_lib.cc#L124) - Type Inference: +* This function specifies how custom operator infers output data types using input data types. + +MXReturnValue inferType( +std::map attrs, +std::vector , +std::vector ) + +* [inferShape](./gemm_lib.cc#L143) - Shape Inference: +* This function specifies how custom operator infers output tensor shape using input shape. + +MXReturnValue inferShape( +std::map attrs, +std::vector> , +std::vector> ) + +* [forward](./gemm_lib.cc#L56) - Forward function: +* This function specifies the computation of forward pass of the operator. + +MXReturnValue forward( +std::map attrs, +std::vector inputs, +std::vector outputs, +OpResource res) + +* [REGISTER_OP(my_op_name) Macro](./gemm_lib.cc#L169): +* This macro registers custom operator to all MXNet APIs by its name, and you need to call setters to bind the above functions to the registered operator. + +REGISTER_OP(my_op_name) +.setForward(forward) +.setParseAttrs(parseAttrs) +.setInferType(inferType) +.setInferShape(inferShape); + +Also there are some optional functions you can specify: + +* [backward](./gemm_lib.cc#L90) - Backward Gradient function: +* This function specifies the computation of backward pass of the operator. + +MXReturnValue backward( +std::map attrs, +std::vector inputs, +std::vector outputs, +OpResource res) + +* [mutateInputs](./gemm_lib.cc#L214) - Specify mutable input: +* This function allows you to mark some inputs to be mutable inputs, useful when using aux parameters for BatchNorm-like operators. + +MXReturnValue mutateInputs( +std::map attrs, +std::vector _indices) + +Let’s take a closer look at those registry functions: + +* **parseAttrs**: This function takes 3 arguments. 1st argument is an input, which is the attributes passed all the way from Python code. When user calls `mx.nd.my_op_name(s,t,keyword=1)`, the keyword is passed to the attributes as an entry of the map. 2nd
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366112812 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: + +1. run `make gemm_lib`, the Makefile will generate a dynamic library **libgemm_lib.so** compiled from gemm_lib.cc. This is the library you are going to load that contains everything of the custom gemm operator. +2. run `python test_gemm.py`, and it’ll first load the above .so library, find operators, register them in the MXNet backend, print "Found x operators"; then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file **include/mxnet/lib_api.h** from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invoke the operator using both ndarray and symbol API, and print outputs of forward and backward pass. The outputs should be the same as the regular MXNet gemm operator. + +## Writing Custom Operators: + +### Regular Custom Operator: + +There are several basic building blocks for making a (stateless) custom operator: + +* [parseAttrs](./gemm_lib.cc#L118) - Attribute Parser: +* This function specifies number of input and output tensors for the custom operator; also this is where a custom operator can validate the attributes (ie. options) specified by the user. + +MXReturnValue parseAttrs( +std::map attrs, +int* num_in, +int* num_out) + + +* [inferType](./gemm_lib.cc#L124) - Type Inference: +* This function specifies how custom operator infers output data types using input data types. + +MXReturnValue inferType( +std::map attrs, +std::vector , +std::vector ) + +* [inferShape](./gemm_lib.cc#L143) - Shape Inference: +* This function specifies how custom operator infers output tensor shape using input shape. + +MXReturnValue inferShape( +std::map attrs, +std::vector> , +std::vector> ) + +* [forward](./gemm_lib.cc#L56) - Forward function: +* This function specifies the computation of forward pass of the operator. + +MXReturnValue forward( +std::map attrs, +std::vector inputs, +std::vector outputs, +OpResource res) + +* [REGISTER_OP(my_op_name) Macro](./gemm_lib.cc#L169): +* This macro registers custom operator to all MXNet APIs by its name, and you need to call setters to bind the above functions to the registered operator. + +REGISTER_OP(my_op_name) +.setForward(forward) +.setParseAttrs(parseAttrs) +.setInferType(inferType) +.setInferShape(inferShape); + +Also there are some optional functions you can specify: + +* [backward](./gemm_lib.cc#L90) - Backward Gradient function: +* This function specifies the computation of backward pass of the operator. + +MXReturnValue backward( +std::map attrs, +std::vector inputs, +std::vector outputs, +OpResource res) + +* [mutateInputs](./gemm_lib.cc#L214) - Specify mutable input: +* This function allows you to mark some inputs to be mutable inputs, useful when using aux parameters for BatchNorm-like operators. + +MXReturnValue mutateInputs( +std::map attrs, +std::vector _indices) + +Let’s take a closer look at those registry functions: + +* **parseAttrs**: This function takes 3 arguments. 1st argument is an input, which is the attributes passed all the way from Python code. When user calls `mx.nd.my_op_name(s,t,keyword=1)`, the keyword is passed to the attributes as an entry of the map. 2nd
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366108808 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + Review comment: Sometimes one of the transpilers will complain that this is too short. Recommend making it longer to match the title. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366110115 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: + +1. run `make gemm_lib`, the Makefile will generate a dynamic library **libgemm_lib.so** compiled from gemm_lib.cc. This is the library you are going to load that contains everything of the custom gemm operator. +2. run `python test_gemm.py`, and it’ll first load the above .so library, find operators, register them in the MXNet backend, print "Found x operators"; then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file **include/mxnet/lib_api.h** from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invoke the operator using both ndarray and symbol API, and print outputs of forward and backward pass. The outputs should be the same as the regular MXNet gemm operator. Review comment: ```suggestion * **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invokes the operator using both NDArray and Symbol APIs, and prints outputs of the forward and backward passes. The outputs should be the same as the regular MXNet `gemm` operator. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366112461 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: + +1. run `make gemm_lib`, the Makefile will generate a dynamic library **libgemm_lib.so** compiled from gemm_lib.cc. This is the library you are going to load that contains everything of the custom gemm operator. +2. run `python test_gemm.py`, and it’ll first load the above .so library, find operators, register them in the MXNet backend, print "Found x operators"; then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file **include/mxnet/lib_api.h** from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invoke the operator using both ndarray and symbol API, and print outputs of forward and backward pass. The outputs should be the same as the regular MXNet gemm operator. + +## Writing Custom Operators: + +### Regular Custom Operator: + +There are several basic building blocks for making a (stateless) custom operator: + +* [parseAttrs](./gemm_lib.cc#L118) - Attribute Parser: +* This function specifies number of input and output tensors for the custom operator; also this is where a custom operator can validate the attributes (ie. options) specified by the user. + +MXReturnValue parseAttrs( +std::map attrs, +int* num_in, +int* num_out) + + +* [inferType](./gemm_lib.cc#L124) - Type Inference: +* This function specifies how custom operator infers output data types using input data types. + +MXReturnValue inferType( +std::map attrs, +std::vector , +std::vector ) + +* [inferShape](./gemm_lib.cc#L143) - Shape Inference: +* This function specifies how custom operator infers output tensor shape using input shape. + +MXReturnValue inferShape( +std::map attrs, +std::vector> , +std::vector> ) + +* [forward](./gemm_lib.cc#L56) - Forward function: +* This function specifies the computation of forward pass of the operator. + +MXReturnValue forward( +std::map attrs, +std::vector inputs, +std::vector outputs, +OpResource res) + +* [REGISTER_OP(my_op_name) Macro](./gemm_lib.cc#L169): +* This macro registers custom operator to all MXNet APIs by its name, and you need to call setters to bind the above functions to the registered operator. + +REGISTER_OP(my_op_name) +.setForward(forward) +.setParseAttrs(parseAttrs) +.setInferType(inferType) +.setInferShape(inferShape); + +Also there are some optional functions you can specify: + +* [backward](./gemm_lib.cc#L90) - Backward Gradient function: +* This function specifies the computation of backward pass of the operator. + +MXReturnValue backward( +std::map attrs, +std::vector inputs, +std::vector outputs, +OpResource res) + +* [mutateInputs](./gemm_lib.cc#L214) - Specify mutable input: +* This function allows you to mark some inputs to be mutable inputs, useful when using aux parameters for BatchNorm-like operators. + +MXReturnValue mutateInputs( +std::map attrs, +std::vector _indices) + +Let’s take a closer look at those registry functions: + +* **parseAttrs**: This function takes 3 arguments. 1st argument is an input, which is the attributes passed all the way from Python code. When user calls `mx.nd.my_op_name(s,t,keyword=1)`, the keyword is passed to the attributes as an entry of the map. 2nd
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366112292 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: + +1. run `make gemm_lib`, the Makefile will generate a dynamic library **libgemm_lib.so** compiled from gemm_lib.cc. This is the library you are going to load that contains everything of the custom gemm operator. +2. run `python test_gemm.py`, and it’ll first load the above .so library, find operators, register them in the MXNet backend, print "Found x operators"; then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file **include/mxnet/lib_api.h** from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invoke the operator using both ndarray and symbol API, and print outputs of forward and backward pass. The outputs should be the same as the regular MXNet gemm operator. + +## Writing Custom Operators: + +### Regular Custom Operator: + +There are several basic building blocks for making a (stateless) custom operator: + +* [parseAttrs](./gemm_lib.cc#L118) - Attribute Parser: +* This function specifies number of input and output tensors for the custom operator; also this is where a custom operator can validate the attributes (ie. options) specified by the user. + +MXReturnValue parseAttrs( +std::map attrs, +int* num_in, +int* num_out) + + +* [inferType](./gemm_lib.cc#L124) - Type Inference: +* This function specifies how custom operator infers output data types using input data types. + +MXReturnValue inferType( +std::map attrs, +std::vector , +std::vector ) + +* [inferShape](./gemm_lib.cc#L143) - Shape Inference: +* This function specifies how custom operator infers output tensor shape using input shape. + +MXReturnValue inferShape( +std::map attrs, +std::vector> , +std::vector> ) + +* [forward](./gemm_lib.cc#L56) - Forward function: +* This function specifies the computation of forward pass of the operator. + +MXReturnValue forward( +std::map attrs, +std::vector inputs, +std::vector outputs, +OpResource res) + +* [REGISTER_OP(my_op_name) Macro](./gemm_lib.cc#L169): +* This macro registers custom operator to all MXNet APIs by its name, and you need to call setters to bind the above functions to the registered operator. + +REGISTER_OP(my_op_name) +.setForward(forward) +.setParseAttrs(parseAttrs) +.setInferType(inferType) +.setInferShape(inferShape); + +Also there are some optional functions you can specify: + +* [backward](./gemm_lib.cc#L90) - Backward Gradient function: +* This function specifies the computation of backward pass of the operator. + +MXReturnValue backward( +std::map attrs, +std::vector inputs, +std::vector outputs, +OpResource res) + +* [mutateInputs](./gemm_lib.cc#L214) - Specify mutable input: +* This function allows you to mark some inputs to be mutable inputs, useful when using aux parameters for BatchNorm-like operators. + +MXReturnValue mutateInputs( +std::map attrs, +std::vector _indices) + +Let’s take a closer look at those registry functions: + +* **parseAttrs**: This function takes 3 arguments. 1st argument is an input, which is the attributes passed all the way from Python code. When user calls `mx.nd.my_op_name(s,t,keyword=1)`, the keyword is passed to the attributes as an entry of the map. 2nd
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366111871 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: + +1. run `make gemm_lib`, the Makefile will generate a dynamic library **libgemm_lib.so** compiled from gemm_lib.cc. This is the library you are going to load that contains everything of the custom gemm operator. +2. run `python test_gemm.py`, and it’ll first load the above .so library, find operators, register them in the MXNet backend, print "Found x operators"; then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file **include/mxnet/lib_api.h** from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invoke the operator using both ndarray and symbol API, and print outputs of forward and backward pass. The outputs should be the same as the regular MXNet gemm operator. + +## Writing Custom Operators: + +### Regular Custom Operator: + +There are several basic building blocks for making a (stateless) custom operator: + +* [parseAttrs](./gemm_lib.cc#L118) - Attribute Parser: +* This function specifies number of input and output tensors for the custom operator; also this is where a custom operator can validate the attributes (ie. options) specified by the user. + +MXReturnValue parseAttrs( +std::map attrs, +int* num_in, +int* num_out) + + +* [inferType](./gemm_lib.cc#L124) - Type Inference: +* This function specifies how custom operator infers output data types using input data types. + +MXReturnValue inferType( +std::map attrs, +std::vector , +std::vector ) + +* [inferShape](./gemm_lib.cc#L143) - Shape Inference: +* This function specifies how custom operator infers output tensor shape using input shape. + +MXReturnValue inferShape( +std::map attrs, +std::vector> , +std::vector> ) + +* [forward](./gemm_lib.cc#L56) - Forward function: +* This function specifies the computation of forward pass of the operator. + +MXReturnValue forward( +std::map attrs, +std::vector inputs, +std::vector outputs, +OpResource res) + +* [REGISTER_OP(my_op_name) Macro](./gemm_lib.cc#L169): +* This macro registers custom operator to all MXNet APIs by its name, and you need to call setters to bind the above functions to the registered operator. + +REGISTER_OP(my_op_name) +.setForward(forward) +.setParseAttrs(parseAttrs) +.setInferType(inferType) +.setInferShape(inferShape); + +Also there are some optional functions you can specify: + +* [backward](./gemm_lib.cc#L90) - Backward Gradient function: +* This function specifies the computation of backward pass of the operator. + +MXReturnValue backward( +std::map attrs, +std::vector inputs, +std::vector outputs, +OpResource res) + +* [mutateInputs](./gemm_lib.cc#L214) - Specify mutable input: +* This function allows you to mark some inputs to be mutable inputs, useful when using aux parameters for BatchNorm-like operators. + +MXReturnValue mutateInputs( +std::map attrs, +std::vector _indices) + +Let’s take a closer look at those registry functions: + +* **parseAttrs**: This function takes 3 arguments. 1st argument is an input, which is the attributes passed all the way from Python code. When user calls `mx.nd.my_op_name(s,t,keyword=1)`, the keyword is passed to the attributes as an entry of the map. 2nd
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366110539 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: + +1. run `make gemm_lib`, the Makefile will generate a dynamic library **libgemm_lib.so** compiled from gemm_lib.cc. This is the library you are going to load that contains everything of the custom gemm operator. +2. run `python test_gemm.py`, and it’ll first load the above .so library, find operators, register them in the MXNet backend, print "Found x operators"; then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file **include/mxnet/lib_api.h** from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invoke the operator using both ndarray and symbol API, and print outputs of forward and backward pass. The outputs should be the same as the regular MXNet gemm operator. + +## Writing Custom Operators: + +### Regular Custom Operator: + +There are several basic building blocks for making a (stateless) custom operator: + +* [parseAttrs](./gemm_lib.cc#L118) - Attribute Parser: +* This function specifies number of input and output tensors for the custom operator; also this is where a custom operator can validate the attributes (ie. options) specified by the user. + +MXReturnValue parseAttrs( +std::map attrs, +int* num_in, +int* num_out) + + +* [inferType](./gemm_lib.cc#L124) - Type Inference: +* This function specifies how custom operator infers output data types using input data types. + +MXReturnValue inferType( +std::map attrs, +std::vector , +std::vector ) + +* [inferShape](./gemm_lib.cc#L143) - Shape Inference: +* This function specifies how custom operator infers output tensor shape using input shape. + +MXReturnValue inferShape( +std::map attrs, +std::vector> , +std::vector> ) + +* [forward](./gemm_lib.cc#L56) - Forward function: +* This function specifies the computation of forward pass of the operator. Review comment: ```suggestion * This function specifies the computation of the forward pass of the operator. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366111466 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: + +1. run `make gemm_lib`, the Makefile will generate a dynamic library **libgemm_lib.so** compiled from gemm_lib.cc. This is the library you are going to load that contains everything of the custom gemm operator. +2. run `python test_gemm.py`, and it’ll first load the above .so library, find operators, register them in the MXNet backend, print "Found x operators"; then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file **include/mxnet/lib_api.h** from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invoke the operator using both ndarray and symbol API, and print outputs of forward and backward pass. The outputs should be the same as the regular MXNet gemm operator. + +## Writing Custom Operators: + +### Regular Custom Operator: + +There are several basic building blocks for making a (stateless) custom operator: + +* [parseAttrs](./gemm_lib.cc#L118) - Attribute Parser: +* This function specifies number of input and output tensors for the custom operator; also this is where a custom operator can validate the attributes (ie. options) specified by the user. + +MXReturnValue parseAttrs( +std::map attrs, +int* num_in, +int* num_out) + + +* [inferType](./gemm_lib.cc#L124) - Type Inference: +* This function specifies how custom operator infers output data types using input data types. + +MXReturnValue inferType( +std::map attrs, +std::vector , +std::vector ) + +* [inferShape](./gemm_lib.cc#L143) - Shape Inference: +* This function specifies how custom operator infers output tensor shape using input shape. + +MXReturnValue inferShape( +std::map attrs, +std::vector> , +std::vector> ) + +* [forward](./gemm_lib.cc#L56) - Forward function: +* This function specifies the computation of forward pass of the operator. + +MXReturnValue forward( +std::map attrs, +std::vector inputs, +std::vector outputs, +OpResource res) + +* [REGISTER_OP(my_op_name) Macro](./gemm_lib.cc#L169): +* This macro registers custom operator to all MXNet APIs by its name, and you need to call setters to bind the above functions to the registered operator. + +REGISTER_OP(my_op_name) +.setForward(forward) +.setParseAttrs(parseAttrs) +.setInferType(inferType) +.setInferShape(inferShape); + +Also there are some optional functions you can specify: + +* [backward](./gemm_lib.cc#L90) - Backward Gradient function: +* This function specifies the computation of backward pass of the operator. + +MXReturnValue backward( +std::map attrs, +std::vector inputs, +std::vector outputs, +OpResource res) + +* [mutateInputs](./gemm_lib.cc#L214) - Specify mutable input: +* This function allows you to mark some inputs to be mutable inputs, useful when using aux parameters for BatchNorm-like operators. + +MXReturnValue mutateInputs( +std::map attrs, +std::vector _indices) + +Let’s take a closer look at those registry functions: + +* **parseAttrs**: This function takes 3 arguments. 1st argument is an input, which is the attributes passed all the way from Python code. When user calls `mx.nd.my_op_name(s,t,keyword=1)`, the keyword is passed to the attributes as an entry of the map. 2nd
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366110748 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: + +1. run `make gemm_lib`, the Makefile will generate a dynamic library **libgemm_lib.so** compiled from gemm_lib.cc. This is the library you are going to load that contains everything of the custom gemm operator. +2. run `python test_gemm.py`, and it’ll first load the above .so library, find operators, register them in the MXNet backend, print "Found x operators"; then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file **include/mxnet/lib_api.h** from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invoke the operator using both ndarray and symbol API, and print outputs of forward and backward pass. The outputs should be the same as the regular MXNet gemm operator. + +## Writing Custom Operators: + +### Regular Custom Operator: + +There are several basic building blocks for making a (stateless) custom operator: + +* [parseAttrs](./gemm_lib.cc#L118) - Attribute Parser: +* This function specifies number of input and output tensors for the custom operator; also this is where a custom operator can validate the attributes (ie. options) specified by the user. + +MXReturnValue parseAttrs( +std::map attrs, +int* num_in, +int* num_out) + + +* [inferType](./gemm_lib.cc#L124) - Type Inference: +* This function specifies how custom operator infers output data types using input data types. + +MXReturnValue inferType( +std::map attrs, +std::vector , +std::vector ) + +* [inferShape](./gemm_lib.cc#L143) - Shape Inference: +* This function specifies how custom operator infers output tensor shape using input shape. + +MXReturnValue inferShape( +std::map attrs, +std::vector> , +std::vector> ) + +* [forward](./gemm_lib.cc#L56) - Forward function: +* This function specifies the computation of forward pass of the operator. + +MXReturnValue forward( +std::map attrs, +std::vector inputs, +std::vector outputs, +OpResource res) + +* [REGISTER_OP(my_op_name) Macro](./gemm_lib.cc#L169): +* This macro registers custom operator to all MXNet APIs by its name, and you need to call setters to bind the above functions to the registered operator. Review comment: ```suggestion * This macro registers the custom operator to all of the MXNet APIs by its name. You need to call setters to bind the above functions to the registered operator. ``` Is the last sentence clear enough? I'm not really sure what you mean. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366110828 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: + +1. run `make gemm_lib`, the Makefile will generate a dynamic library **libgemm_lib.so** compiled from gemm_lib.cc. This is the library you are going to load that contains everything of the custom gemm operator. +2. run `python test_gemm.py`, and it’ll first load the above .so library, find operators, register them in the MXNet backend, print "Found x operators"; then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file **include/mxnet/lib_api.h** from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invoke the operator using both ndarray and symbol API, and print outputs of forward and backward pass. The outputs should be the same as the regular MXNet gemm operator. + +## Writing Custom Operators: + +### Regular Custom Operator: + +There are several basic building blocks for making a (stateless) custom operator: + +* [parseAttrs](./gemm_lib.cc#L118) - Attribute Parser: +* This function specifies number of input and output tensors for the custom operator; also this is where a custom operator can validate the attributes (ie. options) specified by the user. + +MXReturnValue parseAttrs( +std::map attrs, +int* num_in, +int* num_out) + + +* [inferType](./gemm_lib.cc#L124) - Type Inference: +* This function specifies how custom operator infers output data types using input data types. + +MXReturnValue inferType( +std::map attrs, +std::vector , +std::vector ) + +* [inferShape](./gemm_lib.cc#L143) - Shape Inference: +* This function specifies how custom operator infers output tensor shape using input shape. + +MXReturnValue inferShape( +std::map attrs, +std::vector> , +std::vector> ) + +* [forward](./gemm_lib.cc#L56) - Forward function: +* This function specifies the computation of forward pass of the operator. + +MXReturnValue forward( +std::map attrs, +std::vector inputs, +std::vector outputs, +OpResource res) + +* [REGISTER_OP(my_op_name) Macro](./gemm_lib.cc#L169): +* This macro registers custom operator to all MXNet APIs by its name, and you need to call setters to bind the above functions to the registered operator. + +REGISTER_OP(my_op_name) +.setForward(forward) +.setParseAttrs(parseAttrs) +.setInferType(inferType) +.setInferShape(inferShape); + +Also there are some optional functions you can specify: + +* [backward](./gemm_lib.cc#L90) - Backward Gradient function: +* This function specifies the computation of backward pass of the operator. Review comment: ```suggestion * This function specifies the computation of the backward pass of the operator. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366109842 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: + +1. run `make gemm_lib`, the Makefile will generate a dynamic library **libgemm_lib.so** compiled from gemm_lib.cc. This is the library you are going to load that contains everything of the custom gemm operator. +2. run `python test_gemm.py`, and it’ll first load the above .so library, find operators, register them in the MXNet backend, print "Found x operators"; then invoke the operator like a regular MXNet operator and output the result. Review comment: ```suggestion 2. Run `python test_gemm.py`. It’ll first load the above .so library, find the operators, register them in the MXNet backend, print "Found x operators", then invoke the operator like a regular MXNet operator and output the result. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366112087 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: + +1. run `make gemm_lib`, the Makefile will generate a dynamic library **libgemm_lib.so** compiled from gemm_lib.cc. This is the library you are going to load that contains everything of the custom gemm operator. +2. run `python test_gemm.py`, and it’ll first load the above .so library, find operators, register them in the MXNet backend, print "Found x operators"; then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file **include/mxnet/lib_api.h** from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invoke the operator using both ndarray and symbol API, and print outputs of forward and backward pass. The outputs should be the same as the regular MXNet gemm operator. + +## Writing Custom Operators: + +### Regular Custom Operator: + +There are several basic building blocks for making a (stateless) custom operator: + +* [parseAttrs](./gemm_lib.cc#L118) - Attribute Parser: +* This function specifies number of input and output tensors for the custom operator; also this is where a custom operator can validate the attributes (ie. options) specified by the user. + +MXReturnValue parseAttrs( +std::map attrs, +int* num_in, +int* num_out) + + +* [inferType](./gemm_lib.cc#L124) - Type Inference: +* This function specifies how custom operator infers output data types using input data types. + +MXReturnValue inferType( +std::map attrs, +std::vector , +std::vector ) + +* [inferShape](./gemm_lib.cc#L143) - Shape Inference: +* This function specifies how custom operator infers output tensor shape using input shape. + +MXReturnValue inferShape( +std::map attrs, +std::vector> , +std::vector> ) + +* [forward](./gemm_lib.cc#L56) - Forward function: +* This function specifies the computation of forward pass of the operator. + +MXReturnValue forward( +std::map attrs, +std::vector inputs, +std::vector outputs, +OpResource res) + +* [REGISTER_OP(my_op_name) Macro](./gemm_lib.cc#L169): +* This macro registers custom operator to all MXNet APIs by its name, and you need to call setters to bind the above functions to the registered operator. + +REGISTER_OP(my_op_name) +.setForward(forward) +.setParseAttrs(parseAttrs) +.setInferType(inferType) +.setInferShape(inferShape); + +Also there are some optional functions you can specify: + +* [backward](./gemm_lib.cc#L90) - Backward Gradient function: +* This function specifies the computation of backward pass of the operator. + +MXReturnValue backward( +std::map attrs, +std::vector inputs, +std::vector outputs, +OpResource res) + +* [mutateInputs](./gemm_lib.cc#L214) - Specify mutable input: +* This function allows you to mark some inputs to be mutable inputs, useful when using aux parameters for BatchNorm-like operators. + +MXReturnValue mutateInputs( +std::map attrs, +std::vector _indices) + +Let’s take a closer look at those registry functions: + +* **parseAttrs**: This function takes 3 arguments. 1st argument is an input, which is the attributes passed all the way from Python code. When user calls `mx.nd.my_op_name(s,t,keyword=1)`, the keyword is passed to the attributes as an entry of the map. 2nd
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366110448 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: + +1. run `make gemm_lib`, the Makefile will generate a dynamic library **libgemm_lib.so** compiled from gemm_lib.cc. This is the library you are going to load that contains everything of the custom gemm operator. +2. run `python test_gemm.py`, and it’ll first load the above .so library, find operators, register them in the MXNet backend, print "Found x operators"; then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file **include/mxnet/lib_api.h** from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invoke the operator using both ndarray and symbol API, and print outputs of forward and backward pass. The outputs should be the same as the regular MXNet gemm operator. + +## Writing Custom Operators: + +### Regular Custom Operator: + +There are several basic building blocks for making a (stateless) custom operator: + +* [parseAttrs](./gemm_lib.cc#L118) - Attribute Parser: +* This function specifies number of input and output tensors for the custom operator; also this is where a custom operator can validate the attributes (ie. options) specified by the user. + +MXReturnValue parseAttrs( +std::map attrs, +int* num_in, +int* num_out) + + +* [inferType](./gemm_lib.cc#L124) - Type Inference: +* This function specifies how custom operator infers output data types using input data types. Review comment: ```suggestion * This function specifies how the custom operator infers output data types using input data types. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366109718 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: + +1. run `make gemm_lib`, the Makefile will generate a dynamic library **libgemm_lib.so** compiled from gemm_lib.cc. This is the library you are going to load that contains everything of the custom gemm operator. Review comment: ```suggestion 1. Run `make gemm_lib`. The Makefile will generate a dynamic library **libgemm_lib.so** compiled from `gemm_lib.cc`. This is the library you are going to load that contains everything for the custom gemm operator. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366110882 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: + +1. run `make gemm_lib`, the Makefile will generate a dynamic library **libgemm_lib.so** compiled from gemm_lib.cc. This is the library you are going to load that contains everything of the custom gemm operator. +2. run `python test_gemm.py`, and it’ll first load the above .so library, find operators, register them in the MXNet backend, print "Found x operators"; then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file **include/mxnet/lib_api.h** from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invoke the operator using both ndarray and symbol API, and print outputs of forward and backward pass. The outputs should be the same as the regular MXNet gemm operator. + +## Writing Custom Operators: + +### Regular Custom Operator: + +There are several basic building blocks for making a (stateless) custom operator: + +* [parseAttrs](./gemm_lib.cc#L118) - Attribute Parser: +* This function specifies number of input and output tensors for the custom operator; also this is where a custom operator can validate the attributes (ie. options) specified by the user. + +MXReturnValue parseAttrs( +std::map attrs, +int* num_in, +int* num_out) + + +* [inferType](./gemm_lib.cc#L124) - Type Inference: +* This function specifies how custom operator infers output data types using input data types. + +MXReturnValue inferType( +std::map attrs, +std::vector , +std::vector ) + +* [inferShape](./gemm_lib.cc#L143) - Shape Inference: +* This function specifies how custom operator infers output tensor shape using input shape. + +MXReturnValue inferShape( +std::map attrs, +std::vector> , +std::vector> ) + +* [forward](./gemm_lib.cc#L56) - Forward function: +* This function specifies the computation of forward pass of the operator. + +MXReturnValue forward( +std::map attrs, +std::vector inputs, +std::vector outputs, +OpResource res) + +* [REGISTER_OP(my_op_name) Macro](./gemm_lib.cc#L169): +* This macro registers custom operator to all MXNet APIs by its name, and you need to call setters to bind the above functions to the registered operator. + +REGISTER_OP(my_op_name) +.setForward(forward) +.setParseAttrs(parseAttrs) +.setInferType(inferType) +.setInferShape(inferShape); + +Also there are some optional functions you can specify: + +* [backward](./gemm_lib.cc#L90) - Backward Gradient function: +* This function specifies the computation of backward pass of the operator. + +MXReturnValue backward( +std::map attrs, +std::vector inputs, +std::vector outputs, +OpResource res) + +* [mutateInputs](./gemm_lib.cc#L214) - Specify mutable input: +* This function allows you to mark some inputs to be mutable inputs, useful when using aux parameters for BatchNorm-like operators. Review comment: ```suggestion * This function allows you to mark some inputs to be mutable inputs. It is useful when using aux parameters for BatchNorm-like operators. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366109555 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: Review comment: ```suggestion You can start getting familiar with custom operators by running some examples provided in the **example/extensions/lib_custom_op** directory. Start with a common linear algebra operator like `gemm` (Generalized Matrix Multiplication). Go to `lib_custom_op` directory and follow these steps: ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366109930 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: + +1. run `make gemm_lib`, the Makefile will generate a dynamic library **libgemm_lib.so** compiled from gemm_lib.cc. This is the library you are going to load that contains everything of the custom gemm operator. +2. run `python test_gemm.py`, and it’ll first load the above .so library, find operators, register them in the MXNet backend, print "Found x operators"; then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has source code implementation of all required components of a custom operator, as well as the registration of the custom operator. Review comment: ```suggestion * **lib_custom_op/gemm_lib.cc**: This file has a source code implementation of all required components of a custom operator, as well as the registration of the custom operator. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366113614 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: + +1. run `make gemm_lib`, the Makefile will generate a dynamic library **libgemm_lib.so** compiled from gemm_lib.cc. This is the library you are going to load that contains everything of the custom gemm operator. +2. run `python test_gemm.py`, and it’ll first load the above .so library, find operators, register them in the MXNet backend, print "Found x operators"; then invoke the operator like a regular MXNet operator and output the result. + Review comment: I think this is missing a transition. How do I go from running this basic example to consuming the following info for my own op? Maybe even a simple example of customization for a particular use case would help. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366111402 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: + +1. run `make gemm_lib`, the Makefile will generate a dynamic library **libgemm_lib.so** compiled from gemm_lib.cc. This is the library you are going to load that contains everything of the custom gemm operator. +2. run `python test_gemm.py`, and it’ll first load the above .so library, find operators, register them in the MXNet backend, print "Found x operators"; then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file **include/mxnet/lib_api.h** from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invoke the operator using both ndarray and symbol API, and print outputs of forward and backward pass. The outputs should be the same as the regular MXNet gemm operator. + +## Writing Custom Operators: + +### Regular Custom Operator: + +There are several basic building blocks for making a (stateless) custom operator: + +* [parseAttrs](./gemm_lib.cc#L118) - Attribute Parser: +* This function specifies number of input and output tensors for the custom operator; also this is where a custom operator can validate the attributes (ie. options) specified by the user. + +MXReturnValue parseAttrs( +std::map attrs, +int* num_in, +int* num_out) + + +* [inferType](./gemm_lib.cc#L124) - Type Inference: +* This function specifies how custom operator infers output data types using input data types. + +MXReturnValue inferType( +std::map attrs, +std::vector , +std::vector ) + +* [inferShape](./gemm_lib.cc#L143) - Shape Inference: +* This function specifies how custom operator infers output tensor shape using input shape. + +MXReturnValue inferShape( +std::map attrs, +std::vector> , +std::vector> ) + +* [forward](./gemm_lib.cc#L56) - Forward function: +* This function specifies the computation of forward pass of the operator. + +MXReturnValue forward( +std::map attrs, +std::vector inputs, +std::vector outputs, +OpResource res) + +* [REGISTER_OP(my_op_name) Macro](./gemm_lib.cc#L169): +* This macro registers custom operator to all MXNet APIs by its name, and you need to call setters to bind the above functions to the registered operator. + +REGISTER_OP(my_op_name) +.setForward(forward) +.setParseAttrs(parseAttrs) +.setInferType(inferType) +.setInferShape(inferShape); + +Also there are some optional functions you can specify: + +* [backward](./gemm_lib.cc#L90) - Backward Gradient function: +* This function specifies the computation of backward pass of the operator. + +MXReturnValue backward( +std::map attrs, +std::vector inputs, +std::vector outputs, +OpResource res) + +* [mutateInputs](./gemm_lib.cc#L214) - Specify mutable input: +* This function allows you to mark some inputs to be mutable inputs, useful when using aux parameters for BatchNorm-like operators. + +MXReturnValue mutateInputs( +std::map attrs, +std::vector _indices) + +Let’s take a closer look at those registry functions: + +* **parseAttrs**: This function takes 3 arguments. 1st argument is an input, which is the attributes passed all the way from Python code. When user calls `mx.nd.my_op_name(s,t,keyword=1)`, the keyword is passed to the attributes as an entry of the map. 2nd
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366110494 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: + +1. run `make gemm_lib`, the Makefile will generate a dynamic library **libgemm_lib.so** compiled from gemm_lib.cc. This is the library you are going to load that contains everything of the custom gemm operator. +2. run `python test_gemm.py`, and it’ll first load the above .so library, find operators, register them in the MXNet backend, print "Found x operators"; then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file **include/mxnet/lib_api.h** from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invoke the operator using both ndarray and symbol API, and print outputs of forward and backward pass. The outputs should be the same as the regular MXNet gemm operator. + +## Writing Custom Operators: + +### Regular Custom Operator: + +There are several basic building blocks for making a (stateless) custom operator: + +* [parseAttrs](./gemm_lib.cc#L118) - Attribute Parser: +* This function specifies number of input and output tensors for the custom operator; also this is where a custom operator can validate the attributes (ie. options) specified by the user. + +MXReturnValue parseAttrs( +std::map attrs, +int* num_in, +int* num_out) + + +* [inferType](./gemm_lib.cc#L124) - Type Inference: +* This function specifies how custom operator infers output data types using input data types. + +MXReturnValue inferType( +std::map attrs, +std::vector , +std::vector ) + +* [inferShape](./gemm_lib.cc#L143) - Shape Inference: +* This function specifies how custom operator infers output tensor shape using input shape. Review comment: ```suggestion * This function specifies how the custom operator infers output tensor shape using input shape. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366108930 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: Review comment: Colons aren't needed in the titles. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366109160 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + Review comment: Introduction? What are we going to accomplish in this example? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc
aaronmarkham commented on a change in pull request #17241: Add CustomOp tutorial doc URL: https://github.com/apache/incubator-mxnet/pull/17241#discussion_r366110345 ## File path: example/extensions/lib_custom_op/README.md ## @@ -0,0 +1,118 @@ +CustomOp Example and Tutorial + + +## Getting Started + +### Have MXNet Ready: + +First you should install MXNet either from compiling from source code or download from nightly build. It doesn’t matter if the build comes with CUDA or MKLDNN. The custom operator doesn’t interact with the execution of other native MXNet operators. + +### Run An Example: + +You can start getting familiar with custom operator by running some examples we provide in the **example/extensions/lib_custom_op** directory. Let’s start with gemm (Generalized Matrix Multiplication) operator, a common linear algebra operator. Go to that directory and follow the steps: + +1. run `make gemm_lib`, the Makefile will generate a dynamic library **libgemm_lib.so** compiled from gemm_lib.cc. This is the library you are going to load that contains everything of the custom gemm operator. +2. run `python test_gemm.py`, and it’ll first load the above .so library, find operators, register them in the MXNet backend, print "Found x operators"; then invoke the operator like a regular MXNet operator and output the result. + +### Basic Files For Gemm Library: + +* **lib_custom_op/gemm_lib.cc**: This file has source code implementation of all required components of a custom operator, as well as the registration of the custom operator. + +* **lib_custom_op/Makefile**: Compile source code to a dynamic shared library, with a header file **include/mxnet/lib_api.h** from MXNet source code. Currently the custom operator is compatible with C++11 onwards. + +* **lib_custom_op/test_gemm.py**: This file calls `mx.library.load(‘libgemm_lib.so’)` to load the library containing the custom operator, invoke the operator using both ndarray and symbol API, and print outputs of forward and backward pass. The outputs should be the same as the regular MXNet gemm operator. + +## Writing Custom Operators: + +### Regular Custom Operator: + +There are several basic building blocks for making a (stateless) custom operator: + +* [parseAttrs](./gemm_lib.cc#L118) - Attribute Parser: Review comment: Should look into the Sphinx plugin that facilitates this, so you don't use a line number that's gonna move. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
