microsoft/onnxruntime-extensions
Publicmirrored fromhttps://github.com/microsoft/onnxruntime-extensionsAvailable
docs/new_operator.md
47lines · modecode
| 1 | # Add a Custom Operator in ONNXRuntime-Extensions |
| 2 | |
| 3 | Before implement a custom operator, you get the ONNX model with one or more ORT custom operators, created by ONNX converters, [ONNX-Script](https://github.com/microsoft/onnx-script), or [ONNX model API](https://onnx.ai/onnx/api/helper.html) and etc.. |
| 4 | |
| 5 | |
| 6 | ## 1. Quick verification with PythonOp (optional) |
| 7 | |
| 8 | Before you actually develop a custom operator for the work, if you want to quickly verify the ONNX model with Python, you can wrap the custom operator with **[PyOp](docs/pyop.md)**. |
| 9 | |
| 10 | ```python |
| 11 | import numpy |
| 12 | from onnxruntime_extensions import PyOp, onnx_op |
| 13 | |
| 14 | # Implement the CustomOp by decorating a function with onnx_op |
| 15 | @onnx_op(op_type="Inverse", inputs=[PyOp.dt_float]) |
| 16 | def inverse(x): |
| 17 | # the user custom op implementation here: |
| 18 | return numpy.linalg.inv(x) |
| 19 | |
| 20 | # Run the model with this custom op |
| 21 | # model_func = PyOrtFunction(model_path) |
| 22 | # outputs = model_func(inputs) |
| 23 | # ... |
| 24 | ``` |
| 25 | |
| 26 | ## 2. Generate the C++ template code of the Custom operator from the ONNX Model (optional) |
| 27 | python -m onnxruntime-extensions.cmd --cpp-gen <model_path> <repository_dir>` |
| 28 | If you are familiar with the ONNX model detail, you create the custom operator C++ classes directly. |
| 29 | |
| 30 | |
| 31 | ## 3. Implement the CustomOp Kernel Compute method in the generated C++ files. |
| 32 | the custom operator kernel C++ code example can be found [operators](../operators/) folder, like [gaussian_blur](../operators/cv2/imgproc/gaussian_blur.hpp). All C++ APIs that can be used in the kernel implementation are listed below |
| 33 | |
| 34 | * [ONNXRuntime Custom API docs](https://onnxruntime.ai/docs/api/c/struct_ort_custom_op.html) |
| 35 | * the third libraries API docs integrated in ONNXRuntime Extensions the can be used in C++ code |
| 36 | - OpenCV API docs https://docs.opencv.org/4.x/ |
| 37 | - Google SentencePiece Library docs https://github.com/google/sentencepiece/blob/master/doc/api.md |
| 38 | - dlib(matrix and ML library) C++ API docs http://dlib.net/algorithms.html |
| 39 | - BlingFire Library https://github.com/microsoft/BlingFire |
| 40 | - Google RE2 Library https://github.com/google/re2/wiki/CplusplusAPI |
| 41 | - JSON library https://json.nlohmann.me/api/basic_json/ |
| 42 | |
| 43 | ## 3. Build and Test |
| 44 | - The unit tests can be implemented as Python or C++, check [test](../test) folder for more examples |
| 45 | - Check [build-package](./development.md) on how to build the different language package to be used for production. |
| 46 | |
| 47 | Please check the [contribution](../README.md#contributing) to see if it is possible to contribute the custom operator to onnxruntime-extensions. |
| 48 | |