microsoft/onnxruntime-extensions
Publicmirrored from https://github.com/microsoft/onnxruntime-extensionsAvailable
docs/pre_post_processing.md
37lines · modeblame
aa846c31Wenbing Li5 years ago | 1 | # The pre/post processing code to ONNX model |
| 2 | | |
| 3 | Most pre and post processing of the DL models are written in Python code, when the user running the converted ONNX model with Python snippets, it would be very efficient and productive to convert these code snippets into the ONNX model, since the ONNX graph is actually a computation graph, it can represent the most programming code, theoretically. | |
| 4 | | |
3806e294Wenbing Li5 years ago | 5 | In the onnxruntime_extensions package, there is a utility to help on that. This tool is to trace the data flow in the processing code and convert all operation in the tracing logging into the ONNX graph, and merge all these graphs into in one single ONNX model. It supports the Python numeric operators and PyTorch's operation APIs (only a subset of the tensor API) |
aa846c31Wenbing Li5 years ago | 6 | |
| 7 | ###Usage | |
06c90225Wenbing Li5 years ago | 8 | In the onnxruntime_extensions.utils, there is an API ```trace_for_onnx```, when it was fed with the input variables in Python code, it start a tracing session to log all operation starting from these variables. Also if there are some PyTorch API calls in the processing code, you need replace the import statement from ```import torch``` to ```from onnxruntime_extensions.onnxprocess import torch_wrapper as torch```, which will enable these PyTorch API can be traced as well. |
aa846c31Wenbing Li5 years ago | 9 | |
| 10 | Overall, it will look like: | |
| 11 | | |
| 12 | ```python | |
06c90225Wenbing Li5 years ago | 13 | from onnxruntime_extensions.onnxprocess import trace_for_onnx |
| 14 | from onnxruntime_extensions.onnxprocess import torch_wrapper as torch # overload torch API if it is needed | |
aa846c31Wenbing Li5 years ago | 15 | |
| 16 | # the raw input, like text, image, or ... | |
| 17 | input_text = ... | |
| 18 | with trace_for_onnx(input_text, names=['string_input']) as tc_sess: | |
| 19 | # The pre or/and post processing code starts | |
| 20 | ... | |
| 21 | ... | |
| 22 | ... | |
| 23 | output = ... | |
| 24 | # Save all trace objects into an ONNX model | |
| 25 | tc_sess.save_to_onnx('<all_in_one.onnx>', output) | |
| 26 | ``` | |
| 27 | | |
| 28 | Then the all-in-one model can be inference from the raw text directly | |
| 29 | ```python | |
3806e294Wenbing Li5 years ago | 30 | from onnxruntime_extensions.eager_op import EagerOp |
aa846c31Wenbing Li5 years ago | 31 | # the input raw text |
| 32 | input_text = ... | |
| 33 | full_model = EagerOp.from_model('<all_in_one.onnx>') | |
| 34 | output = full_model(input_text) | |
| 35 | print(output) | |
| 36 | ``` | |
| 37 | Or you do inference on this model with any other programming ONNXRuntime API, like C++, C#, Java and etc. |