microsoft/onnxruntime-extensions
Publicmirrored fromhttps://github.com/microsoft/onnxruntime-extensionsAvailable
docs/pre_post_processing.md
37lines · modecode
| 1 | # The pre/post processing code to ONNX model |
| 2 | |
| 3 | Most pre and post processing of the DL models are written in Python code, when the user running the converted ONNX model with Python snippets, it would be very efficient and productive to convert these code snippets into the ONNX model, since the ONNX graph is actually a computation graph, it can represent the most programming code, theoretically. |
| 4 | |
| 5 | In the onnxruntime_extensions package, there is a utility to help on that. This tool is to trace the data flow in the processing code and convert all operation in the tracing logging into the ONNX graph, and merge all these graphs into in one single ONNX model. It supports the Python numeric operators and PyTorch's operation APIs (only a subset of the tensor API) |
| 6 | |
| 7 | ###Usage |
| 8 | In the onnxruntime_extensions.utils, there is an API ```trace_for_onnx```, when it was fed with the input variables in Python code, it start a tracing session to log all operation starting from these variables. Also if there are some PyTorch API calls in the processing code, you need replace the import statement from ```import torch``` to ```from onnxruntime_extensions.onnxprocess import torch_wrapper as torch```, which will enable these PyTorch API can be traced as well. |
| 9 | |
| 10 | Overall, it will look like: |
| 11 | |
| 12 | ```python |
| 13 | from onnxruntime_extensions.onnxprocess import trace_for_onnx |
| 14 | from onnxruntime_extensions.onnxprocess import torch_wrapper as torch # overload torch API if it is needed |
| 15 | |
| 16 | # the raw input, like text, image, or ... |
| 17 | input_text = ... |
| 18 | with trace_for_onnx(input_text, names=['string_input']) as tc_sess: |
| 19 | # The pre or/and post processing code starts |
| 20 | ... |
| 21 | ... |
| 22 | ... |
| 23 | output = ... |
| 24 | # Save all trace objects into an ONNX model |
| 25 | tc_sess.save_to_onnx('<all_in_one.onnx>', output) |
| 26 | ``` |
| 27 | |
| 28 | Then the all-in-one model can be inference from the raw text directly |
| 29 | ```python |
| 30 | from onnxruntime_extensions.eager_op import EagerOp |
| 31 | # the input raw text |
| 32 | input_text = ... |
| 33 | full_model = EagerOp.from_model('<all_in_one.onnx>') |
| 34 | output = full_model(input_text) |
| 35 | print(output) |
| 36 | ``` |
| 37 | Or you do inference on this model with any other programming ONNXRuntime API, like C++, C#, Java and etc. |
| 38 | |