tensorflow::ops::QuantizeAndDequantizeV2
tensorflow::ops::QuantizeAndDequantizeV2
#include <array_ops.h>
Quantizes then dequantizes a tensor.
Summary
This op simulates the precision loss from the quantized forward pass by:
- Quantizing the tensor to fixed point numbers, which should match the target quantization method when it is used in inference.
- Dequantizing it back to floating point numbers for the following ops, most likely matmul.
There are different ways to quantize. This version does not use the full range of the output type, choosing to elide the lowest possible value for symmetry (e.g., output range is -127 to 127, not -128 to 127 for signed 8 bit quantization), so that 0.0 maps to 0.
To perform this op, we first find the range of values in our tensor. The range we use is always centered on 0, so we find m such that
- m = max(abs(input_min), abs(input_max)) if range_given is true,
- m =