In conventional encoder control algorithms, the distortion between an original picture s and the reconstructed picture s′ is measured by the sum of squared Errors DSSE(s,s′) or its normalized logarithm, the PSNR. However, it is well known that the PSNR in general does not correlate well with subjective judgment of image quality. We developed an encoder control algorithm that is based on a weighted distortion measure where the weights are spatio-temporally varying and reflect the local visual error sensitivity of the underlying content. For the definition of the weighted distortion measure DWSSE, we partition a video frame into blocks Bk and define
DWSSE(s,s′)=∑kwk(s)⋅DSSE,k(s,s′).
Here, DSSE,k is the sum of squared errors on Bk and wk(s) are local weights. The less activity, measured by a high-pass filter in the spatial or temporal direction, is present on Bk, the larger the weights wk(s) become. This reflects the well known fact that, due to reduced perceptual masking capabilities, low-frequency content is subjectively more sensitive to reconstruction errors than high-frequency content.
The well-established block-based encoder control for the unweighted sum of squared errors can still be used for our perceptual error measure. The only difference is that the Lagrangian parameter is spatio-temporally varying depending on the weights wk. More precisely, if λ is a fixed Lagrangian parameter that defines an operational point on the rate-distortion curve for the error measure DWSSE and if Rk denotes the rate on the block Bk, then under some simplified assumptions, on each block Bk, the encoder needs to minimize the well-known term
DSSE,k(s,s′)+λk⋅Rk,λk=λωk.
The effect of our perceptually optimized encoder control is illustrated below. In the middle image, which arises using a conventional encoder control, highly visible transmission errors in the water on the top right part can be observed. With our perceptutally optimized encoder control, such errors are no longer visible, as can be seen from the bottom picture.

References
- C. R. Helmrich, S. Bosse, M. Siekmann, H. Schwarz, D. Marpe, and T. Wiegand, “Perceptually Optimized Bit Allocation and Associated Distortion Measure for Block-Based Image or Video Coding,” in Proc. IEEE Data Compression Conf. (DCC), Snowbird, Mar. 2019.
- J. Erfurt, C. R. Helmrich, S. Bosse, H. Schwarz, D. Marpe, and T. Wiegand, “A Study of the Perceptually Weighted Peak Signal-to-Noise Ratio (WPSNR) for Image Compression,” in Proc. IEEE Int. Conf. Image Process. (ICIP), Taipei, Sep. 2019.