From a11c16a0b0cadf3a14fa5e7329c2a144a2165bc6 Mon Sep 17 00:00:00 2001 From: Neil Birkbeck Date: Fri, 20 Sep 2013 17:25:40 -0700 Subject: [PATCH] avfilter/vf_psnr: Prevent integer overflow. The 32-bit integer accumulator in MSE computation can overflow for 8-bit frame data. (e.g., for 1080p white frame compared to a black frame can give sum of 255*255*1080*1920 > 2^32). Signed-off-by: Neil Birkbeck Signed-off-by: Michael Niedermayer --- libavfilter/vf_psnr.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavfilter/vf_psnr.c b/libavfilter/vf_psnr.c index 3744579d71..1bcdd7de5c 100644 --- a/libavfilter/vf_psnr.c +++ b/libavfilter/vf_psnr.c @@ -91,7 +91,7 @@ void compute_images_mse(PSNRContext *s, const uint8_t *ref_line = ref_data[c]; const int ref_linesize = ref_linesizes[c]; const int main_linesize = main_linesizes[c]; - int m = 0; + uint64_t m = 0; for (i = 0; i < outh; i++) { for (j = 0; j < outw; j++)