40931 Commits

Author SHA1 Message Date
Timo Rothenpieler
d1275f8a77 avcodec/nvdec: reset bitstream_len/nb_slices when resetting bitstream pointer 2024-03-30 00:32:56 +01:00
Hendrik Leppkes
920cdf960f
avcodec/vdpau_mpeg4: fix order of quant matrix coefficients
The matrix coefficients are stored permutated for the IDCT,
rather then in plain raster order, and need to be un-permutated
for the hardware.
2023-05-15 18:51:16 +02:00
Hendrik Leppkes
4c9b34fce8
avcodec/vdpau_mpeg12: fix order of quant matrix coefficients
The matrix coefficients are stored permutated for the IDCT,
rather then in plain raster order, and need to be un-permutated
for the hardware.
2023-05-15 18:51:16 +02:00
Hendrik Leppkes
23023147e2
avcodec/nvdec_mpeg4: fix order of quant matrix coefficients
The matrix coefficients are stored permutated for the IDCT,
rather then in plain raster order, and need to be un-permutated
for the hardware.
2023-05-15 18:51:16 +02:00
Hendrik Leppkes
8696df1449
avcodec/nvdec_mpeg2: fix order of quant matrix coefficients
The matrix coefficients are stored permutated for the IDCT,
rather then in plain raster order, and need to be un-permutated
for the hardware.
2023-05-15 18:51:16 +02:00
Timo Rothenpieler
0c7dc61052 avcodec/cuviddec: backport extradata fixes 2020-10-01 22:18:05 +02:00
Timo Rothenpieler
968836e233 avcodec/cuviddec: handle arbitrarily sized extradata 2020-09-30 14:18:05 +02:00
Michael Niedermayer
2b7c0ae67c avcodec/dstdec: Replace AC overread check by sample rate check
Real files do skip coding 0 bits at the end, thus this kind of check
does not work reliable.

Fixes: Ticket 8770
Fixes: dst-256fs44-6ch-refdstencoder.dff

The samplerate is specified in ISO/IEC 14496-3:2005(E) as one of 3 fixed
values, this also can be used to limit the duration and avoid the timeout

This reverts commit f6df99dba1ae64b05d08fba8160d13eb9795042f.

(cherry picked from commit 1679f23beb3cfc3639352b3cbe7c08c00189c6b0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 11:51:17 +02:00
Andreas Rheinhardt
6536090792 avcodec/hevc_mp4toannexb_bsf: Check NAL size against available input
The hevc_mp4toannexb bsf does not explicitly check whether a NAL unit
is so big that it extends beyond the end of the input packet; it does so
only implicitly by using the checked version of the bytestream2 API.
But this has downsides compared to real checks: It can lead to huge
allocations (up to 2GiB) even when the input packet is just a few bytes.
And furthermore it leads to uninitialized data being output.
So add a check to error out early if it happens.

Also check directly whether there is enough data for the length field.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit ea1b71e82f5a1752d59d3bfb9704092a79eba6b5)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-02 22:46:35 +02:00
Andreas Rheinhardt
1fde9bf0a0 avcodec/ttaenc: Fix undefined shift
ttaenc contained (1 << unary) - 1 as an argument for a function
expecting an unsigned int. unary can be as big as 31 in this case.
The type of the shift and the whole expression is int, because 1 fits
into an integer, so that the behaviour is undefined if unary == 31
as the result of the shift can't be represented in an int §. Subtraction
by 1 (which makes the result of the whole expression representable in
an int) doesn't change that this is undefined (it usually leads to
signed integer overflow which is undefined, too).

The solution is simple: Make 1 unsigned to change the type of the
whole expression to unsigned int (as the function expects anyway).

Fixes ticket .

§: This of course presupposes the common int range of -2^31..2^31-1

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3ab488a5407f833ecc66e8fa4c537dc4852db720)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-02 10:48:28 +02:00
Andreas Rheinhardt
ba3ff1381e avcodec/jpeg2000dwt: Fix undefined shifts of negative numbers
Affected the vsynth*-jpeg2000 and the vsynth*-jpeg2000-97 FATE tests
(where * ranges over { 1, 2, 3, _lena }) as well as ticket .

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5cf593adcd79a7c9502dc2725e1f0681ada36aef)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-02 10:43:53 +02:00
Andreas Rheinhardt
d4a7500feb avcodec/ituh263dec: Fix undefined left shift of negative number
Fixes ticket .

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 324487b596fbcda0a5753c7bb7b2e96e9d512479)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-02 10:43:53 +02:00
Andreas Rheinhardt
d5f9f2ded0 avcodec/dnxhdenc: Fix undefined left shifts of negative numbers
Affected 61 FATE-tests: 60 vsynth tests and lavf-mxf_opatom.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b7f156e8cbdf3256c7860c62ebb7a6c3002cbb03)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-02 10:43:52 +02:00
Andreas Rheinhardt
44c6865e54 avcodec/exr: Fix undefined left shifts of negative numbers
Affected the FATE-tests exr-rgb-scanline-pxr24-half-uint32-13x9 and
exr-rgb-scanline-pxr24-uint32.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8b0f949906116c40b6f1e55a1bce4447ada3219c)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-02 10:43:52 +02:00
Andreas Rheinhardt
b5ef6bd7f0 avcodec/pcm: Fix undefined shifts
Fixes the acodec-pcm-u16[lb]e FATE-tests.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 69473bec6f38fefc9a433d95f8e00de101299592)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-02 10:43:52 +02:00
Andreas Rheinhardt
42ae8d5100 avcodec/wavpackenc: Fix undefined shifts
Fixes ticket  and the acodec-wavpack FATE-test.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 84974c6fb542cf019904016c2165d9a62db9f312)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-02 10:43:52 +02:00
Andreas Rheinhardt
ec99e0a54c avcodec/ac3enc: Fix invalid shift
Fixes the FATE-tests unknown_layout-ac3, ac3-fixed-encode, ac3-encode
and eac3-encode. It furthermore fixes the ac3-encoder bugs mentioned in
tickets ,  and .

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2f289ec914cc7e8133858d4f9e8d91dab685ae44)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-02 10:42:09 +02:00
Andreas Rheinhardt
0b4fe2155e avcodec/tdsc: Fix undefined shifts
Fixes the tdsc FATE-test.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 86bef10e7aee338a4df60c57904c16e33509e76e)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-02 10:42:09 +02:00
Andreas Rheinhardt
3858c755d9 avcodec/cinepakenc: Fix invalid shifts
Fixes: left shift of 1 by 31 places cannot be represented in type 'int'.
Affected the FATE-tests vsynth1-cinepak, vsynth2-cinepak and
vsynth_lena-cinepak. Also fixes ticket .

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e3650dcfc9cde6ceccf7bbc225962da196e2a386)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-02 10:15:07 +02:00
Andreas Rheinhardt
41fba5d5a0 avcodec/cbs_h2645: Fix potential out-of-bounds array access
The maximum allowed index for an array access is FF_ARRAY_ELEMS - 1; yet
the current code allowed FF_ARRAY_ELEMS. This wasn't dangerous in practice,
as parameter sets with invalid ids were already filtered out during
reading.

Found via PVS-Studio (see ticket ).

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit f3333c3c67e8825a4468120bb8aa0943c72c03f3)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-02 10:15:07 +02:00
Andreas Rheinhardt
2bcf99b8fa h264_redundant_pps: Fix memleak in case of errors
Now the fragment is uninitialized and the input packet freed in case of
errors.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@googlemail.com>
Signed-off-by: Mark Thompson <sw@jkqxz.net>
(cherry picked from commit 40b74abfca39bf514333c3ebb6d6e946975057c3)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-02 10:15:07 +02:00
Andreas Rheinhardt
71215c5c8a avcodec/cavsdsp: Fix undefined left shifts of negative numbers
Affected the ffmpeg-filter_colorkey FATE-test (but only if the C version
of idct8_add is used and not e.g. the x86 SSE2 version).

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Fixes: left shift of negative value -107
Fixes: 20398/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CAVS_fuzzer-5725389278412800

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0f0f2ab0c3b3d04e904db97b07ae829c72c91778)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-02 10:15:06 +02:00
Andreas Rheinhardt
ff8a49d4b0 avcodec/ra144enc: Fix invalid left shift of negative number
by replacing it with a multiplication. Said multiplication can't
overflow an int32_t because lpc_coefs is limited to 16 bit precision.

Fixes the FACE-test acodec-ra144 as well as part of .

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e3fb9af6f1353f30855eaa1cbd5befaf06e303b8)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-02 10:05:29 +02:00
Andreas Rheinhardt
ef0e524660 avcodec/adxenc: Avoid undefined left shift of negative numbers
Replace "((a << shift) + b) >> shift" by "a + (b >> shift)". This avoids
a left shift which also happens to trigger undefined behaviour in case "a"
is negative. This affected the FATE-tests acodec-adpcm-adx and
acodec-adpcm-adx-trellis; it also fixes ticket .

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 59a9d65e0d790821f88527a82569f56eb2f8a9be)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-02 10:05:29 +02:00
Andreas Rheinhardt
06bc2002c4 avcodec/adpcm: Fix undefined left shifts of negative numbers
Affected the adpcm-afc, adpcm-ea-1, adpcm-ea-2, adpcm-ea-maxis-xa,
adpcm-thp and ea-cdata FATE-tests. Also fixes ticket .

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3ad8af51b7c0a968ac3fd62964780d4ff9136c5a)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-02 10:05:29 +02:00
Andreas Rheinhardt
1e37b66992 avcodec/proresenc_anatoliy: Fix invalid left shift of negative number
This fixes ticket  as well as the vsynth*-prores_# FATE-tests
(where * ranges over { 1, 2, 3, _lena } and # over { , _int, _444,
_444_int }).

(Given that prev_dc is in the range -0xC000..0x3FFF, no overflow can
happen upon multiplication with 2.)

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 198081efb7c7343349f0a7acc836f001c511e990)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-02 10:04:24 +02:00
Andreas Rheinhardt
ce4b305d90 avcodec/bitstream: Don't check for undefined behaviour after it happened
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 5e196dac22cc510db104922f99626a03b453ef4a)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-01 16:58:07 +02:00
Michael Niedermayer
98980ce38f avcodec/pngdec: Check for fctl after idat
Fixes: out of array access
Fixes: 23554/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APNG_fuzzer-4796622520451072.fuzz

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 65b1ba680fb67902a9c876a49d0146eaae5a1c3d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00
Michael Niedermayer
ba0c96ecd3 avcodec/wmalosslessdec: fix overflow with pred in revert_cdlms
Fixes: signed integer overflow: 2048 + 2147483646 cannot be represented in type 'int'
Fixes: 23538/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMALOSSLESS_fuzzer-5227567073460224

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 21598d711d894081d0566282473044ba4f378f33)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00
Michael Niedermayer
26b097f06c avcodec/ffwavesynth: Avoid undefined operation on ts overflow
Alternatively these conditions could be treated as errors
Fixes: 23147/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FFWAVESYNTH_fuzzer-5639254549200896
Fixes: signed integer overflow: 9223372036854775807 + 1 cannot be represented in type 'int64_t' (aka 'long')

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 584d334afd59714ed04637a9227a4f1368c26166)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00
Michael Niedermayer
5ea288cf5a avcodec/mpeg4videodec: Fix 2 integer overflows in get_amv()
Fixes: signed integer overflow: -144876608 * 16 cannot be represented in type 'int'
Fixes: 22782/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-6039584977977344

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e361785ee05cc75d3caacf2f254160b0336f5358)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00
Michael Niedermayer
2e29f3023c avcodec/lossless_audiodsp: Fix undefined overflows in scalarproduct_and_madd_int16_c()
Fixes: signed integer overflow: 2142077091 + 6881070 cannot be represented in type 'int'
Fixes: 22737/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMALOSSLESS_fuzzer-5958388889681920

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c0dfe134beefde4070d43910518b1f4a58f01794)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00
Michael Niedermayer
b8a1a3cc92 avcodec/sonic: Fix several integer overflows
Fixes: signed integer overflow: 2129689466 + 2129689466 cannot be represented in type 'int'
Fixes: 20715/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SONIC_fuzzer-5155263109922816

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 75d520e33704447f1b29ac47fd9e40994a6bc659)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00
Michael Niedermayer
eee5414f21 avcodec/mpeg4videodec: avoid invalid values and reinitialize in format changes for studio profile
Fixes: out of array access
Fixes: 23327/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-5134822992510976

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e53235f06c229a23d3241b47e32647019161fb7c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00
Michael Niedermayer
0b6c2d3dbd avcodec/pixlet: Fix log(0) check
Fixes: passing zero to clz(), which is not a valid argument
Fixes: 23337/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_PIXLET_fuzzer-5179131989065728

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bd0f81526d3f4c23ecd0a399829103be2445c011)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00
Michael Niedermayer
807bcc9a1e avcodec/iff: Fix off by x error
Fixes: out of array access
Fixes: 23245/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5723121327013888.fuzz

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 51225dee0a6266780d26d43bd6802bbcf736327e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00
Michael Niedermayer
c25bfbe37d avcodec/wmalosslessdec: Check block_align maximum
Fixes: Assertion failure
Fixes: 22737/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMALOSSLESS_fuzzer-5958388889681920

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 314d10f7a60f1786c85da30a569be61e2b906fef)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00
Michael Niedermayer
9942728b61 avcodec/loco: Fix signed integer overflow in loco_get_rice()
Fixes: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
Fixes: 22975/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LOCO_fuzzer-5658160970072064

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit aa88cdfd90f5da0683cd6556c75a5ba5740a1c27)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00
Michael Niedermayer
1f110be565 avcodec/mpeg12dec: remove outdated comments
Found-by: Kieran
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 48de8f5816aa54dc584aeb2dbbf63a0e880279e2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00
Michael Niedermayer
c8927529d0 avcodec/snowdec: Avoid integer overflow with huge qlog
Fixes: integer overflow
Fixes: 22285/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SNOW_fuzzer-5682428762128384

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 38fbf33c7255b503453052c32ab5ae4fb151b29e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00
Michael Niedermayer
7d1f349590 avcodec/mpeg12dec: Fix got_output
This makes got_output consistent with the code in slice_end() which sets the output

in slice_end()
 if (s->pict_type == AV_PICTURE_TYPE_B || s->low_delay) {
     int ret = av_frame_ref(pict, s->current_picture_ptr->f);
     ...
 } else {

Fixes: assertion failure
Fixes: 22178/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG1VIDEO_fuzzer-5664234440753152

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4f33a9803a3068ce2d52289fc1db60375dc8b7a5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00
Michael Niedermayer
780b721022 avcodec/lzf: Consider the needed size in reallocation
Fixes: NULL pointer dereference
Fixes: 22381/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_NOTCHLC_fuzzer-5659879921680384.fuzz

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 292b9b93a50aa0622e33013de9f2ddc130bef671)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00
Michael Niedermayer
79ca86c1ec avcodec/adpcm: XA: Check shift similar to filter
Fixes: negative shift
Fixes: 22499/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ADPCM_XA_fuzzer-5765452130418688

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6d96bae9c480e020e9f51fabd5642d7ae6020943)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00
Michael Niedermayer
f3dca389bc avcodec/huffyuvdec: Test vertical coordinate more often
Fixes: out of array access
Fixes: 22892/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HYMT_fuzzer-5135996772679680.fuzz

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a1223ddc5692772198a02600ecff2545f32b37be)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00
Michael Niedermayer
14fe71f5dc avcodec/hq_hqa: Check info size
Fixes: assertion failure
Fixes: 21079/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HQ_HQA_fuzzer-5737046523248640

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cf28521fee22dbe2f7eeb8ab0306c0fd0802c48a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00
Michael Niedermayer
9f1cb0b890 avcodec/wmalosslessdec: Fix integer overflow in mclms_predict()
Fixes: signed integer overflow: 2147483636 + 2048 cannot be represented in type 'int'
Fixes: 22016/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMALOSSLESS_fuzzer-5109395618004992

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c42ed06695848617350a94543823e850f190b3ab)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00
Michael Niedermayer
63d5023257 avcodec/vp9dsp_template: Fix integer overflow(s) in iadst16_1d()
Fixes: signed integer overflow: 1080285923 - -1130879337 cannot be represented in type 'int'
Fixes: 22002/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VP9_fuzzer-6260237310099456

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 071e2937236945c168ab99d3e3b01539194466dd)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00
Michael Niedermayer
50c508ef65 avcodec/h264dec: Disable forced small_padding on flag2 fast
Fixes: 20978/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_H264_fuzzer-5746381832847360

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00
Michael Niedermayer
65cfae33e2 avcodec/pnmdec: Use unsigned for maxval rescaling
Fixes: signed integer overflow: 65535 * 55335 cannot be represented in type 'int'
Fixes: 21955/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_PGMYUV_fuzzer-5669206981083136

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 49459aca47d4803b2188fbf12b758bd2b01e91d7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00
Michael Niedermayer
0c903ef05a avcodec/ivi: Clear got_p_frame before decoding a new frame using it
Fixes: assertion failure
Fixes: 21666/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_INDEO4_fuzzer-5706468994318336

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1d633e6a0a61118c9b2d1785d96bdebaa8c38592)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-01 13:33:45 +02:00