30855 Commits

Author SHA1 Message Date
James Almer
7e297b4bc6 avcodec/wavpackenc: print channel count in av_log call
Fixes a warning with -Wformat-extra-args
(cherry picked from commit 17e7fdf61a04f52c499e2d06eab2cf2d22343aa9)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:54:00 +01:00
Michael Niedermayer
1c99807cb7 avcodec/mpeg4video: Check time_incr
Fixes assertion failure
Fixes out of memory access

Fixes: test_casex.ivf

Found-by: Tyson Smith <twsmith@mozilla.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7c97946d6131b31340954a3f603b6bf92590a9a5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:39:04 +01:00
Michael Niedermayer
4865d66b76 avcodec/wavpackenc: Check the number of channels
They are stored in a byte, thus more than 255 is not possible

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 59c915a403af32c4ff5126625b0cc7e38f4beff9)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:39:04 +01:00
Michael Niedermayer
71b640230e avcodec/wavpackenc: Headers are per channel
Fixes: 1b8b83a53bfa751f01b1daa65a4758db/signal_sigabrt_7ffff6ae7cb7_7488_403f71d1a2565b598d01b6cb110fac8f.aiff
Fixes: assertion failure

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 26757b0279b4b93c6066c2151d4d3dbd2ec266bf)

Conflicts:

	libavcodec/wavpackenc.c

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:39:04 +01:00
Michael Niedermayer
1d02458c06 avcodec/dvdec: Fix "left shift of negative value -254"
Fixes: dvdec_left_shift.avi

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 93ac72a98dff592ffc174cfb36a8975dfbf145ae)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:39:04 +01:00
Michael Niedermayer
e2285e7cc8 avcodec/mjpegdec: Fix negative shift
Fixes: mjpeg_left_shift.avi

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d86d7b2486cd5c31db8e820d8a89554abf19567e)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:39:04 +01:00
Michael Niedermayer
4c6b48a08e avcodec/mss2: Check for repeat overflow
Fixes: mss2_left_shift.wmv

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e273dade78943e22b71d0ddb67cd0d737fc26edf)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:39:04 +01:00
Michael Niedermayer
2111d51619 avcodec/mpegvideo_enc: Clip bits_per_raw_sample within valid range
Fixes out of array read
Fixes: test_case-mdc.264 (b47be15a120979f5a1a945c938cbef33)

Found-by: Tyson Smith <twsmith@mozilla.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 13f266b50cc7554028d22480b7e4383968e64a63)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:39:04 +01:00
Michael Niedermayer
1b3dffc0c7 avcodec/motion_est: Fix mv_penalty table size
Fixes out of array read

Found-by: Tyson Smith <twsmith@mozilla.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5b4da8a38a5ed211df9504c85ce401c30af86b97)

Conflicts:

	libavcodec/motion_est.h
2016-01-15 17:39:04 +01:00
Michael Niedermayer
d472b83abc avcodec/h264_slice: Fix integer overflow in implicit weight computation
Fixes mozilla bug 1230423

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7cc01c25727a96eaaa0c177234b626e47c8ea491)

Conflicts:

	libavcodec/h264_slice.c

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:39:04 +01:00
Michael Niedermayer
1363d0759d avcodec/put_bits: Always check buffer end before writing
This causes a overall slowdown of 0.1 % (tested with mpeg4 single thread encoding of matrixbench at QP=3)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cccb0ffccc3723acc7aab3a859b24743596dd9c0)

Conflicts:

	libavcodec/put_bits.h
2016-01-15 17:39:04 +01:00
Andreas Cadhalpun
f48392197d mjpegdec: extend check for incompatible values of s->rgb and s->ls
This can happen if s->ls changes from 0 to 1, but picture allocation is
skipped due to s->interlaced.

In that case ff_jpegls_decode_picture could be called even though the
s->picture_ptr frame has the wrong pixel format and thus a wrong
linesize, which results in a too small zero buffer being allocated.

This fixes an out-of-bounds read in ls_decode_line.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 7ea2db6eafa0a8a9497aab20be2cfc8742a59072)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:39:04 +01:00
Michael Niedermayer
a5b4b80775 avcodec/h264_refs: Fix long_idx check
Fixes out of array read
Fixes mozilla bug 1233606

Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b92b4775a0d07cacfdd2b4be6511f3cb362c977b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:39:04 +01:00
Michael Niedermayer
8cd0e23be8 avcodec/mpeg4videodec: also for empty partitioned slices
Fixes assertion failure
Fixes: id_acf3e47f864e1ee4c7b86c0653e0ff31e5bde56e.m4v

Found-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 70f13abb4f9a376ddc0d2c566739bc3c6a0c47e7)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-20 15:43:21 +01:00
Andreas Cadhalpun
06edade7ee rawdec: only exempt BIT0 with need_copy from buffer sanity check
Otherwise the too small buffer is directly used in the frame, causing
segmentation faults, when trying to use the frame.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 699e68371ec7e381e5cc48e3d96e29c669261af7)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-20 15:40:21 +01:00
Andreas Cadhalpun
882391f975 xwddec: prevent overflow of lsize * avctx->height
This is used to check if the input buffer is large enough, so if this
overflows it can cause a false negative leading to a segmentation fault
in bytestream2_get_bufferu.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9d38f06d05efbb9d6196c27668eb943e934943ae)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-20 15:40:09 +01:00
Andreas Cadhalpun
79d75f1554 exr: fix out of bounds read in get_code
This macro unconditionally used out[-1], which causes an out of bounds
read, if out is the very beginning of the buffer.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 90b99a81071d10e6b5efe86a4602d54d4f45bbcb)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-20 15:40:01 +01:00
Andreas Cadhalpun
105be66545 on2avc: limit number of bits to 30 in get_egolomb
More don't fit into the integer output.

Also use get_bits_long, since get_bits only supports reading up to 25
bits, while get_bits_long supports the full integer range.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 4d5c3b02e9d2c9a630ca433fabca43285879e0b8)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-20 15:39:57 +01:00
Andreas Cadhalpun
46e7a63b6e sonic: make sure num_taps * channels is not larger than frame_size
If that is the case, the loop setting predictor_state in
sonic_decode_frame causes out of bounds reads of int_samples, which has
only frame_size number of elements.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9637c2531f7eb040ad1c3cb46cb40a63dfc77b80)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-20 15:39:20 +01:00
Andreas Cadhalpun
31d6900161 opus_silk: fix typo causing overflow in silk_stabilize_lsf
Due to this typo max_center can be too large, causing nlsf to be set to
too large values, which in turn can cause nlsf[i - 1] + min_delta[i] to
overflow to a negative value, which is not allowed for nlsf and can
cause an out of bounds read in silk_lsf2lpc.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit f61d44b74aaae1d306d8a0d38b7b3d4292c89ced)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-20 15:39:16 +01:00
Andreas Cadhalpun
26b74a7213 aaccoder: prevent crash of anmr coder
If minq is negative, the range of sf_idx can be larger than
SCALE_MAX_DIFF allows, causing assertion failures later in
encode_scale_factors.

Reviewed-by: Claudio Freire <klaussfreire@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 7a4652dd5da0502ff21c183b5ca7d76b1cfd6c51)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-20 15:38:32 +01:00
Andreas Cadhalpun
d52b5f85f2 mjpegdec: consider chroma subsampling in size check
If the chroma components are subsampled, smaller buffers are allocated
for them. In that case the maximal block_offset for the chroma
components is not as large as for the luma component.

This fixes out of bounds writes causing segmentation faults or memory
corruption.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 5adb5d9d894aa495e7bf9557b4c78350cbfc9d32)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-07 01:34:15 +01:00
Michael Niedermayer
f0ae57aca4 avcodec/hevc: Check max ctb addresses for WPP
Fixes out of array read
Fixes: 2f95ddd996db8a6281d2e18c184595a7/asan_heap-oob_192fe91_3330_58e4441181e30a66c19f743dcb392347.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dad354f38ddc9bfc834bc21358a1d0ad41532ca0)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Conflicts:

	libavcodec/hevc.c

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:16:35 +01:00
Michael Niedermayer
4a91e424da avcodec/vp3: ensure header is parsed successfully before tables
Fixes assertion failure
Fixes: 266ee543812e934f7b4a72923a2701d4/signal_sigabrt_7ffff6ae7cc9_7322_85218d61759d461bdf7387180e8000c9.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 26379d4fddc17cac853ef297ff327b58c44edbad)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:08:26 +01:00
Michael Niedermayer
f217aca5ba avcodec/jpeg2000dec: Check bpno in decode_cblk()
Fixes: undefined shift
Fixes: c409ef86f892335a0a164b5871174d5a/asan_heap-oob_1dff564_2159_162b7234616deab02b544410455eb07b.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a85b02dcf70f62a6a433a607143f1f78fa5648bb)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:08:26 +01:00
Michael Niedermayer
e4f90cbd0d avcodec/pgssubdec: Fix left shift of 255 by 24 places cannot be represented in type int
Fixes: b293a6479bb4b5286cff24d356bfd955/asan_generic_225c3c9_7819_cc526b657450c6cdef1371b526499626.mkv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4f2419888ba49245761f4ab343679c38e7880cfe)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:08:26 +01:00
Michael Niedermayer
5c0be549ef avcodec/hevc: Fix integer overflow of entry_point_offset
Fixes out of array read
Fixes: d41d8cd98f00b204e9800998ecf8427e/signal_sigsegv_321165b_7641_077dfcd8cbc80b1c0b470c8554cd6ffb.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 214085852491448631dcecb008b5d172c11b8892)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:08:26 +01:00
Michael Niedermayer
9ce8308ee3 avcodec/dirac_parser: Check that there is a previous PU before accessing it
Fixes out of array read
Fixes: 99d142c47e6ba3510a74b872a1a2ae72/asan_heap-oob_11b36f4_3811_0f5c69e7609a88a580135678de1df844.dxa

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a08681f1e614152184615e2bcd71c3d63835f810)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:08:26 +01:00
Michael Niedermayer
87dbcf7cd2 avcodec/dirac_parser: Add basic validity checks for next_pu_offset and prev_pu_offset
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c7d6ec947c053699950af90f695413a5640b3872)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:08:26 +01:00
Michael Niedermayer
752a6591b2 avcodec/dirac_parser: Fix potential overflows in pointer checks
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 79798f7c57b098c78e0bbc6becd64b9888b013d1)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:08:26 +01:00
Michael Niedermayer
38c1ab17ea avcodec/wmaprodec: Check bits per sample to be within the range not causing integer overflows
Fixes: 549d5aab1480d10f2a775ed90b0342f1/signal_sigabrt_7ffff6ae7cc9_5643_96bbb0cfe3e28be1dadfce1075016345.wma

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 66e05f6ff5e5c105bdd7bf3a49234ddac1b592c5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:08:26 +01:00
Michael Niedermayer
4b77e0a33e avcodec/wmaprodec: Fix overflow of cutoff
Fixes: 129ca3e28d73af7b1e24a9d4118e7a2d/signal_sigabrt_7ffff6ae7cc9_836_762b310fc3ef6087bd7771e5d8e90b9b.asf

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0c56f8303e676556ea09bfac73d881c6c9057259)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:08:26 +01:00
Michael Niedermayer
f41c1d3bfe avcodec/vp3: Fix "runtime error: left shift of negative value"
Fixes: 5c6129154b356b80bcab86f9e3ee5d29/signal_sigabrt_7ffff6ae7cc9_7322_d26ac6d7cb6567db1b8be0159b387d0b.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 18268f761bffb37552f59f87542fef3d5c80618c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:08:26 +01:00
Michael Niedermayer
eff24a1f13 avcodec/apedec: Check length in long_filter_high_3800()
Fixes out of array read
Fixes: 0a7ff0c1d93da9cef28a315ec91b692a/asan_heap-oob_4a52e5_3604_9c56dbb20e308f4faeef7b35f688521a.ape

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cd7524fdd13dc8d0cf22e2cfd8300a245542b13a)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:08:26 +01:00
Michael Niedermayer
9b28bbb10c avcodec/vp3: always set pix_fmt in theora_decode_header()
Fixes assertion failure
Fixes: d0bb0662da342ec65f8f2a081222e6b9/signal_sigabrt_7ffff6ae7cc9_5471_82964f0a9ac2f4d3d59390c15473f6f7.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a814f1d364ba912adf61adef158168c5f7604e93)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:08:25 +01:00
Michael Niedermayer
73870147f8 avcodec/mpeg4videodec: Check available data before reading custom matrix
Fixes: out of array read
Fixes: 76c515fc3779d1b838667c61ea13ce92/asan_heap-oob_1fc0d07_8913_794a4629a264ebdb25b58d3a94ed1785.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 891dc8f87536ac2ec695c70d081345224524ad99)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:08:25 +01:00
Michael Niedermayer
e5d162ae9b avcodec/utils: Clear dimensions in ff_get_buffer() on failure
Fixes out of array access
Fixes: 482d8f2fd17c9f532b586458a33f267c/asan_heap-oob_4a52b6_7417_1d08d477736d66cdadd833d146bb8bae.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit abee0a1c60612e8638640a8a3738fffb65e16dbf)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:08:25 +01:00
Michael Niedermayer
6e28d0d047 avcodec/utils: Use 64bit for aspect ratio calculation in avcodec_string()
Fixes integer overflow
Fixes: 3a45b2ae02f2cf12b7bd99543cdcdae5/asan_heap-oob_1dff502_8022_899f75e1e81046ebd7b6c2394a1419f4.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4f03bebc79f76df3a3e5bb9e1bc32baabfb7797c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:08:25 +01:00
Michael Niedermayer
2b9e849c37 avcodec/vp3: Clear context on reinitialization failure
Fixes null pointer dereference
Fixes: 1536b9b096a8f95b742bae9d3d761cc6/signal_sigsegv_294aaed_2039_8d1797aeb823ea43858d0fa45c9eb899.ogv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6105b7219a90438deae71b0dc5a034c71ee30fc0)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-05 23:48:52 +01:00
Michael Niedermayer
d8c0546ecd avcodec/hevc: allocate entries unconditionally
Fixes out of array access
Fixes: 08664a2a7921ef48172f26495c7455be/asan_heap-oob_23036c6_3301_523388ef84285a0270caf67a43247b59.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d85aa76115214183e7e3b7d65e950da61474959a)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Conflicts:

	libavcodec/hevc.c
2015-12-05 23:48:52 +01:00
Michael Niedermayer
f9d6d02fd1 avcodec/hevc_cabac: Fix multiple integer overflows
Fixes: 04ec80eefa77aecd7a49a442cc02baea/asan_heap-oob_19544fa_3303_1905796cd9d8e15f86d664332caabc00.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d5028f61e44b7607b6a547f218f7d85217490a5b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-05 23:48:52 +01:00
Michael Niedermayer
6bc2f5cc0b avcodec/jpeg2000dwt: Check ndeclevels before calling dwt_encode*()
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit feb3f39614b88c113211a98dda1bc2fe5c3c6957)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-05 23:48:52 +01:00
Michael Niedermayer
551e18aa92 avcodec/jpeg2000dwt: Check ndeclevels before calling dwt_decode*()
Fixes out of array access
Fixes: 01859c9a9ac6cd60a008274123275574/asan_heap-oob_1dff571_8250_50d3d1611e294c3519fd1fa82198b69b.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 75422280fbcdfbe9dc56bde5525b4d8b280f1bc5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-05 23:48:52 +01:00
Michael Niedermayer
c6b81e252e avcodec/hevc: Check entry_point_offsets
Fixes out of array read
Fixes: 007c4a36608ebdf27ee260ad60a81184/asan_heap-oob_32076b4_2243_116b1cb29d91cc4974d6680e3d10bd91.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ef9f7bbfa47317f9d46bf46982a394d2be78503c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-05 23:48:52 +01:00
Michael Niedermayer
d3de02d9d4 avcodec/cabac: Check initial cabac decoder state
Fixes integer overflows
Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2340_591e9810c7b09efe501ad84638c9e9f8.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Found-by: xiedingbao (Ticket4727)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8000d484b83aafa752d84fbdbfb352ffe0dc64f8)

Conflicts:

	libavcodec/cabac.h

Conflicts:

	libavcodec/h264_cabac.c
	libavcodec/h264_slice.c
2015-12-05 23:48:52 +01:00
Michael Niedermayer
cc80478463 avcodec/cabac_functions: Fix "left shift of negative value -31767"
Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2340_591e9810c7b09efe501ad84638c9e9f8.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Found-by: xiedingbao (Ticket4727)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a1f6b05f5228979dab0e149deca7a30d22e98af5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-05 23:48:52 +01:00
Michael Niedermayer
fa3df7ae41 avcodec/ffv1dec: Clear quant_table_count if its invalid
Fixes deallocation of corrupted pointer
Fixes: 343dfbe142a38b521ed069dc4ea7c03b/signal_sigsegv_421427_4074_ffb11959610278cd40dbc153464aa254.avi
No releases affected

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e04126072e984f8db5db9da9303c89ae01f7d6bb)

Fixes ticket #5052.
2015-12-02 12:02:51 +01:00
Michael Niedermayer
b622d6f6f6 avcodec/ffv1dec: Print an error if the quant table count is invalid
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a8b254e436dce2f5c8c6459108dab4b02cc6b79b)
2015-12-02 12:00:22 +01:00
Michael Niedermayer
9ba0cb7c27 avcodec/h264_slice: Limit max_contexts when slice_context_count is initialized
Fixes out of array access
Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2049_f2192b6829ab6e0eefcb035329c03c60.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4ea4d2f438c9a7eba37980c9a87be4b34943e4d5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-26 16:58:33 +01:00
Michael Niedermayer
608f928c74 avcodec/vp8: Do not use num_coeff_partitions in thread/buffer setup
The variable is not a constant and can lead to race conditions

Fixes: repro.webm (not reproducable with FFmpeg alone)

Found-by: Dale Curtis <dalecurtis@google.com>
Tested-by: Dale Curtis <dalecurtis@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dabea74d0e82ea80cd344f630497cafcb3ef872c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-26 16:58:33 +01:00