70966 Commits

Author SHA1 Message Date
Michael Niedermayer
d6ce1cb140 Update for 2.6.6
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
n2.6.6
2015-12-22 17:42:21 +01:00
Andreas Cadhalpun
b2cb7db14f nuv: sanitize negative fps rate
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit f6830cf5ba03fdcfcd81a0358eb32d4081a2fcce)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-20 15:06:35 +01:00
Andreas Cadhalpun
462afe5d00 rawdec: only exempt BIT0 with need_copy from buffer sanity check
Otherwise the too small buffer is directly used in the frame, causing
segmentation faults, when trying to use the frame.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 699e68371ec7e381e5cc48e3d96e29c669261af7)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-20 15:06:31 +01:00
Andreas Cadhalpun
c51800d324 mlvdec: check that index_entries exist
This fixes NULL pointer dereferencing.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9fcfe4a3cdf9a5af0c37758b178965b7b99582d4)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-20 15:06:27 +01:00
Andreas Cadhalpun
4d2f936803 nutdec: reject negative value_len in read_sm_data
If it is negative, it can cause the byte position to move backwards in
avio_skip, which in turn makes sm_size negative and thus size larger
than the size of the packet buffer, causing invalid writes in avio_read.

Also fix potential overflow of avio_tell(bc) + value_len.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ce10f572c12b0d172c72d31d8c979afce602bf0c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-20 15:06:23 +01:00
Andreas Cadhalpun
c4b2985d9a xwddec: prevent overflow of lsize * avctx->height
This is used to check if the input buffer is large enough, so if this
overflows it can cause a false negative leading to a segmentation fault
in bytestream2_get_bufferu.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9d38f06d05efbb9d6196c27668eb943e934943ae)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-20 15:06:19 +01:00
Andreas Cadhalpun
ea5ac146cd nutdec: only copy the header if it exists
Fixes ubsan runtime error: null pointer passed as argument 2, which is
declared to never be null

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9f82506c79874edd7b09707ab63d9e72078de8f9)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-20 15:06:14 +01:00
Andreas Cadhalpun
350f1f4919 exr: fix out of bounds read in get_code
This macro unconditionally used out[-1], which causes an out of bounds
read, if out is the very beginning of the buffer.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 90b99a81071d10e6b5efe86a4602d54d4f45bbcb)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-20 15:06:10 +01:00
Andreas Cadhalpun
564e63a033 on2avc: limit number of bits to 30 in get_egolomb
More don't fit into the integer output.

Also use get_bits_long, since get_bits only supports reading up to 25
bits, while get_bits_long supports the full integer range.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 4d5c3b02e9d2c9a630ca433fabca43285879e0b8)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-20 15:06:05 +01:00
Michael Niedermayer
c1b7e9fe52 avcodec/mpeg4videodec: also for empty partitioned slices
Fixes assertion failure
Fixes: id_acf3e47f864e1ee4c7b86c0653e0ff31e5bde56e.m4v

Found-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 70f13abb4f9a376ddc0d2c566739bc3c6a0c47e7)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:28 +01:00
Michael Niedermayer
d0d9fad2ba avcodec/h264_refs: Fix long_idx check
Fixes out of array read
Fixes mozilla bug 1233606

Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b92b4775a0d07cacfdd2b4be6511f3cb362c977b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:28 +01:00
Michael Niedermayer
3f7bbb0ded avfilter/vf_mpdecimate: Add missing emms_c()
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 997de2e8107cc4256e50611463d609b18fe9619f)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:28 +01:00
Michael Niedermayer
10395b4738 avformat/mxfenc: Do not crash if there is no packet in the first stream
Fixes: Ticket4914

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b51e7554e74cbf007a1cab83c7bed3ad9fa2793a)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:28 +01:00
Rainer Hochecker
bc264004e0 avformat/utils: estimate_timings_from_pts - increase retry counter, fixes invalid duration for ts files with hevc codec
Fixes a mpegts file with hevc that fails estimating duration. Increasing number of
retries fixes the issue.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2d8c2f1a28073d451c7db31291c333cb15ca3d0b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:28 +01:00
Michael Niedermayer
8e053f61f5 avformat/matroskaenc: Check codecdelay before use
Fixes CID1238790

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e6971db12b8ae49712b77378fa8141de4904082b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:28 +01:00
Michael Niedermayer
f821441795 avutil/mathematics: Fix division by 0
Fixes: CID1341571

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bc8b1e694cc395fdf5e2917377ef11263c937d85)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:28 +01:00
Michael Niedermayer
3e8d9eade6 avcodec/hevc: Check max ctb addresses for WPP
Fixes out of array read
Fixes: 2f95ddd996db8a6281d2e18c184595a7/asan_heap-oob_192fe91_3330_58e4441181e30a66c19f743dcb392347.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dad354f38ddc9bfc834bc21358a1d0ad41532ca0)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:28 +01:00
Michael Niedermayer
63c50350b6 avcodec/vp3: ensure header is parsed successfully before tables
Fixes assertion failure
Fixes: 266ee543812e934f7b4a72923a2701d4/signal_sigabrt_7ffff6ae7cc9_7322_85218d61759d461bdf7387180e8000c9.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 26379d4fddc17cac853ef297ff327b58c44edbad)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:28 +01:00
Michael Niedermayer
7294d95689 avcodec/jpeg2000dec: Check bpno in decode_cblk()
Fixes: undefined shift
Fixes: c409ef86f892335a0a164b5871174d5a/asan_heap-oob_1dff564_2159_162b7234616deab02b544410455eb07b.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a85b02dcf70f62a6a433a607143f1f78fa5648bb)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:28 +01:00
Michael Niedermayer
8681296bb9 avcodec/pgssubdec: Fix left shift of 255 by 24 places cannot be represented in type int
Fixes: b293a6479bb4b5286cff24d356bfd955/asan_generic_225c3c9_7819_cc526b657450c6cdef1371b526499626.mkv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4f2419888ba49245761f4ab343679c38e7880cfe)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:28 +01:00
Michael Niedermayer
0fa22a42d6 swscale/utils: Fix for runtime error: left shift of negative value -1
Fixes: c106b36fa36db8ff8f3ed0c82be7bea2/asan_heap-oob_32699f0_6321_467b9a1d7e03d7cfd310b7e65dc53bcc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 325b59368dae3c3f2f5cc39873002b4cf133ccbc)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
929fd61602 avcodec/hevc: Fix integer overflow of entry_point_offset
Fixes out of array read
Fixes: d41d8cd98f00b204e9800998ecf8427e/signal_sigsegv_321165b_7641_077dfcd8cbc80b1c0b470c8554cd6ffb.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 214085852491448631dcecb008b5d172c11b8892)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
09305dca59 avcodec/dirac_parser: Add basic validity checks for next_pu_offset and prev_pu_offset
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c7d6ec947c053699950af90f695413a5640b3872)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
cc88d7a640 avcodec/dirac_parser: Fix potential overflows in pointer checks
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 79798f7c57b098c78e0bbc6becd64b9888b013d1)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
e12f07b475 avcodec/wmaprodec: Check bits per sample to be within the range not causing integer overflows
Fixes: 549d5aab1480d10f2a775ed90b0342f1/signal_sigabrt_7ffff6ae7cc9_5643_96bbb0cfe3e28be1dadfce1075016345.wma

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 66e05f6ff5e5c105bdd7bf3a49234ddac1b592c5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
5164067861 avcodec/wmaprodec: Fix overflow of cutoff
Fixes: 129ca3e28d73af7b1e24a9d4118e7a2d/signal_sigabrt_7ffff6ae7cc9_836_762b310fc3ef6087bd7771e5d8e90b9b.asf

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0c56f8303e676556ea09bfac73d881c6c9057259)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
354fa47018 avformat/smacker: fix integer overflow with pts_inc
Fixes: ce19e41f0ef1e52a23edc488faecdb58/asan_heap-oob_2504e97_4202_ffa0df1baed14022b9bfd4f8ac23d0cb.smk

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7ed47e97297fd5ef473d0cc93f0455adbadaac83)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
ad09bebe0c avcodec/vp3: Fix "runtime error: left shift of negative value"
Fixes: 5c6129154b356b80bcab86f9e3ee5d29/signal_sigabrt_7ffff6ae7cc9_7322_d26ac6d7cb6567db1b8be0159b387d0b.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 18268f761bffb37552f59f87542fef3d5c80618c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Timo Teräs
e8fa0e3ec6 mpegencts: Fix overflow in cbr mode period calculations
ts->mux_rate is int (signed 32-bit) type. The period calculations
will start to overflow when mux_rate > 5mbps. This fixes overflows
by converting first to 64-bit type.

Fixes #5044.

Signed-off-by: Timo Teräs <timo.teras@iki.fi>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 64f7db554ee83846f207e82a08946a6a5a6acfe2)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
2e77ab8f10 avutil/timecode: Fix fps check
The fps variable is explicitly set to -1 in case of some errors, the check must
thus be signed or the code setting it needs to use 0 as error code
the type of the field could be changed as well but its in an installed header

Fixes: integer overflow
Fixes: 9982cc157b1ea90429435640a989122f/asan_generic_3ad004a_3799_22cf198d9cd09928e2d9ad250474fa58.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b46dcd5209a77254345ae098b83a872634c5591b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
00fde782a1 avutil/mathematics: return INT64_MIN (=AV_NOPTS_VALUE) from av_rescale_rnd() for overflows
Fixes integer overflow
Fixes: mozilla bug 1229167

Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f03c2ceec174877e03bb302f5971fbe9ffbe4856)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
0350f373f7 avcodec/apedec: Check length in long_filter_high_3800()
Fixes out of array read
Fixes: 0a7ff0c1d93da9cef28a315ec91b692a/asan_heap-oob_4a52e5_3604_9c56dbb20e308f4faeef7b35f688521a.ape

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cd7524fdd13dc8d0cf22e2cfd8300a245542b13a)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
4d8d2f1991 avcodec/vp3: always set pix_fmt in theora_decode_header()
Fixes assertion failure
Fixes: d0bb0662da342ec65f8f2a081222e6b9/signal_sigabrt_7ffff6ae7cc9_5471_82964f0a9ac2f4d3d59390c15473f6f7.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a814f1d364ba912adf61adef158168c5f7604e93)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
100dbe1bf0 avutil/mathematics: Do not treat INT64_MIN as positive in av_rescale_rnd
The code expects actual positive numbers and gives completely wrong
results if INT64_MIN is treated as positive
Instead clip it into the valid range that is add 1 and treat it as
negative

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 25e37f5ea92d4201976a59ae306ce848d257a7e6)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
d8a66a150a avutil/integer: Fix av_mod_i() with negative dividend
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3a9cb18855d29c96a5d9d2f5ad30448cae3a2ddf)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
93e168705d avformat/dump: Fix integer overflow in av_dump_format()
Fixes part of mozilla bug 1229167

Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8e7f4520226d2d9ad6a58ad6c32d1455a8b244b2)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
a0a2bfbfd8 avcodec/utils: Clear dimensions in ff_get_buffer() on failure
Fixes out of array access
Fixes: 482d8f2fd17c9f532b586458a33f267c/asan_heap-oob_4a52b6_7417_1d08d477736d66cdadd833d146bb8bae.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit abee0a1c60612e8638640a8a3738fffb65e16dbf)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
80a24e88e3 avcodec/utils: Use 64bit for aspect ratio calculation in avcodec_string()
Fixes integer overflow
Fixes: 3a45b2ae02f2cf12b7bd99543cdcdae5/asan_heap-oob_1dff502_8022_899f75e1e81046ebd7b6c2394a1419f4.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4f03bebc79f76df3a3e5bb9e1bc32baabfb7797c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
67b29888f4 avcodec/vp3: Clear context on reinitialization failure
Fixes null pointer dereference
Fixes: 1536b9b096a8f95b742bae9d3d761cc6/signal_sigsegv_294aaed_2039_8d1797aeb823ea43858d0fa45c9eb899.ogv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6105b7219a90438deae71b0dc5a034c71ee30fc0)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
c84ec8d5c9 avcodec/hevc: allocate entries unconditionally
Fixes out of array access
Fixes: 08664a2a7921ef48172f26495c7455be/asan_heap-oob_23036c6_3301_523388ef84285a0270caf67a43247b59.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d85aa76115214183e7e3b7d65e950da61474959a)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
4916aa2383 avcodec/hevc_cabac: Fix multiple integer overflows
Fixes: 04ec80eefa77aecd7a49a442cc02baea/asan_heap-oob_19544fa_3303_1905796cd9d8e15f86d664332caabc00.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d5028f61e44b7607b6a547f218f7d85217490a5b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
40d5f570d8 avcodec/jpeg2000dwt: Check ndeclevels before calling dwt_encode*()
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit feb3f39614b88c113211a98dda1bc2fe5c3c6957)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
20a96b9d8c avcodec/jpeg2000dwt: Check ndeclevels before calling dwt_decode*()
Fixes out of array access
Fixes: 01859c9a9ac6cd60a008274123275574/asan_heap-oob_1dff571_8250_50d3d1611e294c3519fd1fa82198b69b.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 75422280fbcdfbe9dc56bde5525b4d8b280f1bc5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
6157938408 avcodec/hevc: Check entry_point_offsets
Fixes out of array read
Fixes: 007c4a36608ebdf27ee260ad60a81184/asan_heap-oob_32076b4_2243_116b1cb29d91cc4974d6680e3d10bd91.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ef9f7bbfa47317f9d46bf46982a394d2be78503c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
1f6aea2cc4 avcodec/cabac: Check initial cabac decoder state
Fixes integer overflows
Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2340_591e9810c7b09efe501ad84638c9e9f8.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Found-by: xiedingbao (Ticket4727)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8000d484b83aafa752d84fbdbfb352ffe0dc64f8)

Conflicts:

	libavcodec/cabac.h

Conflicts:

	libavcodec/h264_cabac.c
	libavcodec/h264_slice.c
2015-12-20 10:59:27 +01:00
Michael Niedermayer
2da8c53386 avcodec/cabac_functions: Fix "left shift of negative value -31767"
Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2340_591e9810c7b09efe501ad84638c9e9f8.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Found-by: xiedingbao (Ticket4727)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a1f6b05f5228979dab0e149deca7a30d22e98af5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
dd4c2fe74a avcodec/h264_slice: Limit max_contexts when slice_context_count is initialized
Fixes out of array access
Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2049_f2192b6829ab6e0eefcb035329c03c60.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4ea4d2f438c9a7eba37980c9a87be4b34943e4d5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:26 +01:00
Michael Niedermayer
292842a0ed avcodec/vp8: Do not use num_coeff_partitions in thread/buffer setup
The variable is not a constant and can lead to race conditions

Fixes: repro.webm (not reproducable with FFmpeg alone)

Found-by: Dale Curtis <dalecurtis@google.com>
Tested-by: Dale Curtis <dalecurtis@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dabea74d0e82ea80cd344f630497cafcb3ef872c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:26 +01:00
Michael Niedermayer
43bd666fb4 avcodec/mpeg4videodec: Check available data before reading custom matrix
Fixes: out of array read
Fixes: 76c515fc3779d1b838667c61ea13ce92/asan_heap-oob_1fc0d07_8913_794a4629a264ebdb25b58d3a94ed1785.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 891dc8f87536ac2ec695c70d081345224524ad99)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-17 21:52:22 +01:00
Andreas Cadhalpun
a90967013b mjpegdec: consider chroma subsampling in size check
If the chroma components are subsampled, smaller buffers are allocated
for them. In that case the maximal block_offset for the chroma
components is not as large as for the luma component.

This fixes out of bounds writes causing segmentation faults or memory
corruption.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 5adb5d9d894aa495e7bf9557b4c78350cbfc9d32)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-17 21:51:33 +01:00