29327 Commits

Author SHA1 Message Date
Gwenole Beauchesne
559261ce49 vaapi: Fix VC-1 decoding (reconstruct bitstream TTFRM correctly).
Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit 53efb758c045900f512c947074900c0dbc988685)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:36 +01:00
Mans Rullgard
f9d17e6f54 4xm: fix signed overflow
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 84dda407628e298f33d610e9e04a8b2945d24665)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:36 +01:00
Mans Rullgard
0b1ac7bf4f wmavoice: fix a signed overflow
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit ba3f07d0611d9a6c10eaa90b3c058ecdffe76676)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:36 +01:00
Mans Rullgard
af0a56e6ef mpegvideo_enc: fix a signed overflow
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 05795f35be4b479bfa8d60ed3eb13e0f89e439c0)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:36 +01:00
Mans Rullgard
5e3ba60e6f crc: fix signed overflow
This fixes a signed overflow from i << 24 when i == 255 by
making i unsigned.  The result of the shift is already
assigned to an variable of unsigned type.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 8b19ae07616bbd18969b94cbf5d74308a8f2bbdf)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
48f9a80072 mpeg12enc: use sign_extend() function
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 2f329db90e5d72ad383a0ba05fde3641a34ef73b)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
2c99aa48d7 lavf: fix signed overflow in avformat_find_stream_info()
On the first iteration through this code, last_dts is always
INT64_MIN (AV_NOPTS_VALUE) and the subtraction overflows in
an invalid manner.  Although the result is only used if the
input values are valid, performing the subtraction is still
not allowed in a strict environment.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit a31e9f68a426f634e002282885c6c2eb1bfbea44)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
fdc669fcbb vp8: fix signed overflows
In addition to avoiding undefined behaviour, an unsigned type
makes more sense for packing multiple 8-bit values.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit bb59156606e00057a706ed30165bc7329db3823f)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
fe3314a413 motion_est: fix some signed overflows
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit e708afd3c026a9eb547dab07781320a7e2564312)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
58afe6061a dca: fix signed overflow in shift
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 559c244d42be7a02c23976216b47fd63b80d6c7f)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
8c2ae575ad aacdec: fix undefined shifts
Since nnz can be zero, this is needed to avoid a shift by 32.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit d12294304acd82cb219e3f66ca9cd6efb2194fa4)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Laurent Aimar
9c78fe9360 bink: Check for various out of bound writes
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit a00676e48e49a3d794d6d2063ceca539e945a4a4)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Laurent Aimar
c98d7882d8 bink: Check for out of bound writes when building tree
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 24adf7832b8370f3c1febbef6c686f574d360d32)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
e52e85ac3a put_bits: fix invalid shift by 32 in flush_put_bits()
If flush_put_bits() is called when the 32-bit buffer is empty,
e.g. after writing a multiple of 32 bits, and invalid shift by
32 is performed.  Since flush_put_bits() is called infrequently,
this additional check should have negligible performance impact.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit ac6eab1496aad6f8b09deabbef4fe5fd829e142d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Alex Converse
4faa00b256 mpegps: Use av_get_packet() instead of poorly emulating it.
(cherry picked from commit 98ef887a759c66febcb612407c6bb361c4d50bcb)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Janne Grunau
90d7146511 motionpixels: decode only the 111 complete frames for fate
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit c2f2dfb3dd20e036b8b08c0fd1486a3044e8f02a)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
59050c0629 mpc8: Check out of bound bands limit
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 9bd854b1ff342f82efa6d2ad4e8fefddce5fa731)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
be2404b06d xan: Prevent NULL dereference with missing palette
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 7d17a794f0348ba40d5cda7d969564cb83981001)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
49007b494e xan: Check for out of bound reads in xan_huffman_decode()
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 3db3fdf4c669aed9379be430c17f151d4d0697c5)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
0277c82de2 xan: Fixed out of bound accesses in xan_unpack()
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 3e0757c2a87c8cf3e452f67bca279001c64cedff)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
5fa8e43b54 motionpixels: Prevent calling init_vlc() with invalid parameters
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 1cd0a5516396bd6fb54e4df1e7c88ed18416299b)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
737bea21b6 shorten: Fix out of bound writes in fix_bitshift()
The data pointers s->decoded[*] already take into account s->nwrap.

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 5f05cf4ea9aaafed8edcabe785c2719786103ec1)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
aa9e308580 dsicinav: Check for out of bounds writes
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 172060328771d149b076f00352b004b5b5272d38)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
d57d039e04 tiertexseqv: Check for out of bound reads
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 64263dd526ec25ede1591fc1144715a20cc7bc4e)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
97a1ab4bce quickdraw: Check for out of bound reads
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 4fd56f842cbaecf74df94c38f9c10452342f436a)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
914b9b0b2b dsicinav: Check for out of bounds reads
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit e3ca9b93d9f464861638dda3280fcf65e402466a)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
39de0e008d motionpixels: Fix the size of workspace buffers
Some buffers must be mod 4 in width and/or height.

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 210c80331e0604edf9c800865c26ba06ed3c2082)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
f2f2a00d39 motionpixels: Clear FF_INPUT_BUFFER_PADDING_SIZE bytes at the end of the temporary buffer
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit d337dd3a907110b32c6305bb65e4beca5b830c5d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
905d0633a6 wmavoice: Check for corrupted extra data
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit d99427cb8ba099375d8cce6df808d4acf045ab43)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
95605595b5 wmavoice: Check for out of bound writes
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 1c1449b548a2a0bf0295a522051b04107286653c)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
fb20141563 xan: Prevent NULL dereferences with missing reference frame
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 06be075cda0a6ba8bab8f543571b380884f562ac)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
c5766b55c4 bink: Prevent NULL dereferences with missing reference frame
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit c7e631986b4a326a71a20a1a51000f3fbf6e64e7)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:30 +01:00
Laurent Aimar
d646cce15f wavpack: Reset internal state on corrupted blocks
wavpack_decode_block() supposes that it is called back with the exact
same buffer unless it has returned with an error. With multi-channels
files, wavpack_decode_frame() was breaking this assumption.

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 2c6cf1394096d08396faadc6e7c0b404fd6df006)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:30 +01:00
Laurent Aimar
04b71cdedd wmapro: Validate the number of audio channels before using it
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 2c1ba7994190fa2f1ad430594551070a49353bd1)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:30 +01:00
Laurent Aimar
fce03f8783 mpc8: Fix return value on EOF
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 1e3336de69d1c4c28a5e306fab20555f4078f2d7)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:30 +01:00
Laurent Aimar
22949c42ed shorten: Prevent block size from increasing
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 95010d18b2d808db9a49377e41bc2f7cf4dfa03e)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:30 +01:00
Laurent Aimar
8751941030 xan: Prevent out of bound accesses
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 124a16f678ddcffe8f1825efb29a6e8da1d580ac)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:30 +01:00
Laurent Aimar
3e1b5981ba vp56: Release old pictures after a resolution changes
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 3d09d0017d10a0d738141a955c75c555133e41b2)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:30 +01:00
Laurent Aimar
efe3fb13a7 vp56: Check for missing reference frame data
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 0ec6d6e9b682318b5b5b5457e09fbf3c4ca41335)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:30 +01:00
Laurent Aimar
987f5dc55e cinepak: Fix invalid read access on extra data
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit d239d4b447885cb7c5eee9ce359f34ad6b64f373)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:30 +01:00
Laurent Aimar
5bb9ce755b cook: Fix js_vlc_bits value validation for joint stereo
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 3a742470a845c24e7c3a40c0a228705ca951e673)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:30 +01:00
Laurent Aimar
ea5a5f0908 segafilm: Check for memory allocation failures in segafilm demuxer.
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 1775b92fee43f0527e2f5892a5a30450fa929722)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:30 +01:00
Laurent Aimar
619aab2f41 Fixed deference of NULL pointer in motionpixels decoder.
Some of the arguments given to init_vlc() come from the stream
and can be corrupted.

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 69a0bce753a5d5556d5bc0888afe390e22611dd8)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:30 +01:00
Ronald S. Bultje
8099d77ca4 mpegvideo: set correct offset for edge emulation buffer.
Using the old code, half of it was unused and the other half was too
small for e.g. >8bpp interlaced data, causing random buffer overruns.
(cherry picked from commit 330deb75923675224fb9aed311d3d6ce3ec52420)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:30 +01:00
Ronald S. Bultje
bb7fd94eeb mpegvideo: fix position of bottom edge.
It was wrong in colorspaces where horizontal and vertical chroma
subsampling are not the same, e.g. 422.
(cherry picked from commit 0884dd5a1b87aff6c8a06e6492dece5cef8f3978)

Conflicts:

	libavcodec/mpegvideo.c

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:30 +01:00
Chris Rankin
ea311af23d qcelpdec: fix the return value of qcelp_decode_frame().
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit bde25700134b98068e2ad21c1f92955a4b489cdc)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:30 +01:00
Justin Ruggles
4562f95ba8 sipr: fix the output data size check and only calculate it once.
(cherry picked from commit 1b5a189f06879338088809b3049ea7620f4e7e78)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:27 +01:00
Justin Ruggles
fc0e151cdc mpc8: check output buffer size before decoding
(cherry picked from commit 5674d4b0a35a34b75e3533a8580e0b5a0a8895a7)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:25 +01:00
Justin Ruggles
56fe62ec94 mpc7: return error if packet is too small.
(cherry picked from commit 8290d1f38b438f1b070de67645c8b4a42014c7ac)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:22 +01:00
Justin Ruggles
ce3e0d48f8 mpc7: check output buffer size before decoding
(cherry picked from commit c8b5c4d27409dfdcec80868686b173ba446c998b)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:20 +01:00