14523 Commits

Author SHA1 Message Date
Dustin Brody
056e9efc8e vp6: partially propagate huffman tree building errors during coeff model parsing and fix misspelling
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit f913eeea43078b3b9052efd8d8d29e7b29b39208)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-09-22 01:22:21 +02:00
Laurent Aimar
cf43508eb3 Check for huffman tree building error in vp6 decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7c249d4fbaf4431b20a90a3c942f3370c0039d9e)
2011-09-22 01:19:27 +02:00
Laurent Aimar
c9c6e5f4e8 Release old pictures after a resolution change in vp5/6 decoder
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dba20b84784a7931b7eac50ced1d43e86801bde9)
2011-09-22 01:19:21 +02:00
Laurent Aimar
a5a02ea3f2 Check for missing reference in vp5/6 decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6a0e78929aa7d6b2c6b598c1589fb0e48fccb132)
2011-09-22 01:19:15 +02:00
Laurent Aimar
69b6248327 Check for invalid slices offsets in RV30/40 decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b64269ce5528bdbec8af671042f97af1242cf044)
2011-09-22 01:19:07 +02:00
Laurent Aimar
533dbaa55b Check output buffer size in nellymoser decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 741ec30bd2385f794efa9fafa84d39a917f2574e)
2011-09-22 01:19:01 +02:00
Michael Niedermayer
ec7f0b527c Merge remote-tracking branch 'khirnov/release/0.7' into release/0.8
* khirnov/release/0.7: (64 commits)
  rv34: Check for invalid slice offsets
  rv34: Fix potential overreads
  rv34: Avoid NULL dereference on corrupted bitstream
  rv10: Reject slices that does not have the same type as the first one
  lavf: Fix context pointer in av_open_input_stream when avformat_open_input fails
  oggdec: fix out of bound write in the ogg demuxer
  Fixed size given to init_get_bits().
  smacker: fix a few off by 1 errors
  Check for invalid VLC value in smacker decoder.
  Check and propagate errors when VLC trees cannot be built in smacker decoder.
  Fixed off by one packet size allocation in the smacker demuxer.
  Check for invalid packet size in the smacker demuxer.
  ape demuxer: fix segfault on memory allocation failure.
  xan: Add some buffer checks (cherry picked from commit 0872bb23b4bd2d94a8ba91070f706d1bc1c3ced8)
  Fixed size given to init_get_bits() in xan decoder. (cherry picked from commit 393d5031c6aaaf8c2dda4eb5d676974c349fae85)
  smacker demuxer: handle possible av_realloc() failure.
  Fixed segfault with wavpack decoder on corrupted decorrelation terms sub-blocks.
  cljr: init_get_bits size in bits instead of bytes (cherry picked from commit 0c1f5b93d9b97c4cc3684ba91a040e90bfc760d2)
  indeo2: fail if input buffer too small (cherry picked from commit b7ce4f1d1c3add86ece7ca595ea6c4a10b471055)
  indeo2: init_get_bits size in bits instead of bytes (cherry picked from commit 68ca330cbd479111db9cb7649d7530ad59f04cc8)
  ...

Conflicts:
	ffmpeg.c
	libavdevice/alsa-audio.h
	libavformat/gxf.c
	libswscale/x86/swscale_template.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2011-09-22 01:10:24 +02:00
Carl Eugen Hoyos
bf7dc6b29d Fix dxva2 decoding for some H264 samples. 2011-09-21 23:47:34 +02:00
Laurent Aimar
d2c5904cab Check for invalid slice offsets in real decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8716c178ddabe28a01a9fe81fff68d117b779333)
2011-09-21 21:04:51 +02:00
Laurent Aimar
738c17b3a6 rv10: Reject slices that does not have the same type as the first one
This prevents crashes with some corrupted bitstreams.

Signed-off-by: Martin Storsjö <martin@martin.st>
2011-09-21 21:03:11 +02:00
Laurent Aimar
60a1384013 Avoid NULL dereference on corrupted bitstream with real decoder.
rv34_decode_slice() can return without allocating any pictures.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 140dbcff357351570426efd0e0bc00e380f5436e)
2011-09-21 20:56:53 +02:00
Laurent Aimar
b59919afe2 Reject slices that does not have the same type than the first one in RV10/RV20 decoder.
This prevents crashes with some corrupted bitstreams.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d788af6cf696d547a442c47e1ce6f93bc9fc97b6)
2011-09-21 20:56:53 +02:00
Michael Niedermayer
764ffdd0ec check all svq3_get_ue_golomb() returns.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 979bea13003ef489d95d2538ac2fb1c26c6f103b)
2011-09-21 20:56:53 +02:00
Michael Niedermayer
ed9e561490 rv34: check for size mismatch
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 35f38b3ab9d755aede5bce8abbe1cb9c07027f8a)
2011-09-21 20:56:53 +02:00
Laurent Aimar
4d8330d095 Fix js_vlc_bits value validation when joint stereo is used in cook decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 352c878de6fca57a3f2a6369e27ca442fa5f9518)
2011-09-21 19:50:08 +02:00
Laurent Aimar
30d7dce94f Fix potential overreads in rv34 decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9a0a64cb26bc6aaa7f29106bded8a27f3db918ee)
2011-09-21 19:50:03 +02:00
Michael Niedermayer
fa3f7391be h264: allow disabling bitstream overread protection by using the fast flag.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 205c13685f36a17f406b7cd07e14b6d2a61f396a)
2011-09-21 19:49:30 +02:00
Alex Converse
b7000d0517 xan: Add some buffer checks
(cherry picked from commit 0872bb23b4bd2d94a8ba91070f706d1bc1c3ced8)
2011-09-21 19:47:12 +02:00
Alex Converse
169e634457 xan: Remove extra trailing newline
(cherry picked from commit 350f57bd7b77b36c150540ad2aae9de42cc4dfca)
2011-09-21 19:47:06 +02:00
Laurent Aimar
053bc4ce8b Fixed size given to init_get_bits() in xan decoder.
(cherry picked from commit 393d5031c6aaaf8c2dda4eb5d676974c349fae85)
2011-09-21 19:47:00 +02:00
Michael Niedermayer
e952ff6981 smacker: fix a few off by 1 errors
stereo & 16bit is untested due to lack of samples

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d67e74929c2c0d09383c2f4a9c904bfdd839c67c)
2011-09-21 19:46:34 +02:00
Michael Niedermayer
9cee26dfde smacker: add forgotten *
found by fenrir

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f98edc73c599badaa0c075fbffb519a150d03d80)
2011-09-21 19:46:23 +02:00
Kostya Shishkov
ad6177e52c rv34: check that subsequent slices have the same type as first one.
This prevents some crashes when corrupted bitstream reports e.g. P-type
slice in I-frame. Official RealVideo decoder demands all slices to be
of the same type too.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 23a1f0c59241465ba30103388029a7afc0ead909)
2011-09-21 19:36:53 +02:00
Laurent Aimar
c9316b7c6d Fixed invalid read access on extra data in cinepak decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dc255275f6293a060518271a151e1ce75499e874)
2011-09-21 18:02:40 +02:00
Laurent Aimar
2bf9a09a2c Fixed segfaults on corruped smacker streams in the decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d07ac1853da29ea696243160e02154ebf758d1ee)
2011-09-21 18:02:29 +02:00
Michael Niedermayer
54544100a3 h264: prevent an out of array read in decode_nal_units()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ea0ac11e52b9cf5264f3d4eb7543e760c2a5fbb4)
2011-09-21 18:02:18 +02:00
Michael Niedermayer
97437dada6 h264dec: Prevent CABAC and CAVLC bitsteram overreading
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 23f5cff92cdcfa55a735c458fcb5f95c0e0f3b1f)
2011-09-21 18:02:13 +02:00
Art Clarke
c8736de331 libspeex encoder wraper
taken from svn head of xuggle
(cherry picked from commit a52cdcd296c40882c3b4f88958990c56f0ce3019)
2011-09-21 18:01:25 +02:00
Joakim Plate
92f1b5df32 dvbsubdec: don't hardcode subtitle colors count in dvbsubdec to 16
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4a3294ef0069c898a495c783dfdb2f0bf1dc6583)
2011-09-21 18:01:20 +02:00
Laurent Aimar
82e4fd193f Fixed segfault with wavpack decoder on corrupted decorrelation terms sub-blocks.
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 8bfea4ab4e2cb32bc7bf6f697ee30a238c65d296)
2011-09-21 18:01:13 +02:00
Alex Converse
3a0649ddeb cljr: init_get_bits size in bits instead of bytes
(cherry picked from commit 0c1f5b93d9b97c4cc3684ba91a040e90bfc760d2)
2011-09-21 18:01:09 +02:00
Alex Converse
9f05400ea8 indeo2: fail if input buffer too small
(cherry picked from commit b7ce4f1d1c3add86ece7ca595ea6c4a10b471055)
2011-09-21 18:01:02 +02:00
Alex Converse
09cfd6f597 indeo2: init_get_bits size in bits instead of bytes
(cherry picked from commit 68ca330cbd479111db9cb7649d7530ad59f04cc8)
2011-09-21 18:00:54 +02:00
Michael Niedermayer
b2af83a9ed cabac test: Change input to test, so a wider range of states is tested.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1eb805ed70a0aac35523c7a308145b6b8d05441e)
2011-09-21 18:00:40 +02:00
Michael Niedermayer
f38b2a6be8 cabac test: match encode and decode side
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 830d7d5c4fbf81265ca5260d59fe8467ac97b2e6)
2011-09-21 18:00:36 +02:00
Michael Niedermayer
db93a5a0c8 cabac: fix cabac encoder
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 89653ea728bd59da2b88967ad648cce685950879)
2011-09-21 18:00:18 +02:00
Laurent Aimar
b5fe6bee01 Fixed deference of NULL pointer in motionpixels decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 824f98f442996eaee9204b132752cf5114fc94cf)
2011-09-21 18:00:10 +02:00
Laurent Aimar
7181adab80 Fixed size given to init_get_bits().
init_get_bits() takes a number of bits and not a number of bytes as
its size argument.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e5e0580b93a5bda34f62a5df50c1b15e610d4ad1)
2011-09-21 17:59:48 +02:00
Laurent Aimar
bac822025e Fixed size given to init_get_bits() in ffv1 decoder.
init_get_bits() takes a number of bits and not a number of bytes as
its size argument.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8362a0ffed0b15a9977fcde947bcb49dfc06638b)
2011-09-21 17:59:43 +02:00
Alex Converse
8a8aafd2b9 wavpack: Check error codes rather than working around error conditions.
(cherry picked from commit dba2b63a98bdcac7bda1a8a2c48950518c075e17)
2011-09-21 17:59:36 +02:00
Michael Niedermayer
a13ef61051 rc: finetune convergence failure fix
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 73e0ec2ff453799e361c67395b93df11986609b3)
2011-09-21 17:59:30 +02:00
Michael Niedermayer
4fbc35cd53 rc: fix convergence failure
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ed14517c234299387d401959c6037f25e0937f02)
2011-09-21 17:59:25 +02:00
Laurent Aimar
558cf502ac Fixed invalid writes in wavpack decoder on corrupted bitstreams.
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 0aedab03405849962b469277afe047aa2c61a87f)
2011-09-21 17:57:33 +02:00
Chris Rankin
b0da6a744a qcelpdec: fix the return value of qcelp_decode_frame().
(cherry picked from commit 04c13dca8812e8302686887b6e8201d4ad25b7d8)
2011-09-21 17:57:01 +02:00
Michael Niedermayer
d99613bad6 jpeglsdec: fix infinite loop
Fixes Ticket331

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bd358e128f47c14c7bc9dfceb8dedb3e5697f017)
2011-09-21 17:57:01 +02:00
Kostya Shishkov
c026f336b9 wavpack: fix wrong return value in wavpack_decode_block()
This function should return number of samples decoded, not number of bytes
decoded.
Spotted by Uoti Urpala.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit bcd4aa8bec19a17bd633e3db13357784c05b6ca5)
2011-09-21 17:56:15 +02:00
Reimar Döffinger
5c2d684986 Check extradata size on resolution change.
Ignore resolution change if resolution not defined in extradata.

Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
(cherry picked from commit 09c5f990bc7629dfbee8c760fd485936c60a7b40)
2011-09-21 17:56:15 +02:00
Gavin Kinsey
30442fa217 jpegdec: set color_range
(cherry picked from commit 2f870e262e953815060de0ebea9b6653133b203e)
2011-09-21 17:56:15 +02:00
Michael Niedermayer
e7d10f5a90 mpeg4: fix typo in mpeg4_encode_gop_header()
Found-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f5bda9fcbb54c9c27503ab1bdb86838eb6f602f1)
2011-09-21 17:56:15 +02:00
Michael Niedermayer
ca5dfd1550 h264: clean all non null elements of delayed_pic[]
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 66ce282df54edb33515d9ef3015b737428b6e0c3)
2011-09-21 17:56:14 +02:00