flcoeffs2[pos] should be the log2 of flcoeffs1[pos].
flcoeffs1[0] can be 0 here, thus flcoeffs2[pos] gets set to -inf,
causing problems further down.
This seems to have been copied from imc_decode_level_coefficients in
commit 4eb4bb3 without updating the position.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 75fd5ce4c1c0b2d96d71c74b650cefaaef519d27)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 27816fb9ef26af1fa820c434079491f155e67471)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
It is used as get_bits argument and reading 0 bits isn't supported.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 05cc8c8e4b7008ff3be8ec477c901b2ceca4b16b)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit f06d9dced4c7267008beae45f28ce4e1848ef403)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
It asserts that the frame linesize is larger than 37, but it can be
smaller and decoding such frames works.
Before commit cc884a35 src_stride > 7*MB_SIZE was necessary, because the
blocks were interleaved in the tmp buffer and the last block was added
with an offset of 6*MB_SIZE.
It was changed for src_stride <= 7*MB_SIZE to write the blocks
sequentially, hence the larger tmp_step.
After that the assert was only necessary to make sure that the buffer
remained large enough.
Since commit bd2b6b33 s->scratchbuf is used as tmp buffer.
As part of commit 86e107a7 the minimal scratchbuf size was increased to
256*7*MB_SIZE, which is enough for any src_stride <= 7*MB_SIZE.
Also add a comment explaining the tmp_step calculation.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 3526a120f92929cb0a4009e403ee2f141030c487)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 26cb351452e76dc4a61028f277cf11b8d7abd74b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The loop always needs at least 8 bytes for chunk_type and chunk_size.
If fewer are left, bytestream2_get_le32 just returns 0 without
reading any bytes, leading to an infinite loop.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0762152f7af6cd93bc8f504d5503723500c3f369)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 762a5878a6b0bef923ef97c15fdb8274a0351278)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
More than 32 bits can't be stored in an integer and get_bits should not
be used with more than 25 bits.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit d0eff8857ceff2601f85037c930cbe61a88b611e)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit f0af6e705f3b30f7f5afa3c24db27433af6b1bfc)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This is the first part of the fix for ticket #4370.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 32a5b631267e1f8bf279e407039b9a99d012d033)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit acfad331adde5b4ae247bf5748211e8fdb6b4ef5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9a345802edf7f430b3335f486aecdd8552f8367b)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 43f8a422b34302d94ba76eb9a5be75ffffbc9881)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
get_bits should not be used for more than 25 bits.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit f9883a669c3df05a5c453428e080298c6511a17e)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 95bd0f3a4b65e1641079f8bbdf391d0a2bfcd27a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
get_bits should not be used with more than 25 bits.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1f1e0a2971b2a01f275bb5088c2e36166514be64)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 2e1226a695dd2faada939e0bdc3ee166b05c3579)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This fixes a segfault when decoding multi-channel MP3onMP4 files.
This is similar to commit cb72230d for MPADSPContext.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 151dbe4579601a81662b4b366d0e10df3c00027a)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit f66d2bf94909904109f6b0609516c10f9f3f3db4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
avoids null pointer dereference
Fixes: signal_sigsegv_d5de40_964_vp80-00-comprehensive-010.ivf with memlimit of 1048576
Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 599d746e07319dc792ed2e511b666fe482f1ff88)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signal_sigsegv_d5de23_967_vp80_00_comprehensive_010.ivf with memlimit 524288
Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a84f0e8d8f293df3c535f9b893730a835bed6520)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes null pointer dereference
Fixes: signal_sigsegv_b02a96_280_RL_420p_ffdirac.drc with memlimit of 67108864
Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1c5b712c0a643a039d6f34269b4102de313a050a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Avoids leaving stale pointers
Fixes: signal_sigabrt_7ffff70eccc9_819_sabtriple.rm with memlimit 536870912
Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3197c0aa87a3b7190e17d49e6fbc7b554e4b3f0a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: b4b47bc2b3fb7ca710bfffe5aa969e37_signal_sigabrt_7ffff70eccc9_744_nc_sample2.avi with memlimit of 4194304
Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f1a38264f20382731cf2cc75fdd98f4c9a84a626)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes null pointer dereferences
Fixes: af1a5a33e67e479f439239097bd0d4fd_signal_sigsegv_7ffff713351a_152_Dolby_Rain_Logo.pmp with memlimit of 8388608
Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5d346feafa817c4fbc30f7ed0b93b2dad6cef15b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes crash in 1288a2fe8e9ae6b00ca40e089d08ca65_signal_sigsegv_7ffff71426a7_354_accident.san with allocation limit 65536
Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 39bbdebb1ed8eb9c9b0cd6db85afde6ba89d86e4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 06a4edb39ad8a9883175f9bd428334a2_signal_sigsegv_7ffff713351a_706_mov__alac__ALAC_6ch.mov
Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f7068bf277a37479aecde2832208d820682b35e6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This prevents possible infinite loops with the calling code along the
lines of while (bytestream2_get_bytes_left()) { ... }, where the reader
does not advance.
CC: libav-stable@libav.org
(cherry picked from commit 86eee85daddb682fa072c2e2657c90a514b855e3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes Assertion failure
Found-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7ef6656b1e5bfbc7499013d3b38b093b6b2f31ec)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This ensures that no stale pointers leak through on any path
Fixes: signal_sigsegv_c3097a_991_xtrem_e2_m64q15_a32sxx.3gp
Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b160fc290cf49b516c5b6ee0730fd9da7fc623b1)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Otherwise a frame with non-zero nb_samples but without any data can be
returned.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 42e7a5b3c704985c2c18970cc94a837b413df9d9)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit f9020d514e9ed5043496a710b36daba1ab182e97)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
the later is not optimal when the buffer size is well known at allocation time
This avoids a memcpy()
Overall 2.5% speedup with a random 1920x1080 video
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 47496eb97cff8130991313d1b7292613620d8592)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2e13a45b1a9a69456631e582bbb06954d169eb55)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array access
Fixes: signal_sigsegv_3670fc0_2818_cov_2307326154_moon.mux
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 79a98294da6cd85f8c86b34764c5e0c43b09eea3)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array access
Fixes: asan_heap-oob_4d5bb0_682_cov_3124593265_Fraunhofer__a_driving_force_in_innovation__small.mp4
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 330863c9f19a23c500ba7901a23f1cc377b353bb)
Conflicts:
libavcodec/h264_slice.c
the max value of the lookup in expanded form is:
(((1 << 7) - 1) << 1) - 65 + 1 + 64 = 254
add one entry of padding to inv_map_table[] to prevent out of bounds
access with non-conforming / fuzzed bitstreams
Signed-off-by: James Zern <jzern@google.com>
Reviewed-by: "Ronald S. Bultje" <rsbultje@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e91f860ea74e11e9178500fe8794c47f57dbf48c)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b54ac8403bfea4e7fab0799ccfe728ba76959a38)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This is required by the APNG spec
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a1736926e9ae51706b537776722e816a4d1f6b36)
Conflicts:
libavcodec/pngdec.c
Multiple IHDR chunks are forbidden in PNG
Fixes inconsistency and out of array accesses
Fixes: asan_heap-oob_4d5c5a_1738_cov_2638287726_c-m2-8f2b481b7fd9bd745e620b7c01a18df2.png
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 47f4e2d8960ca756ca153ab8e3e93d80449b8c91)
Conflicts:
libavcodec/pngdec.c
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Claiming to have decoded more bytes than the packet size is wrong.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 2a4700a4f03280fa8ba4fc0f8a9987bb550f0d1e)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes undefined behavior and segfault
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8edc17b639c4ac47913c467107ffb43c67c64890)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Also use the frame pixel format instead of the one from the codec
context, which is more robust.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit fdc64a104410f5fcc7f35b62287b0ae502b7061a)
Conflicts:
libavcodec/h264_slice.c
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Otherwise it can be 0 in sonic_decode_frame, causing SIGFPE crashes.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 58995f647b5fa2e1efa33ae4f8b8a76a81ec99df)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
If one of the dimensions is larger than 8176, s->mb_width or
s->mb_height is larger than 511, leading to an int16_t overflow of
s->mv_max.{x,y}. This then causes av_clip to be called with amin > amax.
Changing the type to int avoids the overflow and has no negative
effect, because s->mv_max is only used in clamp_mv for clipping.
Since mv_max.{x,y} is positive and mv_min.{x,y} negative, av_clip can't
increase the absolute value. The input to av_clip is an int16_t, and
thus the output fits into int16_t as well.
For additional safety, s->mv_{min,max}.{x,y} are clipped to int16_t range
before use.
Reviewed-by: Ronald S. Bultje <rsbultje@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 6fdbaa2b7fb56623ab2163f861952bc1408c39b3)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Otherwise the check 'tile_size < size' treats a negative size as
unsigned, causing the check to pass. This subsequently leads to
segmentation faults.
This was originally fixed as part of Libav commit 72ca83, so the
original author is one of the following developers:
Anton Khirnov <anton@khirnov.net>
Diego Biurrun <diego@biurrun.de>
Luca Barbato <lu_zero@gentoo.org>
Martin Storsjö <martin@martin.st>
Reviewed-by: Ronald S. Bultje <rsbultje@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit b18eac7ff22332c9344769af15f7b245dd13cc64)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
libopenjpeg can return images with components without data.
This fixes segmentation faults.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 3ef5702926c495232ffe685303ba8661bdff1149)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
If chan2 is not smaller than the number of channels, it can cause
segmentation faults due to dereferencing a NULL pointer.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 05c57ba2f42324da2fdc93d83d65bb68dd637613)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The AVFrame values are closer to the AVFrame bitmap changed instead of
the AVCodecContext values, so this should be more robust
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit aef0e0f009802f1a5e21eb6465498632071e4475)
Conflicts:
libavcodec/h264_slice.c
This avoid potential out of array accesses
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 12ba1b2b4d5592c0e27b0fcc83db929e8d6a8eee)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
See: vlc ticket 14649
Reported-by: carl
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3c803ed9cb23e5a8d76b6c31d8a8c71cac27e769)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c2657633187e325a439e3297fd9ccd0522ab2e39)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6b6ae7c3ead5dee786a4aea929820076a7c82da4)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
If opj_image_create fails to allocate an image it returns NULL, which
causes a segmentation fault at 'img->x0 = 0'.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1577526b47439f33a999339efdec5d624b70e1da)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 291ad5cc9cf815eb110b062487980fab2d107936)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8f5ffed183e099128a732a00976f69fdc641d093)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 561d3a57aaa95c7e8e65e96b36dd069100603650)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e4c2ec879b1121c02279cd60a54643da0d249e40)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
