A negative codec_id cannot be handled by the found_decoder API of
AVStream->info: if the codec_id is not recognized, found_decoder is set
to -codec_id, which has to be '<0' according to the API documentation.
This can cause NULL pointer dereferencing in try_decode_frame.
Also make sure the codec_type matches the expected one for codec_id.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ecf63b7cc2)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Also correct the check to reject log < 7, because UPDATE_CACHE only
guarantees 25 meaningful bits.
This fixes undefined behavior:
runtime error: shift exponent is negative
Testing with START/STOP timers in get_ue_golomb, one for the first
branch (A) and one for the second (B), shows that there is practically no
slowdown, e.g. for the cavs decoder:
With the check in the B branch:
629 decicycles in get_ue_golomb B, 4194260 runs, 44 skips
433 decicycles in get_ue_golomb A,268434102 runs, 1354 skips
Without the check:
624 decicycles in get_ue_golomb B, 4194273 runs, 31 skips
433 decicycles in get_ue_golomb A,268434203 runs, 1253 skips
Since the B branch is executed far less often than the A branch, this
change is negligible, even more so for the h264 decoder, where the ratio
B/A is a lot smaller.
Fixes: mozilla bug 1230239
Fixes: fbeb8b2c7c996e9b91c6b1af319d7ebc/asan_heap-oob_195450f_2743_e8856ece4579ea486670be2b236099a0.bit
Found-by: Tyson Smith
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 22e960ad47)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If the input contains too many too large values, the imdct can overflow.
Even if it didn't, the output would be larger than the valid range of 29
bits.
Note that this is a very delicate limit: Allowing values up to 1<<25
does not prevent input larger than 1<<29 from arriving at
sbr_sum_square, while limiting values to 1<<23 breaks the
fate-aac-fixed-al_sbr_hq_cm_48_5.1 test.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit fdc94db37e)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This also as a sideeffect fixes the non aligned case
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a066ff89bc)
The transpose_4x4H is wrong which cost me much time to find this bug. The orders of r2 and r3 are wrong,
this bug waste me much time while I make aarch64 arm instruction which used the function.
(cherry picked from commit c18176bd55)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes a mpegts file with hevc that fails estimating duration. Increasing number of
retries fixes the issue.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2d8c2f1a28)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
If the chroma components are subsampled, smaller buffers are allocated
for them. In that case the maximal block_offset for the chroma
components is not as large as for the luma component.
This fixes out of bounds writes causing segmentation faults or memory
corruption.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 5adb5d9d89)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
VP8E_UPD_ENTROPY, VP8E_UPD_REFERENCE, VP8E_USE_REFERENCE were removed
from libvpx and the remaining values were never used here
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Zern <jzern@google.com>
(cherry picked from commit 6540fe04a3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: d41d8cd98f00b204e9800998ecf8427e/signal_sigsegv_321165b_7641_077dfcd8cbc80b1c0b470c8554cd6ffb.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2140858524)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 99d142c47e6ba3510a74b872a1a2ae72/asan_heap-oob_11b36f4_3811_0f5c69e7609a88a580135678de1df844.dxa
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a08681f1e6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
ts->mux_rate is int (signed 32-bit) type. The period calculations
will start to overflow when mux_rate > 5mbps. This fixes overflows
by converting first to 64-bit type.
Fixes#5044.
Signed-off-by: Timo Teräs <timo.teras@iki.fi>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 64f7db554e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The fps variable is explicitly set to -1 in case of some errors, the check must
thus be signed or the code setting it needs to use 0 as error code
the type of the field could be changed as well but its in an installed header
Fixes: integer overflow
Fixes: 9982cc157b1ea90429435640a989122f/asan_generic_3ad004a_3799_22cf198d9cd09928e2d9ad250474fa58.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b46dcd5209)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes integer overflow
Fixes: mozilla bug 1229167
Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f03c2ceec1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 0a7ff0c1d93da9cef28a315ec91b692a/asan_heap-oob_4a52e5_3604_9c56dbb20e308f4faeef7b35f688521a.ape
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cd7524fdd1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array read
Fixes: 76c515fc3779d1b838667c61ea13ce92/asan_heap-oob_1fc0d07_8913_794a4629a264ebdb25b58d3a94ed1785.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 891dc8f875)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The code expects actual positive numbers and gives completely wrong
results if INT64_MIN is treated as positive
Instead clip it into the valid range that is add 1 and treat it as
negative
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 25e37f5ea9)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 59bb925e90201fa0f87f0a31945d43b5/asan_heap-oob_4a52e5_3388_66027f11e3d072f1e02401ecc6193361.jvt
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit aa427537b5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array access
Fixes: 482d8f2fd17c9f532b586458a33f267c/asan_heap-oob_4a52b6_7417_1d08d477736d66cdadd833d146bb8bae.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit abee0a1c60)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 2f95ddd996db8a6281d2e18c184595a7/asan_heap-oob_192fe91_3330_58e4441181e30a66c19f743dcb392347.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dad354f38d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
