Metadata streams have priv_data set to NULL.
Reviewed-by: Josh de Kock <josh@itanimul.li>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit fdb8c455b637f86e2e85503b7e090fa448164398)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
A negative extradata size for example gets passed to memcpy in
avcodec_parameters_from_context causing a segmentation fault.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1c7da19a4b45f5623cb3955b29b9a581026e3c61)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ce3147eb198770b558acf6c05f33cb807a413707)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
channel_index can be -1.
This problem was introduced in commit
2dd7b46132e2801ef34fe1b5c27e0113cdcfa2f9.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ffdc5d09e498bee8176c9e35df101c01c546a738)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 3c0328d58d98664b05efdd377d3fe66a569d385e)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
They are not valid and can cause problems/crashes for API users.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit a86ebbf7f641bc797002ddea7fb517759722cd1b)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This is similar to commit 9ac61e73d0843ec4b83f4e3d47eded73234e406e.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 0edd569466eb45b134690b9f4efbb57eda86f58d)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This prevents a division by zero in read_packet.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 25012c56448a48487cdc9699465e640871dbcd60)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Otherwise put_bits can be called with a value that doesn't fit in the
sample_len, causing an assertion failure.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit cdb5479c9ddc886f0b8661db585405ebab343e80)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes a heap-buffer-overflow detected by AddressSanitizer.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 360bc0d90aa66cf21e9f488e77d21db18e01ec9c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
It can read less than the requested amount, in which case buf contains
uninitialized data, causing problems like segmentation faults later on.
Also make sure that image->size is positive, so that it can't match a
negative error code.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 89eb398c7fc4cb9a15e55bdf2ab6435b5332e377)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This problem was introduced in commit
4b90dcb8493552c17a811c8b1e6538dae4061f9d.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit c82b8ef0e4f226423ddd644bfe37e6a15d070924)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The implicit checks via v_data_size and a_data_size don't work in the case
'(hdr_size > 7) && !ctx->alpha_info'.
This fixes segmentation faults due to invalid reads.
This problem was introduced in commit
547c2f002a87f4412a83c23b0d60364be5e7ce58.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1e33035ee7a8d9fb7a4b8b6cc54842e72b36ed70)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
It causes a cb_depth of 32, leading to assertion failures in get_bits.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 226d35c84591f1901c2a13819031549909faa1f5)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit d54c95a1435a8a3fcd599108ec85b7f56a0fcbf9)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 467eece1bea5c8325c6974190ba61f1bba88a3f3)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1bbb18fe82fc77a10d45fa53bd2738d2c54de6c6)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The code assumes that s->streams[0] is valid.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ff100c9dd97d2f1f456ff38b192edf84f9744738)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The API does not allow returning AVERROR codes.
It triggers an assert in av_parser_parse2.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 5249706e9d2ec5ed1b07d8ffdbb8fb9104261f6d)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Metadata streams have priv_data set to NULL.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 0efb6106118c17308b3fdc3190f5e5bf84b01d5c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes heap-use-after-free detected by AddressSanitizer.
Reviewed-by: Luca Barbato <lu_zero@gentoo.org>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit bb6a7b6f75ac544c956e3eefee297700ef4d3468)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This was broken by the following Libav commit:
4c387c7 ppc: dsputil: do unaligned block accesses correctly
The following tests fail due to this:
fate-checkasm
fate-vsynth1-dnxhd-2k-hr-hq fate-vsynth1-dnxhd-edge1-hr
fate-vsynth1-dnxhd-edge2-hr fate-vsynth1-dnxhd-edge3-hr
fate-vsynth1-dnxhd-hr-sq-mov fate-vsynth1-dnxhd-hr-hq-mov
fate-vsynth2-dnxhd-2k-hr-hq fate-vsynth2-dnxhd-edge1-hr
fate-vsynth2-dnxhd-edge2-hr fate-vsynth2-dnxhd-edge3-hr
fate-vsynth2-dnxhd-hr-sq-mov fate-vsynth2-dnxhd-hr-hq-mov
fate-vsynth3-dnxhd-2k-hr-hq fate-vsynth3-dnxhd-edge1-hr
fate-vsynth3-dnxhd-edge2-hr fate-vsynth3-dnxhd-edge3-hr
fate-vsynth3-dnxhd-hr-sq-mov fate-vsynth3-dnxhd-hr-hq-mov
Fixes trac ticket #5508.
Reviewed-by: Carl Eugen Hoyos <ceffmpeg@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 3932ccc472ad4f4d370dcfc1c2f574b0f3acb88c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes out-of-bounds reads by the bitstream reader.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 60178e78f2fe9a7bfb9da0abc985835e2ebfd2f1)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The number of channels is used as divisor in decode_frame, so it must
not be zero to avoid SIGFPE crashes.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 5540d6c1343e6d1e06d6601b7d35884761711e3e)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes out-of-bounds reads.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 14e4e26559697cfdea584767be4e68474a0a9c7f)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Otherwise another frame gets referenced into picture, triggering an assert
(from commit 13aae8) in av_frame_ref.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit a92f8edf0c51781e152651cce2e753ad6e359eb2)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes asserts (from commit 13aae8) in av_frame_ref and
av_frame_move_ref.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1966ea012fd72abc8003e95dc3c8ad9e9f197913)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Also check for possible overflows.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit b0a043f51b8cc3b420dc3ceaa38fe9aa344799aa)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
It can be unset in avcodec_parameters_from_context and a value of 0
causes SIGFPE crashes.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 93c39db5f1544d1220488cfeb93bfe812a52f374)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Negative values don't make sense and too large values can cause
overflows. For AV_CODEC_ID_ADPCM_THP this leads to a too small extradata
buffer being allocated, causing out-of-bounds writes.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ee5f0f1d355fa0fd9194ac97a2c8598c93ed328b)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If probe_codec is called with pkt == NULL, it sets probe_packets to 0
and request_probe to -1.
However, request_probe can change when calling s->iformat->read_packet
and thus a probe_packets value of 0 doesn't guarantee a request_probe
value of -1.
In that case calling probe_codec again is necessary to prevent
triggering the assert.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit a5b4476a602f31e451b11ca0c18bc92be130a50e)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit bc7e128a6e8e2a79d0ff7cab5e8a799b3ea042ea)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9959a52b14bcfa3e5baeb3fc8a86c04bbc0d3d5d)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit c143a9c96ff907a8fe4598529664aec7cb156708)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If realloc fails, the pointer is overwritten and the previously allocated
buffer is leaked, which goes against the expected behavior of keeping the
packet unchanged in case of error.
Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 574929d8b6de32ae712fcca7ab09f01a3e4616be)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit fecb3e82a4ba09dc11a51ad0961ab491881a53a1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This function must be called from the mb or slice encoding loop and MMX state may not
be clean there
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 03ec6b780cfae85b8bf0f32b2eda201063ad061b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4f96f9d1118e073d346d16be157fa5075434e7f2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This code is not correct for git master
Reviewed-by: Stefano Sabatini <stefasab@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2bd99564540a365d5b80d9aad6c19264b15955af)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The calculation of width/height should round up, not round down to
prevent setting width or height to 0.
Also image->comps[compno].w is unsigned (at least in openjpeg2), so the
calculation could silently wrap around without the explicit cast to int.
Reviewed-by: Michael Bradshaw <mjbshaw@gmail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 56706ac0d5723cb549fec2602e798ab1bf6004cd)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
openjpeg 2 sets the data pointers of the image components to NULL,
causing segfaults if the image is reused.
Reviewed-by: Michael Bradshaw <mjbshaw@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 69c8505f3bf54f316e9dc8bec1c71dfa1febec63)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Use check_lib2 to test the header together with the function. This is
necessary, because '-DOPJ_STATIC' changes what the included header does.
Also add '-DOPJ_STATIC' to CPPFLAGS, so that it isn't necessary to
hardcode this in libavcodec/libopenjpeg{dec,enc}.c.
Finally, check for non-static openjpeg 2.1, too.
Reviewed-by: Michael Bradshaw <mjbshaw@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 7a65aef00d113a38e0d1a54df49eead9df6aa15c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Moritz Barsnick <barsnick@gmx.net>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 99d68d462fbd777cfd3fe055d4181a6f7c03fac7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Moritz Barsnick <barsnick@gmx.net>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3e5d27d7a7350e096eac9f8999d02bf48c3b3a69)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Moritz Barsnick <barsnick@gmx.net>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f4e4bde1f4cff99d4ec59ed361ff9228b2050e6b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
