Affected the ffmpeg-filter_colorkey FATE-test (but only if the C version
of idct8_add is used and not e.g. the x86 SSE2 version).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Fixes: left shift of negative value -107
Fixes: 20398/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CAVS_fuzzer-5725389278412800
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0f0f2ab0c3b3d04e904db97b07ae829c72c91778)
by replacing it with a multiplication. Said multiplication can't
overflow an int32_t because lpc_coefs is limited to 16 bit precision.
Fixes the FACE-test acodec-ra144 as well as part of #8217.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e3fb9af6f1353f30855eaa1cbd5befaf06e303b8)
Replace "((a << shift) + b) >> shift" by "a + (b >> shift)". This avoids
a left shift which also happens to trigger undefined behaviour in case "a"
is negative. This affected the FATE-tests acodec-adpcm-adx and
acodec-adpcm-adx-trellis; it also fixes ticket #8008.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 59a9d65e0d790821f88527a82569f56eb2f8a9be)
Affected the adpcm-afc, adpcm-ea-1, adpcm-ea-2, adpcm-ea-maxis-xa,
adpcm-thp and ea-cdata FATE-tests. Also fixes ticket #8487.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3ad8af51b7c0a968ac3fd62964780d4ff9136c5a)
This fixes ticket #7997 as well as the vsynth*-prores_# FATE-tests
(where * ranges over { 1, 2, 3, _lena } and # over { , _int, _444,
_444_int }).
(Given that prev_dc is in the range -0xC000..0x3FFF, no overflow can
happen upon multiplication with 2.)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 198081efb7c7343349f0a7acc836f001c511e990)
The documentation of both avio_open() as well as avio_open2() states
that on failure, the pointer to an AVIOContext given to this function
(via a pointer to a pointer to an AVIOContext) will be set to NULL. Yet
it didn't happen upon failure of ffurl_open_whitelist() or when allocating
the internal buffer failed. This commit changes this.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 220846f74fc33fc037147e54535bcaef8caad28b)
Fixes: left shift of 1 by 31 places cannot be represented in type 'int'.
Affected the FATE-tests vsynth1-cinepak, vsynth2-cinepak and
vsynth_lena-cinepak. Also fixes ticket #8220.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e3650dcfc9cde6ceccf7bbc225962da196e2a386)
Affected every usage of vf_xbr, e.g. the FATE-tests filter-2xbr,
filter-3xbr, filter-4xbr.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4294dc3589a3ab973b10a85b576ff15e3ffb000d)
Affected every usage of this filter; in particular, it affected the
FATE-tests filter-2xbr, filter-3xbr and filter-4xbr.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit fa211943265ca991548a4cc2f85a6df9cedcd092)
Affected the vsynth*-jpeg2000 and the vsynth*-jpeg2000-97 FATE tests
(where * ranges over { 1, 2, 3, _lena }) as well as ticket #7983.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5cf593adcd79a7c9502dc2725e1f0681ada36aef)
Affected the FATE-tests vsynth_lena-dv-411, vsynth1-dv-411,
vsynth2-dv-411 and hevc-paramchange-yuv420p.yuv420p10.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e2646e23be69bdef1e41d4decee1a4298701b8d1)
This affected many FATE-tests: The number of failing tests went down
from 663 to 344. (Both numbers exclude tests that failed because of
unaligned accesses in code that is inside #if HAVE_FAST_UNALIGNED.)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 736c7c20e7819811dc59f43490563789b192eb6e)
Affected the FATE-tests exr-rgb-scanline-pxr24-half-uint32-13x9 and
exr-rgb-scanline-pxr24-uint32.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8b0f949906116c40b6f1e55a1bce4447ada3219c)
Fixes the movenc FATE-test.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 646799b42fd59ee79920e472795bf881b78bb5ce)
Fixes the acodec-pcm-u16[lb]e FATE-tests.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 69473bec6f38fefc9a433d95f8e00de101299592)
Fixes ticket #8161 and the acodec-wavpack FATE-test.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 84974c6fb542cf019904016c2165d9a62db9f312)
The pointer arguments to memcpy (and several other functions of the
C standard library) are not allowed to be NULL, not even when the number
of bytes to copy is zero. An AVEncryptionInitInfo's data pointer is
explicitly allowed to be NULL and yet av_encryption_init_info_add_side_data
unconditionally used it as a source pointer to copy from. This commit changes
this so that copying is only done if the number of bytes to copy is > 0.
Fixes ticket #8141 as well as a part of ticket #8150.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Tomas Härdin <tjoppen@acc.umu.se>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e6018fda14d7cfe2c890fb336c9264c4ea0b6c5c)
Fixes the FATE-tests unknown_layout-ac3, ac3-fixed-encode, ac3-encode
and eac3-encode. It furthermore fixes the ac3-encoder bugs mentioned in
tickets #7994, #8144 and #8159.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2f289ec914cc7e8133858d4f9e8d91dab685ae44)
Fixes the FATE-tests g723_1-dec-1, g723_1-dec-2 and g723_1-dec-4.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 54e1efda6d67afb11e21e40109c10b447f8d7f3c)
Fixes the tdsc FATE-test.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 86bef10e7aee338a4df60c57904c16e33509e76e)
ttaenc contained (1 << unary) - 1 as an argument for a function
expecting an unsigned int. unary can be as big as 31 in this case.
The type of the shift and the whole expression is int, because 1 fits
into an integer, so that the behaviour is undefined if unary == 31
as the result of the shift can't be represented in an int §. Subtraction
by 1 (which makes the result of the whole expression representable in
an int) doesn't change that this is undefined (it usually leads to
signed integer overflow which is undefined, too).
The solution is simple: Make 1 unsigned to change the type of the
whole expression to unsigned int (as the function expects anyway).
Fixes ticket #8153.
§: This of course presupposes the common int range of -2^31..2^31-1
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3ab488a5407f833ecc66e8fa4c537dc4852db720)
The code for GAB2 subtitles predates refcounting AVPackets. So in order
to transfer the ownership of a packet's data pkt->data was simply stored
and the packet zeroed; in the end (i.e. in the read_close-function) this
data was then simply freed with av_freep(). This of course leads to a leak
of an AVBufferRef and an AVBuffer. It has been fixed by keeping and
eventually unreferencing the packet's buf instead.
Additionally, the packet is now reset via av_packet_unref().
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit da44bbefaabeb2fdb58a03fe533a44aa150486fc)
3469cfab added a check for whether the extradata coincided with the
beginning of the packet's data in order not to add extradata to packets
that already have it. But the check used was buggy for packets whose
size is smaller than the extradata's size. This commit fixes this.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a88a3cdb4b166c83b823d34abe8a7a6743c7ebd5)
by changing the type to unsigned.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8ae026d74f599b2d00b91798af1c1067a879007c)
Fixes#8314.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Peter Ross <pross@xvid.org>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 373c1c9b691fd4c6831b3a114a006b639304c2af)
The Dash muxer uses submuxers and when one such submuxer has been allocated,
it is initially only stored in a temporary variable. Therefore it leaks
if an error happens between the allocation and storing it permanently.
This commit changes this.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: "Jeyapal, Karthick" <kjeyapal@akamai.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8aeab0dbc159d65a20bb4cc89c2524b482fd867a)
by freeing it a bit earlier.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bb20f3dd730689c3a99f7820cff8b74b06992fff)
A Seek element in a Matroska SeekHead should contain a SeekID and a
SeekPosition element and upon reading, they should be sanitized:
Given that IDs are restricted to 32 bit, longer SeekIDs should be treated
as invalid. Instead currently the lower 32 bits have been used.
For SeekPosition, no checks were performed for the element to be
present and if present, whether it was excessively large (i.e. the
absolute file position described by it exceeding INT64_MAX). The
SeekPosition element had a default value of -1 which means that a check
seems to have been intended; but it was not implemented. This commit adds
a check for overflow to the calculation of the absolute file position of
the referenced level 1 elements.
Using -1 (i.e. UINT64_MAX) as default value for SeekPosition implies that
a Seek element without SeekPosition will run afoul of this check.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 7c243eece3427bc5a6d54657d488d5c0c2985a8e)
Matroska (or actually EBML) uses variable-length numbers where only
seven bits of every byte is usable for the length; the other bits encode
the length of the variable-length number. So in order to find out how
many bytes one needs to encode a given number one can use a loop like
while (num >> 7 * bytes) bytes++; the Matroska muxer effectively did this.
Yet it has a disadvantage: It is impossible for the result of a single
right shift of an unsigned number with most significant bit set to be
zero, because one can only shift by 0..(width - 1). On some
architectures like x64 it is not even possible to do it with undefined
right shifts in which case this leads to an infinite loop.
This can be easily avoided by switching to a loop whose condition is
(num >>= 7). The maximum value the so modified function can return
is 10; any value > 8 is invalid and will now lead to an assert in
put_ebml_num() or in start_ebml_master() (or actually in
put_ebml_size_unknown()).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 9b0f9003dfab6a230d46aaa94091bf509d889f37)
For FLAC, Speex, Opus and VP8 the Ogg muxer allocates two buffers
for building the headers: The first for extradata in an Ogg-specific
format and the second contains a Vorbiscomment. These buffers are
reachable via pointers in the corresponding AVStream's priv_data.
If an error happens during building the headers, the AVStream's
priv_data would be freed. This is pointless in general as it would be
freed generically anyway, but here it is actively harmful: If the second
of the aforementioned allocations fails, the first buffer would leak
upon freeing priv_data.
This commit stops freeing priv_data manually, which allows the muxer to
properly clean up in the deinit function.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 0fcf74f4357e949f5971d39b04a128103b8949bb)
avformat_find_stream_info() may decode some frames to get stream
information. And when it does this for subtitles, the decoded subtitles
leak.
(Decoding subtitles was added in b1511e00f6fefde6cb31b2e17f7812cfac1c8bd6
for PGS subtitles. When PGS subtitles originate from a container that
exports every segment as a packet of its own, no output will be
generated when decoding a packet, because not enough input is available.
Yet when used with PGS subtitles in the Matroska form a single packet
contains enough data to generate output. Yet said output is not freed,
hence this leak.)
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit d026fef999f7e2491721b3878bea6e6da8b9f92e)
The data of an attachment file is put into an AVCodecParameter's
extradata. The corresponding size field has type int, yet there was no
check for the size to fit into an int. As a consequence, it was possible
to create extradata with negative size (by using a big enough max_alloc).
Other errors were also possible: If SIZE_MAX < INT64_MAX (e.g. on 32bit
systems) then the file size might be truncated before the allocation;
and avio_read() takes an int, too, so one would not have read as much
as one desired.
Furthermore, the extradata is now padded as is required.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 889ad93c8839e5ac1ec28bc8e1fea6df71b9bf80)
vobsub_read_packet() didn't check whether an array of AVPackets was
valid and therefore used uninitialized values.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit a39536caee6607f481e9075bfb11937f46a47489)
The WebM DASH Manifest muxer only supports VP8, VP9, Vorbis and Opus,
but there was no check for this. The codec type is used to get a pointer
to a string containing the codec name or NULL if it is not one of those
four codecs. Said pointer has then been used without further checks as
string for the %s conversion specifier in an avio_printf()) call which
is undefined behaviour.
This commit adds a check for the supported codec types.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit cbea58b2b35c6409e062c929f0b2ab763b8661eb)
The classical ptr = av_realloc(ptr, size).
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 418e468699868a0265f8b439beedf64bb643b088)
Do it only if the packet has been successfully allocated in
av_new_packet() -- otherwise on error a completely uninitialized packet
would be unreferenced later.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6bd8bcc2ac4c64577d964552317989e61db794d8)
The classical ptr = av_realloc(ptr, size), just with av_fast_realloc().
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5acef1206144554a48f699b421e8d739e752d8ab)
If an AVFormatContext could be allocated, but white-/blacklists couldn't
be copied, the AVFormatContext would leak as it was only accessible
through a local variable that goes out of scope when one goes to fail.
Furthermore, in case writing a header of a submuxer failed, the options
used for said call could leak.
Both of these memleaks have been fixed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit abbb466368c51285ca27d5e3959a16a9591e9a4c)
Don't read a 64bit number before having checked that the data is at
least 8 bytes long.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6e9cc964293bf1e0cca6a52b2938a20d711e4146)
Sometimes it has not been checked whether opening the dynamic buffer for
writing Tags fails; this might have led to segfaults.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b4f300f8ea209b861865ef495b298a88b48f3913)
This is needed especially for AV1: If a reformatting error happens (e.g.
if the length field of an OBU contained in the current packet indicates
that said OBU extends beyond the current packet), the data pointer is
still NULL, yet the size is unchanged, so that writing the data leads
to a segmentation fault.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 58428bef4b2c053f47dce35157fb96833ba8efea)
That way one doesn't have to free later. In this case (concerning TTA
extradata), this also fixes a memleak when the output samplerate is
invalid.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit f7bf59b431e0921d5f318154f64c78bb226e33b9)
The unsharp filter uses an array of arrays of uint32_t, each of which is
separately allocated. These arrays also need to freed separately; but
before doing so, one needs to check whether the array of arrays has
actually been allocated, otherwise one would dereference a NULL pointer.
This fixes#8408.
Furthermore, the array of arrays needs to be zero-initialized so that
no uninitialized pointer will be freed in case an allocation of one of
the individual arrays fails.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 710ab136931ff228b355d87512b0d4ca4e94656a)
