Fixes: runtime error: pointer index expression with base 0x000000000000 overflowed to 0xfffffffffffffff8
Fixes: 58440/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HEVC_fuzzer-5956015530311680
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a0f4d4e65093a4cb627f05d09b19c922e88cfac1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 29 + 2147483640 cannot be represented in type 'int'
Fixes: 25413/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HEVC_fuzzer-5697909331591168
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 106f11f68af643ad1f372b840d38a0a30c6e9bcf)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d2d8e797cc4f3cea3470d464bd5f51cd097fe371)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: NULL pointer dereference and out of array access
Fixes: 13871/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HEVC_fuzzer-5746167087890432
Fixes: 13845/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HEVC_fuzzer-5650370728034304
This also fixes the return code for explode mode
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 54655623a82632e7624714d7b2a3e039dc5faa7e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes deadlocks when decoding packets containing more than one of the aforementioned
slices when using frame threads.
Tested-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 70c8c8a818f39bc262565ec29fae2baffb3e1660)
Fixes: runtime error: signed integer overflow: 2147483647 + 6 cannot be represented in type 'int'
Fixes: 2263/clusterfuzz-testcase-minimized-4800359627227136
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1edbf5e20c75f06d6987bc823e63aa4e649ccddd)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bc406744620710911de9157eafa3e61d0246566f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: d41d8cd98f00b204e9800998ecf8427e/signal_sigsegv_321165b_7641_077dfcd8cbc80b1c0b470c8554cd6ffb.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 214085852491448631dcecb008b5d172c11b8892)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 2f95ddd996db8a6281d2e18c184595a7/asan_heap-oob_192fe91_3330_58e4441181e30a66c19f743dcb392347.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dad354f38ddc9bfc834bc21358a1d0ad41532ca0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array access
Fixes: 08664a2a7921ef48172f26495c7455be/asan_heap-oob_23036c6_3301_523388ef84285a0270caf67a43247b59.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d85aa76115214183e7e3b7d65e950da61474959a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 007c4a36608ebdf27ee260ad60a81184/asan_heap-oob_32076b4_2243_116b1cb29d91cc4974d6680e3d10bd91.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ef9f7bbfa47317f9d46bf46982a394d2be78503c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
At least the new videotoolbox decoder does not actually set a frame if
end_frame fails. This causes the API to return success and signals that
a picture was decoded, even though AVFrame->data[0] is NULL.
Fix this by propagating end_frame errors.
* commit '077b55943330150db0eafd36bbee614697cabd98':
hevc: handle a NULL sps in set_sps() properly
Merged-by: Michael Niedermayer <michael@niedermayer.cc>
* commit '18156b53f9b642b71c182c5c9818175a61572d2b':
hevc: do not pass an entire HEVCContext into export_stream_params()
Merged-by: Michael Niedermayer <michael@niedermayer.cc>
* commit 'd82e1adc2019135a2fc45372e0ed0b5ef107cdd0':
hevc: move splitting the packet into NALUs into a separate function
Conflicts:
libavcodec/hevc.c
libavcodec/hevc.h
libavcodec/hevc_parse.c
Merged-by: Michael Niedermayer <michael@niedermayer.cc>
* commit 'ae05b4865514fd71b5e9431e93aa0d03d7ba7751':
hevc: eliminate the second call to hls_nal_unit()
Conflicts:
libavcodec/hevc.c
Merged-by: Michael Niedermayer <michael@niedermayer.cc>
* commit 'd7bebe4805193783f0b6f292f9127a75709fb7d9':
hevc: skip invalid/ignored NALUs when splitting the packet
Merged-by: Michael Niedermayer <michael@niedermayer.cc>
Also, make hls_nal_unit() work only on the provided NAL unit, without
requiring a whole decoding context.
This will allow splitting this code for reuse by the parser.
It is used as get_bits argument and reading 0 bits doesn't make sense.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
It is used as get_bits argument and reading 0 bits isn't supported.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
* commit '69ab9f53f901eac6a649e22d28cf093357870627':
hevc: split bitstream unescaping to a separate file
Conflicts:
libavcodec/Makefile
libavcodec/hevc.c
See: afa93d198aaf2cc661c4df6d4095cd030265d30a
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit 'fd124d8357b1becfde3ac8d5e3320127cf97a5b7':
hevc_ps: split the code for parsing the SPS and exporting it into the context
Conflicts:
libavcodec/hevc.c
libavcodec/hevc_ps.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
This change introduces basic support for HEVC decoding through vdpau.
Right now, there are problems with the nvidia driver/library implementation
that mean that frames are incorrectly laid out in memory when they are
returned from the decoder, and it is normally impossible to recover the
complete decoded frame due to loss of data from alignment inconsistencies.
I obviously hope that nvidia will be fixing it in due course - I've verified
the problems exist with their example application.
As such, this support is not useful for any real world application, but I
believe that it is correct (with the caveat that the mangled frames may hide
problems) and will work properly once the nvidia problem is fixed.
Right now it appears that any file encoded by x265 or nvenc is decoded
correctly, but that's because these files don't use a bunch of HEVC
features.
Quick summary:
Features that seem to work:
1) Short Term References
2) Scaling Lists
3) Tiling
Features with known problems:
1) Long Term References
It's hard to tell what's going on here. After I read the nvidia example
app that does not set the IsLongTerm flag on LTRs, and changed my code,
a bunch of frames using LTR started to display correctly, but there
are still samples with glitches that are related to LTRs.
In terms of real world files, both x265 and nvenc only use short term
refs from this list. The divx encoder seems similar.
Signed-off-by: Philip Langdale <philipl@overt.org>
Today, we track the short term RPS size for DXVA, but only if the
SliceHeader RPS is being used. Otherwise it's left uninitialized.
NVIDIA's VDPAU implementation requires that the size be accurately
tracked even if an SPS RPS is being used. In this case, it's really
counting the size of the RPS idx information, but you end up with
mangled output if the value is not accurate.
VDPAU also needs the size of the long term RPS.
Signed-off-by: Philip Langdale <philipl@overt.org>
