174 Commits

Author SHA1 Message Date
Michael Niedermayer
b60895a13b avcodec/hevc: Fix typo in num_entry_point_offsets check
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3051e7fa712dfe2136f19b7157211453895f2a3c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-01 23:25:21 +02:00
wm4
fea9ed39e6 hevc: make avcodec_decode_video2() fail if get_format() fails
Personally, I need the decoder to back out if get_format() returns no
usable pixel format. This didn't work because the error code was not
propagated down the call chain. This in turn happened because the
variable declaration removed in this patch shadowed the variable, whose
value is returned at the end of the function. Consequently, failures of
decode_nal_unit() were ignored in this place.

Reviewed-by:  Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cc5e4bb48476a89cc8ce0c41bc2bd2e8fda9b37c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-01 23:25:21 +02:00
Michael Niedermayer
57b7a009c0 avcodec/hevc: Check num_entry_point_offsets
Fixes CID1239099 part 2

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1c6ae98d4a9ff9ea607df87908393eda4ebdf4e8)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-01 23:25:20 +02:00
Michael Niedermayer
f1b0d65237 avcodec/hevc: Check offset_len
Fixes CID1239099 part 1

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3e9d5e16ad9799f6b6faae4f21120d23146b84c9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-01 23:25:20 +02:00
Michael Niedermayer
43924a8e99 avcodec/hevc: Fix handling of skipped_bytes() reallocation failures
Fixes CID1260704

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e172f5e53ae4dbbcdcf81c9a3b962dc9f5a8a98d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-13 20:48:09 +01:00
Michael Niedermayer
02d979d007 Merge commit '88411b87b4bb3c5820ec232f26ba4a284c11a7f9' into release/2.4
* commit '88411b87b4bb3c5820ec232f26ba4a284c11a7f9':
  display: fix order of operands

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-01-18 03:22:15 +01:00
Michael Niedermayer
9d43e51eec Merge commit '20d6ae4626a42fbc6e7b9c0040bdd64397b23e11' into release/2.4
* commit '20d6ae4626a42fbc6e7b9c0040bdd64397b23e11':
  hevc: always clip luma_log2_weight_denom

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-01-17 23:56:07 +01:00
Vittorio Giovara
88411b87b4 display: fix order of operands
CC: libav-stable@libav.org
Bug-Id: CID 1238828 / CID 1238832
(cherry picked from commit b1b1a7370e141c912e3d0bbaa668dcee05c3ad67)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2015-01-13 00:16:42 +01:00
Vittorio Giovara
20d6ae4626 hevc: always clip luma_log2_weight_denom
Its value shall be between 0 and 7 according to the specifications.

CC: libav-stable@libav.org
Bug-Id: CID 1257502
2015-01-12 23:16:05 +01:00
Michael Niedermayer
de882ec2b1 avcodec/hevc: clear filter_slice_edges() on allocation
This avoids use of uninitialized memory
Fixes: asan_static-oob_17aa046_582_cov_212287884_DBLK_G_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8aa8d12554868c32436750f881954193087219c8)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-17 20:02:06 +01:00
Michael Niedermayer
1d99adc953 avcodec/hevc: fix chroma transform_add size
Fixes accessing misaligned address
Fixes: signal_sigsegv_1feb99c_10_signal_sigsegv_2d1d35c_79_cov_691940146_EXT_A_ericsson_3.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit df8394c693d3c1e13b7ebf6af01c5e55321cf952)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 23:52:29 +02:00
Mickaël Raulet
684d0a0b23 avcodec/hevc: fix dead code
fix CID 1231985

cherry picked from commit 745a35a777eaa5f77b8660b44098110a29916aae
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-06 20:36:27 +02:00
Christophe Gisquet
dad7f15567 hevcdsp: remove more instances of compile-time-fixed parameters
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-22 15:22:42 +02:00
Christophe Gisquet
d4f44b66d3 hevcdsp: remove compilation-time-fixed parameter
The dststride parameter is always MAX_PB_SIZE.

Reviewed-by: Mickaël Raulet <mraulet@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-22 14:57:37 +02:00
Michael Niedermayer
2278146a0c Merge commit 'd1b1c3bb5ef1bee409c06c3bcacfb8674e1cb574'
* commit 'd1b1c3bb5ef1bee409c06c3bcacfb8674e1cb574':
  hevc: reorder loops

Conflicts:
	libavcodec/hevc.c

See: eca1957c4cbee66a0b1f3dcabaffe68d61885f16
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-12 15:59:33 +02:00
Christophe Gisquet
cf6090dc62 hevc: use intreadwrite
When dealing with MVs, both components may be processed at a time.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-08-12 10:03:20 +00:00
Christophe Gisquet
d1b1c3bb5e hevc: reorder loops
iterate over memory in a more continuous order

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-08-12 09:57:05 +00:00
Christophe Gisquet
5ec85c9750 hevc: do generic validation of bitstream
After finishing parsing VPS/SPS/PPS/slice header, check remaining bits,
and if an overconsumption occurred, report invalid data.

Liked-by: BBB
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-11 02:47:05 +02:00
Ronald S. Bultje
bfffce4d08 hevc: add missing comma in log message.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-11 02:27:27 +02:00
Christophe Gisquet
7117547298 hevc: fix incorrect sao buffer size
It previously used the output, cropped size, causing overreads/writes.

Fixes ticket #3839.

This issue was introduced by d249e682, which is not part of any release

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-10 12:34:57 +02:00
Michael Niedermayer
3f2495d98c Merge commit '70211539a39ca3854f8a9e97d51dc27caa079943'
* commit '70211539a39ca3854f8a9e97d51dc27caa079943':
  hevc: deobfuscate slice/tile boundary handling for DBF

Conflicts:
	libavcodec/hevc.c
	libavcodec/hevc_filter.c

See: a5c621aa852522c79146035b2db4b89d2e096d3c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-09 21:01:55 +02:00
Anton Khirnov
70211539a3 hevc: deobfuscate slice/tile boundary handling for DBF
Use named constants instead of magic numbers, avoid using variables with
inverse meaning from what their name implies.
2014-08-09 16:13:48 +00:00
Anton Khirnov
4d1ff2a489 hevc: calculate the dbf strength in hls_pcm_sample() only if dbf is enabled 2014-08-09 16:13:11 +00:00
Michael Niedermayer
3fa9692ae2 avcodec/hevc: move HEVCLocalContext declaration into loop
Found-by: CSA
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 05:15:57 +02:00
Michael Niedermayer
243236a6f5 avcodec/hevc: fix "discards const qualifier from pointer target type" warning
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-28 14:34:47 +02:00
Michael Niedermayer
772dfd5f6e avcodec/hevc: add some const to cbf arrays
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-28 14:28:56 +02:00
Mickaël Raulet
42ffa226f9 hevc: clean up non relevant TODO
cherry picked from commit a2ef95f043d89d05482af19460c9e9afdadec685

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-28 14:27:04 +02:00
Anton Khirnov
a5c621aa85 hevc: rename variable in boundary strength to b more explicit
Signed-off-by: Mickaël Raulet <mraulet@insa-rennes.fr>

cherry picked from commit 348bebedc0012aae201419669fca1eb61ec93ca6
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-28 14:20:19 +02:00
Michael Niedermayer
e0492311c8 Merge commit 'e76f2d11970484266e67a12961f2339a5c2fccf9'
* commit 'e76f2d11970484266e67a12961f2339a5c2fccf9':
  hevc: eliminate the last element from TransformTree

Conflicts:
	libavcodec/hevc.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-28 14:02:03 +02:00
Michael Niedermayer
4a73fa19ca Merge commit '4aa80808bcc2a30fcd7ce5b38594319df3a85b36'
* commit '4aa80808bcc2a30fcd7ce5b38594319df3a85b36':
  hevc: eliminate unnecessary cbf_c{b,r} arrays

Conflicts:
	libavcodec/hevc.c

See: 77ef9fd1e938010600620c1d32cabd15e7d5b853
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-28 13:54:51 +02:00
Michael Niedermayer
0a1ffc5788 Merge commit '0daa2554636ba1d31f3162ffb86991e84eb938a8'
* commit '0daa2554636ba1d31f3162ffb86991e84eb938a8':
  hevc: do not store the transform inter_split flag in the context

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-28 13:09:28 +02:00
Michael Niedermayer
2fb8aa9b10 Merge commit '53a11135f2fb2123408b295f9aaae3d6f861aea5'
* commit '53a11135f2fb2123408b295f9aaae3d6f861aea5':
  hevc: simplify splitting the transform tree blocks

Conflicts:
	libavcodec/hevc.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-28 13:03:59 +02:00
Michael Niedermayer
453224f10b Merge commit 'e36a2f4c5280e2779b0e88974295a711cf8d88be'
* commit 'e36a2f4c5280e2779b0e88974295a711cf8d88be':
  hevc: eliminate an unnecessary array

Conflicts:
	libavcodec/hevc.c

See: 255086a7e06417d98417cea192053b8a8531eb24
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-28 12:54:14 +02:00
Anton Khirnov
e76f2d1197 hevc: eliminate the last element from TransformTree
Replace it by passing an additional parameter to transform_unit()
2014-07-28 08:10:35 +00:00
Anton Khirnov
4aa80808bc hevc: eliminate unnecessary cbf_c{b,r} arrays
They are replaced by passing additional parameters to the transform
functions.
2014-07-28 08:09:18 +00:00
Anton Khirnov
0daa255463 hevc: do not store the transform inter_split flag in the context
It does not need to be preserved.
2014-07-28 08:05:47 +00:00
Anton Khirnov
53a11135f2 hevc: simplify splitting the transform tree blocks 2014-07-28 08:04:19 +00:00
Anton Khirnov
e36a2f4c52 hevc: eliminate an unnecessary array
We do not need to store the value of the split flag.
2014-07-28 08:03:53 +00:00
Anton Khirnov
77ef9fd1e9 hevc: eliminate unnecessary cbf_c{b,r} arrays
- They are be replaced by passing additional parameters to the transform
functions.
- Adaptation to 4:2:2

Signed-off-by: Mickaël Raulet <mraulet@insa-rennes.fr>
cherry picked from commit f518bb22531c648f1c37f978b0c7ad2e71e04c25
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-28 00:55:44 +02:00
Mickaël Raulet
772f7f4edd hevc: fix skip_flag
cherry picked from commit 509ae456551005b9bf9b4d9730b6247c97fdd990

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-28 00:47:28 +02:00
Mickaël Raulet
3b777db132 hevc: remove non necessary parameters to ff_hevc_set_qpy
cherry picked from commit 6f58c111ad9920d983bb18eacf901193bac5d937

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-28 00:39:13 +02:00
Mickaël Raulet
23480da0aa hevc: add support for bumping process
cherry picked from commit 8aa2fb7df3cffc67a3fd03a3a7eb49dbed4094c7

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-26 15:40:34 +02:00
Mickaël Raulet
f6e218a02d hevc: fix RAP_B_Bossen
cherry picked from commit 903236292f066bc321080e3e2192b64f8943d960

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-26 02:01:04 +02:00
Timothy Gu
c508adb223 hevc: propagate error code from set_sps()
Signed-off-by: Timothy Gu <timothygu99@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-24 16:25:55 +02:00
Mickaël Raulet
fa0506e413 hevc/rext: update boundary strength for chroma deblocking filter
cherry picked from commit 7d05c95ac5a63d7675bf645e74b4cf1fffff4796

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-22 13:00:14 +02:00
Mickaël Raulet
950a4f8326 hevc: fix offset for sao temporary frame
fixes fate on alpha

cherry picked from commit 0a8ce1cbdaff7cd026bcf95ee3787bc7b4c063ee

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-20 02:48:53 +02:00
Christophe Gisquet
7a4a5515b0 hevc: use intreadwrite
When dealing with MVs, both components may be processed at a time.

On Win64, 560 to 539 cycles for derive_spatial_merge_candidates.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-19 15:27:08 +02:00
Michael Niedermayer
ffa0b510cd avcodec/hevc: fix pict_type
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-18 22:59:03 +02:00
Mickaël Raulet
d249e6828e hevc/sao: optimze sao implementation
- adding one extra pixel all around the frame
- do not copy when SAO is not applied

5% improvement

cherry picked from commit 10fc29fc19a12c4d8168fbe1a954b76386db12d0
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-18 22:46:50 +02:00
Michael Niedermayer
cf92cc8751 avcodec/hevc: clear HEVClcList[i] on allocation
Fixes fate failure with --enable-memory-poisoning && make THREAD_TYPE=slice THREADS=7  fate-hevc-conformance-ENTP_C_Qualcomm_1

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-16 01:34:57 +02:00