1912 Commits

Author SHA1 Message Date
Michael Niedermayer
f0b6705e1a swscale/utils: Detect and skip unneeded sws_setColorspaceDetails() calls
This avoids running various table inits unnecessarily

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cc538e9dbd14b61d1ac8c9fa687d83289673fe90)

Conflicts:

	libswscale/utils.c
2016-01-15 17:23:55 +01:00
Michael Niedermayer
1ba90ce49e swscale/yuv2rgb: Increase YUV2RGB table headroom
This makes SWS more robust
Fixes: 07650a772d98aa63b0fed6370dc89037/asan_heap-oob_27ddeaf_2657_2c81ff264dee5d9712cb3251fb9c3bbb.264
Fixes: out of array read

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8f3a9a8c278acf886f70a1d743bc07b6f9c7b51a)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:23:55 +01:00
Michael Niedermayer
4bafd67795 swscale/yuv2rgb: Factor YUVRGB_TABLE_LUMA_HEADROOM out
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5e5f82a28737fba4402259617500911cc37e3674)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:23:55 +01:00
Michael Niedermayer
dbe194002a swscale/utils: Use normal bilinear scaler if fast cannot be used due to tiny dimensions
Fixes Ticket4960

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1edf129cbc897447a289ca8b045853df5df1bab3)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:23:54 +01:00
Michael Niedermayer
bb70069e6a swscale/utils: Fix intermediate format for cascaded alpha downscaling
Fixes Ticket4926

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b83d8be6bff7d645469a623aee0b380541da15cf)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:23:54 +01:00
Michael Niedermayer
0fa22a42d6 swscale/utils: Fix for runtime error: left shift of negative value -1
Fixes: c106b36fa36db8ff8f3ed0c82be7bea2/asan_heap-oob_32699f0_6321_467b9a1d7e03d7cfd310b7e65dc53bcc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 325b59368dae3c3f2f5cc39873002b4cf133ccbc)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:27 +01:00
Michael Niedermayer
26e98f9ae5 swscale/x86/rgb2rgb_template: Fallback to mmx in interleaveBytes() if the alignment is insufficient for SSE*
This also as a sideeffect fixes the non aligned case

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a066ff89bcbae6033c2ffda9271cad84f6c1b807)
2015-12-15 11:18:39 +01:00
Michael Niedermayer
45430c3753 swscale/x86/rgb2rgb_template: Do not crash on misaligend stride
Fixes Ticket5013

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 80bfce35ccd11458e97f68f417fc094c5347070c)
2015-12-15 11:18:37 +01:00
Michael Niedermayer
fb47b6cb4b swscale/utils: Clear pix buffers
Fixes use of uninitialized memory
Fixes: a96874b9466b6edc660a519c7ad47977_signal_sigsegv_7ffff713351a_744_nc_sample.avi with memlimit 2147483648

Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a5d44d5c220e12ca0cb7a4eceb0f74759cb13111)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-20 04:43:41 +02:00
Michael Niedermayer
49ad1658b2 swscale/swscale_unscaled: Fix rounding difference with RGBA output between little and big endian
Fixes fate/dds-rgb16 on big endian

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f6ab967eae497733f6adc12b30075980fd6eea98)

Conflicts:

	tests/ref/fate/dds-rgb16
2015-07-20 04:43:40 +02:00
James Almer
48127e2ad6 swscale/x86/rgb2rgb_template: fix signedness of v in shuffle_bytes_2103_{mmx,mmxext}
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit e22edbfd413242dda720dc5191fc00a51c24d74c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-20 04:43:40 +02:00
James Almer
860dc1e19b swscale/x86/rgb2rgb_template: add missing xmm clobbers
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 910eeab48026060b5f7780b2560445c069eb4d6b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-20 04:43:40 +02:00
Michael Niedermayer
e920af2cdf swscale/rgb2rgb_template: Fix signedness of v in shuffle_bytes_2103_c()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7604358018229f345dfdf88b16c8930a67984435)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-20 04:43:40 +02:00
Michael Niedermayer
12e166dacb swscale/rgb2rgb_template: Implement shuffle_bytes_0321_c and fix shuffle_bytes_2103_c on BE
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit abb833c5681b84d7025c083e2191140eaa30dca7)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-20 04:43:40 +02:00
Michael Niedermayer
80cec4e496 swscale/rgb2rgb_template: Disable shuffle_bytes_2103_c on big endian
The function is specific to little endian

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4df3cf90bf7a54793e90304bd1b6c7599673f36a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-20 04:43:40 +02:00
Nick Lewycky
a5167b4d66 libswscale/x86/hscale_fast_bilinear_simd.c: Include BX in the clobber list on x86_64, because it isn't implicitly included when PIC is on.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 48e9f68384a2af257b9ca7633bf14f0c2748edc6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-05-15 10:04:51 +02:00
Rong Yan
be1b665dec swscale/ppc/swscale_altivec.c: POWER LE support in yuv2planeX_8() delete macro GET_VF() it was wrong
GCC tool had a bug of PPC intrinsic interpret, which has been fixed in GCC 4.9.1. This bug lead to
errors in two of our previous patches. We found this when we update our GCC tools to 4.9.1 and by
reading the related info on GCC website. We fix our previous error in two separate commits

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 603c839398f89d3ef15c47530470fbb0051632c0)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-05-14 19:07:33 +02:00
Michael Niedermayer
ae20682f6b swscale: Add prefix to updateMMXDitherTables()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-27 19:23:46 +01:00
Michael Niedermayer
007498fc1a swscale/utils: clear formatConvBuffer on allocation
Fixes use of uninitialized memory
Fixes: asan_heap-oob_35ca682_1474_cov_3230122439_aletrek_tga_16bit.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-27 03:13:14 +01:00
Michael Niedermayer
fb8e5044b4 swscale/utils: add asserts to check filterpos
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-24 01:00:24 +01:00
Michael Niedermayer
1895d414aa swscale/utils: More carefully merge and clear coefficients outside the input
Fixes out of array read
Fixes: asan_heap-oob_35ca682_1474_cov_3230122439_aletrek_tga_16bit.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-24 01:00:24 +01:00
Michael Niedermayer
03bffb68f6 swscale: Use av_clip_uintp2()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-21 14:49:17 +01:00
Michael Niedermayer
b20426398c swscale/yuv2rgb: Use av_clip_uint8()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-21 14:44:47 +01:00
Michael Niedermayer
d501b986a9 swscale/bayer_template: Add () to protect the argument of BAYER_READ()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-18 01:55:15 +01:00
Michael Niedermayer
2a1b79d7e6 Merge commit 'e51f22122d23589e93ac4f0b3e570bb925755915'
* commit 'e51f22122d23589e93ac4f0b3e570bb925755915':
  swscale: Check memory allocations

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-17 22:22:17 +01:00
Federico Tomassetti
e51f22122d swscale: Check memory allocations
CC: libav-stable@libav.org
Bug-Id: CID 1267888 / CID 1267890
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
2015-02-17 12:16:43 -05:00
Michael Niedermayer
692b22626e swscale/utils: Limit filter shifting so as not to read from prior the array
Fixes out of array read
Fixes: asan_heap-oob_1fb2f9b_3780_cov_3984375136_usf.mkv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-05 00:33:50 +01:00
Michael Niedermayer
f30798584f swscale/input: fix rgba64 alpha non native
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-01-21 21:21:00 +01:00
Michael Niedermayer
95d04690aa swscale/input: Fix alpha of YA16 input
Fixes Ticket4278

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-01-21 21:21:00 +01:00
Vittorio Giovara
2df7277711 swscale: fix gbrap to gbrap alpha scaling 2015-01-16 02:47:15 +01:00
Vittorio Giovara
89df3fd49e swscale: fix warning about incompatible function pointer type 2015-01-16 02:47:10 +01:00
Michael Niedermayer
27513846f6 Merge commit '928061670e873e816daa14827853b7e11221ff5f'
* commit '928061670e873e816daa14827853b7e11221ff5f':
  libswscale: GBRAP input & output and GBRAP16 input support

Conflicts:
	libswscale/input.c
	libswscale/swscale_internal.h
	libswscale/swscale_unscaled.c
	libswscale/utils.c
	tests/ref/fate/filter-pixdesc-gbrap
	tests/ref/fate/filter-pixfmts-copy
	tests/ref/fate/filter-pixfmts-null
	tests/ref/fate/filter-pixfmts-scale
	tests/ref/fate/filter-pixfmts-vflip

See: 5c057433ccd32f37ddc828d3f82c5b1ad7d3e3db
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-01-15 05:47:25 +01:00
Paul B Mahol
928061670e libswscale: GBRAP input & output and GBRAP16 input support
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
2015-01-14 17:17:24 +01:00
Michael Niedermayer
90de28befd Merge commit '1dd797e3c9f179f957316a0becbec048b42df8aa'
* commit '1dd797e3c9f179f957316a0becbec048b42df8aa':
  swscale: check memory allocations

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-01-06 21:44:50 +01:00
Vittorio Giovara
1dd797e3c9 swscale: check memory allocations
CC: libav-stable@libav.org
Bug-Id: CID 1257779
2015-01-05 15:35:39 +01:00
Kieran Kunhya
18982f084c swscale: Pass through chroma positions in sws_getCachedContext
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-26 17:55:12 +01:00
Michael Niedermayer
3a2bed1f9e swscale/yuv2rgb: Fix width % 4 != 0 with bgr4_byte/rgb4_byte
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-24 00:33:21 +01:00
Michael Niedermayer
3848512d48 swscale/yuv2rgb: Fix width % 4 != 0 with bgr4/rgb4
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-24 00:32:53 +01:00
Michael Niedermayer
3d00ba2688 swscale/yuv2rgb: fix width % 8 != 0 and rgb/bgr8 output
Fixes Ticket2570

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-24 00:23:49 +01:00
Michael Niedermayer
2a983ff7fe swscale: increase yuv2rgb table headroom
Fixes out of array access
Fixes: case2_bad_read_yuv2rgbx32.mp4
Found-by: Michal Zalewski <lcamtuf@coredump.cx>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-16 23:37:26 +01:00
Michael Niedermayer
e827f65618 Merge commit '33c827f632f95ffe3399b695a5a0d47b366b6e20'
* commit '33c827f632f95ffe3399b695a5a0d47b366b6e20':
  swscale: Properly scale YUV

Conflicts:
	libswscale/swscale_unscaled.c
	tests/ref/fate/filter-pixdesc-yuv420p10be
	tests/ref/fate/filter-pixdesc-yuv420p10le
	tests/ref/fate/filter-pixdesc-yuv420p9be
	tests/ref/fate/filter-pixdesc-yuv420p9le
	tests/ref/fate/filter-pixdesc-yuva420p10be
	tests/ref/fate/filter-pixdesc-yuva420p10le
	tests/ref/fate/filter-pixdesc-yuva420p9be
	tests/ref/fate/filter-pixdesc-yuva420p9le
	tests/ref/fate/filter-pixfmts-copy
	tests/ref/fate/filter-pixfmts-null
	tests/ref/fate/filter-pixfmts-scale
	tests/ref/fate/filter-pixfmts-vflip

See: abe0b8e9f378a0f8781c1a3da6714d20cfd19594
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-06 01:30:08 +01:00
Derek Buitenhuis
33c827f632 swscale: Properly scale YUV
Only shift limited range luma, and always only shift chroma
for upconversion.

Based off a patch by Michael Niedermayer.

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2014-12-05 13:03:49 +00:00
Michael Niedermayer
8524558858 swscale/x86/rgb2rgb_template: fix crash with tiny size and nv12 output
Fixes Ticket4151

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-03 20:21:56 +01:00
Michael Niedermayer
4388e78a0f swscale/x86/rgb2rgb_template: handle the first 2 lines with C in rgb24toyv12_*()
This avoids out of array accesses
Should fix Ticket3451

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-20 01:20:31 +01:00
Michael Niedermayer
2f6bb86f85 swscale/utils: support bayer input + scaling, and bayer input + any supported output
Fixes Ticket4053

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-12 23:59:37 +01:00
Rong Yan
e74e14608f libswscale/ppc/swscale_altivec.c : fix hScale_altivec_real() yuv2planeX_16_altivec() yuv2planeX_8() for little endian
add marcos GET_LS() GET_VF() LOAD_FILTER() LOAD_L1() GET_VF4() FIRST_LOAD() UPDATE_PTR() LOAD_SRCV() LOAD_SRCV8() GET_VFD() for POWER LE

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-12 15:15:03 +01:00
Kieran Kunhya
b546023b93 swscale: fix yuv2yuvX_8 assembly on x86
use_mmx_vfilter check/fix by commiter
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-09 23:52:17 +01:00
Michael Niedermayer
fa040c2247 Merge commit 'fc1eda543c6ef043300612db90da5cfd972af650'
* commit 'fc1eda543c6ef043300612db90da5cfd972af650':
  swscale: fix sign extensions in yuv planar conversion

Conflicts:
	libswscale/rgb2rgb_template.c

See: a07e9d72a1d97dc6a4d57e1f7b708e54e7db83f5
See: a30972609ca39b791ce1e4e5cc6c3dd6cb9c9b12
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-18 23:40:50 +02:00
Vittorio Giovara
fc1eda543c swscale: fix sign extensions in yuv planar conversion
Casting the left-most byte to unsigned avoids an undefined
result of the shift by 24 if bit 7 is set.

yuvPlanartouyvy_c and yuvPlanartoyuy2_c are affected.

CC: libav-stable@libav.org
Bug-Id: CID 732281 / CID 732282
2014-10-18 16:15:10 +01:00
Michael Niedermayer
fba894615d swscale: support internal scaler cascades
Fixes Ticket3170

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-11 05:05:17 +02:00