Merge pull request #510 from underlines/feature/color_space_conversion

Feature/color space conversion

README.md

@@ -1,3 +1,4 @@
+
 ![demo-gif](demo.gif)
 
 
@@ -173,6 +174,147 @@ options:
 
 Looking for a CLI mode? Using the -s/--source argument will make the run program in cli mode.
 
+### Webcam mode on Windows 11 using WSL2 Ubuntu
+If you want to use WSL2 on Windows 11 you will notice, that Ubuntu WSL2 doesn't come with USB-Webcam support in the Kernel. You need to do two things: Compile the Kernel with the right modules integrated and forward your USB Webcam from Windows to Ubuntu with the usbipd app. Here are detailed Steps:
+
+This tutorial will guide you through the process of setting up WSL2 Ubuntu with USB webcam support, rebuilding the kernel, and preparing the environment for the Deep-Live-Cam project.  
+  
+#### 1. Install WSL2 Ubuntu  
+  
+Install WSL2 Ubuntu from the Microsoft Store or using PowerShell:  
+  
+#### 2. Enable USB Support in WSL2  
+  
+1. Install the USB/IP tool for Windows:  
+[https://learn.microsoft.com/en-us/windows/wsl/connect-usb](https://learn.microsoft.com/en-us/windows/wsl/connect-usb)  
+  
+2. In Windows PowerShell (as Administrator), connect your webcam to WSL:  
+  
+```powershell  
+usbipd list  
+usbipd bind --busid x-x # Replace x-x with your webcam's bus ID  
+usbipd attach --wsl --busid x-x # Replace x-x with your webcam's bus ID  
+```  
+ You need to redo the above every time you reboot wsl or re-connect your webcam/usb device.
+ 
+#### 3. Rebuild WSL2 Ubuntu Kernel with USB and Webcam Modules  
+  
+Follow these steps to rebuild the kernel:  
+  
+1. Start with this guide: [https://github.com/PINTO0309/wsl2_linux_kernel_usbcam_enable_conf](https://github.com/PINTO0309/wsl2_linux_kernel_usbcam_enable_conf)  
+  
+2. When you reach the `sudo wget [github.com](http://github.com/)...PINTO0309` step, which won't work for newer kernel versions, follow this video instead or alternatively follow the video tutorial from the beginning: 
+[https://www.youtube.com/watch?v=t_YnACEPmrM](https://www.youtube.com/watch?v=t_YnACEPmrM)  
+  
+Additional info: [https://askubuntu.com/questions/1413377/camera-not-working-in-cheese-in-wsl2](https://askubuntu.com/questions/1413377/camera-not-working-in-cheese-in-wsl2)  
+  
+3. After rebuilding, restart WSL with the new kernel. 
+  
+#### 4. Set Up Deep-Live-Cam Project  
+ Within Ubuntu:
+1. Clone the repository:  
+  
+```bash  
+git clone [https://github.com/hacksider/Deep-Live-Cam](https://github.com/hacksider/Deep-Live-Cam)  
+```  
+  
+2. Follow the installation instructions in the repository, including cuda toolkit 11.8, make 100% sure it's not cuda toolkit 12.x.  
+   
+#### 5. Verify and Load Kernel Modules  
+  
+1. Check if USB and webcam modules are built into the kernel:  
+  
+```bash  
+zcat /proc/config.gz | grep -i "CONFIG_USB_VIDEO_CLASS"  
+```  
+  
+2. If modules are loadable (m), not built-in (y), check if the file exists:  
+  
+```bash  
+ls /lib/modules/$(uname -r)/kernel/drivers/media/usb/uvc/  
+```  
+  
+3. Load the module and check for errors (optional if built-in):  
+  
+```bash  
+sudo modprobe uvcvideo  
+dmesg | tail  
+```  
+  
+4. Verify video devices:  
+  
+```bash  
+sudo ls -al /dev/video*  
+```  
+  
+#### 6. Set Up Permissions  
+  
+1. Add user to video group and set permissions:  
+  
+```bash  
+sudo usermod -a -G video $USER  
+sudo chgrp video /dev/video0 /dev/video1  
+sudo chmod 660 /dev/video0 /dev/video1  
+```  
+  
+2. Create a udev rule for permanent permissions:  
+  
+```bash  
+sudo nano /etc/udev/rules.d/81-webcam.rules  
+```  
+  
+Add this content:  
+  
+```  
+KERNEL=="video[0-9]*", GROUP="video", MODE="0660"  
+```  
+  
+3. Reload udev rules:  
+  
+```bash  
+sudo udevadm control --reload-rules && sudo udevadm trigger  
+```  
+  
+4. Log out and log back into your WSL session.  
+  
+5. Start Deep-Live-Cam with `python run.py --execution-provider cuda --max-memory 8` where 8 can be changed to the number of GB VRAM of your GPU has, minus 1-2GB. If you have a RTX3080 with 10GB I suggest adding 8GB. Leave some left for Windows.
+
+#### Final Notes  
+  
+- Steps 6 and 7 may be optional if the modules are built into the kernel and permissions are already set correctly.  
+- Always ensure you're using compatible versions of CUDA, ONNX, and other dependencies.  
+- If issues persist, consider checking the Deep-Live-Cam project's specific requirements and troubleshooting steps.  
+  
+By following these steps, you should have a WSL2 Ubuntu environment with USB webcam support ready for the Deep-Live-Cam project. If you encounter any issues, refer back to the specific error messages and troubleshooting steps provided.
+
+#### Troubleshooting CUDA Issues  
+
+If you encounter this error:  
+  
+```  
+[ONNXRuntimeError] : 1 : FAIL : Failed to load library [libonnxruntime_providers_cuda.so](http://libonnxruntime_providers_cuda.so/) with error: libcufft.so.10: cannot open shared object file: No such file or directory  
+```  
+  
+Follow these steps:  
+  
+1. Install CUDA Toolkit 11.8 (ONNX 1.16.3 requires CUDA 11.x, not 12.x):  
+[https://developer.nvidia.com/cuda-11-8-0-download-archive](https://developer.nvidia.com/cuda-11-8-0-download-archive)  
+  select: Linux, x86_64, WSL-Ubuntu, 2.0, deb (local)
+2. Check CUDA version:  
+  
+```bash  
+/usr/local/cuda/bin/nvcc --version  
+```  
+  
+3. If the wrong version is installed, remove it completely:  
+[https://askubuntu.com/questions/530043/removing-nvidia-cuda-toolkit-and-installing-new-one](https://askubuntu.com/questions/530043/removing-nvidia-cuda-toolkit-and-installing-new-one)  
+  
+4. Install CUDA Toolkit 11.8 again [https://developer.nvidia.com/cuda-11-8-0-download-archive](https://developer.nvidia.com/cuda-11-8-0-download-archive), select: Linux, x86_64, WSL-Ubuntu, 2.0, deb (local)
+  
+```bash  
+sudo apt-get -y install cuda-toolkit-11-8  
+```  
+
 ## Want the Next Update Now?
 If you want the latest and greatest build, or want to see some new great features, go to our [experimental branch](https://github.com/hacksider/Deep-Live-Cam/tree/experimental) and experience what the contributors have given.
 

modules/capturer.py

@@ -1,16 +1,28 @@
 from typing import Any
 import cv2
+import modules.globals  # Import the globals to check the color correction toggle
 
 
 def get_video_frame(video_path: str, frame_number: int = 0) -> Any:
     capture = cv2.VideoCapture(video_path)
+
+    # Set MJPEG format to ensure correct color space handling
+    capture.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc(*'MJPG'))
+    
+    # Only force RGB conversion if color correction is enabled
+    if modules.globals.color_correction:
+        capture.set(cv2.CAP_PROP_CONVERT_RGB, 1)
+    
     frame_total = capture.get(cv2.CAP_PROP_FRAME_COUNT)
     capture.set(cv2.CAP_PROP_POS_FRAMES, min(frame_total, frame_number - 1))
     has_frame, frame = capture.read()
+
+    if has_frame and modules.globals.color_correction:
+        # Convert the frame color if necessary
+        frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
+
     capture.release()
-    if has_frame:
-        return frame
-    return None
+    return frame if has_frame else None
 
 
 def get_video_frame_total(video_path: str) -> int:

modules/globals.py

@@ -17,6 +17,7 @@ keep_fps = None
 keep_audio = None
 keep_frames = None
 many_faces = None
+color_correction = None  # New global variable for color correction toggle
 nsfw_filter = None
 video_encoder = None
 video_quality = None

modules/predicter.py

@@ -1,6 +1,8 @@
 import numpy
 import opennsfw2
 from PIL import Image
+import cv2  # Add OpenCV import
+import modules.globals  # Import globals to access the color correction toggle
 
 from modules.typing import Frame
 
@@ -10,10 +12,16 @@ MAX_PROBABILITY = 0.85
 model = None
 
 def predict_frame(target_frame: Frame) -> bool:
+    # Convert the frame to RGB before processing if color correction is enabled
+    if modules.globals.color_correction:
+        target_frame = cv2.cvtColor(target_frame, cv2.COLOR_BGR2RGB)
+        
     image = Image.fromarray(target_frame)
     image = opennsfw2.preprocess_image(image, opennsfw2.Preprocessing.YAHOO)
     global model
-    if model is None: model = opennsfw2.make_open_nsfw_model()
+    if model is None: 
+        model = opennsfw2.make_open_nsfw_model()
+        
     views = numpy.expand_dims(image, axis=0)
     _, probability = model.predict(views)[0]
     return probability > MAX_PROBABILITY

modules/processors/frame/face_swapper.py

@@ -49,6 +49,10 @@ def swap_face(source_face: Face, target_face: Face, temp_frame: Frame) -> Frame:
 
 
 def process_frame(source_face: Face, temp_frame: Frame) -> Frame:
+    # Ensure the frame is in RGB format if color correction is enabled
+    if modules.globals.color_correction:
+        temp_frame = cv2.cvtColor(temp_frame, cv2.COLOR_BGR2RGB)
+        
     if modules.globals.many_faces:
         many_faces = get_many_faces(temp_frame)
         if many_faces:

modules/ui.py

@@ -93,6 +93,11 @@ def create_root(start: Callable[[], None], destroy: Callable[[], None]) -> ctk.C
     many_faces_switch = ctk.CTkSwitch(root, text='Many faces', variable=many_faces_value, cursor='hand2', command=lambda: setattr(modules.globals, 'many_faces', many_faces_value.get()))
     many_faces_switch.place(relx=0.6, rely=0.65)
 
+    # Add color correction toggle button
+    color_correction_value = ctk.BooleanVar(value=modules.globals.color_correction)
+    color_correction_switch = ctk.CTkSwitch(root, text='Fix Blueish Cam\n(force cv2 to use RGB instead of BGR)', variable=color_correction_value, cursor='hand2', command=lambda: setattr(modules.globals, 'color_correction', color_correction_value.get()))
+    color_correction_switch.place(relx=0.6, rely=0.70)
+
 #    nsfw_value = ctk.BooleanVar(value=modules.globals.nsfw_filter)
 #    nsfw_switch = ctk.CTkSwitch(root, text='NSFW filter', variable=nsfw_value, cursor='hand2', command=lambda: setattr(modules.globals, 'nsfw_filter', nsfw_value.get()))
 #    nsfw_switch.place(relx=0.6, rely=0.7)