{ "cells": [ { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "view-in-github" }, "source": [ "\"Open" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "663IVxfrpIAb" }, "source": [ "#◢ [ DeOldify-video\n", "\n", "##This colbab notebook colorizes video in four steps\n", "1. Upload source media or specify media URL - YouTube, Twitter, MySpace, etc.\n", "2. Extract single images from media\n", "3. Process images with [DeOldify](https://github.com/jantic/DeOldify) \n", "4. Rebuild the video from **colorized** images\n", "\n", "I'm on twitter [@tradica](https://twitter.com/tradica)\n", "\n", "\n", "---\n", "\n", "\n", "Thanks [@citnaj](https://twitter.com/citnaj) for creating DeOldify and thanks to Matt Robinson for his [notebook](https://colab.research.google.com/github/jantic/DeOldify/blob/master/DeOldify_colab.ipynb). It helped make DeOldify approachable.\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "ZjPqTBNoohK9" }, "source": [ "\n", "\n", "---\n", "\n", "\n", "#◢ [ Set Runtime type to Python 3/GPU\n", "In the Runtime menu above be sure:\n", "* Runtime Type = Python 3\n", "* Hardware Accelerator = GPU **<-------------- IMPORTANT **\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": {}, "colab_type": "code", "id": "00_GcC_trpdE" }, "outputs": [], "source": [ "from os import path\n", "import torch\n", "print(torch.__version__)\n", "print(torch.cuda.is_available())" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "gaEJBGDlptEo" }, "source": [ "#◢ [ Git clone and install DeOldify" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": {}, "colab_type": "code", "id": "-T-svuHytJ-8" }, "outputs": [], "source": [ "!git clone -b FastAIv1 --single-branch https://github.com/jantic/DeOldify.git DeOldify\n", "#!git clone https://github.com/jantic/DeOldify.git DeOldify\n", "!cd DeOldify" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "BDFjbNxaadNJ" }, "source": [ "#◢ [ Setup" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": {}, "colab_type": "code", "id": "Lsx7xCXNSVt6" }, "outputs": [], "source": [ "!pip install PyDrive\n", "!pip install ffmpeg-python\n", "!pip install youtube-dl\n", "!pip install tensorboardX" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": {}, "colab_type": "code", "id": "MsJa69CMwj3l" }, "outputs": [], "source": [ "import os\n", "from pydrive.auth import GoogleAuth\n", "from pydrive.drive import GoogleDrive\n", "from google.colab import auth\n", "from oauth2client.client import GoogleCredentials\n", "from google.colab import drive\n", "from IPython.display import Image\n", "import fastai\n", "from fastai import *\n", "from fastai.vision import *\n", "from fastai.callbacks import *\n", "from fastai.vision.gan import *\n", "from fasterai.dataset import *\n", "from fasterai.visualize import *\n", "from fasterai.tensorboard import *\n", "from fasterai.loss import *\n", "from fasterai.filters import *\n", "from fasterai.generators import *\n", "from pathlib import Path\n", "from itertools import repeat\n", "from google.colab import drive\n", "from google.colab import files\n", "torch.backends.cudnn.benchmark=True" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!mkdir \"models\"" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": {}, "colab_type": "code", "id": "uqJcwLG80fVe" }, "outputs": [], "source": [ "!wget https://www.dropbox.com/s/1ReNu8QCgi4vfyDovgYGMs7tlUGrnz72V/colorize_gen.pth -O ./models/colorize_gen.pth" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": {}, "colab_type": "code", "id": "ye5zoAYS2q2q" }, "outputs": [], "source": [ "!mkdir /content/WORKFOLDER\n", "!mkdir /content/WORKFOLDER/monochromatics\n", "!mkdir /content/WORKFOLDER/colorized\n", "!mkdir /content/WORKFOLDER/ANSWER\n", "\n", "import ffmpeg\n", "\n", "# COLAB PROGRESS BAR\n", "from IPython.display import HTML, display\n", "\n", "def progress(value, max=100):\n", " return HTML(\"\"\"\n", " \n", " {value}\n", " \n", " \"\"\".format(value=value, max=max))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "results_dir=Path('/content/WORKFOLDER/colorized')\n", "\n", "#Adjust this if image doesn't look quite right (max 64 on 11GB GPU). The default here works for most photos. \n", "#It literally just is a number multiplied by 16 to get the square render resolution. \n", "#Note that this doesn't affect the resolution of the final output- the output is the same resolution as the input.\n", "#Example: render_factor=21 => color is rendered at 16x21 = 336x336 px. \n", "render_factor=21" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": {}, "colab_type": "code", "id": "tzHVnegp21hC" }, "outputs": [], "source": [ "vis = get_colorize_visualizer(results_dir=results_dir, render_factor=render_factor)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": {}, "colab_type": "code", "id": "UQWKTCA43DYZ" }, "outputs": [], "source": [ "%cd \"/content/WORKFOLDER\"\n", "media_file = \"/content/WORKFOLDER/media_file\"\n", "\n", "def ugly_get_fps():\n", " media_data = ffmpeg.probe(media_file)\n", " stream_data = media_data['streams']\n", " stream_data_zero = str(stream_data).rsplit('avg_frame_rate\\': \\'', 1)[1]\n", " fps_nums=stream_data_zero.rsplit(\"'\")[0]\n", " fps_list = fps_nums.rsplit(\"/\")\n", " return int(fps_list[0])/int(fps_list[1])" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "HhjJhmq8ptZJ" }, "source": [ "---\n", "#◢ [ 1 ] Upload Source Video or Specify Media URL" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "WkY6IysOc02E" }, "source": [ "###◢ [ Upload" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": {}, "colab_type": "code", "id": "qmGDdYOK3WqT" }, "outputs": [], "source": [ "#Run this to make an upload widget appear\n", "source_media = files.upload()\n", "\n", "os.system('ln -s /content/WORKFOLDER/' + list(source_media.keys())[0] + ' /content/WORKFOLDER/media_file')\n", "source_media_name = list(source_media.keys())[0]" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "z5rSDjZbTntY" }, "source": [ "###◢ [ Specify URL\n", "YouTube, Imgur, Twitter, MySpace, Reddit ... most work" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": {}, "colab_type": "code", "id": "TkGgyI9rR4P3" }, "outputs": [], "source": [ "#@title Paste media URL or leave blank if uploading\n", "media_url = '' #@param {type:\"string\"}\n", "source_media_name = media_url.rsplit('/', 1)[-1]\n", "os.system('youtube-dl ' + media_url + ' -o ' + source_media_name)\n", "os.system('ln -s /content/WORKFOLDER/' + str(source_media_name) + ' /content/WORKFOLDER/media_file')" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "PviCbt7fptco" }, "source": [ "#◢ [ 2 ] Extract Images from Source Media" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": {}, "colab_type": "code", "id": "_6e9pwxw7Ufq" }, "outputs": [], "source": [ "os.system('ffmpeg -i media_file -qscale:v 2 /content/WORKFOLDER/monochromatics/%5d.jpg')\n", "framecount = len(os.listdir('/content/WORKFOLDER/monochromatics'))" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "sUQrbSYipiJn" }, "source": [ "#◢ [ 3 ] DeOldify / Colorize" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": {}, "colab_type": "code", "id": "6ZGtxdrnBAgO" }, "outputs": [], "source": [ "render_factor = 5 #@param {type: \"slider\", min: 5, max: 42}\n", "#progress bar\n", "prog = 0\n", "out = display(progress(0, 100), display_id=True)\n", "for img in os.listdir(\"/content/WORKFOLDER/monochromatics\"):\n", " img_path = str(\"/content/WORKFOLDER/monochromatics/\") + img\n", " if os.path.isfile(img_path):\n", " qqq = vis.get_transformed_image(img_path, render_factor)\n", " qqq.save(\"/content/WORKFOLDER/colorized/\" + img)\n", " prog += 1\n", " out.update(progress(prog, framecount))" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "nFI8LVP2B-jE" }, "source": [ "#◢ [ 4 ] Build Video" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": {}, "colab_type": "code", "id": "csaF_PHzB9mM" }, "outputs": [], "source": [ "answer_file = '/content/WORKFOLDER/ANSWER/' + source_media_name + '-RF' + str(render_factor) + 'FR' + str(round(ugly_get_fps(), 2)) + '.mp4'\n", "os.system('ffmpeg -f image2 -framerate ' + str(ugly_get_fps()) + ' -i /content/WORKFOLDER/colorized/%05d.jpg -c:v libx264 -crf 0 ' + answer_file)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "A5WMS_GgP4fm" }, "source": [ "###◢ [ Download\n", "* In the Menu on the left, click **Files**\n", "* It's in /content/WORKFOLDER/ANSWER/\n", "* ( Or use this with its zero feedback :) )" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": {}, "colab_type": "code", "id": "-PpnqBCUPw-Z" }, "outputs": [], "source": [ "#Find your video in Files > /content/WORKFOLDER/ANSWER/ (It's better than this)\n", "files.download(answer_file)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "s-6-z0oKjU-l" }, "source": [ "###◢ [ Build 50/50 Split Screen original / DeOldified video\n", "(need to complete step four)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": {}, "colab_type": "code", "id": "s7yfhaR3fTGx" }, "outputs": [], "source": [ "os.system('ffmpeg -i ' + str(media_file) + ' -i ' + str(answer_file) + ' -filter_complex \"[0]crop=iw/2:ih:0:0, pad=iw*2:ih[left]; [1]crop=iw/2:ih:iw/2:0[right]; [left][right]overlay=w\" -strict -2 /content/WORKFOLDER/ANSWER/split-RF' + str(render_factor) + '.mp4')" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "VqIrDA2bDMPv" }, "source": [ "---\n", "#⚙ [ Delete Workfiles" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": {}, "colab_type": "code", "id": "btv1Az5ODNjl" }, "outputs": [], "source": [ "# Delete extracted\n", "!rm /content/WORKFOLDER/monochromatics/*jpg" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": {}, "colab_type": "code", "id": "6XFxW2vfDNm7" }, "outputs": [], "source": [ "# Delete Colorized\n", "!rm /content/WORKFOLDER/colorized/*jpg" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": {}, "colab_type": "code", "id": "h5oY6FjXDNq5" }, "outputs": [], "source": [ "!rm /content/WORKFOLDER/media_file" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 34 }, "colab_type": "code", "executionInfo": { "elapsed": 768, "status": "ok", "timestamp": 1548878396597, "user": { "displayName": "Robert Bell", "photoUrl": "https://lh6.googleusercontent.com/-HcTrynSPUPc/AAAAAAAAAAI/AAAAAAAAbuk/mDAP2a19CmQ/s64/photo.jpg", "userId": "08010292799749788749" }, "user_tz": 420 }, "id": "YW7rAXBlYDNk", "outputId": "94584eea-8288-47a6-9edf-0198d6a2e62a" }, "outputs": [ { "data": { "text/plain": [ "0" ] }, "execution_count": 21, "metadata": { "tags": [] }, "output_type": "execute_result" } ], "source": [ "# Delete Source Media \n", "os.system('rm /content/WORKFOLDER/' + source_media_name)" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 34 }, "colab_type": "code", "executionInfo": { "elapsed": 1139, "status": "ok", "timestamp": 1548878397566, "user": { "displayName": "Robert Bell", "photoUrl": "https://lh6.googleusercontent.com/-HcTrynSPUPc/AAAAAAAAAAI/AAAAAAAAbuk/mDAP2a19CmQ/s64/photo.jpg", "userId": "08010292799749788749" }, "user_tz": 420 }, "id": "6bU6Ov9aYes6", "outputId": "e8634a16-a579-4fc1-a305-b6b7b9ebf88f" }, "outputs": [ { "data": { "text/plain": [ "0" ] }, "execution_count": 22, "metadata": { "tags": [] }, "output_type": "execute_result" } ], "source": [ "# Delete Answer\n", "os.system('rm ' + answer_file)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "X7Ycv_Y9xAHp" }, "source": [ "---\n", "#⚙ [ \n", "* [/r/Nickelodeons/](https://www.reddit.com/r/Nickelodeons/)\n", "* https://twitter.com/silentmoviegifs " ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "3CUeJf2XF-1X" }, "source": [ "https://twitter.com/silentmoviegifs/status/1087282910288363522\n", "If you click the date at the top of a tweet you can copy the URL from the browser" ] } ], "metadata": { "accelerator": "GPU", "colab": { "collapsed_sections": [], "name": "DeOldify-video.ipynb", "provenance": [], "toc_visible": true, "version": "0.3.2" }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.0" } }, "nbformat": 4, "nbformat_minor": 2 }