
LivePortraitã£ãŠããŒçŸåœ¹æåŒ·å£ãã¯AIã®å®å šãŸãšã(5äžå/幎ã®ç¯çŽ)
çæ§ã©ã
ãŒãããã¶ãã¡ã
:åå¿äººã§ãïŒ
â
ãŸãã¯ãã¡ããã芧ãã ãã(ãããããªãð®)Â

â å°å ¥ã§ãŒãïŒ
è¿å¹ŽAIã¿ã¬ã³ããšãVtuberã倧é墿®ããŠãŸããããŒïŒ
ãã®äžã§äœ¿ãããŠããããããå£ãã¯æè¡(RipSync:ãªããã·ã³ã¯)ã®æåŒ·ãªãŒãã³ãœãŒã¹ãçºè¡šãããŸããããã®åãLivePortraitã§ãã
åãããããœãŒã¹ãšåããäžããåç»ãã¡ã€ã«ã ãã§ã¹ããããŸãð
â ä»ãŸã§ç§ã詊ããRipSyncã®AIã¢ãã«ã¯äžèšãããããã¯ããã«åé§ã
ã»SadTalker
ã»Hallo
ã»avatarity
ã»Stablediffusion Multi OpenPoseãããªãããçµã¿åãã
ã¶ã£ã¡ããLivePortraitãããã°HeyGenãªã©ã«ãéãæãã®ç¡é§ã§ãã
â»çŸã«ç§ã®Youtubeãªã©ã®åç»ã§èª¬æããŠãããã¶åã¡ããã¯ç¡æã§åããŠãŸãã
å°å ¥ãããªãç°¡åã«ãªã£ãŠããã®ã§ç°å¢èšå®ãšäœ¿ã£ãŠã¿ãææãæ®ããŠãããŸãã
â äœ¿ãæ¹ã¯ãã¡ã
ðGithubã®ãœãŒã¹ã³ãŒãã¯ãã¡ããã¯ãªãã¯ð
ð¥ä»åã¯å
šéšå
¥ãã®Live Portraits_videoã§èª¬æããŠãããŸã(noteã§å)
ãvideoãšä»ããã¢ãã«ã¯åç»ãã綺éºã«ãªããã·ã³ã¯ãã§ããŸããïŒ
ç°å¢ã¯ãGetting StartedããèŠããšãªããšãªãåºæ¥ãŸãããååŠè
ã®æ¹ãèºããããããªããã€ã³ãããŸãšããŠèšããŸãã
äºåã«ã¿ããªå€§å¥œãffmpegã䜿ããããã«ããŠãããŸããã
â»ããŠã³ããŒãããŠç°å¢å€æ°ãèšå®ããã ã(binãã¡ã€ã«ã®ãã¹ããŒïŒ)
git clone https://github.com/KwaiVGI/LivePortrait
cd LivePortrait
# ã¢ãã³ã³ãã®ç°å¢ãã€ããã
(ä»åã¯ãããç°¡å)
conda create -n LivePortrait python==3.9.18
conda activate LivePortrait
# pipã§ã®ç°å¢ã(Pytorchã®GPUèšå®ã ãæ³šæ)
pip install -r requirements.txt
# for macOS with Apple Silicon users
pip install -r requirements_macOS.txtããšã¯äºååŠç¿ã¢ãã«ã®ããŠã³ããŒãã ããŒããŒã¡ãŒã¡éã(3GB)ã
# 容éããŒã¡ãŒã¡ããã®ã§git lfsãã€ã³ã¹ããŒã«ããŠããå®è¡
git clone https://huggingface.co/KwaiVGI/liveportrait pretrained_weightsð¥ããã§pretrained_wightsãã£ã¬ã¯ããªãæ¢ã«ããããŒã£ãŠæããã人ãããããã§ãããã®å Žåã¯æ ãŠãpretrained_wightsãåé€ããŠå®è¡ã§ãªã±ãŒã
#ãŠã§ãã¢ããª(Gradio)ã§è©Šãããæ¹ã¯ãã(现ããèšå®ã¯åºæ¥ãªãããŒ)
python app.py
# WindowsãšLinuxãŠãŒã¶ãŒã¯ãã¡ãã§å®è¡
python inference.py -s assets/examples/source/s9.jpg -d assets/examples/driving/d0.mp4 --no_flag_pasteback
# ãã¶ãã¡ã
ã®èšå®ã¯ãããªåŒæ°
python inference.py
# MacãŠãŒã¶ãŒã¯ããã
PYTORCH_ENABLE_MPS_FALLBACK=1 python inference.pyRTX3090(24GB)ãä¹ã£ã端æ«ã§è©ŠããŠ10ç§åç»ã§3åãããã§ã¯ã宿ïŒ
â»æšå¥šã¯NVIDIA GeForce RTX 2060çšåºŠã®6GB以äžãªã®ã§ã泚æ

â åºæ¥äžããããã¡ãã§ã(鿢ç»ãã)

ãããšéæ¢ç»ãä¿¡ããããªããããæ»ããã«ã¹ããã£ãŠãããŸã
â»ä»ãŸã§ã®èŠåŽãªããã£ãŠãã
â ããã«ïŒåç»ããªããã·ã³ã¯ããããŸãïŒ
ææ°æ
å ±ãªã®ã§æå€ãšç¥ãããŠãªãåç»ãããªããã·ã³ã¯ãããæ¹æ³ã§ãïŒ
â»èº«äœãåããŠããåç»ãå£ãã¯ãããã€ã¡ãŒãžã§ãã
Source Videoã®ã¿ãã«ããŠåãæ§ã«éžãã§å®è¡ããã ãïŒ
â ã¯ããåºæ¥äžããããã¡ãã§ã(åç»ãã)

åç»ããå£ãã¯åç»ãäœãããšããäºã¯è²ããªäºãã§ããããã«ãªããŸã
ã»ãªã³ããã³ãæè²ã®èª¬æå¡ãäœã
ã»Youtubeãªã©ã®åç»ã§æã身äœã®åããå ãããã
ã»ççã«èµ°ã£ãŠã人ã®åç»ãå£ãã¯ãããŠæ¥œãã
身äœã®åããå
¥ã£ãŠããèšå Žæã®ããåç»ã§ãšã³ãžã§ã€ããŠãã ããã
ã¡ãªã¿ã«ãªã¢ã«ã¿ã€ã ã§Webã«ã¡ã©ã«é©çšäœ¿ããã¢ãã«ã詊ããŸãããã
茪éã®åºå®ãªã©ããäžæ©ã§ããã®ã§ç²ŸåºŠã奜転ããã°å±éããŸããïŒ
ðäžèšåç»ã®æåŸã«åºãŠããããã¶åãã¡ãããåãæè¡ã§åããŠãŸãð
â æåŸã«ãåŸãªæ å ±
ç§ã¯å¹Žéã®ãµãã¹ã¯æéã30äžåãè¶
ãããã ã®ã¢ãã§ãããLivePortraitã®ãããã§HeyGenãªã©ã®å©çšããªããªã5äžå/幎ã»ã©ãåŸã«ãªããŸããã
ä»ã«ããããªãåŸãªããŒã«ãããããããã®ã§è¿æ¥å±éããããŸãã
ãã®ããã«è¿å¹Žã®AIã¯ãªãŒãã³ãœãŒã¹ã®ã¬ãã«ããšãã§ããªãäžãã£ãŠããã®ã§ãææãµãŒãã¹ã䜿ããªããŠãæ§ã
ãªããžãã¹ã·ãŒã³ã§æŽ»çšã§ããŸãã
ä»åŸãçæ§ãå°ãã§ããåŸã«ããããŠçšŒããããã«æ
å ±çºä¿¡ãç¶ããŸãïŒ
ææãµãŒãã¹ããåªè¯ãªãŒãã³ãœãŒã¹ãð€ïŸïŸ
ïŸïŸïœ°ð