
Google Colab ã§è©Šã Stable Cascade ã§ã®æ°æä»£ã®ããã¹ãç»åçæ
2024幎2æ13æ¥ãStability AIããæ°ã¢ãã«ãStable CascadeãããªãªãŒã¹ãããŸããã
https://ja.stability.ai/blog/stable-cascade
ðãæ°ç»å Žãæ°ããããã¹ãããç»åãžã®å€æã¢ãã«ãStable Cascadeã玹ä»ããŸããWÃŒrstchenã¢ãŒããã¯ãã£ãåºã«ã誰ã§ãç°¡åã«ãã¬ãŒãã³ã°ãšåŸ®èª¿æŽãã§ããããèšèšãããŠããŸããðšâŠ pic.twitter.com/hKbaVvyKgH
â Stability AI Japan (@StabilityAI_JP) February 13, 2024
ðãæ°ç»å Žãæ°ããããã¹ãããç»åãžã®å€æã¢ãã«ãStable Cascadeã玹ä»ããŸããWÃŒrstchenã¢ãŒããã¯ãã£ãåºã«ã誰ã§ãç°¡åã«ãã¬ãŒãã³ã°ãšåŸ®èª¿æŽãã§ããããèšèšãããŠããŸããðš
éåçšå©çšã«éãããã§ãã¯ãã€ã³ããšæšè«ã¹ã¯ãªãããæäŸã埮調æŽãControlNetãLoRAãã¬ãŒãã³ã°ã®ããã®ã¹ã¯ãªãããå ¬éäžïŒð
ð¡é«å質ãæè»æ§ãå¹çæ§ãéèŠããããŒããŠã§ã¢ã®å¶çŽãè¶ ãããã®é©æ°çã¢ãããŒãããã²äœéšããŠãã ããã
詳现ã¯ãã¡ãâ https://ja.stability.ai/blog/stable-cascade
Stable Cascade ã¢ãã«ã«ãŒããã
ãã®ã¢ãã«ã¯ãWÃŒrstchenã¢ãŒããã¯ãã£ã«åºã¥ããŠæ§ç¯ãããŠãããStable Diffusionã®ãããªä»ã®ã¢ãã«ãšã®äž»ãªéãã¯ãããå°ããªæœåšç©ºéã§åäœããŠããããšã§ããããã¯ãªãéèŠãªã®ã§ããããïŒæœåšç©ºéãå°ãããã°å°ããã»ã©ãæšè«ãããéãå®è¡ããããšãã§ãããã¬ãŒãã³ã°ãããäœã³ã¹ãã«ãªããŸãã
æœåšç©ºéã¯ã©ã®ãããå°ããã®ããšãããšãStable Diffusionã¯å§çž®ç8ã䜿çšãã1024x1024ã®ç»åã128x128ã«ãšã³ã³ãŒãããŠããŸããStable Cascadeã¯42ã®å§çž®çãéæãã1024x1024ã®ç»åã24x24ã«ãšã³ã³ãŒãããããšãã§ããŸãããããŠãããã¹ãæ¡ä»¶ã¢ãã«ã¯ãé«åºŠã«å§çž®ãããæœåšç©ºéã§åŠç¿ãããŸãããã®ã¢ãŒããã¯ãã£ã®ä»¥åã®ããŒãžã§ã³ã§ã¯ãStable Diffusion 1.5ãšæ¯èŒããŠ16åã®ã³ã¹ãåæžãéæããŸããã
ãããã£ãŠããã®çš®ã®ã¢ãã«ã¯å¹çãéèŠãªçšéã«é©ããŠããŸããããã«ããã¡ã€ã³ãã¥ãŒãã³ã°ãLoRAãControlNetãIP-AdapterãLCMãªã©ã®æ¢ç¥ã®æ¡åŒµã¯ãã¹ãŠããã®æ¹æ³ã§ãå¯èœã§ãã
ã¢ãã«ã®è©³çް
Stable Cascadeã¯ãããã¹ãããã³ããããç»åãçæããããã«åŠç¿ãããæ¡æ£ã¢ãã«ã§ãã
éçºè
ïŒStability AI
è³éæäŸè
ïŒStability AI
ã¢ãã«ã®çš®é¡ïŒããã¹ãç»åçæã¢ãã«
ã¢ãã«ãœãŒã¹
ç ç©¶ç®çã«ã¯ãStabilityCascadeã®Githubãªããžããª
ïŒhttps://github.com/Stability-AI/StableCascade ïŒããå§ãããŸãã
ãªããžããª: https://github.com/Stability-AI/StableCascade
è«æïŒhttps://openreview.net/forum?id=gU58d5QeGv
"WÌrstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Models", Pablo Pernias, Dominic Rampas, Mats Leon Richter, Christopher Pal, Marc Aubreville, Published: 16 Jan 2024, ICLR 2024 oral
ã¢ãã«ã®æŠèŠ
Stable Cascadeã¯3ã€ã®ã¢ãã«ã§æ§æãããŠããŸãïŒ Stage AãStage BãStage Cã®3ã€ã®ã¢ãã«ã§æ§æãããç»åãçæããããã®ã«ã¹ã±ãŒãã衚çŸããŠããŸããã¹ããŒãžAãšã¹ããŒãžBã¯ãStable Diffusionã«ãããVAEã®ä»äºãšåæ§ã«ãç»åãå§çž®ããããã«äœ¿çšãããŸãããããããã®ã»ããã¢ããã§ã¯ãããé«ãç»åå§çž®ãéæããããšãã§ããŸããStable Diffusionã¢ãã«ã空éå§çž®ä¿æ°8ã䜿çšãã1024 x 1024ã®è§£å床ã®ç»åã128 x 128ã«ãšã³ã³ãŒãããã®ã«å¯ŸããStable Cascadeã¯42ã®å§çž®ä¿æ°ãéæããŸããããã«ããã1024 x 1024ã®ç»åã24 x 24ã«ãšã³ã³ãŒãããªãããç»åãæ£ç¢ºã«ãã³ãŒãããããšãã§ããŸããããã«ã¯ãåŠç¿ãšæšè«ãããå®äŸ¡ã«è¡ãããšãã倧ããªå©ç¹ããããŸããããã«ãã¹ããŒãžCã¯ãããã¹ãããã³ãããäžãããããšã24 x 24ã®å°ããªæœåšèœåãçæãã圹å²ãæãããŸãã
次ã®å³ã¯ãããèŠèŠçã«ç€ºããŠããŸãã

ä»åã®ãªãªãŒã¹ã§ã¯ãã¹ããŒãžCã®ãã§ãã¯ãã€ã³ãã2ã€ãã¹ããŒãžBã®ãã§ãã¯ãã€ã³ãã2ã€ãã¹ããŒãžAã®ãã§ãã¯ãã€ã³ãã1ã€çšæããŠããŸããã¹ããŒãžCã«ã¯10åãš36åã®ãã©ã¡ãŒã¿ã»ããŒãžã§ã³ããããŸããã埮調æŽã«æãåŽåãè²»ããã36åããŒãžã§ã³ã®äœ¿çšã匷ããå§ãããŸããã¹ããŒãžBã¯7åãš15åã®ãã©ã¡ãŒã¿ãŒãã©ã¡ããçŽ æŽãããçµæããããããŸããã15åã®æ¹ãå°ãã现ãããã£ããŒã«ã®åæ§ç¯ã«åªããŠããŸããåŸã£ãŠãããããã®å€§ããæ¹ã®ããŒãžã§ã³ã䜿çšããæ¹ããæè¯ã®çµæãåŸãããŸããæåŸã«ãã¹ããŒãžAã¯2,000äžåã®ãã©ã¡ãŒã¿ãå«ã¿ããµã€ãºãå°ããããåºå®ãããŠããŸãã

Stability AIã®è©äŸ¡ã«ãããšãStable Cascadeã¯ããã³ããã®ã¢ã©ã€ã¡ã³ããšçŸçå質ã®äž¡æ¹ã«ãããŠãã»ãŒãã¹ãŠã®æ¯èŒã§æé«ã®ããã©ãŒãã³ã¹ãçºæ®ããŸãããäžã®å³ã¯ãparti-promptsïŒãªã³ã¯ïŒãšå¯©çŸçãªããã³ãããããã¯ã¹ãããã®ã䜿çšãã人éã«ããè©äŸ¡ã®çµæã§ããå ·äœçã«ã¯ãStable CascadeïŒæšè«ã¹ãããæ°30ïŒãPlayground v2ïŒæšè«ã¹ãããæ°50ïŒãSDXLïŒæšè«ã¹ãããæ°50ïŒãSDXL TurboïŒæšè«ã¹ãããæ°1ïŒãWÃŒrstchen v2ïŒæšè«ã¹ãããæ°30ïŒãšæ¯èŒããŸããã
ã³ãŒãäŸ
â ïž éèŠïŒ 以äžã®ã³ãŒããåäœãããã«ã¯ãPRãWIPã§ããéã«ãã®ãã©ã³ããããã£ãã¥ãŒã¶ãŒãã€ã³ã¹ããŒã«ããå¿
èŠããããŸãã
â»å®éã«åäœããGoogle Colabã®ã³ãŒãã¯ãã¡ãïŒAICUã®ãªããžããªïŒ
https://j.aicu.ai/StableCascade
çšé
çŽæ¥äœ¿çš
ãã®ã¢ãã«ã¯ä»ã®ãšããç ç©¶çšã§ããæ³å®ãããç ç©¶åéãšèª²é¡ã¯ä»¥äžã®éãã§ãã
ã»çæã¢ãã«ã®ç ç©¶
ã»æå®³ãªã³ã³ãã³ããçæããå¯èœæ§ã®ããã¢ãã«ã®å®å
šãªå±éã
ã»çæã¢ãã«ã®éçãšãã€ã¢ã¹ã®èª¿æ»ãšçè§£ã
ã»èžè¡äœåã®çæãšããã¶ã€ã³ããã®ä»ã®èžè¡çããã»ã¹ã«ããã䜿çšã
ã»æè²çãŸãã¯åµé çãªããŒã«ãžã®å¿çšã
é€å€ãããçšéã¯ä»¥äžã®éãã
ç¯å²å€ïŒOut-of-scopeïŒã®äœ¿çšïŒæ³å®ã®ç¯å²å€
ãã®ã¢ãã«ã¯ã人ç©ãåºæ¥äºã®äºå®ãŸãã¯çå®ã®è¡šçŸãšãªãããã«èšç·ŽãããŠããªãããããã®ãããªã³ã³ãã³ããçæããããã«ã¢ãã«ã䜿çšããããšã¯ããã®ã¢ãã«ã®èœåã®ç¯å²å€ã§ãããã®ã¢ãã«ã¯ãStability AI ã®å©çšèŠå®ã«éåãããããªæ¹æ³ã§äœ¿çšããªãã§ãã ããã
å¶éãšãã€ã¢ã¹
å¶éäºé
ã»é¡ã人ç©ã¯äžè¬çã«æ£ããçæãããªãå¯èœæ§ããããŸãã
ã»ã¢ãã«ã® autoencoding éšåã¯éå¯éã§ãã
æšå¥šäºé
ãã®ã¢ãã«ã¯ç ç©¶ç®çã§ã®ã¿äœ¿çšã§ããŸãã
ã¢ãã«ãå§ããã«ã¯
https://github.com/Stability-AI/StableCascade ããã§ãã¯ããŠãã ããã
AICU mediaç·šééšãã
WÃŒrstchenïŒãŽã«ã¹ããã§ã³ïŒãšã¯å°ãããœãŒã»ãŒãžã®ããšã§ãð
å
¬åŒã®ã¢ãã«ã«ãŒãã®ã³ãŒããããŒã¹ã«TextToImageãå®è¡ã§ãã Google Colabã§åäœããã³ãŒãã詊éšå
¬éããŸãã
https://j.aicu.ai/StableCascade
ïŒäžèšã®ã©ã€ã»ã³ã¹ã«åŸãç ç©¶ç®çã«éããŸãïŒ
â»ããŒã«ã«ç°å¢ã§ã® å
¬åŒ notebook ã¯ãã¡ãã«ååšããŸãã
https://github.com/Stability-AI/StableCascade/blob/master/inference/readme.md
T4 GPUã§ã®å®éš

V100ã§ã®å®éš

V100 ãã€ã¡ã¢ãª ã§ã®å®éš

A100ã§ã®å®éš

ãµã³ãã«ã³ãŒãã®t2iããã³ããã¯ãã®ããã«æå®ãããŠããŸããã
prompt = "Anthropomorphic cat dressed as a pilot"
negative_prompt = ""

ããªãé«ç²Ÿçްãªç»åãçæããŠããŸããé«å§çž®ã«ãé¢ããããã§ãã
ããããæ¢ã£ãŠãããŸããã1024x1024ã®ç»åçæèªäœã¯2ç§ã§çæããŠããŸãïŒA100ç°å¢ïŒã

prior_output ã®ç»åãµã€ãºãå€ããããšã§æšªé·ç»åãç°¡åã«çæã§ããŸãã
ãŸããã¬ãã£ãããã³ãããæšæºè£
åã§ãã

ãã£ããã®ãã¬ã³ã¿ã€ã³ãªã®ã§ããããªæãã§çæããŠãããŸãã




髪ã®ãªã ã©ã€ããçæ¯ããŸã€æ¯ã®æåãçŒé¡ã®åéæã®åœ±ãé¡ã«èœã¡ãŠããããããçŽ æŽãããã§ããã

GPUæ¶è²»éã¯å°ãã¥ã€äžãã£ãŠãããŸãã
ååã«ããŒãããã¢ãã«ã§äœåºŠã§ãçæã§ããäžã«ãç»è³ªãé«ç²Ÿçްã§ãã
çŸåšã¯ç ç©¶ç®çã«éå®ãããŠããŸãããä»åŸãåçšã©ã€ã»ã³ã¹ãæåŸ
ãããŸããã
ãŸããã®èšäºã§ã¯TextToImageã®ã¿ç޹ä»ããããŸãããããªããžããªã®äžã«ã¯ãªãªãŒã¹æ
å ±ã«ããããã«ãåŸæ¥ã® Stable Diffusion ã§ãªãŒãã³ã«éçºãããŠãããæ§ã
ãªãŠãŒã¹ã±ãŒã¹ã®ãã¬ãŒãã³ã°ã¹ã¯ãªãããå®è¡äŸãååšããŠããŸããéåžžã«ã·ã³ãã«ãªãœãŒã¹ã³ãŒã矀ã«ãªã£ãŠããŸãã
ã»Text to Image
ã»ControlNet
ã»LoRA
ã»ç»ååæ§æ
Stable Cascadeã¯Stage AãšBã䜿ã£ãŠç»åãå§çž®ããStage Cã¯ããã¹ãæ¡ä»¶åŠç¿ã«äœ¿ãããŸãããããã£ãŠãLoRAãControlNetãStage Cã®ããã ãã«ãã¬ãŒãã³ã°ããã®ã¯çã«ããªã£ãŠããŸãã
ã³ãŒãããŒã¹ã¯éçºåæã§ããäºæãã¬ãšã©ãŒãæé©åãããŠããªããã¬ãŒãã³ã°ãæšè«ã³ãŒãã«ééãããããããŸãããããããããè©«ã³ç³ãäžããŸãããããèå³ãããã°ãææ°ã®æ¹åãšæé©åãç®æããŠã¢ããããŒããç¶ããŸããããã«ãã¢ã€ãã¢ããã£ãŒãããã¯ããããã¯ã¢ããããŒãã«è²¢ç®ããŠãã ããæ¹ã ããã®ãé£çµ¡ããåŸ ã¡ããŠãããŸãã
@misc{pernias2023wuerstchen,
title={Wuerstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Models},
author={Pablo Pernias and Dominic Rampas and Mats L. Richter and Christopher J. Pal and Marc Aubreville},
year={2023},
eprint={2306.00637},
archivePrefix={arXiv},
primaryClass={cs.CV}
}远èšïŒmkshingããããèè é£ã«ãç¥ãã®ãã€ãŒã
Stable Cascade (WÃŒrstchen v3) by @dome_271 and @pabloppp is out!
â mkshing (@mk1stats) February 13, 2024
I've been a big fan of WÃŒrstchen since v1 but v3 is absolutely the best as their results are shown.
code: https://t.co/C42ZIBDo3n
Big congrats to both of you and @StabilityAI! https://t.co/ELbwEdxbf7 pic.twitter.com/an5sODdpyM
Stable Cascadeã Google Colab T4 16GBã§åãã³ãŒãã
Also, I hacked to run the Stable Cascade on Colab free plan (T4 16GB)!
â mkshing (@mk1stats) February 14, 2024
Enjoy! #stablecascade
colab: https://t.co/C33RSn5uSy
ããããšãããããŸãïŒ