
Solverã®ä»çµã¿ïŒCFRãšã¯äœãïŒ
ãããŒã«ãŒãœã«ããŒã®å
éšã§äžçªãã䜿ãããŠããã¢ã«ãŽãªãºã ã®ã²ãšã€ããCFRïŒCounterfactual Regret MinimizationïŒã§ãã
ãååã¯é£ããèŠããŸããããèªå·±å¯ŸæŠãç¹°ãè¿ããªãããåŸæãæžãããŠããããšã§ããã·ã¥åè¡¡ã«è¿ã¥ãä»çµã¿ããšæã£ãŠããã°ã ãããåã£ãŠããŸãã
ããã§ã¯ãæ°åŒã¯æå°éã«ããŠãäœãããŠããã¢ã«ãŽãªãºã ãªã®ããããã¬ã€ã€ãŒç®ç·ã§ã¯ã©ãçè§£ããŠããã°ããã®ãããæŽçããŸãã
â 1ïŒCFRãšã¯äœãããã¢ã«ãŽãªãºã ãïŒ
ã»CFRã¯äžèšã§ããã°
ããå€äººæ°ã®é çªã²ãŒã ïŒããŒã«ãŒã®ãããªæšæ§é ã²ãŒã ïŒã§ãããã·ã¥åè¡¡ã«è¿ãæŠç¥ãèŠã€ããèªå·±å¯ŸæŠã¢ã«ãŽãªãºã ã
ã»ãã£ãŠããããšã®æµãã¯ã·ã³ãã«ã§ã
ã1ïŒé©åœãªæŠç¥ããã¹ã¿ãŒãããŠèªå·±å¯ŸæŠãã

ã2ïŒãæ¬åœã¯ãã®ã¢ã¯ã·ã§ã³ãéžãã§ããã°ãã£ãšåŸã ã£ãã®ã«ããšããâåŸæâããåã¹ãããã»åã¢ã¯ã·ã§ã³ããšã«èšé²ãã
ã3ïŒåŸæã倧ããã£ãã¢ã¯ã·ã§ã³ã»ã©ã次ã®å埩ã§éžã³ãããïŒç¢ºçãäžããïŒ

ã4ïŒãããäœåã»äœäžåãšç¹°ãè¿ããšããã©ã®ã¢ã¯ã·ã§ã³ã«ã€ããŠã倧ããªåŸæãæ®ããªãæŠç¥ãã«è¿ã¥ããŠãã
ãâ ãã®å¹³åæŠç¥ããããã·ã¥åè¡¡æŠç¥ã®è¿ãã«èœã¡çã
ã»ãã€ã³ãã¯ããäžçºã§æ£è§£ãè§£ããã®ã§ã¯ãªã
ããèªå·±å¯ŸæŠããªããå°ããã€âåŸæâãæžãããŠããããšããè¶³ãç®ã§åè¡¡ã«å¯ããŠããããšã
æçµçã«ã¯åŸæïŒEVå·®ïŒdEVïŒã 0 ã«è¿ã¥ããŸã§èšç®ãåããŸãããæéããããããããããããçšåºŠã®ãšããã§æã¡åãèšå®ã«ãªã£ãŠããŸãããã£ããèšãã°ããã®æŠç¥ã«å¯ŸããŠçžæãå®å šãªãã¹ãã¬ã¹ãã³ã¹ãæã£ããšãã«å¥ªããEVã®äžéãããããã® 0.25ïŒ ä»¥äžã«ãªã£ããæ¢ããããšããæå³ã§ãã詳ããã¯ãç¥ããªããšæ¥ããããdEVââ0.01bbå·®ã®æå³ããã芧ãã ããã
â 2ïŒãåŸæããšãç¡åŸææŠç¥ã
ã»ã²ãšã€ã®ãæ
å ±ã»ããããèããŸãã
ãæ
å ±ã»ãããšã¯ããã®ãã¬ã€ã€ãŒããèŠããæ
å ±ãåãã«ãªãç¶æ³ãã®ããšã§ãïŒäŸïŒèªåã®ãã³ããšããŒãã¯èŠããŠããããçžæã®ãã³ãã¯èŠããªãããªã©ïŒã
ã»ããã«ã¯ãããããããã§ãã¯ãããã©ãŒã«ããã®ããã«è€æ°ã®ã¢ã¯ã·ã§ã³ããããŸãã
ã»å埩çªå·ã (t) ãšãããã®å埩ã§å®éã«éžãã ã¢ã¯ã·ã§ã³ã (a_t)ããã®ãšãã®EVã (v(a_t)) ãšããŸãã
ãããã代ããã«ã¢ã¯ã·ã§ã³ (a) ãéžãã§ããããã®ä»®æ³EVã (v(a)) ãšãããšããã¢ã¯ã·ã§ã³ (a) ã«ã€ããŠã®âåŸæâã¯
ã»å埩ãéãããšãã¢ã¯ã·ã§ã³ (a) ã«ã€ããŠã®çޝç©åŸæã¯
ã»Regret Matching ãšåŒã°ããæŽæ°ã«ãŒã«ã§ã¯ãåæ å ±ã»ããããšã«
ãã»æ£ã®çޝç©åŸæã倧ããã¢ã¯ã·ã§ã³ã»ã©ã次å以éã®ç¢ºçãé«ããã
ãã»çޝç©åŸæãè² ããŒãã®ã¢ã¯ã·ã§ã³ã«ã¯ç¢ºçãå²ãæ¯ããªãïŒãŸãã¯æ¥µç«¯ã«å°ããããïŒ
ãšããåœ¢ã§æŠç¥ãæŽæ°ããŠãããŸãã
ã»ãã£ããèšãã°ã
ãéå»ã«ãããããã£ãŠããã°ããã£ãâŠããšåŸæããæã»ã©ã次ããã¯åºçªãå¢ããã
ãšããããšãŠã人éã£ãœãããžãã¯ãæ°åŒã«èœãšãããã®ã§ãã
â 3ïŒãCounterfactualïŒæ¡ä»¶ä»ãã®ïŒããšããéšå
ã»ããŒã«ãŒã§ã¯ãããæ
å ±ã»ããã«å°éãã確çã¯
ãçžæã®æŠç¥ãããããŸã§ã®ã¢ã¯ã·ã§ã³ã®æµãã«åŒ·ãäŸåããŸãã
ã»CFRã§ã¯æ¬¡ã®ãããªâæ¡ä»¶ä»ãã®äžçâãèããŸãã
ãèªåãããæŠç¥ã䜿ããçžæã¯çŸæç¹ã®æŠç¥ã䜿ã£ãŠãããšä»®å®ãããšãã
ããã®æ å ±ã»ããã«å°éãã確çã¯ã©ãããããïŒã
ã»ãã®æ¡ä»¶ä»ã確çãéã¿ãšããŠäœ¿ãã
ãã»ãã®æ
å ±ã»ããã«å°éãããšä»®å®ãããšãã®EV
ãã»ãã®ãšãã«å¥ã¢ã¯ã·ã§ã³ãéžãã§ããå Žåã®EV
ãèšç®ããŠãåŸæãæŽæ°ããŸãã
ããã Counterfactual RegretïŒåå®ä»®æ³ã®åŸæïŒã§ãã
ã»çŽæçã«ã¯ã
ãããŸããŸãã®ã¹ãããã«ããŸãæ¥ãªãã£ãããåŸæãå°ãããã®ã§ã¯ãªãã
ããã®ã¹ãããã«æ¥ãâãšä»®å®ããäžã§âãã©ã®ã¢ã¯ã·ã§ã³ãããã£ããã
ã ãã«éäžããŠåŸæã枬ã£ãŠããããšããããšã§ãã
â 4ïŒã¢ã«ãŽãªãºã ã®ãã£ããæé
ã»ããŒã«ãŒçšã®CFRãããªãç²ããŸãšãããšã1å埩ãããã¯æ¬¡ã®ãããªæµãã§ãã
ã1ïŒçŸåšã®æŠç¥ãããã¡ã€ã«ïŒåãã¬ã€ã€ãŒã®æŠç¥ïŒã§ã²ãŒã ããªãŒãå·¡åãã
ã2ïŒåæ
å ±ã»ããã§ãå
šã¢ã¯ã·ã§ã³ã® counterfactual valueïŒæ¡ä»¶ä»ãEVïŒãèšç®ãã
ã3ïŒå®éã«éžãã ã¢ã¯ã·ã§ã³ãšã®å·®ããããã®æ
å ±ã»ããå
ã®åã¢ã¯ã·ã§ã³ã®ãåŸæããæŽæ°ãã
ã4ïŒåæ
å ±ã»ããããšã«
ãããã»æ£ã®çޝç©åŸæã«æ¯äŸããŠã¢ã¯ã·ã§ã³ç¢ºçãå²ãåœãŠã
ãããã»æ£èŠåããŠæ°ããæŠç¥ã«ãã
ã5ïŒå埩ãéããªããããã®æç¹ã®æŠç¥ãã§ã¯ãªããæŠç¥ã®å¹³åãããšã£ãŠãã
ãããïŒãã®å¹³åæŠç¥ãããã·ã¥åè¡¡ã«åæããããšããçè«çµæãããïŒ
ã»å®éã®å®è£ ã§ã¯ãã²ãŒã ããªãŒãå šéšãªããã®ã¯éãããã®ã§ã
ãã»ãµã³ããªã³ã°CFRïŒã©ã³ãã ã«éšåããªãŒã ã蟿ãïŒ
ã»CFR+ãDCFR ãªã©ã®æ¹è¯ç
ã䜿ãããŠããŸãããæ ¹æ¬ã®ã¢ã€ãã¢ã¯äžã®æµãã§ãã
â 5ïŒCFRãããŒã«ãŒã§å¬ããçç±
ã»ããŒã«ãŒã¯ãããªãŒæ§é ã®ã²ãŒã ãã§ãããç¶æ æ°ãéæ¹ããªãå€ãã®ã§ã1åã§å šäœã®æé©è§£ãèšç®ããã®ã¯çŸå®çã§ã¯ãããŸããã
CFRã¯
ã»åæ
å ±ã»ããã屿çã«èŠãŠãåŸæãæžãããäœæ¥ãã²ãããç©ã¿äžãã
ã»ãã®çµæãšããŠãå
šäœãšããŠãç¡åŸææŠç¥ãã«è¿ã¥ããŠãã
ã»ç¡åŸææŠç¥ã®å¹³åããããã·ã¥åè¡¡ã«åæããïŒãšããå®çãããïŒ
ãšããæ§é ãæã£ãŠããã®ã§ã
ã巚倧ãªã²ãŒã ããªãŒããèªå·±å¯ŸæŠããªããè¿äŒŒçã«GTOãžå¯ããŠããã
ããã®ã¢ã«ãŽãªãºã ãšããŠéåžžã«çžæ§ãè¯ãããšããããã§ãã
å®éã®ããŒã«ãŒAIïŒPluribusãDeepStackç³»ãªã©ïŒãåçš®ãœã«ããŒã¯ã
ãã®CFRãã¡ããªãŒïŒCFR+ãDCFRãExternal Sampling CFR ãªã©ïŒãåå°ã«ããŠã
ã»ããªãŒãéšåçã«ãµã³ããªã³ã°ãã€ã€
ã»Regretã®æŽæ°ãé«éåã
ã»çŸå®çãªæéãšã¡ã¢ãªã§ãã»ãŒGTOãã«è¿ãæŠç¥ãçæããŠããã
ãšã€ã¡ãŒãžããŠããã°å€§ããã¯å€ããŸããã
â 6ïŒãã¬ã€ã€ãŒç®ç·ã§ã®CFRã®çè§£
ãã¬ã€ã€ãŒãCFRã®å
éšå®è£
ãå®å
šã«çè§£ããå¿
èŠã¯ãããŸããã
æŒãããŠããããã®ã¯ã次ã®2ç¹ã ãã§ãã
â ãèªå·±å¯ŸæŠãç¹°ãè¿ããªããâåŸæâãæžãããŠãã â ç¡åŸææŠç¥ â ããã·ã¥åè¡¡ä»è¿ããšããæµãã«ãªã£ãŠããããšã
â¡åã¹ãããã§ããã®ã¢ã¯ã·ã§ã³ãããŠããã°ããã£ãããšããåŸæãèšé²ãããã®å€§ããã«æ¯äŸããŠæ¬¡åã®é »åºŠãå€ããŠããããšã
ã»ãã®ã€ã¡ãŒãžãæã£ãŠããã ãã§ã
ããœã«ããŒããªãâããã£ãœãâããã¯ã¹ããµã€ãºãåºããŠããã®ãã
ãæèŠçã«çŽåŸãããããªããŸãã
ã»ãœã«ããŒã¯âç¥èšâã§ã¯ãªããCFRã®ãããªã¢ã«ãŽãªãºã ã§
ãã»èªå·±å¯ŸæŠ
ãã»åŸæã®èç©
ãã»æŠç¥ã®æŽæ°
ãã²ãããç¹°ãè¿ãããçµæãšããŠã®æŠç¥ããèŠããŠãããŠããã ãã§ãã
ãã®æå³ã§ãããœã«ããŒã®ä»çµã¿ïŒCFRã®èãæ¹ããäžåºŠæŽçããŠãããšã
åºåã®èŠãæ¹ãâãºã©ãæ¹âã®æèŠãã ãã¶å€ãã£ãŠããã¯ãã§ãã
PioSolverïŒææ°çïŒ
Pioã®äžèº«ã¯å ¬åŒã«ã¯çްããå ¬éãããŠãªããã©ã
ããšããšãCFRããŒã¹ã®ãœã«ããŒããšããŠç¥ãããŠãã
Pioã®äœè ãæãããCFRã®æ¹è¯çïŒããããæé©åããŠãããããã®èª¬æãããŠãã
ã€ã¡ãŒãžãšããŠã¯ãCFRãåå°ã«ããç¬èªæ¹è¯çã ãšèããŠåé¡ãªãã
GTO+ / GTO+2
ãã¡ããåºæ¬è·¯ç·ã¯ CFRããŒã¹ã®å埩ã¢ã«ãŽãªãºã ã
GTO+åŽã¯ããã¥ã¡ã³ãã§ããCFRïŒregret minimizationïŒã䜿ã£ãŠè§£ããŠãããç³»ã®èª¬æãããŠããŠãGTO+2 ã§ãããšã³ãžã³åŒ·åïŒåšèŸºæ©èœè¿œå ãã£ãŠæãã§ãã¢ã«ãŽãªãºã ã®ç³»çµ±ããŸã£ããå¥ç©ã«å€ãã£ãããšãã話ã¯åºãŠããªãã
GTO Wizard
å ¬éãããŠããç¯å²ã ãšïŒ
ããªããããïŒãã¹ãããããã® GTO è§£ã¯CFR ç³»ãœã«ããŒïŒèªåãšã³ãžã³ïŒã§äºåã«å·šå€§DBãšããŠè§£ããŠãããŠããããã¯ã©ãŠãããåŒã£åŒµã£ãŠããŠããããšããäœãã
ãªã®ã§ãGTO Wizard ïŒ ãã®å Žã§CFRãåããŠããããŒã«ãã§ã¯ãªã
ãCFRã§äºåã«è§£ããããœãªã¥ãŒã·ã§ã³ã®ãã¥ãŒã¢ïŒè£å©æ©èœïŒAIã
ãšããã€ã¡ãŒãžãè¿ãã
Simple ç³»ãœã«ããŒ
Simple Postflop / Simple Preflop / Simple 3-way ãªã©ïŒ
å ¬éæ å ±ã»æåã»èšç®ã¹ã¿ã€ã«ããèŠãŠ
â CFRç³»ïŒïŒãµã³ããªã³ã°ã»æ¹è¯çïŒ ãããŒã¹ã«ããŠããã¿ã€ããPio / GTO+ ãšåããæšæ§é ã²ãŒã ãèªå·±å¯ŸæŠã§åè¡¡ã«å¯ããã系統
ICMIZER / HRC ã¿ãã㪠ICMã»push/foldå°çšããŒã«
â 䞻圹㯠CFR ã§ã¯ãªããICMã¢ãã«ïŒå
šæ¢çŽ¢ïŒåçèšç»æ³ïŒç·åœ¢èšç»