èæ¯
ãé èªå¯ŸçŸ©èªããã¡ãã£ãšé¢çœããšæã£ãã®ã§ãWord2Vecãå¿çšããŠæ©æ¢°ã«äœãããŠã¿ããïŒãšãã詊ã¿ã
ããããããšã¯ã以äžã®ãããªã®ã£ã°å¯ŸçŸ©èªãèªåçæããããšã
- ãèµ€ã®ä»äººãâãçœãæäººã
- ããŠãµã®ã¯å¯ãããšæ»ã¬ãâããŽãªã©ã¯å€ç¬ãèè² ãçãæãã
- ãçããããªãã¯çŸãããâ ãæ»ããã¹ã
- ãå·ããäžè¯å§ããŸãããâ ããã§ãã¯ããèŸããŸããã
- ãã³ã¢ã©ã®ããŒãã â ããŽãªã©ã®ã¬ã¯ã€ãšã ã
- ãããææ ¢ã â ããã倧æŽãã
- ãçççã«ç¡çã â ãçè«äžã¯å¯èœã
- ãã²ã¹ã®æ¥µã¿ä¹å¥³ã â ãã»ãã®ããã¥ã¢èŠªç¶ã
- ãé±åå°å¹Žãžã£ã³ããâãæåè人ã¹ã¯ã¯ããã
- ããæ¯ãããšäžç·ã â ããç¶ããã¯å¥å± ã
- ããããªããã瀟äŒã«åºãŠããéçšããªããã â ãããã ãã®åãããã°å¹Œçšåã§ã¯ç¡æµã ããã
æãããŠWord2VecãæŽ»çšããŠããã®ãããªãŠãŒã¢ã¢ãçã¿åºããã®ãïŒïŒ
æ¬æçš¿ã®å 容
- é èªå¯ŸçŸ©èªã¯ãŠãŒã¢ã¢ã§ãããæ£ç¢ºæ§ããé¢çœããåªå ããŸãã
- 詊è¡é¯èª€ããåãçµã¿ãèå¯ã®æµããèšé²ãããç®çã§ããã綺éºãªã³ãŒãã¯åºãŠããŸããã
- ãšããã€ã€ã詊ããããšãå šãŠæžãããšã¯é£ãããããªã端æã£ãŠããŸãã
- 倿°åãšãããããã®ã¯æ°ã«ããªãã»ã»ã»ã
- ç°å¢ã¯Windows10 + Python3 +JupyterNotebook ãåæãšããŠããŸãã
- Word2Vecã®ã¢ãã«ã¯åŠç¿æžã¿ã®ãã®ã䜿ãã®ã§æ¯èŒçç°¡åã«åãããŸãã
- Mecab(圢æ çŽ è§£æ)ãPythonã®ç°å¢æ§ç¯ã«ã€ããŠã¯è§ŠããŸããã
- ã³ãŒããèŠãŠããã§ã¯ãªããæ¬æïŒå®è¡çµæã ãã§èªã¿ç©çã«æ¥œãããæçš¿ãç®æããŸãã
æåã®èšç»
Word2Vecãšã¯ïŒ
- åèªããã¯ãã«åããŠã以äžã®ãããªèšç®ãå¯èœã«ãªãä»çµã¿
- ãçæ§ã ㌠ãç·ã ïŒ ã女ã = ã女çã
詳ããã¯ãä»ã«å€æ°ã®è¯èšäºããããããã°ã°ãã°ããåããã®ã§ãæ¬çš¿ã§ã¯å²æããã
ããã䜿ãã°å¯ŸçŸ©èªãªããŠç°¡åïŒ
ãããããããããäŸãã°ããŠãµã®ãã®æ£åå¯Ÿãæ±ããããšãããšã
以äžã®ããã«ãå
šãé¢ä¿ã®ãªãèšèãåºãŠããã
> model.most_similar(negative=["ãŠãµã®"], topn=7)
[('ç·çž®', 0.649797797203064),
('æžæ¡', 0.6217541694641113),
('æŽåæ°å¹¹ç·', 0.6201282143592834),
('è£æ£äºç®', 0.6149194836616516),
('åŽäœ¿äº€æž', 0.6115709543228149),
('æè¡', 0.6105344295501709),
('äºç®æ¡', 0.605506420135498)]
æ©æ¢°ã«ãšã£ãŠããç¡é¢ä¿ããªãã®ããã究極ã®å察ã§ããã
ããŠãµã®ãã«å¯ŸããŠããŽãªã©ããªã©ã¯ã
å®ã¯ç©åãè¿ãèšèãšããŠèªèãããŠããã
奜ãã®å察ã¯å«ãã§ã¯ãªãç¡èŠã奜ããªåã«ã¯ããããããç·ã®åã®ããã§ããã
Word2Vecã®åŒ±ç¹ ïŒ å¯ŸçŸ©èªã«åŒ±ã
ãããªWord2Vecã«ã匱ç¹ããããWord2Vecã¯å¯ŸçŸ©èªã«åŒ±ãã®ã ã
çç±ã¯ç°¡åã§ããç§ã¯ããªãã®ããšã奜ãã§ãããšãç§ã¯ããªãã®ããšãå«ãã§ããã®ããã«ã察矩èªã¯åãæèã§ç»å Žããããã ã
ãªã®ã§ããã察矩èªãéèŠãªæå³ããã€æ©èœãWord2Vecã§éçºããããšæãã®ã§ããã°å°ãæ€èšãçŽããã»ããè¯ãã ããã
by https://deepage.net/bigdata/machine_learning/2016/09/02/word2vec_power_of_word_vector.html
äžèšã®èšäºã«ãããããã«ãWord2Vecã¯æ¬æ¥å¯ŸçŸ©èªã«åŒ±ãã·ã¹ãã ã
æ€è𿮵éã§æ¢ã«å
šåŠå®ãããŠããããããã©ãããã®ã»ã»ã»ïŒ
å·¥å€«ã®æ¹åæ§
ãçæ§ã ㌠ãç·ã ïŒ ã女ã = ã女çããã®äŸãå
ã«ã
ãçæ§ãã®å¯ŸçŸ©èªãšããŠã女çããåºãæµããèããŠã¿ãã
ãŸãã å
ã«å¯ŸçŸ©èªã®ãªã¹ããšããŠããç·ïŒå¥³ããªã©ã®ãªã¹ããçšæããŠããã
ãçæ§ãâãç·ããªã©ãå
¥åãããèªå¥ã«è¿ã察矩èªã®çµãèŠã€ããŠã
äžèšã®æŒç®ãè¡ãããšããæµãã¯ã©ãã ãããïŒ
äŸãã°ãããŠãµã®ãã¯ã匱ããã®ã§
ããŠãµã®ãïŒã匱ãïŒã匷ããã§ãŽãªã©ãšãã«ãªããªããïŒ
> model.most_similar(positive = ["ãŠãµã®", "匷"], negative=["匱"], topn=15)
[('ããã¡', 0.8022700548171997),
('ã«ãšã«', 0.7846293449401855),
('ã¯ã', 0.7720282673835754),
('ãã¯ããŠ', 0.7702590823173523),
('ã¯ã', 0.7532936930656433),
('ãªãªã«ã', 0.7518944144248962),
('ã¯ã·', 0.7504986524581909)]
ããŒãããããã¡ãããã«ãšã«ãã¯ããŸã匷ããã§ã¯ãªãã
ãã¯ãããã¯ããããªãªã«ãã ã¯çµæ§ããç·ãã£ãŠãããããªæ°ãããã
ãã®æç¹ã§ããŸããŽã£ããã®çµæã«ãªã£ãŠããªãããé£ãããã§ã¯ãããã
ãããã工倫/調æŽãããã°èŠèŸŒã¿ã¯ãããããªã¬ãã«ããŸããã£ãŠã¿ããã
äºåæºå
æºåïŒã察矩èªãªã¹ãã®æºå
ãŸãã以äžã®ããã«å
ãšãªã察矩èªãªã¹ããäœæã100ïœ300åãããã®çµãçšæããã
WebããããçšåºŠååŸã§ãããããããããã®åæ°ãªãã°æã§äœã£ãŠãè¯ãã
import pandas as pd
df = pd.read_csv('ïœãã¡ã€ã«ãã¹çç¥ïœ/Taigigolist.csv')
print (df[:3]) # show 3 column
# org, taigigo ãšããèŠåºãã§csvã¯ä¿åãããŠãããã
org taigigo
0 å®å¿ äžå®
1 å®å
š å±éº
2 å§çް æŠç¥
å®å¿âäžå®ããšãäžå®âå®å¿ãã®äž¡æ¹ã®ãã¿ãŒã³ãåæ§ã«æ±ãããã®ã§ã
以äžã®ããã«ããŠãéã«ããããŒã¿ãã¬ãŒã ãšçµåããã
ãããã£ãå°ããªå å·¥ã®ç©ã¿éããããŠãããšã詊è¡é¯èª€ãé²ããããã
import pandas as pd
df = pd.read_csv('ïœãã¡ã€ã«ãã¹çç¥ïœ/Taigigolist.csv')
# dfãšå
šãåããã®ãäœæãã³ã¬ã«éã«ããããŒã¿ã远å ããã
df2 = pd.DataFrame(index=[], columns = df.columns)
# å
ã®dfãããrowçªå·ãéã«([[0],[1]]ã[[1],[0]]ã«)ããŠappendããŠããã
for index, rows in df.iterrows():
#ããŒã¿ãã¬ãŒã ããšã®åã®ã¯ã©ã¹ã¯ãpd.Seriesãçšãã
series = pd.Series([rows[1], rows[0]], index = df2.columns )
df2 = df2.append(series, ignore_index = True)
# indexãæã®ãŸãŸã«ãªãã®ã§ãäžå¿reset_indexã§æ¯ããªãããŠããã
# ïŒã¡ãªãdrop=Trueã«ããªããšindexãå¥ã«ã©ã ãšããŠç»å ŽããïŒ
df_alltaigigo = df_alltaigigo.reset_index(drop=True)
print(df_alltaigigo[:3])
æºåïŒãWord2Vecã®åŠç¿æžã¿ã®ã¢ãã«ã®æºå
Word2Vecã®ã¢ãã«ã¯ãåŠç¿æžã¿ã®ã¢ãã«ãããŠã³ããŒãããŠããŒã«ã«ã«ä¿åããŠããã
以äžïŒäŸïŒ
- çœã€ã®ã³ãŒãã¬ãŒã·ã§ã³/WikipediaããïŒ
http://aial.shiroyagi.co.jp/2017/02/japanese-word2vec-model-builder/ - fasttext-vector ã®åŠç¿æžã¿ã¢ãã«ïŒ
https://qiita.com/Hironsan/items/513b9f93752ecee9e670
from gensim.models.word2vec import Word2Vec
import gensim
# è€æ°ã®èŸæžãåæã«æ±ããšéããªããããæ¬åœã¯äœ¿ããã®ã ãã«ããæ¹ãããã
# æ¥æ¬èªwikipedia
modelW_path = './word2vec_models/latest-ja-word2vec-gensim-model/word2vec.gensim.model'
modelW = Word2Vec.load(modelW_path)
# fasttext-vector
modelV_path = './word2vec_models/vector/model.vec'
modelV = gensim.models.KeyedVectors.load_word2vec_format(modelV_path, binary=False)
#### ããŒããããçµæã®ãã¹ã
# ãããªæãã§å¿
èŠãªæã«åãæ¿ããŠãè€æ°ã®èŸæžã詊ã
model = modelW
# é¡äŒŒåºŠã衚瀺ãã
sim_do = model.similarity(u'京éœ', u'æ±äº¬')
print(sim_do)
# 0.719850214818 ãªã©ãšã³ãµã€ã³é¡äŒŒåºŠã衚瀺ããã
åœåèšç»ãå®è£ ããŠã¿ãããããã»ã»ã»
察矩èªãªã¹ãäžãããæããã³ã¢ã©ãã«è¿ãèšèãæ¢ã
ã³ã¢ã©ã®å¯ŸçŸ©èªãæ±ããããã«ã
ãŸãã³ã¢ã©ã®ã€ã¡ãŒãžã«è¿ã察矩èªã®çµãæ¢ãæ¹éã§ãã£ãã
äŸãã°ãã³ã¢ã©ãïŒãããšãªããããèŠã€ãããïŒä»åã®ãªã¹ãäžã«ã¯ãã®èªå¥ã¯ç¡ãã£ããïŒ
ããšãªãããâãèã
ããããšãã察矩èªã®çµã§ããã°ã
ãã³ã¢ã©ãïŒãããšãªãããïŒãèã
ãããïŒïŒïŒãã«ãã£ãŠã
ã³ã¢ã©ãèããããããªåç©ãïŒãã³ã¢ã©ã®å¯ŸçŸ©èªããšããããšã
ããã§ã察矩èªãªã¹ãã®èªå¥ãšå
¥åèªå¥ã®è¿äŒŒåºŠãèšç®ããŠã
倧ããé ã«äžŠã¹ãããã«ããæ©èœãäœãã
ããã§ã¡ãã£ãšé¢åãªç¹ã¯ãäœæãã察矩èªãªã¹ãã®èšèã¯ã
å¿
ãããå
šãŠWord2Vecã®èŸæžã«å
¥ã£ãŠããããã§ã¯ç¡ããšããããšã
Word2Vecã®æŒç®ãåºæ¥ãªãèšèãçµæ§åºãŠããŠããŸãã®ã§ã
察矩èªãªã¹ãã調æŽããŠãé€å€ããå¿
èŠãããã
ãŸããWord2Vecã¢ãã«å
ã®ãã¯ãã«ãã€ãã€ãã«èšå®ãããŠããŠã
æŒç®ããŠãäžæã察矩èªãäœããªãçµãªã©ãååšãããããããªãã
ãããã£ãããã°ãšã©ãŒèªå¥ãé€å€ããŠããå¿
èŠãããã
ãã®ãããæåã¯æåã§å¯ŸçŸ©èªãªã¹ããã¡ã³ããã³ã¹ããŠã
ãã¥ãŒãã³ã°ããŠããæ¹æ³ãæå¹ã ãšèããŠããã
ããããããã§ã¯ãWord2Vecã®ã¢ãã«ã倿Žãããã³ã«èª¿æŽãçºçãã
ç¹å®ã®Word2Vecã¢ãã«ã«äŸåãããã¥ãŒãã³ã°ã«ãªã£ãŠããŸãããã«ã
詊è¡é¯èª€ã®æ®µéãšããŠã¯æãŸãããªãã
ããã§ãWord2Vecã®èŸæžã«å«ãŸããªãèšèãèªåã§é€å€ããããã«ããŠã
Word2Vecã®ã¢ãã«ã®å€æŽãã察矩èªãªã¹ãã®è¿œå 倿Žã«å¯Ÿå¿ããããããããã
# 察矩èªããŒã¿å
ã§ãèŸæžããŒã¿ã«ãããã®ã®ã¿ãæç²ïŒåæ§ç¯
def create_filterd_taigigo_list(input_word):
# 空ã®ããŒã¿ãã¬ãŒã ãçæããã
df_alltaigigo_filterd = pd.DataFrame(index=[], columns = ["org","taigigo","sim_inp_org","sim_inp_tai","sim_org_tai"])
for index, rows in df_alltaigigo.iterrows():
#ããŒã¿ãã¬ãŒã ããšã®åã®ã¯ã©ã¹ã¯ãpd.Seriesãçšãã
try:
#ããããã®åèªã®éã®é¡äŒŒåºŠã衚瀺ãã
sim_INP_ORG = model.similarity(input_word, rows[0])
sim_INP_TAI = model.similarity(input_word, rows[1])
sim_ORG_TAI = model.similarity(rows[0], rows[1])
except KeyError:
#ããŒãšã©ãŒãçºçããå Žåã¯æ¡çšããªãã
#Word2Vecã®èŸæžã«ç¡ãçšèªãåºãŠãããšãšã©ãŒã«ãªãããã«ããã®ãšã©ãŒåŠçã¯éèŠã
continue
#ãšã©ãŒãç¡ãå Žåã¯åŠçãç¶ç¶ããŠä»¥äžã宿œ
series = pd.Series([rows[0], rows[1], sim_INP_ORG, sim_INP_TAI, sim_ORG_TAI], index = df_alltaigigo_filterd.columns )
df_alltaigigo_filterd = df_alltaigigo_filterd.append(series, ignore_index = True)
return df_alltaigigo_filterd
# æ¬æ¥ã¯ããã®ããã«åèªããšã«å®æœããå¿
èŠã¯ãªããŠã
# Word2Vecã®ã¢ãã«ã察矩èªãªã¹ã ã®ããŒãæã«äžåºŠã ã宿œããã°è¯ãéšåããããã
# å°æ¥ãåèªã«ãã£ãŠã䜿ãã¢ãã«/ãªã¹ããéãããã«ãããããããªãã®ã§ãåèªããšã«å®æœããŠãããã
# ããã³ã¢ã©ãã§å®éšããã
df_alltaigigo_filterd = create_filterd_taigigo_list("ã³ã¢ã©")
# ããå
ã®ããŒã¯ãŒãããšé¡äŒŒåºŠãè¿ãé ã«äžŠã³æ¿ããããä»ã®äžŠã³æ¿ãã«ããã«ã¯ãœãŒãããŒã倿Žããã°ãããã
df_alltaigigo_filterd = df_alltaigigo_filterd.sort_values(by='sim_inp_org', ascending=False)
# TOP5ã衚瀺ãããã
print (df_alltaigigo_filterd[:5])
org taigigo sim_inp_org sim_inp_tai sim_org_tai
120 ç±åž¯ å¯åž¯ 0.463996 0.306327 0.855372
318 æºè
¹ 空è
¹ 0.391780 0.269278 0.795424
116 è»è³ª 硬質 0.362629 0.170545 0.780534
387 人工 èªç¶ 0.360179 0.256472 0.565806
128 販売 賌å
¥ 0.355862 0.180285 0.697335
ãç±åž¯ããäžçªè¿ãèšèã§ããã®é¡äŒŒåºŠã¯ã0.463996ã
ããã¯ããçæ§ãã«è¿ãèšèïŒãç·ããèŠã€ããããšããéšåã«çžåœãã
ãã³ã¢ã©ãã«è¿ãèšèïŒãç±åž¯ãã ã£ããšããããšãïŒå¯ŸçŸ©èªãªã¹ãã«æžããèªå¥ã®äžã§ïŒ
ãç±åž¯ãããæºè
¹ããã¯ãŸããŸãçè§£ã§ãããããªãã人工ãã®æ¹ããèªç¶ãããè¿ãã®ãã
ã販売ããäžäœãªã®ãè¬ã§ããããã®ããããæ¢ã«Word2Vecã¢ãã«ã®ç²ŸåºŠã®é床ãå£éèŠãã
ãã³ã¢ã©ãïŒãç±åž¯ãïŒãå¯åž¯ãïŒ ã«ãªãã©ã¯ãŒïŒïŒ
> model.most_similar(positive = ["ã³ã¢ã©", "å¯åž¯"],negative=["ç±åž¯"], topn = 15)
[('ã«ãªãã©ã¯ãŒ', 0.7316122055053711),
('ãã€ã·ã³ã¹', 0.7241581678390503),
('ã€ããŠãºã©', 0.7234511375427246),
('æªæé', 0.7190210819244385),
('ã³ãããºã¯', 0.7122239470481873),
('åã°ã', 0.7119472026824951),
('ããªãºã', 0.7114987373352051),
('ã€ããŠ', 0.7092882394790649),
('è¥èœ', 0.7086647748947144),
('ã·ã§ãŒã³ã»ãã¯ã€ã', 0.7073724865913391),
('ã¿ããª', 0.7060376405715942),
('èæ€ç©', 0.7052432298660278),
('ã¢ã³ã·ããã§ãŠ', 0.7018641233444214),
('å
µåº«çé«ç åž', 0.7012842893600464),
('ã€ã¯ãŠãã¯', 0.699478030204773)]
ã«ãªãã©ã¯ãŒïŒïŒãã€ã·ã³ã¹ïŒïŒ
æ€ç©ã«ãªã£ãŠããŸã£ããããé«ç åžããªã©ã«ãªã¹ãªçµæã«ã
ãã³ã¢ã©ãã®å¯ŸçŸ©èªã¯ãã«ãªãã©ã¯ãŒãã§ããã ãšã·ã¥ãŒã«ãããã®ã§ãã¡ã
éã®èšç®ãªãã°ãããå°ããã·ãªçµæã«
äºæ³ä»¥äžã«ãé ããèšèã«ãªã£ãŠããŸã£ãããã
ãã³ã¢ã©ãã«è¿ãèšèã«æã£ãŠããããã«ã
éã«ãç±åž¯ãåŽãè¶³ãç®ã«äœ¿ã£ããã©ããïŒãšèããã
> model.most_similar(positive = ["ã³ã¢ã©", "ç±åž¯"],negative=["å¯åž¯"], topn = 15)
[('倿©åç©å
Œ', 0.7533067464828491),
('ãããã§ã¯ã°ã', 0.7488281726837158),
('ãããã·ã§ãã', 0.7408534288406372),
('ç¬ç«', 0.7375373840332031),
('ã»ãã»ã€ã€ã³ã³', 0.7364336252212524),
('ããã', 0.7325634360313416),
('ã¢ã©ã€ã°ã', 0.7280879020690918),
('ç±åž¯é', 0.7206511497497559),
('ã¬ããµãŒãã³ã', 0.713334321975708),
('ãã©ãã³ãŽ', 0.7120300531387329),
('ã·ã£ã', 0.7114540338516235),
('ã€ã«ã«', 0.7058818340301514),
('ã»ã€ãŠã', 0.7048503160476685),
('ãã³ã¯', 0.7012484073638916),
('ã¢ããªã«ãŸãŠ', 0.7004272937774658)]
ã倿©åç©å
¬åããäžäœã«åºãŠããã®ã¯æ°ã«ãªããã
æ€ç©ã«ãªã£ãŠããŸãããã¯è¯ãçµæã«èŠããã
ããããã§ã¯ã°ãããªã©ã¯å€å°çŽåŸæãããã
ä»ã®èšèã§ããå
šäœçã«ããéãã«äœ¿ã£ãæ¹ãããåŸåããã£ãã
ãã®æç¹ã§åœåã®æ§æ³ã¯ä¿®æ£ãå¿
èŠã«ãªã£ãã
ã€ãŸãã
ãçæ§ã ㌠ãç·ã ïŒ ã女ã = ã女çãã§ã¯ãªãã
ãçæ§ã ㌠ã女ã ïŒ ãç·ã = ãïŒïŒãã®ãããªèšç®ã®æ¹ãè¯ããšããããšã ã
ãããããå
ã®èšèãšè¿ãèšèãåŒããŠããŸããšã
å
ã®èšèããé¢ããããŠããŸãããã«ãæå³äžæã«ãªã£ãŠããŸãããšãåå ã
察矩èªãšããã©ãããçšåºŠå
ã®èšèã«è¿ãèšèãéžã¶ã¹ããšããããšã
ãŸããã倿©åç©å ¬åãã®ãããªè€åããèšèã¯é€å€ããå¿ èŠãããã
課é¡ïŒãæªããèšèãäžäœã«å ¥ãåé¡
ãä¹å¥³ãã®å¯ŸçŸ©èªãååŸããããšãããã
ãå€ãªå€ãªããšããç ããããæ¯ããããªã©ãäžäœã«å
¥ã£ãŠããã
åè©ã®å€å®ã宿œãããããã¯é€å€ããªããã°ãªããªãã
åè©ã®å€å®ä»¥å€ã®é¢ã§ããä¹å¥³ã«å¯ŸããŠå€ãªå€ãªæ¯ããã®ã¯ã±ã·ã«ã©ã³ã
> model.most_similar(positive = ["ä¹å¥³", "質å"],negative=["åç"], topn = 15)
[('ãå§«æ§', 0.8300929665565491),
('å€ãªå€ãª', 0.8079479932785034),
('åŠç²Ÿ', 0.8011484146118164),
('ãã¥ãŒããã', 0.7944810390472412),
('人é', 0.7932041883468628),
('ãµãã¥ãã¹', 0.7867072224617004),
('ç©äºº', 0.7798398733139038),
('ç ãã', 0.7614409923553467),
('æ¯ãã', 0.7611073851585388),
('ãã³ã', 0.7570786476135254),
('é»ç«', 0.7554555535316467),
('é女', 0.7538139820098877),
('çœã', 0.7514758110046387),
('çŸãã', 0.7502076625823975),
('æ»é', 0.7494140863418579)]
ãŸã ãŸã ããä¹å¥³ãâããªã€ãžããç®åºããã®ã¯é£ãããã ã
æåã®çµæ
課é¡ã¯ãããã®ã®ããšããããäžéãå®è£ ããŠã¿ãã
äžããããåèªã«å¯ŸããŠãè¿ã察矩èªïŒAâBïŒãæ±ããã
ãäžããããåèªããŒãBãïŒãAãã®æŒç®ã§ãé èªå¯ŸçŸ©èªãæ±ããã
ãã®å®è£
ã颿°åãããïŒå®è£
ã¯ã»ãŒäžè¿°ã®éãã®ããçç¥ïŒ
ãããŠã以äžã®ããã«å¯ŸçŸ©èªãæ±ããã
- å ¥åäŸïŒãã³ã¢ã©ã®ããŒãã
- MeCabïŒåœ¢æ çŽ è§£æïŒã䜿ã£ãŠåè§£
- ã³ã¢ã©ã/ãã®ã/ãããŒã
- ã³ã¢ã©ãâã察矩èªå
- ã®ãããâãïŒåè©å€å®ã§ãå©è©ãªã©ã¯å¯ŸçŸ©èªåããªãïŒ
- ããŒããâã察矩èªå
- ãã³ã¢ã©ãã®å¯ŸçŸ©èªãïŒãã®ãïŒããããŒããã®å¯ŸçŸ©èª
åææ®µéã§ã®çµæäŸ
- ã³ã¢ã©ã®ããŒã â 倿©åç©å
¬åã®ã¬ã¹ããŒã«
ãïŒæåŸ ïŒãŽãªã©ã®ã¬ã¯ã€ãšã ïŒ - ã¯ããŒãºã¢ããçŸä»£ â ç¿»åŒè¿ä»£
ãïŒæåŸ ïŒãŒãããèŠãéå»ïŒ - ãæ¯ãããšäžç· â ãã°ãã¡ãããšåé
ãïŒæåŸ ïŒãç¶ããã¯å¥å± ïŒ - ãã©ããçãæªé â ãŽã§ã«ãµãŒããäžçããµã¿ã³
ãïŒæåŸ ïŒããŸãããè±ãã å€©äœ¿ïŒ - ããããããããã§è©Šåçµäºã§ãã â å¿ããããããŸããŸã§è©Šåçµç€åŸ©æŽ»ã§ãã
ãïŒæåŸ ïŒåã£ããšæã£ããæãæããªããïŒ - èµ€ã®ä»äºº â 黿åã®è²§ä¹äºº
ãïŒæåŸ ïŒçœãæäººïŒ - ãªã¢å
ççºãã â ãã¢ä¿èªçããªã
ãïŒæåŸ ïŒãã¢ãªã¿åçž®ããïŒ - ããã 京éœè¡ãã â ããã æµ
èæ¥ãã
ãïŒæåŸ ïŒããããããïŒå€§éªãæ¥ãïŒïŒ - ã²ã¹ã®æ¥µã¿ä¹å¥³ â ã³ã©ãæ²ã®èªæèé女
ãïŒæåŸ ïŒã»ãã®ããã¥ã¢èŠªç¶ïŒ
åæçµæãžã®èå¯
ãããªãã«**ããŠãŒã¢ã¢ãã®ããçµæ**ã¯åŸããããããŸã ãŸã 調æŽãå¿
èŠã ã
è¯ãæå³ãåãããªãåèªãæ¥ãå Žåãå€ãããåè©ãªã©ã®å€æã¯é£ããã
èŸæžãå
¥ãæ¿ããããã¡ãã£ãšãã調æŽã§ããã«çãã¯å€ããã
ããšãäžçºã§è¯ãçµæãåºããªããŠãã
ä¹±æ°ãå
¥ããŠäœåºŠãããã°ãé¢çœãåè£ãåºãæãããããšããã¬ãã«ãïŒ
ä¿®æ£ã®èšç»ãšããã®æºå
å·¥å€«ã®æ¹åæ§
- 察矩èªãªã¹ãã®äžãããã©ã®çµã䜿ãã®ãããã£ãšãã¥ãŒãã³ã°ãã
- ç¹å®ã®çµã ãã䜿ããšãå šéšãæªããèšèã«ãªãå¯èœæ§ããããè€æ°ã®çµã䜿ã
- çµæã®äžãããåãåè©ã®èšèã ããæãåºãããã«ãã
- å ã®åèªãšãããçšåºŠäŒŒãŠããåèªãéžãã ã»ããè¯ãçµæã«ãªã
- ã倿©åç©å ¬åããªã©ã®è€æ°èªã¯æ¡çšããªãããã«ãã
- å€å°ã®ä¹±æ°ãå ¥ããè€æ°ã®åè£ãåºãããã«ãã
æºåïŒãåè©ïŒè€åèªã®å€å®çšã®æ©èœ
ã倿©/åç©/å
¬åããªã©ã®è€æ°èªãé€ãããŸããå
ã®èªå¥ãšéãåè©ãæé€ããããã
MecabïŒåœ¢æ
çŽ è§£æïŒãçšããŠãèªå¥ã®åè§£ãšãåè©å€å®ã宿œããæ©èœãæºåããã
åæ ŒãšãªãçµæãšããŠã¯ãè¿åŽããããªã¹ãã®é·ãããïŒãã§ããããšïŒïŒåèªãšããããšïŒãšã
ãã®åè©ããå
ã
å
¥åããèªå¥ã®åè©ãšåäžã«ãªã£ãŠããããšããïŒç¹ãå
±ã«æºããããšã
ãã®ãã§ãã¯ã®ããã«ãå
¥ååèªãåè§£ïŒåè©ãã§ãã¯ããã颿°ãäœãã
import MeCab
def check_hinsi(taisyou_word):
mecab = MeCab.Tagger("-Ochasen")
WordHinsi = []
#ååŠçïŒããªã ãªã©ãäŸã\tãã¯ããªã ããŠãããå
šè§ã¹ããŒã¹å
trline = taisyou_word.replace(u'\t', u'ã')
#Mecabã§ã®è§£æã宿œ
parsed_line = mecab.parse(trline)
#åæçµæããïŒè¡ïŒåèªïŒããšã«åå²ãããæ¹è¡ã³ãŒãã§åå²ããã
wordsinfo_list = parsed_line.split('\n')
#ç¶æ³ãèŠãå Žåã¯ã³ã¡ã³ãè§£é€
#print(taisyou_word+"ã¯"+str(len(wordsinfo_list))+"åèªã«åãããŸãã")
#åèªããšã«ãè§£æåŠçã«å
¥ããŠããã
for wordsinfo in wordsinfo_list:
#print("wordsinfo= " + wordsinfo)
# ååèªã®æ
å ±ãã¿ãã§åå²ããã
# ããç«,ã¿ãã,ããç«,åè©-äžè¬ ãªã©ã®ãããªïŒè¡ããlist圢åŒã«ãªãã
info_list = wordsinfo.split('\t')
#print(info_list)
#空çœãªã©ã®å Žåãäžçªç®ã®èŠçŽ ãç¡ãå Žåãããããã
#æåã«ãã®ãªã¹ãã®é·ãããã§ãã¯
#以äžã®ããã«ç¡æå³ã«ïŒåå²ããããããåè©æ
å ±ãååšããªããã®ã¯ç¡èŠããã
#['ã³ã¢ã©', 'ã³ã¢ã©', 'ã³ã¢ã©', 'åè©-äžè¬', '', '']
#['EOS']
#['']
if(len(info_list)>2):
WordHinsi.append((info_list[0], info_list[3]))
return WordHinsi
check_hinsi("倿©åç©å
Œ")
[('倿©', 'åè©-åºæåè©-å°å-äžè¬'), ('åç©', 'åè©-äžè¬'), ('å
¬å', 'åè©-äžè¬')]
工倫ãšããŠã以äžã®éšåã§ãMeCabã®è¿œå èŸæžãçšããŠããªãããšã
MeCab.Tagger("-Ochasen")
éåžžã¯ãMeCabã®ç²ŸåºŠããããããã«ããã匷åãªèŸæžãçšããã»ããè¯ãã®ã ãã
ä»åã®ç®çããèšããšãé«ç²ŸåºŠã®èŸæžã䜿ã£ãŠããŸããšã
ã倿©åç©å
¬åããåºæåè©ãšããŠç»é²æžã¿ã«ãªã£ãŠãäžèªãšå€å®ãããŠããŸãã
åè§£ã®åºŠåãèšå®ããæ¹æ³ãè¯ãã®ãããããªããã
åè§£ãããããŠãå°ãã®ã§ãä»åã¯**ãäžéå端ãªç¶æ
ã®èŸæžã**ãšããŠ
æ¢ããŠããã©ã«ãã®èŸæžã䜿ãããšã«ããŠããã
æºåïŒããªã¹ãäžã®åèªãã䌌ãŠããé ã«äžŠã³æ¿ããæ©èœ
ãã³ã¢ã©ãã«é¢ããŠæ§ã
ãªæŒç®ãããŠããæã«ã
ã¡ãã£ãšæ¹åã誀ããšå
šãŠã®èšèãå€ãªæ¹åã«è¡ã£ãŠããŸãããšãåãã£ãã
äŸãã°ãå
šãã³ã¢ã©ã®å¯ŸçŸ©èªã£ãœãç¡ãåèªãåºãŠãã
ãã³ã¢ã©ãïŒãåæãïŒãåè§£ãïŒãâ çŸä»£çŸè¡å®¶ã â¡é 圢äœå®¶ãâ¢ãã³ãžãŒ
äžæ¹ãæŒç®ã®æ¹åãéã«ããå Žåã¯ãåç©ãšããæå³ã§è¯ãçµæãåºãã
ãã³ã¢ã©ãïŒãåè§£ãïŒãåæãïŒãâ ã·ãã¡ã³ãã§ãŠãâ¡ãã©ãžã«ã â¢ãã¬ã¢
ãã¡ããéã«ãããæ¯åäžæããããšã¯éããªãã
ã³ã¢ã©ãæ€ç©ã«ãªã£ããé³¥ã«ãªã£ãããåç©å
¬åã«ãªã£ãããçæ°ã®å®éšã
ããã§ãäžã€ã®å¯ŸçŸ©èªãéžãã§æŒç®ããã®ã§ã¯ãªãã
ããã€ãã®å¯ŸçŸ©èªãéžãã§ããã®ããããã®çµæã®äžäœïŒäœãïŒäœãéããŠã
éããäžã§ãäžçªå
ã®èšèã«äŒŒãŠããèªå¥ãéžã¹ã°ã
ç¡é¢ä¿ãªæ¹åã«è¡ã£ãŠããŸããªã¹ã¯ãäœæžã§ããå®å®æãäžãããšèããã
è€æ°ã®èªå¥ããããèšèã«äŒŒãŠããé ã«äžŠã¹ããšããæ©èœãäœãã
# ç¹å®ã®åèªã®äžãã䌌ãŠãããã®ãéžã³ãé çªã«äžŠã¹ã
# å
šãŠã®åèªãèŸæžã«ããåæã®åäœãªã®ã§ããšã©ãŒææ³šæ
def most_niteiru(input_word, entries_list):
similarities = [model.similarity(input_word, entry) for entry in entries_list]
results = [[similarity, entry] for (entry, similarity) in zip(entries_list, similarities)]
return sorted(results, reverse=True)
# æã䌌ãŠããåèªãäžã€è¿ãã ããªãã°ä»¥äžã®ããã«ããã
# return sorted(results, reverse=True)[0][1]
most_niteiru('èžè¡', ['鳿¥œ', 'çµµç»', 'æ°åŠ', 'æžé', '俳å¥', 'ç§åŠ', 'æŽå²', 'æ¿æ²»', 'éå'])
äžèšã¯ããèžè¡ãã«è¿ãé çªã«ããªã¹ãã®äžã®åèªã䞊ã¹ããšããäŸã
詊ããŠã¿ããšããªãã»ã©ããèžè¡ãã«è¿ãé çªã«äžŠãã§ããæ°ãããã
[[0.7420505983478638, 'çµµç»'],
[0.69370409674484712, '鳿¥œ'],
[0.60828528861396158, 'æžé'],
[0.59956030567022522, '俳å¥'],
[0.57690359730963992, 'ç§åŠ'],
[0.55100553510535721, 'æŽå²'],
[0.50427923095606197, 'æ¿æ²»'],
[0.46252147384040676, 'æ°åŠ'],
[0.45263270924380355, 'éå']]
ãããŠãããããããããã¿ããããž
ãªããŠããã€æ®µèœã¿ã€ãã«ã
ãåèªãåäœã§å¯ŸçŸ©èªãå°ã颿°ãå®è£ ïŒæ¬çš¿ã®æ¬äœïŒ
ãããŸã§ã®çµæãç·åå¡ããŠã
ããåèªã«å¯ŸããŠãé èªå¯ŸçŸ©èªãåºåãã颿°ãäœãã
ãã®åšåã詊ããŠã¿ããã
# åæâ ãäºåã«model ã«word2vecã®modelãããŒãããŠããåæ
# åæâ¡ããœãŒãæžã¿ã®å¯ŸçŸ©èªãªã¹ããã°ããŒãã«å€æ°ããå©çšããåæ
df_alltaigigo_filterd = df_alltaigigo_filterd.sort_values(by='sort_val', ascending=False)
def get_natural_taigigo_NEO(input_word, input_hinsi):
FinalKouhoList = []
for index, rows in df_alltaigigo_filterd.iterrows():
# input_word + rows['taigigo'] - rows['org']ãã®Word2Vecã®æŒç®ã宿œããã
#â
ãããããã€ãŸã§ååŸãããã¯éèŠãªãã¥ãŒãã³ã°ãã©ã¡ãŒã¿
enzan_kekka_list = model.most_similar(positive = [input_word, rows['taigigo']], negative=[rows['org']], topn=15)
KouhoList = []
#䌌ãŠããèªå¥ã®äžäœããé çªã«foræã«å
¥ããŠããã
for enzan_kekka in enzan_kekka_list:
#è€æ°èªå€å®ïŒåè©å€å®ã®é¢æ°ã«å
¥ãã
enzan_kekka_hinsi_list = check_hinsi(enzan_kekka[0])
# 以äžã®ãããªçµæãè¿ã£ãŠãã
# [('倿©', 'åè©-åºæåè©-å°å-äžè¬'), ('åç©', 'åè©-äžè¬'), ('å
¬å', 'åè©-äžè¬')]
#åè©ãªã¹ããäœææã«ãè€æ°ã«åãããå Žåãããããäžèªã§ã¯ãªãã®ã§NGâé·ããïŒã®æã ãåŠçãç¶ç¶
#倿©åç©å
¬åã¯é·ããïŒãªã®ã§ç¡èŠããããšããã¯ã±
if( len(enzan_kekka_hinsi_list) == 1 ):
#åè©ãã§ãã¯ã¯ãåé ã®ïŒæåãŸã§äžèŽããŠããã°OKãšãã
#ïŒåè©ïœïœã®ããšã¯ããŸãæ°ã«ããªãïŒ
# å³å¯ã«ãã§ãã¯ããå Žå â if(input_hinsi == enzan_kekka_hinsi_list[0][1]):
if(input_hinsi[:2] == enzan_kekka_hinsi_list[0][1][:2]):
#åè©ãäžèŽãããããåè£ã«ç»é²ããã
KouhoList.append(enzan_kekka_hinsi_list[0][0])
#åè£ãªã¹ããäœãããŠããŠãïŒå以äžããã°ããã®äžäœïŒåãååŸãããïŒãªã¹ããäœãããªãå ŽåãããããïŒ
#äžåãããªãå Žåãããšããšã€ãã€ããªãªã¹ãã§ãã£ããããååŸã¯é¿ããã
if( len(KouhoList)>1 ):
#éäžçµéïŒäžäœã§ååŸããããŒã¯ãŒãïŒãèŠãå Žåã³ã¡ã³ããå€ãã
#print(KouhoList)
#ããããšã®åè£ãªã¹ããžè¿œå ã
FinalKouhoList.append( KouhoList[0] )
FinalKouhoList.append( KouhoList[1] )
#äžå®éã®ããããšã®åè£ãããŸã£ãããå
šéšæ€çŽ¢ããå¿
èŠã¯ãªããããå
šæã«ãŒããä¿®äºããã
#â
ããã€ã®åè£ãŸã§ååŸãããã¯éèŠãªãã¥ãŒãã³ã°ãã©ã¡ãŒã¿
if(len(FinalKouhoList) > 10):
break
#è€æ°ã®å¯ŸçŸ©èªãªã¹ããã«ãŒãããŠã©ãã©ãåè£ã远å ããŠãã£ãçµæã»ã»ã»ã»
#ãããäœãå
¥ã£ãŠããªãå Žåã¯ãããšã®ããŒã¯ãŒããå
¥ããŠãã(倿ããªãã®ã§ã)
if(len(FinalKouhoList)<1):
FinalKouhoList.append( input_word )
#åãèšèãäœåºŠãå
¥ãå Žåããããããäœæãããªã¹ããããéè€ãé€å»ããã
#print(FinalKouhoList)
FinalKouhoList_unique = list(set(FinalKouhoList))
#åè£ãªã¹ãã®äžãããæãINPUTã«äŒŒãã¯ãŒããã䞊ã¶ããã«äžŠã³æ¿ããè¡ãã
#äžäœNä»¶ãæœåºããã(ããŸãäžäœãååŸããŠãæå³ããªããããããçšåºŠçµã)
#â
ãã®çµãæ¹ãéèŠãªãã¥ãŒãã³ã°ãã©ã¡ãŒã¿
#æçµçã«ã¯ãrandom.choice(list)ã§çµæã®äžããã©ã³ãã ã«ååŸããäºå®ã§ããããã
nita_word_list = most_niteiru(input_word, FinalKouhoList_unique)[:3]
return nita_word_list
get_natural_taigigo_NEO("ããã","åè©-äžè¬")
[[0.92463033052039401, 'ã¿ãã'],
[0.8783315085813681, 'èº'],
[0.86586162812338352, 'ãã']]
ããã¯çµæ§è¯ãçµæã§ã¯ãªããããïŒïŒ
ãããããã«å¯ŸããŠ**ãã¿ãããã¯å€§æ¬åœ**ã ãã
ããããããããªãå¹ããŠããŠé¢çœãã
ãããããåºãŠãããšãªããé¢çœãæ°ãããŠããŸãã®ã¯ãªãã ããã
å¥ãªèšèã§ã詊ããŠã¿ã
print(get_natural_taigigo_NEO("ã³ã¢ã©","åè©-äžè¬"))
print(get_natural_taigigo_NEO("éŽäž","åè©-äžè¬"))
print(get_natural_taigigo_NEO("ã«ãŒãã³","åè©-äžè¬"))
print(get_natural_taigigo_NEO("ä¹å¥³","åè©-äžè¬"))
print(get_natural_taigigo_NEO("ã¹ããŒããã©ã³","åè©-äžè¬"))
print(get_natural_taigigo_NEO("åç©","åè©-äžè¬"))
print(get_natural_taigigo_NEO("çŒé¡","åè©-äžè¬"))
print(get_natural_taigigo_NEO("ã«ã¬ãŒ","åè©-äžè¬"))
print(get_natural_taigigo_NEO("ã©ãŒã¡ã³","åè©-äžè¬"))
print(get_natural_taigigo_NEO("ããŒã«","åè©-äžè¬"))
print(get_natural_taigigo_NEO("é£è¡æ©","åè©-äžè¬"))
[[0.79610911695616471, 'ã»ãã»ã€ã€ã³ã³'], [0.785405187925722, 'ãã³ã¯'], [0.76901927799464209, 'ãããã§ã¯ã°ã']]
[[0.91507178306705739, 'æè¢'], [0.90161941201164741, 'é©éŽ'], [0.8959922878688289, 'ã»ãŒã¿ãŒ']]
[[0.85584726609771655, 'ãœãã¡ãŒ'], [0.83979639140314122, 'ãã©ã³ã¯'], [0.83008699669693065, 'ããŒã']]
[[0.87598038376876186, '人é'], [0.86406344018281578, 'åŠç²Ÿ'], [0.77857242742516286, 'ã«ãŒãã³ã¯ã«']]
[[0.92941233856523409, 'Android'], [0.89327022466870609, 'ãã£ãŒãã£ãŒãã©ã³'], [0.88803100750540598, 'ã¢ããª']]
[[0.91011603498044402, 'ç¬è«é¡'], [0.90208324646657068, 'åºä¹³é¡'], [0.88445087859468874, 'ã€ã']]
[[0.90876897001328705, 'ã¡ã¬ã'], [0.85207171387504843, 'ãã©ãžã£ãŒ'], [0.81841303414529509, 'éŽäž']]
[[0.86714373685808754, 'çŽ
è¶'], [0.85008168395949824, 'ã¹ãã²ããã£'], [0.84403310700446899, 'ã±ãã£ãã']]
[[0.95296741693813036, 'ãã©ã'], [0.91918231065962774, 'ããçŒã'], [0.90987101568564399, 'é€å']]
[[0.89793376386512991, 'ãŠã€ã¹ããŒ'], [0.89059171717745844, '飲æ'], [0.88606741254739563, 'çä¹³']]
[[0.83567369949489889, 'é£è¡è¹'], [0.82413943992220307, 'æ°ç'], [0.82146012956360659, 'ã°ã©ã€ããŒ']]
çŒé¡ãšã¡ã¬ããå¥ãªèšèæ±ãã«ãªã£ãŠããããšããçªã£èŸŒã¿ã©ããã¯ãããã®ã®ã
éŽäžvsæè¢ãã©ãŒã¡ã³vsãã©ãããªã©ã¯å€§å€è¯ãã
ã«ã¬ãŒã¯é£²ã¿ç©ãšããŠã屿§ãæã£ãŠããããçŽ
è¶ãªã®ãïŒïŒ
ã¹ããŒããã©ã³vsãã£ãŒãã£ãŒãã©ã³ã¯ïŒäœã§ããã綺éºãªçµæã
çŒé¡vsãã©ãžã£ãŒãã å
±ã«äºã€ã®å圢ãšããå
±éç¹ããããè峿·±ãçµæã ã
ãçŒé¡ãããããã®å¯ŸçŸ©èªãããã©ãžã£ãŒãå€ãããªãã°ãå€ãããšãããããèŠãããšã§ãããã®ãïŒ
æçµçïŒãèµ€ã®ä»äººãã¯ãæ°Žè²ã®èªåãã«ãªã£ã
ããããããæç« ãã«å¯ŸããŠäžèšã®å¯ŸçŸ©èªåŠçãçµã¿èŸŒã
ãããŸã§ããã°ããæç« ãã«å¯ŸããŠå®æœããã®ã¯é£ããã¯ãªãã
ããã¡ãã£ãšé·ããªã£ãŠããããäžçªéèŠãªå·¥å€«ã¯ãåèªãã«å¯ŸããåŠçãªã®ã§ã
ããã³ãŒãç¡ãã§çµæã ãæžãããšã«ããã
ãããæ¬æçš¿ãžã®ãããããã300éæãããã
ãæç« 倿ãã®å
šéã®ã³ãŒããšãããã€ã端æã£ãéšåãèŒããŠæŽæ°ããŸãïŒ
ãã£ãšãã£ãããã®åŸã®ç²ŸåºŠæ¹åãšãµãŒãã¹å
¬éã®æ€èšãããŸãïŒ
ã£ãŠå®£èšããŠãããªããšãèå³ãæã£ãŠããã人ãããŠãé¢åãã£ãŠæ¹åããªããªã£ãŠããŸã
â éæããããã远èšããŸããã
æçµçµæãµã³ãã«
ä¹±æ°ã§äžåºŠã«ïŒåã®çµæãåºããŠããããããã®äžã§ã¡ãã£ãšé¢çœããã®ãèŒããŠã¿ãã
- èµ€ã®ä»äºº â æ°Žè²ã®èªå
- ã³ã¢ã©ã®ããŒã â ãããã§ã¯ã°ãã®ã«ãŒ
- ã¯ããŒãºã¢ããçŸä»£ â åãæ²æ±°å€å ž
- ãæ¯ãããšäžç· â ãããããšåé
- ãã©ããçãæªé â ã°ãããç©¿ããæ»ç¥
- ããããããããã§è©Šåçµäºã§ãã â å¿ãããããã¡ãã¡ã§ã¹ããŒãªã³ã°ç¶ç¶ã§ãã
- ã¡ã³ãŸãåã¡ãã â ãã«ãŸãå«ãããã
- ã¢ãŠããªãçŒäž â ã»ãŒããšã³ãã€ã¢ãã©ã¿
- ã²ã¹ã®æ¥µã¿ä¹å¥³ â ããããŒã®ç¡åžžäººé
- çœç·ã®å åŽãŸã§ãäžãããã ãã â æããã®äžç«¯ãŸã§ãäžããããã ã
- ããŒã«ã¯åé â ãããã¯åœŒæ°
- åã¯æ°äžçã®ç¥ã«ãªã â ä¿ºã¯æ°å®å®ã®å¥³ç¥ã«å ¥ãæ¿ãã
- æ©åæŠå£«ã¬ã³ãã â 察空éªç¥ãžã§ã€ãœã³è²¯æ°Žæ±
- ãŠãµã®ãšã«ã¡ â ãã³ãšããªããºã
- 鲿ã®å·šäºº â éåŽã®ã€ã¯ã«ã
- éæ³å°å¥³ â 粟éçŸå°å¹Ž
- æ»ã«ç©çã â æ²ããéç£
- çççã«ç¡ç â ã¢ã¬ã«ã®ãŒçŽç²ã«ãŸãšã
- çŸãã¡ã®æ²é» â é¶éã®é©æ
- å·ããäžè¯å§ããŸãã â ãªã ã©ã€ã¹å¿ããŸãã
- ãŠãµã®ã¯å¯ãããšæ»ã¬ â ã«ãšã«ã¯èŸããšçµ¶ãã
- æã®ã²ãã倪éœã« â å£å ãå°çã«
- é±åå°å¹Žãžã£ã³ã â æå幌女å®è¿ã
- ããææ ¢ â ãšãããéäž
- è¬è§£ãã¯ãã£ããŒã®åŸã§ â æ£äœèŠã€ãåºãã¯æé£ã®çŽåã§
æçµçµæèå¯
ãªããªã人ãäœã£ã以äžã«é¢çœããã®ã¯åºæ¥ãªããã®ã®ã
é©åœã«èªå¥ãå
¥ããŠããããçšåºŠïŒïŒïŒïŒ
ãããïŒïŒå¯ŸçŸ©èªã£ãœããã®ãè¿ã£ãŠããã
ãïŒäžèšã®çµæãµã³ãã«ã¯ãå
šãèªèãããããããšããæ®ãïŒïŒïŒ
ãé€ãã
ããå€å°ã€ãŸããªããã®ãã粟床ã®ãµã³ãã«ãšããæå³ã§æ²èŒïŒ
ãæ¯ãããšäžç·ãªã倧äžå€«ã ãããããããšåéã£ãŠèªæããã¡ããããã
æããã®äžç«¯ãŸã§äžãããã®ç¡æå³ãå æžãããã
ããŒã«ãåéã«ããç·ã®åããããªãããããã圌æ°ã«ãã女ã®åãããŠããããããããªãã
ãã³ãšããªããºãã¯ãå šããäºãç¡é¢ä¿ãã€ãã¬ãããã ã
é¶éã®é©æã¯ããšãŠãå°ããªããšã«é©ããŠããããã§åŸ®ç¬ãŸããã
ãªã ã©ã€ã¹ã«ããã£ãŠã¯ãå§ãã以åã«å¿ããã ãããïŒ
å£å ãå°çã«ãã¯ç¶ºéºã«å¯ŸçŸ©èªãªæããããã
æå幌女å®è¿ããã¯äžéšã«å€§å€ãŠã±ãããªéèªããææ¬ã®ãæåè人ã¹ã¯ã¯ãããããé¢çœãã
ïœïœãéããããã®ã³ã¡ã³ããŒã¿ãŒ
ãã¯ãããåè©ããäžçªæ±ããããããåè©ãã¯é£ãããšããå°è±¡ã
ãèµ€ã®ä»äººããããçœãæäººããŸã§ãåçŸããããšã¯åºæ¥ãªãã£ããã
ãããŸã§ããã«ã¯åæµ·éã®éèã«ã€ããŠãåŠç¿ãããå¿
èŠãããããã
ãæ°Žè²ã®èªåããŸã§ãä»åã®éçãªæ°ãããã
å®ã¯ãæ¶ç©ºå¯ŸçŸ©èªbotãªãå
é§è
æ§ãããã£ãããããšãèªèããŠãããã
ã¢ãããŒãã®ä»æ¹ã䌌ãŠããããã§ç°ãªããåºæ¥ãçµæãšããŠãã
åŸæäžåŸæãããªãç°ãªãçµæãšãªã£ããšèŠãŠããã
æ¹åããæ¹åãšããŠã¯ã
Word2Vecã®ã¢ãã«åã³MeCabã®ç²ŸåºŠã«äŸåãããšããã倧ããããã
匷åãªèŸæžã䜿ããã°ç²ŸåºŠãäžããå¯èœæ§ãããã
ç¹ã«ãè¡šèšæºãã®ä¿®æ£ããåè©ã®æ±ãæ¹ã®å·¥å€«ããåºæ¥ããšæãŸããã
ïŒåè©ã®æŽ»çšã«ã€ããŠã¯ç¹ã«é¢åã ãïŒ
ãŸããä»åã®å¯ŸçŸ©èªãªã¹ããã·ã³ãã«ãªãã®ã§ããããã
ãã£ã𿬿 Œçãªå¯ŸçŸ©èªãªã¹ããçšæãã
åçš®ãã©ã¡ãŒã¿ãŒã®ãã¥ãŒãã³ã°ããã¡ããšè¡ãã°ã
ãã粟床ãããããããããªãã
ããããã粟床ããšã¯äœããšãããšã
ä»åã®ç®çã¯ãããŸã§ãé¢çœããããšãªã®ã§ã
å€å°ç²ŸåºŠãæªãæ¹ããéã«æããããªãã£ãèšèã«ãªã£ãŠé¢çœããããããªãã
ããŸããèšããããããªã
äŸãã°ãäžèšã®äŸã§ã¯ãã¬ã³ãã ããèªèãããã«ã
ããã ãããè²¯æ°Žæ± ãã«ãªã£ãŠããŸã£ãŠããã
ä»®ã«ãã¬ã³ãã ãããã¶ã¯ãã«å€æããããšããŠããã©ã¡ããé¢çœãã®ã倿ã¯é£ããã
ã察空éªç¥ãžã§ã€ãœã³è²¯æ°Žæ± ãã¯åŒ·ããã ã察空ã«éå®ãããŠããã®ã§äœ¿ãã©ãããé£ãããŠããã
æ£ç¢ºã«åå¯Ÿã®æå³ã«ãªã£ãŠããªãã£ããšããŠãã
ãæ²é»ããã驿ãã«ãªããããªïŒåŸ®åŠã«å€ããïŒã»ã³ã¹ã¯ã
ãªããªã人ã§ã¯æãã€ããªãå³ããããšæãã
ã©ã®ããããé¢çœãã®ãã¯ãå€ãã®äººã®æèŠãèããŠãããã¥ãŒãã³ã°ããæ¹ããããããããªãã
ïœçµããã«ïœãïŒããšã ïŒ
**ãæ¬æ°ãªã¢ã€ãã¢ãã¯ããç¬ãã**ãšãã䌌ãŠããŠã
ã©ã¡ãããæãããããªãã£ããã®å士ãçµã³ã€ãããšã«ãã£ãŠçããã
ãã ããæ¬åœã«ç¡é¢ä¿ãªãã®å士ã¯ãçµãã§ããã£ä»ããªãã®ã§æå³ãç¡ãã
å察ã®äœçœ®ã«ãããªãããããçšåºŠäŒŒããã®ãç¹ããå¿
èŠãããã®ã ã
ä»åã®ããŒãã¯ããŸãã«ãããæ©æ¢°ã§å®çŸãããããšãã詊ã¿ã§ããã
決ããŠã圹ã«ç«ã€ãããŒã«ã§ã¯ãªããã
æãããããªãã£ãçºæ³ãã¢ã€ãã¢ãç¬ãããæ©æ¢°ã«åºãããããšããåŽé«ãªè©Šã¿ãªã®ã ããã¶ãã
é¢çœããšæã£ãæ¹ã¯ãã²ãããããããŠã¿ãŠãã ããã
ããããå€ã ïŒ èå³ãæã€äººãå€ãå Žåãå
šéã³ãŒãã®æ²èŒãããã®åŸã®æ¹åã®è¿œèšæŽæ°ãããŸãã
ããã«æ³å®ãã倧å¹
ã«å€ããã°ã粟床æ¹åïŒãµãŒãã¹ãšããŠã®å
¬éãæ€èšããŸãã
以äžã
倿°ã®ããããžã®åŸ¡ç€Œè¿œèšïŒäŸæè¿œå ç
2018/5/8 æ¬ç« ã远èšïŒ
äºæ³ãã¯ããã«è¶
ããé床ã§ã300ã©ãããæ¢ã«äºæ¥ãããã«500ããããé ããŠããŸããã
ã¿ãªããŸããã®ã声æŽãã³ã¡ã³ããããã«åŸ¡ç€Œç³ãäžããŸãã
äºåã«å®£èšã®éããæªèšèŒã®ã³ãŒããªã©ã远èšããäºå®ã§ãã
ããæ¥éãããŠèšäºäœæãéã«åããªãã®ã§ãåœåº§ãã®ããšããŠã
æžãã®ã楜ãªã¯ãªã«èŠãŠé¢çœãã®ã§
ããå°ããããããªèšèãå
¥ããŠã¿ãŠãäœåäŸãå¢ãããŠã¿ãããšã«ããŸããïŒ
ãããããªèšèã詊ããŠåŸåãèŠãªããšãè¯ãç¹æªãç¹ã¯åãããªããã®ã§ãã
- åºåæ¹æ³ã¯åååæ§ã«ãä¹±æ°ä»ãã§ïŒåçæãããŠããã®äžã§äžçªé¢çœãããªãã®ãèšèŒ
ïŒäžçºåè² ã§ã¯äœäŸã»ã©ã®é¢çœãã«ãªããªãã®ã§ãæ®éã«äœ¿ãããã«ã¯ããã¡ãã粟床åäžãå¿ èŠïŒ
远å äœåäŸâ ïŒ
ã¡ãã£ãšé£ããããªé·ãã®èšèãæŒ«ç»ãªã©ã«ãææŠã
- ç¬ãæ©ãã°æ£ã«åœãã â ç«ã楜ããã°ããŒãã«æºãã
- (â»ç¬ãç«ã¯ãã£ã±ãã©ã€ãã«)
- ã©ãããã®èæ¯ã¹ â ã²ããã®éŒ»èŒã¹
- (â»ããã¯å¯ŸçŸ©ã«ãªã£ãŠããªãã§ããã倧çç¬ããããäžçªãïŒïŒ)
- ç³æ©ãå©ããŠæž¡ã â å±±åŽãæ¯ãåããŠèŸ¿ã
- (â»ç³æ©ã人åçã«æ±ãããŠããæš¡æ§ã)
- ããã ãå¬ãããã çããåã³ãããšãèžã®å·ãçãã§ã â ããã ãããããããã£ãããã è²ãæ²ãã¿ããã¡ããçæ¯ã®ç«å·ãæ²»ãã§ã
- (â»çæ¯ã®ç«å·ããæ¬æ°)
- 空ãèªç±ã«é£ã³ãããªãã¯ããã¿ã±ã³ãã¿ãŒ â çŸé¢šãå¹³çã«é£ã³åããããªããŸãããã©ããã
- (â»å¹³çããŸããã®ãããã®ãã³ã·ã§ã³ã®äœãã奜ã)
- ããã®ãŸãŸä»èµ·ãã£ãããšã話ãã â ããã®ãŸãŸãã€èµ·ãããäºãèãã
- (â»äœã§ãèããçãªæãã«)
- äžæéä¹ã¯æµ
ååãæããŠããŸããäžçäžã®èª°ãããã â åæ¡éŠç¹ã¯æ©ååãæããŠãããŸããåœå
å€ã®ãªã«ãããã
- (â»éŠç¹ã£ãŠèª°ã ã!?æããã®ãã!?åœå å€ã®äœãããã£ãŠåããã©ããããšãçªã£èŸŒã¿ã©ããå€ã)
- ããŸãããã³ããŒïŒã ïŒïŒ â ããããã°ã©ã³ããŒãªã¹ã¢ïŒã ïŒïŒ
- (â»ã·ã¥ãŒã«ãããŠçç¬ããªãã§ã°ã©ã³ããŒãªã¹ã¢ïŒãªã®ïŒïŒããžãŒã¿ããã)
- éã¯åœããéãïŒ â å ±å¥šã¯æšéãã軜ãïŒ
- (â»å ±å¥šãäžèŠéããã«èŠããŠå®ã¯è»œãããã ã ã ãè³åé»ç€ºé²ã«ã€ãžã®ããã¬ã¯ã®èšèãã)
- ãã£ãã²ãšã€ã®çå®èŠæããèŠãç®ã¯åäŸãé è³ã¯å€§äººããã®åã¯åæ¢åµã³ãã³ â ãã£ãäžçš®ã®ççžéšããæè§Šãã¯å€§äººãç¥åã¯å¹Œå
ãäœããã®ç·å¢ã¯ååäºé圢
- (â»ä»åã®åäœãããé圢ã®ãšã£ã€ããããããæè§ŠãããGood)
- ç§ã®æŠéåã¯53äžã§ã â ããªãã®ç©ºæŠèœåã¯46å
ã§ã
- (â»å¯Ÿç©ºéªç¥ãžã§ã€ãœã³è²¯æ°Žæ± ã®èœåã倿ïŒïŒ)
é·ãã®èšèãåè©ãªã©ã¯ããã¯ãã¡ãã£ãšé£æåºŠãäžããæ°ãããŸãã
远å äœåäŸâ¡ïŒïŒäŒæ¥ç³»ïŒ
äŒæ¥ç³»ã¯ãã©ã€ãã«ç€Ÿãªã©ãåºãå Žåãå€ããšå€æã
- ãå£ã®æäººãããã â ãå°äŸ¿ã®å¹Œãªãã¿ãéªç¥ã¿ã€ã¬ãŒã¹
- (â»å£ããå ¥ã£ãŠæåŸã¯å°äŸ¿ã«ããšããããšïŒã©ã€ãã«ã¯çå£ã§åºã)
- ããªããšãã³ã³ãã«ããã¡ããªãŒããŒã â ç§ãšãããªãªã«ãããŒãœã³
- (â»ããªãªã®ïŒäººç®ã¯èª°ã ã)
- ã»ãã³ã€ã¬ãã³ãããæ°å â ãŠã«ãã©ã¬ããªã¥ãŒã·ã§ã³ãã€ãŸããªãå€ç¬
- (â»ãŠã«ãã©ãã³ã»ãã³ããïŒïŒ)
- ã€ã³ãã«å
¥ã£ãŠã â IBMç§»ãã¡ãã£
- (â»ç§»ãããšã«ãªã£ã)
- ãããããªãããšãŸããªããã«ã«ããŒããã£ã±ãã³ãã â å§ãããããªãããšãŸã£ãªãããµãããããŒã«ãåã«ã³ã§ã
- (â»ãåã«ã³ã§ãããžã®é£ã³å ·å)
- ã«ã©ãã«ããŒã¹ãã«ã«ãã¹ â ãšãããã«ããã¯ã¹ãããŠãŒããŒ
- (â»ã«ã©ãããå¿ã®ãšãããã«ãã«ã«ãã¹ãšãã¥ãŒããŒã¯ãªããããã£ãœã)
- ã³ã³ããæºã¿ã³ã«ãã³ã¹ã¢ç³æ²¹ â ããã©ãæºã©ã ã«ãäžæŽé»æ©
- (â»ã¿ã³ãã¯èãšããŠèªèãããå¯èœæ§ã)
è£è¶³ã³ãŒãã¯è¿æ¥ãïŒæäžã«ã¯èšèŒããŸãããã
âèšèŒããŸããã
埡瀌远èšãã®â¡ïŒæç« åŠçç
äºåã«å®£èšã®éãããæç« 倿ãã®å
šéã³ãŒããæ²èŒããŸãã
ïŒâ»ãã宣èšããŠããªãã£ããå€åãµãã£ãп޿°ããŠããªãã£ãã®ã§ãå¿æŽããŠãããæ¹ã«æè¬ïŒ
ãŸããããå°ãç²ŸåºŠã®æ¹åãé¢çœãUPã®æ¹æ³ãæ€èšäžã§ãâå¥èšäºã«ãªãäºå®ã
æç« å šäœã«å¯ŸããŠã察矩èªãçæãã颿°
åŠçã®æµããšããŠã¯ãMecabã§åœ¢æ
çŽ è§£æãããŠã
åºãŠããåèªãäžéšã§äœæããŠãããããåèªãâã察矩èªãªã¹ããã«ãã颿°
ïŒãããããããã¿ãããã«ãã颿°ïŒã«é çªã«å
¥ããŠããã
import MeCab
import random
def get_Taigigo_bun(inputtext):
mecab = MeCab.Tagger("-Ochasen")
#èªåœãå€ãããèŸæžã䜿ããšã察矩èªã«ãããã¯ãŒãã
#äžèªãšèªèãããŠããŸãå Žåãå¢ããŠããŸãã®ã§ããããŠèŸæžã¯äœ¿ããªã
#mecab = MeCab.Tagger(r"-Ochasen -d .\mecab-ipadic-neologd")
resulttext = ""
#ååŠçïŒããªã ãªã©ãäŸã\tãã¯ããªã ããŠå
šè§ã¹ããŒã¹å
trline = inputtext.replace(u'\t', u'ã')
#####ã¡ã€ã³åŠçïŒ
#Mecabã§ã®è§£æã宿œ
parsed_line = mecab.parse(trline)
#éäžçµéãèŠãŠã¿ãå Žåã¯ä»¥äžã®printã§ã
#parseçµæã¯ãæ¹è¡åºåãã§å
šåèªã®åè©æ
å ±ãæ ŒçŽãããŠããã
#print("== parseã³ãã³ãã®çµæ ==")
#print(parsed_line)
#åæçµæããïŒè¡ïŒåèªïŒããšã«åå²ãããæ¹è¡ã³ãŒãã§åå²ããã
wordsinfo_list = parsed_line.split('\n')
#åèªããšã«ãè§£æåŠçã«å
¥ããŠããã
for wordsinfo in wordsinfo_list:
#print("wordsinfo= " + wordsinfo)
# ååèªã®æ
å ±ãã¿ãã§åå²ããã
# ããç«,ã¿ãã,ããç«,åè©-äžè¬ ãªã©ã®ãããªïŒè¡ãlist圢åŒã«ããŠèŠçŽ ãåãåºããããã«ã
info_list = wordsinfo.split('\t')
#print(info_list)
#空çœãªã©ã®å Žåãäžçªç®ã®èŠçŽ ãç¡ãå Žåãããããã
#æåã«ãåºæ¥ããªã¹ãã®é·ãããã§ãã¯
if(len(info_list)>2):
#äžéšã®åè©ããäžéšã®äŸå€åèªã¯ã倿åŠçåŠçã宿œããªãã
if( info_list[3].startswith('å©è©') or
info_list[3].startswith('å¯è©-å©è©é¡æ¥ç¶') or
info_list[3].startswith('åè©-ç¹æ®-å©åè©èªå¹¹') or
info_list[3].startswith('ãã£ã©ãŒ') or
info_list[3].startswith('åè©-æ¥å°Ÿ') or
info_list[3].startswith('èšå·') or
info_list[3].startswith('æ¥é è©') or
info_list[3].startswith('å©åè©') or
info_list[0]==u'ã®'
):
#äžèšãã¿ãŒã³ã¯ãäœãããªã
resulttext+=info_list[0]
else :
#åè©ã®å Žåãååã«å¯ŸããŠå¯ŸçŸ©èªãåãããå
ã
ã®å
¥åã«å¯ŸããŠå¯ŸçŸ©èªãåããïŒéãããã
#ã©ãã察矩èªåãããšãã«å€æã¯ããããããå€å°ã®ããã¯ç¡èŠããŠååã察象ãšããã
#ïŒåè©ä»¥å€ã¯ãååãå
ã
ã®å
¥åãå€ãããªãã¯ããªã®ã§åœ±é¿ããªãïŒ
#å
ã
ã®å
¥åã䜿ãå Žå
#word = info_list[0]
#ååã䜿ãå Žå
word = info_list[2]
#åè©ãä¿åãã
word_hinsi = info_list[3]
try:
#ããŒã¯ãŒããèŸæžã«å
¥ã£ãŠããã確èªããã
#äžã®ãã¯ãã«ãã®ãã®ãåºããïŒãªããã°ãšã©ãŒã«ïŒ
out = model[word]
#以äžã§ãå¥éäœæã®ãåèªâ察矩èªã®ãªã¹ããè¿ã颿°ãåŒã³åºã
taigigo_kouho_list = get_natural_taigigo_NEO(word, word_hinsi)
#ä¹±æ°ã§ãã®ãªãããäžèªãéžã¶
taigigo = random.choice(taigigo_kouho_list)[1]
resulttext += taigigo
except KeyError:
# ã¢ãã«å
ã«ããŒã¯ãŒããç¡ãå Žå
#ãå
šäœããšã©ãŒã«ãããªãããããšããããå
ã®èšèãå
¥ããŠãã
#print(word+"ã§ãšã©ãŒçºç")
#print(KeyError)
resulttext += word
#åè¡ãåèªåå²ããçµæããšã®Foræã®çµäº
#print(resulttext)
return resulttext
# äžèšã®é¢æ°ã¯ãã©ã³ãã ã§äžã€ãè¿ããããä»»æã®åæ°ç¹°ãè¿ãããã
# ãããã°çšã«è¡šç€ºãã颿°ãã€ããŠããã
def get_Taigigo_bun_kurikaesi(input_bun, kurikaesi):
print(input_bun)
print("== çµæ ==")
for i in range(kurikaesi):
print(get_Taigigo_bun(input_bun))
print(" ")
# ãã¹ã
get_Taigigo_bun_kurikaesi(u'èµ€ããŠäžžããŠããããããã',5)
== å
¥å ==
èµ€ããŠäžžããŠããããããã
== çµæ ==
é»ããŠå¹³ãããŠããŸãèº
éããŠå¹³ãããŠããŸãã¿ãã
é»ããŠåè§ããŠçŸå³ããèº
çœããŠçްé·ããŠãããããã
éããŠçްé·ããŠããŸãèº
äžã®ããã«ã©ã³ãã ã§ïŒååºãã®ã§ããã®å Žåã
ãçœããŠçްé·ããŠããããããããªããŠå¥è·¡çã«äžæãåºãŠããã®ã§ã
ããã€ãããããããã ãããšã«ããŸããããã åºåäŸãšããŠèšèŒãããïŒ
ã€ãŸãã以äžã®ããã«åŠçããŠããã
- èµ€ãâèµ€ããâé»ããéããçœã
- äžžãâäžžããâå¹³ãããåè§ãã现é·ã
- ãããããâãçŸå³ãããããããïŒããããå€æã¯æ¬æ¥ã¯é€å€ãã¹ãïŒãããŸã
- ããããâãã¿ãããèºããã
æç« åŠçã®çµæèå¯
ãã®äŸã¯ã匱ç¹ãšè¯ãç¹ãåãããããã§ãŠããã
è¯ãç¹ãšããŠã¯ãåå¥ã®å€æåŠçïŒç¹ã«åè©ïŒã«å¯ŸããŠã¯ã
äžæãåããŠããããšã ãšæãã
äžæ¹ã匱ç¹ã¯ä»¥äž
- 匱ç¹
- åè©ã圢容è©ãªã©ã®æŽ»çšåœ¢ããäžæŠååã«æ»ããŠãããããã¡ãã£ãšèªå°Ÿãããããã
- ããã¯å°éå®¶ãªãçŽããŠããããïŒæ¬çš¿ã®è¶£æšãšã¡ãã£ãšãããåãçµã¿ã«ãªãããæ·±è¿œããã
誰ãçŽããŠããããå¬ãããª
- çŸå³ãããããããããªã©ã»ãŒåæ§ã®æå³ã®åèªãèŸæžã«å«ãŸãããããåºãŠããŸãã
- å
šãåäžã®èªã¿æ¹ã®ãã®ã¯ãäŸå€åŠçã§é€å€ã¯å¯èœã
é¢åãªã®ã§ãã£ãŠããªãã ã - éãæŽ»çšã®ãã®ããååãæ¯èŒããŠé€å€å¯èœãäžãšåãããŠæ¬æ¥ã¯ãã£ãã»ããããã
- å
šãåäžã®èªã¿æ¹ã®ãã®ã¯ãäŸå€åŠçã§é€å€ã¯å¯èœã
- åèªããšã«è§£æããããããçœããããããã¯ãã ã®å¶ç¶
- ãéãããèºãããªã©ãåºçŸããŠããã
- ååŸã®åèªãšã®é¢é£æ§ãèãããšãããèªç¶ãªè¡šçŸã«ãªããããããªãã
- ãã ããã©ã¡ããé¢çœã衚çŸã«ãªããã¯å šãåãããªã
- Mecabã®åè§£æ§èœã«äŸåïŒäžèšäŸã§ã¯ãªããïŒ
- ã¬ã³ãã âãã¬ã³ãããã ãã«å¯ŸããŠããžã§ã€ãœã³ããšãè²¯æ°Žæ± ããåºãäŸ
- åè§£ããå Žæã«ãã£ãŠãããããªå€æã«ãªã£ãŠããŸãã
- ãã ããã©ã¡ããé¢çœã衚çŸã«ãªããã¯å šãåãããªã
- åè©ã圢容è©ãªã©ã®æŽ»çšåœ¢ããäžæŠååã«æ»ããŠãããããã¡ãã£ãšèªå°Ÿãããããã
ãã¥ãŒãã³ã°ã®ãã€ã³ã
ãã¥ãŒãã³ã°ã«ãã£ãŠçµæãå€ãããšããã¯å€æ°ããã
æ¬è³ªçãªç®æã¯â
- å
šäœ
- â Mecabã®èŸæž
- â
Word2Vecã®ã¢ãã«
- â»æ¬æ¥ã¯Word2Vecã¢ãã«ãåç©ã«ãããMecabã§äœãã°èŸæžã¯åã
- â 察矩èªã®èŠæ¬è¡šcsvïŒä»åã¯200èªïœ300èªã®ç°¡æçãªãã®ïŒ
- æç« â察矩èªã®é¢æ°ã«ã€ããŠïŒèšäºäžã®é¢æ°ã®ã³ã¡ã³ãã«ãèšèŒïŒ
- â åè©ãäŸå€èªå¥ã®èšå®ãæ±ãæ¹
- åè©ã圢容è©ã«å¯ŸããŠãååãåŠçorå ã®å ¥åå€ãåŠç
- çæåŠçæã«äœåçæãããïŒïŒä»åã¯ïŒåãã€åºããŠããïŒ
- åèªâ察矩èªãªã¹ãã®é¢æ°ã«ã€ããŠïŒèšäºäžã®é¢æ°ã®ã³ã¡ã³ãã«ãèšèŒïŒ
- â 察矩èªã®èŠæ¬è¡šïŒcsvïŒã«å¯ŸããŠãäœããœãŒãããŒã«ãããïŒå ¥ååŸãšã®é¡äŒŒåºŠãªã©ããæŒç®ïŒ
- èšèã®æŒç®åŠçæã«ãçµæäžäœtopããã€ãååŸãããïŒ
- ãåè£ããéããéã«ãããã€ã®åè£ãååŸãããïŒ
- å ã®èšèã«è¿ãé ã«éžå®ããæã«ãããã€ã®åè£ãååŸãããïŒ
æ©æ¢°ãäœãé¢çœãã®æ¬è³ªãšã¯ïŒïŒããšã ïŒ
äžèšã®ãã¥ãŒãã³ã°ãã€ã³ããçºããŠããã ãã
ã²ãšã€ãæ¬æçš¿ã®äžã§æå€§ã®ãã€ã³ããšãèšãã¹ã芳ç¹ã¯ã
ãã®ã£ã°ããé¢çœãããšããäžååŠç¿ãããŠããªããšããããšã
ãé¢çœãããšããæ©æ¢°åŠç¿ãããŠã
人éãé¢çœãããšãèšãã®ãšåæ§ã«ã人éã®ç䌌ããããšããã¢ãããŒããã
äžã€ã®ã¢ãããŒããšããŠéèŠã ãšæãã
ãããããé¢çœããïŒãé¢é£æ§ãããã®ã«ãªããªãæãã€ããªãããšããšããŠã
å
šã人éãäœã£ãé¢çœãããå
šãåŠã°ãªãã£ããšããŠãã
æ©æ¢°ã«é¢çœãããšãèšãããããšãåºæ¥ãã®ã ãšæãã
ä»åã®çµæã«éã£ãŠèšãã°ãæ©æ¢°ãäžå/äžååã§ããããã«
ïŒã·ã¥ãŒã«èŠçŽ ãå
¥ã£ãŠïŒé¢çœãã£ããšããçµæããããã
ä»®ã«ãMecabãWord2Vecã®ã¢ãã«ãç©åãé«ç²ŸåºŠã«åºæ¥ãŠããŠãã
ãããé«ç²ŸåºŠã«åºæ¥ãŠããæ¹ããé¢çœãçµæãçãŸãããšæãããã
ããããèšèã®æ¹åæ§ïŒãã¯ãã«ïŒãæ©æ¢°ã¯åçã«æ±ãããã«ã
é垞人éã§ã¯å
šãæããã€ããªãã£ãæ¹åãžèšèãæ»ãæ²ããããšãåºæ¥ãã®ã ã
ç¬ãã ãã§ãªããã¢ã€ãã¢åºãããã¬ã¹ãã«ãããŠãããã®æ§è³ªã¯æçšã ã
æ©æ¢°ã®äž»èгã¯ã人éã®äž»èгãšãŸãéã£ãæèŠãçã¿åºããŠããã
ããããããç®çã«å¿ããŠãã¥ãŒãã³ã°ã§ãããå°æ¥ããçžè«çžæã«ãªãã ããã
ããŸãïŒè¿œå äœåäŸâ¢ïŒ
æè§ãªã®ã§ããã€ã远å äœåäŸãã
- èµ€ãåœæã®ã·ã£ã¢ â éãåæã®ãžã¥ããŒ
- (â»ãžã¥ããŒïŒã·ã£ã¢ãšå¯Ÿç §çããïŒ)
- èã®é³Žããããªå£° â ãŽãããªã®å ããæ§ãªçŸæ¥å¿
- (â»ãŽãããªã®å ãããããªããšããŠããçŸæ¥å¿ãããªãããã)
- ãã€ããäžè©±ã«ãªã£ãŠãããŸã â ã¡ãã£ãšæé 倩ã«å
¥ãæ¿ãããŠãã£ãŸã
- (â»ã©ãªã£ãæ)
- äžå¯žã®è«ã«ãäºåã®é â æŸã®ãŽãããªã«ãåããŒã¹äžæ»
- (â»ïŒïŒãªã®ã«ïŒããŒã¹ïŒïŒãããäžæ»èº«ã®ãŽãããª)
- æµ·èã§é¯ãé£ã â 埳å©ã§é°¹ãé£ã
- (â»äžæ¯ãã£ãŠããæ)
- 鬌ã®å±
ã¬éã«æŽæ¿¯â é·ç¥ã®ãã£ãŠããã¬ããã ã«æŸä¹³
- (â»æŸä¹³ãã¯ãªããªãåºãŠããªã衚çŸ)
- èžèœäººã¯æ¯ãåœâ æå人ã¯é 骚ãå±é£
- (â»é ãæªããšããããã§ã¯ãªãã骚ãå±ãªã)
- å
æ°ããã©ãïŒãªãããã³C â ã«ã©ããã«ãžãïŒãªãããã³A
- (â»ãã«ãžãã£ãŠèª°ã ããã«ã©ããã«ãžãã®ç Žå£å)
- äžäžäžäžå·Šå³å·Šå³BAâ äžäžãããŒã°ããã€è
°éšè
°éšMi
- (â»ãããŒã°ããã€ããããè °ãæ¯ã£ãŠããïŒ)
以äžã