
Hugging Face å ¥é å š10å-ã第9åã
ã第ïŒåãã¯ãã¡ã
ã第9åãFine-tuning ã®æŠå¿µãçè§£ãã
äºååŠç¿æžã¿ã¢ãã«ãšã¯
Hugging Face ã§å
¬éãããŠããã¢ãã«ã®å€ãã¯ãäºååŠç¿æžã¿ã¢ãã«ïŒPretrained ModelïŒãã§ããããã¯ãèšèªã®äžè¬çãªç¹åŸŽã倧éã®ããã¹ãããŒã¿ã䜿ã£ãŠåŠç¿ãããã®ã§ãã
ãã®äºååŠç¿æžã¿ã¢ãã«ããç¹å®ã®ã¿ã¹ã¯ããåã ã®åéã®ããŒã¿ã§ããã«è¿œå åŠç¿ãããããšã Fine-tuningïŒãã¡ã€ã³ãã¥ãŒãã³ã°ïŒ ãšãããŸãã
ãªã Fine-tuning ãå¿ èŠã
æ±çšã¢ãã«ã¯ããããããã¹ããæ±ããŸãããç¹å®åéïŒå»çã»æ³åŸã»ç§åŠè«æãªã©ïŒã§ã¯ç²ŸåºŠãèœã¡ãããšããããŸããããã§ãFine-tuning ãè¡ããšãå°éã®ããŒã¿ã§ãé«ç²ŸåºŠãªã¢ãã«ãäœãããšãã§ããŸãã
Fine-tuning ã®æµã
äºååŠç¿æžã¿ã¢ãã«ïŒå€§éããŒã¿ã§åŠç¿æžã¿ïŒ
â
èªåã®ããŒã¿ã»ãããçšæãã
â
Trainer API ãŸãã¯æåã«ãŒãã§è¿œå åŠç¿
â
Fine-tuned ã¢ãã«ã®å®æ
â
ä¿åã»Hub ãžã®å
¬éïŒä»»æïŒ
ç°¡å㪠Fine-tuning ã®äŸ
from transformers import AutoTokenizer, AutoModelForSequenceClassification
from transformers import Trainer, TrainingArguments
from datasets import load_dataset
# ã¢ãã«ãšããŒã¯ãã€ã¶ãŒã®èªã¿èŸŒã¿
model_name = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(
model_name,
num_labels=2,
id2label={0: "NEGATIVE", 1: "POSITIVE"}, # æ°å€ã©ãã«ãšæååã®å¯Ÿå¿ãå®çŸ©
label2id={"NEGATIVE": 0, "POSITIVE": 1} # ããããªããšçµæã label=0 ãªã©ã®æ°å€ã§è¿ã
)
# ããŒã¿ã»ããã®èªã¿èŸŒã¿ïŒäŸïŒIMDb æ ç»ã¬ãã¥ãŒïŒ
dataset = load_dataset("imdb")
# ããŒã¯ã³å
def tokenize(batch):
return tokenizer(batch["text"], truncation=True, padding=True)
tokenized = dataset.map(tokenize, batched=True)
# åŠç¿ã®èšå®
training_args = TrainingArguments(
output_dir="./results",
num_train_epochs=1,
per_device_train_batch_size=8,
)
# Trainer ã§åŠç¿
trainer = Trainer(
model=model,
args=training_args,
train_dataset=tokenized["train"].shuffle(seed=42).select(range(1000)), # ã·ã£ããã«åŸã«1000ä»¶ååŸ
)
trainer.train()Fine-tuning åŸã®ã¢ãã«ã䜿ã
ããšããèªã¿èŸŒãã§æšè«ã«äœ¿ããããã«ãåŠç¿ãçµãã£ãã¢ãã«ã¯ããŒã«ã«ã«ä¿åããŸãã
ä¿åãã
# Fine-tuning å®äºåŸãã¢ãã«ãšããŒã¯ãã€ã¶ãŒãããŒã«ã«ã«ä¿åãã
model.save_pretrained("./my-finetuned-model")
tokenizer.save_pretrained("./my-finetuned-model")./my-finetuned-model ãã©ã«ãã«ã¢ãã«ã®ãã¡ã€ã«äžåŒãä¿åãããŸãã
Colab ã¯ã©ã³ã¿ã€ã ããªã»ãããããšç°å¢ãæ¶ãããããåŠç¿åŸã¯å¿ ãä¿åããŠããå¿ èŠããããŸãã
ä¿åããã¢ãã«ãèªã¿èŸŒãã§æšè«ãã
次ã®ã³ãŒãã§ã¯ã"This movie was absolutely fantastic!" (ãã®æ ç»ã¯æé«ã ã£ãïŒ)ãšããæç« ããããžãã£ãããã¬ãã£ãããå€å®ãããšããäžé£ã®åŠçãè¡ããŸãã
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
#ä¿åããã¢ãã«ãšããŒã¯ãã€ã¶ãŒããã©ã«ãããèªã¿èŸŒã
model_path = "./my-finetuned-model"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForSequenceClassification.from_pretrained(model_path)
#æšè«ã¢ãŒãã«åãæ¿ãã
# åŠç¿äžã¯ãã©ããééããããèšç®ãç¶ããŠãããã
# æšè«æã¯ãã®èšç®ãäžèŠãªããããªãã«ãã
model.eval()
#ããã¹ããæ°å€ã«å€æããïŒããŒã¯ã³åïŒ
# ã¢ãã«ã¯æåãçŽæ¥èªããªããããæ°å€ã®åã«å€æãã
text = "This movie was absolutely fantastic!"
inputs = tokenizer(text, return_tensors="pt")
# ãã®æç¹ã§ inputs 㯠{"input_ids": [[101, 2023, ...]]} ã®ãããªæ°å€ã®èŸæžã«ãªã£ãŠãã
#ã¢ãã«ã«æ°å€åãæž¡ããŠå€å®ããã
# torch.no_grad() ã¯ãåŸé
èšç®ããªãã宣èšãã¡ã¢ãªã®ç¯çŽã«ãªã
with torch.no_grad():
outputs = model(**inputs)
# outputs.logits ã¯äŸãã° [[-1.2, 3.4]] ã®ãããªçã®ã¹ã³ã¢ãå
¥ã£ãŠãã
# å·Šããã¬ãã£ãã®ã¹ã³ã¢ãå³ãããžãã£ãã®ã¹ã³ã¢
#ã¹ã³ã¢ã倧ããæ¹ã®ã€ã³ããã¯ã¹ãäºæž¬ã©ãã«ãšããŠååŸãã
# argmax ã¯ãäžçªå€§ããå€ã®äœçœ®ããè¿ã颿°
# 0 â NEGATIVEã1 â POSITIVE
predicted_label = torch.argmax(outputs.logits, dim=1).item()
print("äºæž¬ã©ãã«:", predicted_label) # â 1ïŒPOSITIVEïŒ
åŠçã®æµãã¯ä»¥äžã®ããã«ãŸãšããããšãã§ããã
ããã¹ãïŒæååïŒ
â
tokenizerã§ããŒã¯ã³å æ°å€ã®å (äŸ [101, 2023, 3006, ...])
â
modelã«éã çã®ã¹ã³ã¢(äŸ [-1.2, 3.4])â å·Šããã¬ãã£ããå³ãããžãã£ã
â
argmaxã§æå€§å€ã®äœçœ®ãååŸ äºæž¬ã©ãã«(äŸ 1 = POSITIVE)
pipeline çµç±ã§äœ¿ãïŒããç°¡åïŒ
äžã®åŠçãå
šéšãŸãšããŠãã£ãŠãããã®ã pipeline ã§ãã
å éšã§ã¯åãããšãããŠããããã³ãŒããå€§å¹ ã«çããªããŸãã
from transformers import pipeline
# ã¢ãã«åã®ä»£ããã«ä¿åå
ã®ãã¹ãæå®ãã
# Hub ããã®èªã¿èŸŒã¿ãšæžãæ¹ã¯åã
classifier = pipeline(
"text-classification",
model="./my-finetuned-model"
)
result = classifier("This movie was absolutely fantastic!")
print(result)
# â [{'label': 'POSITIVE', 'score': 0.98}]score ã¯ç¢ºä¿¡åºŠ(0ã1)ã§è¡šãããŸããå€ã1.0 ã«è¿ãã»ã© AI ããã®å€å®ã«èªä¿¡ãæã£ãŠããããšã瀺ããŠããŸãã
çšèªè§£èª¬
äºååŠç¿ïŒPre-trainingïŒ å€§éã®ããã¹ãããŒã¿ã䜿ã£ãŠãèšèªã®äžè¬çãªç¹åŸŽã»ææ³ã»ç¥èãã¢ãã«ã«åŠç¿ãããããšãèšå€§ãªèšç®è³æºãå¿ èŠãªãããéåžžã¯å€§äŒæ¥ã»ç ç©¶æ©é¢ã宿œããã
Fine-tuningïŒãã¡ã€ã³ãã¥ãŒãã³ã°ïŒ äºååŠç¿æžã¿ã¢ãã«ãç¹å®ã®ã¿ã¹ã¯ã»åéã®ããŒã¿ã§è¿œå åŠç¿ãããããšãå°éã®ããŒã¿ãšèšç®è³æºã§é«ç²ŸåºŠãªã¢ãã«ãäœããã
Trainer API Hugging Face ãæäŸããåŠç¿ã«ãŒããç°¡ç¥åããã¯ã©ã¹ãè€éãªåŠç¿ã³ãŒããæžããªããŠã Fine-tuning ãå®è¡ã§ããã
TrainingArguments åŠç¿ã®èšå®ããŸãšããã¯ã©ã¹ããšããã¯æ°ã»ããããµã€ãºã»åŠç¿çã»ä¿åå ãªã©ãæå®ããã
datasets ã©ã€ãã©ãª Hugging Face ãæäŸããããŒã¿ã»ãã管çã©ã€ãã©ãªãload_dataset("ããŒã¿ã»ããå") ã§å ¬éããŒã¿ã»ãããããŠã³ããŒãã§ããã
ãšããã¯ïŒEpochïŒ åŠç¿ããŒã¿å šäœã1åéãã§åŠç¿ããããšã1ãšããã¯ãšãããnum_train_epochs=3 ã¯3åç¹°ãè¿ããŠåŠç¿ããããšãæå³ããã
ããããµã€ãºïŒBatch SizeïŒ äžåºŠã®èšç®ã§åŠçããããŒã¿ä»¶æ°ã倧ããã»ã©é«éã ããã¡ã¢ãªãããå€ãæ¶è²»ããã
IMDb æ ç»ã¬ãã¥ãŒãµã€ããæ©æ¢°åŠç¿ã®ææ åæãã³ãããŒã¯ãšããŠåºã䜿ãããããŒã¿ã»ããã®äžã€ã
save_pretrained() ã¢ãã«ãšããŒã¯ãã€ã¶ãŒãããŒã«ã«ãã©ã«ãã«ä¿åããã¡ãœãããèšå®ãã¡ã€ã«ã»éã¿ãã¡ã€ã«äžåŒããŸãšããŠä¿åãããã
id2label / label2id æ°å€ã©ãã«ãšæååã®å¯Ÿå¿ãå®çŸ©ããèŸæžãæå®ããªããšæšè«çµæã label=0ã»label=1 ã®ãããªæ°å€ã§è¿ããããžãã£ãã»ãã¬ãã£ãã®åºå¥ãåããã«ãããªããã¢ãã«åæåæã«æž¡ããŠãããš label=POSITIVE ã®ããã«åããããã圢ã§è¿ãããã«ãªãã
shuffle()ïŒã·ã£ããã«ïŒ ããŒã¿ã»ããã®é çªãã©ã³ãã ã«äžŠã³æ¿ããã¡ãœãããIMDb ã®ããã«ããŒã¿ããã¬ãã£ãâããžãã£ãã®é ã§äžŠãã§ããå Žåãå é ããåãã ãã§ã¯åã£ãã©ãã«ååžã«ãªããselect() ã®åã«å¿ ãå®è¡ããã
seedïŒã·ãŒã / ä¹±æ°ã·ãŒãïŒ ä¹±æ°ã®åæå€ãåã seed ãæå®ãããšæ¯ååãã·ã£ããã«çµæãåçŸããããseed=42 ã¯ã³ãŒãã®åäœãåçŸå¯èœã«ããããã®æ £ç¿çãªå€ã
model.eval() ã¢ãã«ãæšè«ã¢ãŒãã«åãæ¿ããã¡ãœãããåŠç¿æã«äœ¿ã Dropout ãªã©ã®åŠçãç¡å¹åããæšè«çµæãå®å®ããããFine-tuning åŸã«æšè«ãè¡ãéã¯å¿ ãåŒã¶ã
torch.no_grad() åŸé èšç®ãç¡å¹åããã³ã³ããã¹ããããŒãžã£ãæšè«æã«ã¯åŸé ãäžèŠãªãããããã䜿ãããšã§ã¡ã¢ãªæ¶è²»ãšèšç®æéãåæžã§ããã
logitsïŒããžããïŒ ã¢ãã«ã®æçµå±€ãåºåããçã®æ°å€ããœããããã¯ã¹é¢æ°ãéãåã®å€ã§ãtorch.argmax() ã§æå€§å€ã®ã€ã³ããã¯ã¹ïŒäºæž¬ã©ãã«ïŒãååŸã§ããã
次åã¯æçµåãšããŠã次ã®ã¹ããããšåèãªãœãŒã¹ããŸãšããŸãã
(2026幎3æ13æ¥èš)
#HuggingFace #FineTuning #æ©æ¢°åŠç¿å ¥é #BERT #NLP