èªåã®éå»ãã€ãŒãã§rinna/japanese-gpt-neox-3.6bãfinetuningããŠã俺tterããçæãã
TL;DR
以äžã®Windowsç°å¢ã§PEFTãåãããšããQiitaèšäºãåèã«ããã ãã§ã¯ãããŸãã
Windowsç°å¢ã§LLMãPEFTã§ãã¡ã€ã³ãã¥ãŒãã³ã°ããããšãããšããã»ãŒå¿ ãbitsandbytesãšããã©ã€ãã©ãªã®ãšã©ãŒã«æ©ãŸããããšæãã®ã§ããããã¡ãã®èšäºã§ã¯ãã®å¯ŸåŠæ³ãæžããŠãããŸããå©ãããŸããã
ãããŠãnpakaããã®äžèšã®èšäºãåèã«ãGoogle Colabã§ã¯ãªãããŒã«ã«ã§åããããšããæãã§ããÂ
ãã£ã©ã¯ã¿ãŒæ§ãäžè²«ããLLMãäœãããã®æåã®ãã¹ãã«æé©
ãäžè²«ãããã£ã©æ§ãæã£ãåçãããAIãäœãããã
ãã§ãã©ã€ã»ã³ã¹ã®åé¡ããªããã£ã©æ§ãä¿ã£ããŸãŸãããªãã®èŠæš¡ãããããŒã¿ã»ãããªããŠç¡ãã
ãèªåã®ãã€ãŒãã䜿ãã°ããã®ã§ã¯ð¡ã
ãããªããšãèããŠãèªåïŒ@matsu_vrïŒã®éå»ãã€ãŒãã§ãæ¥æ¬èªLLMã®rinna/japanese-gpt-neox-3.6bããã¡ã€ã³ãã¥ãŒãã³ã°ãããšãããæã£ã以äžã«ã俺ã£ãœãããã€ãŒããçæããããšãã§ããŸããã
ãäžè²«ãããã£ã©æ§ãæã£ãåçãããAIãäœãããã
â æŸxR (@matsu_vr) June 8, 2023
ãã§ãã©ã€ã»ã³ã¹ã®åé¡ããªããã£ã©æ§ãä¿ã£ããŸãŸãããªãã®èŠæš¡ãããããŒã¿ã»ãããªããŠç¡ãã
ãèªåã®ãã€ãŒãã䜿ãã°ããã®ã§ã¯ð¡ã
ãšããããã§ãrinna 3.6bãèªåã®éå»ãã€ãŒãã§finetuneãããªã¬ãã¿ãŒã§ããããªã俺ã£ãœãïŒïŒ pic.twitter.com/3k7CpM5HmL
ç§ã®éå»ãã€ãŒãããã£ãšçºããŠã¿ãŠããã ããã°ãããªããããã£ãœããããšãäŒãããããããŸããã
ãã£ã©ã¯ã¿ãŒæ§ããã¡ã€ã³ãã¥ãŒãã³ã°ã§ä»å ããããšãããšããèªåã®ãã€ãŒããšããã®ã¯æš©å©ã®åé¡ããªãã®ã§å©çšããããã§ãããã¹ãçšéãšããŠã¯ãèªåã®ãã€ãŒãã¯æé©ãããããŸããããã®ãã¹ãã®ææã掻ãããŠãèªåãªãã®ãã£ã©ã¯ã¿ãŒãäœã£ãŠããã°ããã®ã§ã
éå»ãã€ãŒãã®ååŸ
Twitterã®èšå®ç»é¢ãããèªåã®éå»ãã€ãŒãã®ã¢ãŒã«ã€ããååŸåºæ¥ãŸããã¡ãªã¿ã«ãã¯ãªãã¯ããŠããã«ååŸåºæ¥ãããã§ã¯ãªãã1æ¥ãããåŸã«ãæºåãåºæ¥ãŸããããšããã¡ãŒã«ãæ¥ãŠãããŠã³ããŒãåºæ¥ãããã«ãªããŸããåŸ
ã¡ãŸãããã

ããŠã³ããŒããããã¡ã€ã«ãè§£åããdataãã©ã«ãã®äžã«ããtweets.jsãã¡ã€ã«ãããªãã®ãã€ãŒãã®ããã¹ãããŒã¿ã§ãããªããJavaScriptã®ãã¡ã€ã«ãªã®ã§ãæåã®window.YTD.tweets.part0 =ãåé€ãããã¡ã€ã«åãtweets.jsonã«ããŸãã

å¿ èŠãªæ å ±ã ããååŸ
ããããŠåŸãããJSONã«ã¯ããã¡ã€ã³ãã¥ãŒãã³ã°ããã«ã¯äœèšãªããŒã¿ããããããããŸãã以äžã®æ§ãªã¹ã¯ãªãããæžããŠãå¿ èŠãªèŠçŽ ã ããåãåºããŸãããã
import json
import re
# JSONãã¡ã€ã«ãèªã¿èŸŒã
with open('tweets.json', 'r', encoding='utf-8') as f:
data = json.load(f)
# åºåãä¿åãããªã¹ããäœæ
output = []
# åãã€ãŒãã«ã€ããŠåŠçãè¡ã
for tweet in data:
full_text = tweet['tweet']['full_text']
# æåã®ããããŸãã¯ãããã§åºåã
match = re.search(r'(.*?[ãã])(.*)', full_text)
# ããããŸãã¯ããããèŠã€ãã£ãå Žå
if match:
first_sentence = match.group(1)
remaining_text = match.group(2)
# æ°ãããã©ãŒãããã«æŽåœ¢
formatted = {
"input": first_sentence,
"completion": remaining_text
}
# çµæããªã¹ãã«è¿œå
output.append(formatted)
# çµæãæ°ããJSONãã¡ã€ã«ã«æžã蟌ã
with open('formatted_tweets.json', 'w', encoding='utf-8') as f:
json.dump(output, f, indent=4)èªåã®ãã€ãŒãã®ããã¹ããæåã«åºãŠããå¥èªç¹ã§åºåããååãinput(instruction)ãåŸåãoutput(completion)ãšããŠåŠç¿ããŒã¿ãšããããšã«ããŸãã
å¿ èŠãªããã±ãŒãžã®ã€ã³ã¹ããŒã«
Google Colabã«ã¯æšæºã§å ¥ã£ãŠãããã©ãããŒã«ã«ã«ã¯å ¥ã£ãŠãªãããã±ãŒãžãå ¥ããå¿ èŠããããŸããWindows11ïŒCuda11.8ã®ç°å¢ã§ã¯ä»¥äžã®æ§ã«ãªããŸãã
pip install -Uqq git+https://github.com/huggingface/peft.git
pip install -Uqq transformers datasets accelerate
pip install sentencepiece
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
pip install scipyã€ã¥ããŠãæåã«æããQiitaèšäºãåèã«ãbitsandbytesã®0.37.0ãã€ã³ã¹ããŒã«ããŸãããã¡ãã¯ãèšäºãåèã«ããŠãã ãããpipã§bitsandbytes==0.37.0ãã€ã³ã¹ããŒã«ããvenvã®ãã¡ã€ã«ãå ¥ã£ããã©ã«ãã®Lib\site-packages\bitsandbytes\cuda_setup\main.pyãèšäºã®ã¬ã€ãã®éãã«æžãæããlibbitsandbytes_cpu.dllãšlibbitsandbytes_cuda116.dllãããŠã³ããŒãããŠããŠãLib\site-packages\bitsandbytes\ã«å ¥ããŸããã¡ãã£ãšæéã§ãããæžããããšããã«ããã°å€§äžå€«ã§ãã
PEFTã«ããåŠç¿ã®å®è¡
以äžã®æ§ãªã¹ã¯ãªãããfinetune.pyãšããŠäœæããŸããã»ãŒnpakaããã®èšäºã®ãŸããŸã§ãïŒæ¬åœã«æè¬ïŒãããŒã¿ã®åãåãã®æã ããèªåã®ãã€ãŒãã®JSONãèªã¿ã«è¡ãããã«å€æŽããŠããŸãã人ã«åãããªãã©ã€ããRTã®ããŒã¿ã¯é€å€ããããã«ããŠããŸãã
# åºæ¬ãã©ã¡ãŒã¿
model_name = "rinna/japanese-gpt-neox-3.6b-instruction-ppo"
peft_name = "lorappo-rinna-3.6b"
output_dir = "lorappo-rinna-3.6b-results"
from transformers import AutoTokenizer
# ããŒã¯ãã€ã¶ãŒã®æºå
tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=False)
CUTOFF_LEN = 256 # ã³ã³ããã¹ãé·
# ããŒã¯ãã€ãºé¢æ°
def tokenize(prompt, tokenizer):
result = tokenizer(
prompt,
truncation=True,
max_length=CUTOFF_LEN,
padding=False,
)
return {
"input_ids": result["input_ids"],
"attention_mask": result["attention_mask"],
}
# ããŒã¿ã»ãããJSONããããŒã
import json
with open("formatted_tweets.json", "r", encoding='utf-8') as f:
loaded_data = json.load(f)
data_1 = [item for item in loaded_data if not item['input'].startswith('RT')]
data = [item for item in data_1 if not item['input'].startswith('@')]
print("ããŒã¿æ°:", len(data))
# ããã³ãããã³ãã¬ãŒãã®æºå
def generate_prompt(data_point):
result = f"""### æç€º:
{data_point["input"]}
### åç:
{data_point["completion"]}
"""
# æ¹è¡â<NL>
result = result.replace('\n', '<NL>')
return result
# ããŒã¿ã»ããã®æºå
VAL_SET_SIZE = 1000
train_dataset = []
val_dataset = []
for i in range(len(data)):
if i % 5 == 0:
x = tokenize(generate_prompt(data[i]), tokenizer)
val_dataset.append(x)
else:
x = tokenize(generate_prompt(data[i]), tokenizer)
train_dataset.append(x)
from transformers import AutoModelForCausalLM
# ã¢ãã«ã®æºå
model = AutoModelForCausalLM.from_pretrained(
model_name,
load_in_8bit=True,
device_map="auto",
)
from peft import LoraConfig, get_peft_model, prepare_model_for_int8_training, TaskType
# LoRAã®ãã©ã¡ãŒã¿
lora_config = LoraConfig(
r= 8,
lora_alpha=16,
target_modules=["query_key_value"],
lora_dropout=0.05,
bias="none",
task_type=TaskType.CAUSAL_LM
)
# ã¢ãã«ã®ååŠç
model = prepare_model_for_int8_training(model)
# LoRAã¢ãã«ã®æºå
model = get_peft_model(model, lora_config)
# åŠç¿å¯èœãã©ã¡ãŒã¿ã®ç¢ºèª
model.print_trainable_parameters()
import transformers
eval_steps = 200
save_steps = 200
logging_steps = 20
# ãã¬ãŒããŒã®æºå
trainer = transformers.Trainer(
model=model,
train_dataset=train_dataset,
eval_dataset=val_dataset,
args=transformers.TrainingArguments(
num_train_epochs=3,
learning_rate=3e-4,
logging_steps=logging_steps,
evaluation_strategy="steps",
save_strategy="steps",
eval_steps=eval_steps,
save_steps=save_steps,
output_dir=output_dir,
save_total_limit=3,
push_to_hub=False,
auto_find_batch_size=True
),
data_collator=transformers.DataCollatorForLanguageModeling(tokenizer, mlm=False),
)
# åŠç¿ã®å®è¡
model.config.use_cache = False
trainer.train()
model.config.use_cache = True
# LoRAã¢ãã«ã®ä¿å
trainer.model.save_pretrained(peft_name)æžããããpython finetune.py ãå®è¡ããŠããã°ããåŸ ã¡ãŸããããGPUãç©ãã§ããŠãããªãåŸ ã€ãšæããŸãã®ã§ããšã©ãŒãåºãŠãªãããšã確èªãããå¯ãŠç¿æã«ç¢ºèªããŠãããã§ãããã
çæã®ãã¹ã
åºæ¥äžãã£ãããã¹ãããŠã¿ãŸãããã以äžã®æ§ãªã¹ã¯ãªãããinference.py ãšããŠäœæããŸãããã¡ããnpakaããã®èšäºã®ã»ãŒãã®ãŸãŸã§ãã
import torch
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "rinna/japanese-gpt-neox-3.6b-instruction-ppo"
peft_name = "lorappo-rinna-3.6b"
output_dir = "lorappo-rinna-3.6b-results"
# ã¢ãã«ã®æºå
model = AutoModelForCausalLM.from_pretrained(
model_name,
load_in_8bit=True,
device_map="auto",
)
# ããŒã¯ãã€ã¶ãŒã®æºå
tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=False)
# LoRAã¢ãã«ã®æºå
model = PeftModel.from_pretrained(
model,
peft_name,
# device_map="auto"
)
# è©äŸ¡ã¢ãŒã
model.eval()
# ããã³ãããã³ãã¬ãŒãã®æºå
def generate_prompt(data_point):
if data_point["input"]:
result = f"""### æç€º:
{data_point["instruction"]}
### å
¥å:
{data_point["input"]}
### åç:
"""
else:
result = f"""### æç€º:
{data_point["instruction"]}
### åç:
"""
# æ¹è¡â<NL>
result = result.replace('\n', '<NL>')
return result
# ããã¹ãçæé¢æ°ã®å®çŸ©
def generate(instruction, input=None, maxTokens=256) -> str:
# æšè«
prompt = generate_prompt({'instruction': instruction, 'input': input})
input_ids = tokenizer(prompt,
return_tensors="pt",
truncation=True,
add_special_tokens=False).input_ids.cuda()
outputs = model.generate(
input_ids=input_ids,
max_new_tokens=maxTokens,
do_sample=True,
temperature=0.7,
top_p=0.75,
top_k=40,
no_repeat_ngram_size=2,
)
outputs = outputs[0].tolist()
# print(tokenizer.decode(outputs))
# EOSããŒã¯ã³ã«ãããããããã³ãŒãå®äº
if tokenizer.eos_token_id in outputs:
eos_index = outputs.index(tokenizer.eos_token_id)
decoded = tokenizer.decode(outputs[:eos_index])
# ã¬ã¹ãã³ã¹å
容ã®ã¿æœåº
sentinel = "### åç:"
sentinelLoc = decoded.find(sentinel)
if sentinelLoc >= 0:
result = decoded[sentinelLoc + len(sentinel):]
return result.replace("<NL>", "\n") # <NL>âæ¹è¡
else:
return 'Warning: Expected prompt template to be emitted. Ignoring output.'
else:
return 'Warning: no <eos> detected ignoring output'
# ããã¹ãçæ
print("èªç¶èšèªåŠçã£ãŠãã\n{0}".format(generate('èªç¶èšèªåŠçã£ãŠãã')))
print("å°é¢šè¿ã¥ããŠããªãã\n{0}".format(generate('å°é¢šè¿ã¥ããŠããªãã')))
python inference.py ãšå®è¡ãããšãçæçµæãåŸãããã¯ãã§ãã
èªç¶èšèªåŠçã£ãŠãã
ãæ¥æ¬èªãæ£ããçè§£ãããã£ãŠãšããããå§ãããããæ¥æ¬èªã®æç« ãæ£ãããŠããæèã話ãæã®æå³ãäŒãããªããšæå³ãéããªããæèãäŒãããããªæèãçæã§ãããããªæè¡ãçãŸããªãããªãŒããšãæã£ãŠã¿ãã
å°é¢šè¿ã¥ããŠããªãã
ä»äºäŒã¿ã«ãªãã®ããª...? https://t.co/GVPcnJhQs7</tmLt>
ç§ã®TwitterãèŠãããšããªã人ã«ã¯ããããªããããããªãã§ãããæžããæ¬äººããèŠããšããªãèªåã£ãœãã§ããèªåã®ãã€ãŒããšæ··ããããŠåºããããã©ããAIã«ãããã®ãããããªããªããããªãããã§ãã
ã¡ãªã¿ã«ãä»ããŠããt.coã®URLã«ã¢ã¯ã»ã¹ããŠã¿ãŠããšãã«äœããããŸãããæ¶ç©ºã®ãŠãŒã¶ãŒã«ã¡ã³ã·ã§ã³ããŠããããšãããªããªãé¢çœãçµæãåºãŠãããšæããŸãã
ã¡ãªã¿ã«ãç§ã®4800äœã®æ°ã®ãã€ãŒãã䜿ã£ãŠãVRAM䜿çšéã¯13GBãããã§ãããæ¬¡ã¯QLoRAã䜿ã£ãŠãããå°ãªãVRAM容éã§ã®åŠç¿ã詊ããŠã¿ãããšæã£ãŠããŸãã