Instructions to use noguchis/medusa-1.0-ELYZA-japanese-Llama-2-7b-instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use noguchis/medusa-1.0-ELYZA-japanese-Llama-2-7b-instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="noguchis/medusa-1.0-ELYZA-japanese-Llama-2-7b-instruct")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("noguchis/medusa-1.0-ELYZA-japanese-Llama-2-7b-instruct") model = AutoModelForCausalLM.from_pretrained("noguchis/medusa-1.0-ELYZA-japanese-Llama-2-7b-instruct", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use noguchis/medusa-1.0-ELYZA-japanese-Llama-2-7b-instruct with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "noguchis/medusa-1.0-ELYZA-japanese-Llama-2-7b-instruct" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "noguchis/medusa-1.0-ELYZA-japanese-Llama-2-7b-instruct", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/noguchis/medusa-1.0-ELYZA-japanese-Llama-2-7b-instruct
- SGLang
How to use noguchis/medusa-1.0-ELYZA-japanese-Llama-2-7b-instruct with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "noguchis/medusa-1.0-ELYZA-japanese-Llama-2-7b-instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "noguchis/medusa-1.0-ELYZA-japanese-Llama-2-7b-instruct", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "noguchis/medusa-1.0-ELYZA-japanese-Llama-2-7b-instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "noguchis/medusa-1.0-ELYZA-japanese-Llama-2-7b-instruct", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use noguchis/medusa-1.0-ELYZA-japanese-Llama-2-7b-instruct with Docker Model Runner:
docker model run hf.co/noguchis/medusa-1.0-ELYZA-japanese-Llama-2-7b-instruct
[
](https://github.com/Open
Access-AI-Collective/axolotl)
medusa-1.0-ELYZA-japanese-Llama-2-7b-instruct
This model is a fine-tuned version of elyza/ELYZA-japanese-Llama-2-7b-instruct on the None dataset. It achieves the following results on the evaluation set:
- Loss: 1.8838
Model description
This is a Medusa-1 created using Medusa.
Intended uses & limitations
Training and evaluation data
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 40
- num_epochs: 2
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 2.4268 | 0.06 | 40 | 2.4129 |
| 2.204 | 0.11 | 80 | 2.2328 |
| 2.1341 | 0.17 | 120 | 2.1939 |
| 2.1774 | 0.23 | 160 | 2.1762 |
| 2.1331 | 0.28 | 200 | 2.1652 |
| 2.1485 | 0.34 | 240 | 2.1537 |
| 2.1608 | 0.4 | 280 | 2.1485 |
| 2.0998 | 0.45 | 320 | 2.1340 |
| 2.1478 | 0.51 | 360 | 2.1233 |
| 2.1374 | 0.57 | 400 | 2.1165 |
| 2.0657 | 0.62 | 440 | 2.0984 |
| 2.1227 | 0.68 | 480 | 2.0834 |
| 2.0573 | 0.74 | 520 | 2.0739 |
| 2.0501 | 0.79 | 560 | 2.0602 |
| 2.0664 | 0.85 | 600 | 2.0431 |
| 2.0231 | 0.91 | 640 | 2.0277 |
| 2.0263 | 0.96 | 680 | 2.0119 |
| 1.8635 | 1.02 | 720 | 1.9990 |
| 1.8844 | 1.07 | 760 | 1.9905 |
| 1.8585 | 1.13 | 800 | 1.9796 |
| 1.8018 | 1.19 | 840 | 1.9728 |
| 1.8134 | 1.24 | 880 | 1.9595 |
| 1.7795 | 1.3 | 920 | 1.9498 |
| 1.7603 | 1.36 | 960 | 1.9371 |
| 1.8302 | 1.41 | 1000 | 1.9262 |
| 1.7909 | 1.47 | 1040 | 1.9172 |
| 1.7787 | 1.53 | 1080 | 1.9083 |
| 1.7476 | 1.58 | 1120 | 1.9010 |
| 1.7897 | 1.64 | 1160 | 1.8945 |
| 1.7795 | 1.7 | 1200 | 1.8895 |
| 1.7329 | 1.75 | 1240 | 1.8864 |
| 1.7221 | 1.81 | 1280 | 1.8846 |
| 1.7624 | 1.87 | 1320 | 1.8840 |
| 1.793 | 1.92 | 1360 | 1.8838 |
| 1.7839 | 1.98 | 1400 | 1.8838 |
Framework versions
- Transformers 4.34.1
- Pytorch 2.1.2
- Datasets 2.16.1
- Tokenizers 0.14.1
- Downloads last month
- 225
Model tree for noguchis/medusa-1.0-ELYZA-japanese-Llama-2-7b-instruct
Base model
elyza/ELYZA-japanese-Llama-2-7b-instruct