< back 日本語版

LoRA Tuning Conversational Japanese Large Language Models using Japanese Instruction Dataset [in Japanese]

Masahiro SUZUKI, Masanori HIRANO, Hiroki SAKAJI

The 20th Text Analytics Symposium, Sep. 6, 2023

---

Conference

The 20th Text Analytics Symposium

Abstract

In this study, we performed LoRA tuning on large language models (LLM) based on both Japanese and English using Japanese instruction tuning and evaluated these models from both quantitative and qualitative perspectives. As a result of the evaluation, the effectiveness of tuning with Japanese instruction data was confirmed. Furthermore, we clarified the challenges in large-scale language models and language resources in Japanese, such as the need for evaluation using a wide range of instruction data and the actual output strings of the models.

Keywords

Large Language Model (LLM); Japanese; Instruction Tuning;

---

Paper

Official page

---

bibtex

@inproceedings{Suzuki2023-textanal20,
  title={{LoRA Tuning Conversational Japanese Large Language Models using Japanese Instruction Dataset [in Japanese]}},
  author={Masahiro SUZUKI and Masanori HIRANO and Hiroki SAKAJI},
  booktitle={The 20th Text Analytics Symposium},
  url={https://ken.ieice.org/ken/paper/20230906NCwg/},
  year={2023}
}

---