Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

如何修改推理阶段的Prompt #6502

Closed
1 task done
HHTao16 opened this issue Dec 31, 2024 · 2 comments
Closed
1 task done

如何修改推理阶段的Prompt #6502

HHTao16 opened this issue Dec 31, 2024 · 2 comments
Labels
solved This problem has been already solved

Comments

@HHTao16
Copy link

HHTao16 commented Dec 31, 2024

Reminder

  • I have read the README and searched the existing issues.

System Info

我在/LLaMA-Factory/src/llamafactory/data/template.py文件中修改了qwen模型的default_system,修改内容如下:

_register_template(
    name="qwen",
    format_user=StringFormatter(slots=["<|im_start|>user\n{{content}}<|im_end|>\n<|im_start|>assistant\n"]),
    format_system=StringFormatter(slots=["<|im_start|>system\n{{content}}<|im_end|>\n"]),
    format_observation=StringFormatter(slots=["<|im_start|>tool\n{{content}}<|im_end|>\n<|im_start|>assistant\n"]),
    format_separator=EmptyFormatter(slots=["\n"]),
    default_system="你是一个专业的美食博主,对于用户的提问你会给出某道菜肴的具体做法,你要详细标明每种原料的用量以及使用的先后顺序。",
    stop_words=["<|im_end|>"],
    replace_eos=True,
    replace_jinja_template=False,
)

Reproduction

在推理阶段,我发现我更改的prompt并没有作用于推理阶段的案例,模型的一个案例输入展示如下:

inputs:
<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
***************<|im_end|>
<|im_start|>assistant

能否指明如何修改推理阶段的prompt,在哪个文件中?

Expected behavior

No response

Others

No response

@github-actions github-actions bot added the pending This problem is yet to be addressed label Dec 31, 2024
@hiyouga
Copy link
Owner

hiyouga commented Dec 31, 2024

重新 build install

@hiyouga hiyouga closed this as completed Dec 31, 2024
@hiyouga hiyouga added solved This problem has been already solved and removed pending This problem is yet to be addressed labels Dec 31, 2024
@HHTao16
Copy link
Author

HHTao16 commented Dec 31, 2024

已成功解决,感谢回答,感谢协助

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
solved This problem has been already solved
Projects
None yet
Development

No branches or pull requests

2 participants