Thinking/non-thinking mode

#3
by bpool - opened

How to enable/disable thinking mode through the prompt?

Is there something like /think, /no_think?

You can find at the end of this Jinja file here:

{%- if add_generation_prompt %}
    {{- role_indicators['assistant'] }}
    {%- if enable_thinking is not defined or enable_thinking is true %}
        {{- "<think>\n" }}
    {%- else %}
        {{- "<think>\n\n</think>\n\n" }}
    {%- endif %}
{%- endif %}

So <think>\n\n</think>\n\n at the end of a prompt determines that no thinking is needed. For example: How many fruits are there?<think>\n\n</think>\n\n

Alternatively, just adding </think> might work too (I haven't tested it): How many fruits are there?</think>

Or if you use Transformers:

enable_thinking=False

By the way it's probably easier to just modify the jinja file.

Sign up or log in to comment