CLI: implemented non interactive mode #18200
Conversation
| if (is_interactive) { | ||
| console::log("\n"); | ||
| console::log("%s\n", LLAMA_ASCII_LOGO); | ||
| console::log("build : %s\n", inf.build_info.c_str()); | ||
| console::log("model : %s\n", inf.model_name.c_str()); | ||
| console::log("modalities : %s\n", modalities.c_str()); | ||
|
|
||
| if (!params.system_prompt.empty()) { | ||
| console::log("using custom system prompt\n"); | ||
| } |
There was a problem hiding this comment.
I think this (and most of the other things, like spinner etc) should be output regardless, but to stderr.
There was a problem hiding this comment.
I think it is ok to output everything to stdout. Other CLI apps like claude code / ollama / gemini do that.
In most cases, outputting to a file via -o should be enough. I don't think we should recommend the use case of piping output from CLI to another application, it is just too error-prone.
There was a problem hiding this comment.
Thank you; tomorrow I will implement it.
IMO we should stop supporting this use case by default as it make the code extremely error-prone. Any contributors in the future may add more debugging / formatting to the output which will definitely break your use case. Instead, specify the input / output via proper arguments:
|
You mean |
|
Yes I mean It is not yet supported in the new CLI, but adding it is easy (feel free to open a PR to add them back) |
andrew-aladev
force-pushed
the
feature/non-interactive-mode-for-cli
branch
from
9910b08 to
6c840fe
Compare
andrew-aladev
force-pushed
the
feature/non-interactive-mode-for-cli
branch
from
6c840fe to
3d5870e
Compare
4 participants
I've implemented non-interactive mode for CLI in the following way:
is_stdin_a_terminalandis_stdout_a_terminalfunctions from run.stdinorstdoutis not a tty.<think>and</think>tags for marking think output.console::logmessages from output in non-interactive mode.Now it is possible to use the following commands:
cli <<< 'input',echo 'input' | cli,cli > output.This PR is a draft; please discuss.