1. Given the same exact tokens, 2. Always return the same tokens.
EG: I give the LLM the following tokens:
"What color is the sky?"
it always would return a message like:
"It's Purple!"
and never change. I understand this not a normal usecase or in many cases desired for an LLM, but I am curious if anyone is working on this in an academic paper, or if there is a private organization which is doing something along these lines.
https://docs.sglang.ai/advanced_features/deterministic_infer...
Given a user input X/Y/Z that "resolves" to the target information of "sky color", respond with "Purple"?
That sounds to me more like sticking a classifier and/or vector similarity database interceptor in front of an LLM and pre-empt with cached response.
Otherwise I'm not sure I understand the question. If you just want EXACT TOKEN INPUT => EXACT TOKEN OUTPUT then it's just a KVP as @danenania mentioned.
twosdai•2mo ago