So I've recently been researching and am getting intereseting to tune the AI models for specialized. use-cases (e.g having knowledge/context about a very specific task or subject). What I am really confused and not sure about is what way is the best to customizing the models for tasks: Fine-Tuning OSS Models on datasets or System Tuning SOTA models like GPT-5, Claude 4.1 Opus, etc by writing detailed system prompts and adding tool calls?

Would ofc really love to hear opinions on it.