It strikes me that LLM's are a subtle combination of knowing the language and knowing some facts. If one were making a chatbot for helping (say) people understand their tax returns then you'd want a 'base' LLM that understood language and then add domain specific knowledge via RAG. So "yes" to knowing that Thursday comes after Wednesday; "no" to knowing how high Everest is; and questions regarding income tax have to reference the RAG. Is this correct?

So are there stock "thin" models for doing this? Is it better to train your own?