Is this a typo? Maybe 5e-4 for pretraining?
Otherwise this goes against all the intuition I have around learning rates and catastrophic forgetting. (a smaller learning rate causing knowledge degredation)
your intuition is sound, but my fingers are not.
palmfacehn•8mo ago
pact_inference•8mo ago
palmfacehn•8mo ago
The model might excel at creating character sheets, after you define a schema. From there you can validate the generated sheets against known lore. You could combine the story telling from the LLM with the formalized character schema to create campaigns. I'm not an expert here, but I suspect you might try asking the model to translate an existing fantasy story dataset into a series of narration/dialogue blocks and character sheets.
Without training, I've experimented with similar approaches for item generation using EBNF.
pact_inference•8mo ago
Definitely! I'm going to start with instruction tuning it for basic question answering, and then add tools to allow it to search the markdown source to cite answers to rules questions. I think adding some dice tooling for proper character sheet creation would be an awesome task to test as well. I'm actually thinking a lot about what tasks I could try that are "trivially" programmatically verifiable in their correctness for stuff like GRPO, so I'm definitely going to use that idea.
> You could combine the story telling from the LLM with the formalized character schema to create campaigns. I'm not an expert here, but I suspect you might try asking the model to translate an existing fantasy story dataset into a series of narration/dialogue blocks and character sheets.
I think probably late this year I'll be able to work on that sort of thing. There's a really interesting approach to story generation https://arxiv.org/abs/2503.22828 here, but modifying ways to translate it into campaign relevant structured objects and "reward" that will take some experimentation.