Hello! I just released wyat-2, a fine-tune of Ouro-1.4B (a looped transformer) trained on a single RTX 4070 Ti.
The link is the report. As far as I know it’s the first decision model built on a looped transformer: not the best, but the first.
I realized I would need heavy data work to make it the best, so I’d prefer not to spend too much time on it for now. Maybe later, if it catches interest.
teilom•46m ago
The link is the report. As far as I know it’s the first decision model built on a looped transformer: not the best, but the first.
I realized I would need heavy data work to make it the best, so I’d prefer not to spend too much time on it for now. Maybe later, if it catches interest.
Model: https://huggingface.co/wyat-io/wyat-2-1.4b