I noticed there weren't boilerplates for AI projects like there were for web apps, so I built one. Same idea - everything you need to get a project up and running quickly. However, instead of web-framework/CSS/DB, it's tools for AI projects: evals, synthetic data gen, fine-tuning, and more.

Kiln is a free, open tool that gives you everything most AI projects need in one integrated package:

- Eval system: including LLM-as-judge evals, eval data generation, human baselines

- Fine-tuning: proxy to many fine-tuning providers like Fireworks/Together/OpenAI/Unsloth

- Synthetic data generation: deeply integrated into evals and fine-tuning

- Model routing: 12 providers including Ollama, OpenRouter, and more

- Git-based collaboration: projects are designed to be synced through your own git server

The key insight is that these tools work much better when they're integrated. For example, the synthetic data generator knows whether you're creating data for evals vs. fine-tuning (which have very different data needs), and evals can automatically test different prompt/model/fine-tune combinations.

It runs entirely locally - your project data stays in local files, and you control your own git repos. No external services required (though it integrates with them if you want).

Main project GitHub: https://github.com/Kiln-AI/Kiln

Demo GitHub where I use it to build a 'natural language to ffmpeg command' demo with evals, fine-tunes, and synthetic data (including demo video): https://github.com/Kiln-AI/demos/blob/main/end_to_end_projec...