{"type":"video","version":"1.0","html":"<iframe src=\"https://www.loom.com/embed/24ba433601de45ba8b63d9fb34c31fd5\" frameborder=\"0\" width=\"1110\" height=\"832\" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>","height":832,"width":1110,"provider_name":"Loom","provider_url":"https://www.loom.com","thumbnail_height":832,"thumbnail_width":1110,"thumbnail_url":"https://cdn.loom.com/sessions/thumbnails/24ba433601de45ba8b63d9fb34c31fd5-fceea1add333597e.gif","duration":631.62,"title":"Getting Started with Eval Protocol for Reinforcement Learning 🚀","description":"In this video, I'm walking you through the Eval Protocol quickstart. I'll show you how it saves you from rewriting your eval logic every time you try a new RL trainer. We'll be working on an agent that generates SVG images, and we're going to use GPT-4.1 as a visual 'judge' to score how well it does on its requirements. First, I'll show you how we run everything locally, where our test uses a RemoteRolloutProcessor to get the SVG code from a server, renders it, and gets it scored by our judge. Once we see that's all working, I'll show you the cool part: we just run a single command, eval-protocol create rft, which automatically packages up our entire evaluator, secrets, and dataset, and kicks off a real training job on Fireworks. To finish, we'll hop into the Fireworks dashboard and check out the training graphs, logs, and the before-and-after visual improvements."}