{"type":"video","version":"1.0","html":"<iframe src=\"https://www.loom.com/embed/9a1271fadf0c4291b263ed8b6bc3a8cb\" frameborder=\"0\" width=\"1108\" height=\"831\" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>","height":831,"width":1108,"provider_name":"Loom","provider_url":"https://www.loom.com","thumbnail_height":831,"thumbnail_width":1108,"thumbnail_url":"https://cdn.loom.com/sessions/thumbnails/9a1271fadf0c4291b263ed8b6bc3a8cb-2776014db3a6693f.gif","duration":300.027,"title":"Self-hosted batching code walkthrough","description":"I was left with some really interesting prompts around running evals on self-hosted models, and took some time to dig into (1) how self-hosting works and (2) how to address the unique scaling problems you hit with self-hosting. To say the least, I was nerdsniped.\n\nThis is part 2 of a 3 part walkthrough. This part covers the meat and potatoes of it, but if you'd like you're more than welcome to chat out the other clips!\n- Part 1: https://www.loom.com/share/003d4facf2254098b4fde8a09f0840d3\n- Part 2: This video!\n- Part 3: https://www.loom.com/share/ddba1d3f42a64f279a64092bcc2829a5"}