dbt_artifacts demo: How to log dbt Cloud jobs for both successes/failures

Hey folks, this is sung speaking here. And today I'm going to demo how the latest version of DBT artifacts works really well with DPD cloud.

And when things fail, there's no longer that annoying quirk where, uh, failures do not make it to your database to log, um, performance, uh, of models that, that executed within that failed job.

It's a right now I'll show you this in the context of a pull request workflow, where I'm willing to make an intentional failure within my first model and see how that looks and feels when I commit and push and kicks off a PR job and see if DPT artifacts works in the first place.

And so I'm going to do this. I'm going to commit and push intentional failure. I'm going to commit those changes.

Okay, cool. And then from there, I'm going to view the pull request and I should see something kicked off. I'm going to refresh this again.

Perfect. I'm going to look at the details and this is going to take maybe a minute or two, cause I have a bunch of other changes, but in the meantime, I'll show you, Hey, what does it take to actually make this work in the first place?

Step one, Make sure to download the right package. I'm just using the latest version for now. Anything. 1.0 above should be a consistent experience for you.

Step two is running DBT depths. So I've already done that here. The next step is making sure to add an on run end hook.

So I already have some on run end hooks, and I just added to it above here. And then I didn't do this optional thing, but for, I feel like 80% of the time people will want to use syntax that looks like this to make sure you're only logging performance for things that are hitting your production, uh, database and schemas, and then make sure you set up the scaffolding within your database.

And I already ran that macro here And it looks something like this creating table. It looks like it's parsing a bunch of macros, but essentially it's using this DDL to create the, the base schemas that you care about in the first place.

And then make sure if you want to persist them in a consistent namespace that you have things up here. You're probably noticing what the heck.

Why did you include it here, Tucson? Uh, that that's just old there, but I'm leaving it in there just in case there's some unintended side effect of, of moving that.

Uh, and then, yeah, that's, uh, that's pretty much it. So let's step back into this. This should fail. I'm gonna refresh this one more time.

Cause I want to see the logs related to this, just to make sure that on run end hook worked as to go down here and Shizam yet that'll always run whether it is a success or failure, that was not true with DBT artifacts with versions less than 1.0, and then I'm going to go over here and then I'm going to do this and refresh this preview.

And I'm going to see, you'll probably notice here, timestamps. I'm going to go here and extend this column out just to make sure it's what I expect to do at the 48 mark.

Yep. That looks about right. For things that failed or succeeded. And then over here, this was skipped and then this was an error, you know, to see her, this is in the context of that exact same invocation from DVD cloud to make this, you know, run in the first place.

And so it's so powerful about this is whenever someone asks like, Hey, can I log the performance? Uh, the historical performance over time of my DBT cloud jobs, whether it's a success or failure.

And now the short answer and the long answer is yes. And you just get the, send them this link and they get to move on with their lives.

And so, yeah, that's about it. See, you.