Dataflow Refresh issues got you down?

Has this ever happened to you? Waking up to find out that a Power BI Dataflow that was scheduled overnight, didn’t happen? Even worse, what if you have a series of Power BI Dataflows (hierarchy) that rely on each other, 1 of them fails and the others don’t; Great, now your data is corrupted, <cue the Sad Trombone>

Then this is the solution for YOU!

Everyone knows that the best combination on Earth is Chocolate and Peanut Butter, that’s settled science.

BUT the Second Best combination is the Durable Function Monitor & Circuit Breaker patterns.

(Yes, it’s even slightly better than The Captain & Tennille)

Introducing the Dataflow Emergency Brake! (Github Repo)

How it Works:

The Azure Durable function orchestrator, implementing the Monitor Pattern, will run for a configurable amount of time and call an Activity function at Configurable intervals (polling). The Activity function uses the DataFlowService (a wrapper for the Power BI REST APIs) to Monitor and Manage the Dataflows. When any Dataflow fails, errors out, or hangs (doesn’t complete in a configurable amount of time) the Activity will either cancel all remaining Dataflows or will restart the Dataflow, depending on configuration.

But Wait, There’s More!

Before you can call any Power BI REST API, you must include a Bearer Token in the header; otherwise, all calls will receive a 401 error (unauthorized)

AuthService.cs – Generates the Bearer Token required for every API call

Act now and we’ll throw in the Retry pattern for free!

Now that you have a bearer token, the Power BI REST APIs world is yours.

DataFlowServices.cs Provides the implementation for interacting with the Power BI REST APIs

• GetDataFlows – Get all the Dataflows in a Workspace (groupid)
  • await _dataFlowServices.GetDataFlows();
• GetDataFlowTransactions – Get all the Transactions in a Dataflow, the first one is the most recent
  • await _dataFlowServices.GetDataFlowTransactions(dataFlowId);
• CancelDataFlow – Cancels a Dataflow
  • await _dataFlowServices.CancelDataFlow(dataFlowId, dataFlowTransactionId);
• RefreshDataFlow – Restarts a Dataflow (Retry capability)
  • await _dataFlowServices.RefreshDataFlow(dataFlowId);

And that’s it!

In No time, you too can Monitor and Manage your Power BI Dataflows

Operators are standing by