Tag Archives: Dataflows (Gen2)

Goodbye Dataflows, For Some?

• , ,

Earlier this month, Microsoft announced the future retirement of gen1 dataflows. Microsoft left unstated whether dataflows would remain, or be removed from, what is included in Microsoft Power BI Pro and Premium Per User (PPU) licenses. This uncertainty has raised serious concerns in the community.

Update: On April 20, Microsoft revised their announcement post, stating that they will “introduce new Dataflow Gen2 paths for Pro/PPU scenarios…” Unfortunately, this still leaves ambiguous whether Pro/PPU users will get to keep dataflows without incurring extra costs. The promised “new paths” could include adding gen2 dataflows to what is included in Pro/PPU licenses (yay!), the “new paths” could simply be better tooling to ease the process of Pro/PPU users moving their dataflows to Fabric (= extra costs)…or something else. Until additional information is announced, the concerns expressed below still apply.

Continue reading →

Less GUIDs: Relative Current Workspace & Items By Name

• , , ,

Power Query dataflows like to use GUIDs to reference workspaces, lakehouses and warehouses. With a GUID reference,  you are authoritatively referencing a specific resource, which is good…but working with GUIDs can entail complications, both for humans and CI/CD, which is not so good.

I know what lakehouse “SomeLakehouse” is, but I’ll have to look up what lakehouse “fb7322d6-781e-4e0a-97b1-1eec69a5547b” is. On the source control side, I find that, due to how GUIDs tend to be intermixed with other code in Power Query, Git often incorrectly wants to merge dev workspace GUIDs to test, or test-specific GUIDs to prod. If not caught by careful human PR/merge review, such inadvertent merges leads to bad things happening.

Can we use names instead of GUIDs? That would resolve at least part of the challenge. Our code could refer to “SomeLakehouse” instead of a GUID (nice!).

However, switching from GUIDs to names still leaves the need to update workspace references when a dataflow is promoted across workspaces (e.g. from dev -> test -> prod.) Is there a way to reference the current workspace, regardless of its name?

The answer is yes to both! Using names instead of GUIDs isn’t new, but hasn’t often been done. On the other hand, relative workspace references is a brand-new, in preview feature.

Let’s explore both, in code!

Continue reading →

What’s With WriteToDataDestination?

• , , , ,

If you examine the refresh details for a Power Query gen2 dataflow, you’ll notice that the activity names which are displayed are identical to the query names in your dataflow except that each has a “_WriteToDataDestination” suffix appended to it. What’s going on with this?

Let’s see what we can uncover! Time for some sleuthing….

Continue reading →