Earlier this month, Microsoft announced the future retirement of gen1 dataflows. Microsoft left unstated whether dataflows would remain, or be removed from, what is included in Microsoft Power BI Pro and Premium Per User (PPU) licenses. This uncertainty has raised serious concerns in the community.
In short:
The retirement. Gen1 dataflows are going away, replaced by gen2 dataflows (currently, both mutually coexist). At the technical level, this is great: in general, gen2 does what gen1 does, but better and has more features. Gen1 -> gen2 is a win for users at the technical level, as well as for Microsoft (e.g. allowing them to retire legacy infrastructure).
The timeline. This retirement will happen at some point in the future. Specific date yet to be announced, but it will be shared at least 12 months ahead of time. (Microsoft’s public announcement seems to suggest that the year-plus notice is promised only for Premium capacity users, but from what Microsoft has told me, that timeframe actually applies to all customers, and that everything will stay working during the promised year-plus window.)
The catch. Currently, gen1 dataflows are included in Power BI Pro and PPU licenses, while gen2 dataflows require Microsoft Fabric. Pro and PPU are flat-rate per user licenses, while Fabric is usage-based. Also, unless you purchase a very large Fabric capacity, Fabric requires that you still maintain at least Pro licenses—so, in many cases, adding Fabric is in addition to existing licenses, not a replacement for them. In short, for a number of customers, the retirement of gen1 dataflows is poised add additional recurring costs—the cost of Fabric—if they want to keep access to dataflows. At the same time, the value of (but not the cost of) their existing Power BI Pro or PPU licenses will drop (because it loses one of its existing, key benefits).
Or so it appears. That last paragraph is based on how gen1 and gen2 are currently sold. Numerous members of the community have publicly expressed concern about this apparent pending loss, and asked about the topic. Microsoft’s replies have avoided saying that, indeed, Pro/PPU users will lose dataflows, but also avoided saying that gen2 dataflows will start being part of Pro/PPU. If the latter were the plan, it seems it would be easy for Microsoft to say so. Doing so promptly would be to their PR advantage, as it would greatly cut down on the backlash they are receiving. Instead, my suspicion is that they don’t plan to bring gen2 dataflows to Pro/PPU but instead are shifting their business model for dataflows towards pay for what you use (e.g. Fabric) vs. flat-rate per user (e.g. Pro/PPU). I hope the large amount of negative reactions that have been received will cause them to rethink this plan.
Is this a big deal?
Yes, for reasons including:
The damaging of trust. When a vendor won’t give you the straight facts, particularly when there is an outcry asking for them, it wounds their reputation. This isn’t a typo in a blog post that remained unnoticed until a moment ago; this is Microsoft choosing to not clearly answer despite much entreaty.
The loss. Moving from gen1 -> gen2 is not a bad thing technically, but the loss of a benefit that users are paying for is. Imagine if Microsoft hinted that Excel was going to be removed from your Microsoft Office 365 license and instead become a pay-per-use add-on. That would hurt; this hurts, too.
The cost. For some, the extra cost of adding Fabric may simply be annoying; for others, it may be prohibitive (imagine a customer with 5 Pro licenses totaling $70 USD/month who now needs to add a F2 or F4 Fabric capacity at $156–526 USD/month…they might not go for it). Compounding the decision is the challenge that determining the size Fabric capacity you need to cover your to-be-migrated dataflows can be hard to predict without actually migrating them. If you try ballpark guessing the net cost increase, you may want to factor in the possibility that your estimate could be a couple Fabric capacity tiers off—and this kind of uncertainty makes decision-making hard. On the other hand, if you decide to discontinue dataflows, there are also costs to consider—such as the financial cost of moving logic from dataflows to other technologies, or the business cost of doing without whatever those dataflows currently do.
The paradigm change. Beyond the money, there’s how this change may affect use, innovation and productivity. Even in a company where the extra costs are more or less seemingly trivial, there is still a tendency to watch costs. Right now, a user in marketing or finance or wherever that wants to automate a business process using a dataflow can do that without it costing extra, and so can freely create dataflows without needing to check with anyone or justify the business benefit vs. cost. However, with a switch to metered dataflows, that freedom will change in at least some companies: If you want to create a dataflow, please ask first; if you have a good enough reason, okay, but keep runtime short so we don’t burn too many capacity units (CUs)…and you’ll need to coordinate your refresh schedule to not interfere so we don’t max out our capacity, etc., etc. A big shift from the simple self-service BI culture that has historically been a significant contributor towards the adoption and success of Power BI.
Final Thoughts
There are consultants, like myself, that can help you sort through the pending impacts, and assist with implementing the requisite migrations or transitions. But this kind of business saddens me. I’d rather be engaged by clients helping them with solutions that add business value, not for them to spend their money on me just to keep what they have still working because a vendor wants them to make a change.
In the spirit of “when you’re given lemons, make lemonade,” perhaps a silver lining in this situation is the opportunity to reevaluate and refocus your BI infrastructure around your business’s current needs; and, at the technical level, the opportunity to refresh and reshape that infrastructure to align with current best practices. If you’re forced to face this situation, why not use it to pay off technical debts, modernize, and hone alignment with strategic business objectives? The silver lining out of this pain could be coming away with BI that is better than ever before.
That being said, this still hurts.