Type Equality

• , , , ,

Type equality is an advanced—and also confusing—Power Query topic. Sometimes, equality comparisons between type values seem to work as intuitively expected; other times, they may not. What’s going on? Is it okay—is it safe—to use the equals operator between type values?

But first: Why the confusion around this subject? The M language specification lays out a brief set of rules for type equality which define the minimum expected behavior. These may then be augmented by additional rules specific to the Power Query implementation you are using. Grasping the full implications of the former isn’t always intuitively obvious and there is almost nothing publicly written documenting the latter—resulting in Power Query type equality being a confusing, little understood advanced niche.

Let’s try to clear things up. Let’s explore the behaviors of Microsoft’s flagship Power Query mashup engine implementation (i.e. the Power Query that ships with Microsoft Power BI and Excel). We’ll examine both behaviors mandated by the M spec (specifically, the relevant subsections of “Types” and “Operators“) and additional ones that are specific to the flagship PQ implementation.

Let’s get started!

Continue reading →

Rounding: Power Query vs. Others

• , , , ,

Did you know that, by default, Power Query may round numbers differently than you learned in grade school? Unlike DAX, Microsoft Excel and Microsoft SQL Server, which use commercial rounding as their main or default approach to rounding, Power Query is different: it defaults to banker’s rounding.

What’s the difference between these two ways to round? What if you don’t want Power Query’s default—can you override it? What if you don’t ever intentionally round numbers, does this topic even matter to you? (Hint: Possibly yes!)

 Let’s explore some details and options.

Continue reading →

What’s With WriteToDataDestination?

• , , , ,

If you examine the refresh details for a Power Query gen2 dataflow, you’ll notice that the activity names which are displayed are identical to the query names in your dataflow except that each has a “_WriteToDataDestination” suffix appended to it. What’s going on with this?

Let’s see what we can uncover! Time for some sleuthing….

Continue reading →

Exploring Power Query Buffering: How Table.Buffer and List.Buffer Work

• , , ,

Table.Buffer and List.Buffer buffer data—but how do they work? How deeply do they buffer? How do they handle errors? And, for that matter, when do they populate?

In a nutshell: Table.Buffer creates a stable copy* of a table’s rows. These rows are fetched only once from the source, regardless of how many times they are read from the buffer. Each time the buffer is accessed, the same* rows are returned in the same order. With List.Buffer, the behavior is identical, except it is list items that are buffered instead of table rows. (*But this may not mean what you think, so keep reading.)

The details have some nuances to them. Let’s explore them, as well as what happens when errors are encountered.

Continue reading →

Power BI’s Pseudo-Time & Pseudo-Date Types

• , ,

Once upon a time, I was bit by a time column which straightforward equality-comparison DAX wouldn’t filter as expected. From this saga, I learned a couple important lessons about Power BI’s data types that are not intuitively obvious from the user interface.

Imagine some simple DAX like the below. When evaluated, the result I’d receive back was blank—even though rows with the matching time could be plainly seen in Power BI’s table view.

CALCULATE (
  COUNTROWS ( Data ),
  Data[Timestamp] = TIME (12, 30, 0)
)
Table view showing value "12:30:00 PM" twice.

Why this strange behavior? Turns out, Power BI’s UI was not displaying the full data values in the column. When viewed in the UI’s table view, it looked like matching values existed in the column, but actually they didn’t. Comparisons were being performed correctly, just not against the time-only values I thought were there.

Continue reading →

Time Values from Dataflow to Power BI via DirectLake

• , ,

Seems simple enough: The table, produced by your Power BI gen2 dataflow, contains a time column. You want to output this table to OneLake, then have Power BI read from it using a DirectLake-powered semantic model.

Sounds easy, but not so fast! When you try to set up the dataflow’s output mappings, columns of type time are not supported for OneLake destinations. Ouch!

So, what do you do?

Try leveraging the fact that Power BI’s time values are, behind the scenes, actually datetime values. For proper “type time” behavior, Power BI expects that the date part of each datetime value is set to Power BI’s epoch date (the date it considers day 0, which is 1899-12-30) and it also needs to know that the column is expected to have time behaviors.

Continue reading →

Render Tables, Lists, Records -> Text

• , ,

Power Query’s simple scalar values—like date, number and logical—can easily be converted to strings. Each has a corresponding type-specific “ToText” method (like Date.ToText or Number.ToText). The generic Text.From can also be used.

But what if you want to render a table, list or record textually? There is no built-in way to convert values of these types directly to text.

However, you can convert them to JSON…and then render that JSON as text!

(input as any) as nullable text =>
  if input = null
  then null
  else Text.FromBinary(Json.FromValue(input))

Handy to render out a complex, nested structure so that you can see all of it at once!

Continue reading →

Privacy Levels in Dataflows: Impacted by Staging?

• , , ,

Staging data is common in the world of Microsoft Power BI dataflows. Computed entities, linked entities and Microsoft Fabric’s “enable staging” options all result in intermediate output being staged to disk and/or a database.

Does this staging affect privacy levels? Yes! It can change which privacy level is being applied.

Continue reading →