You’re a developer. You know how to write code that interacts with databases. Using tools like Entity Framework and LINQ, you can push and pull data to and from data sources. But how comfortable are you directly talking to a relational database in its native dialect—SQL? Maybe you can cobble together a basic
SELECT statement—but when you do, do you feel like a traveler in a strange land where you barely know the language?
To improve our understanding of how Power Query works, let’s step back from the pieces and look at the paradigm—the methodology that M is built around. Conceptually, how does M work? How does it think? What makes it tick? The particulars, like variables, let expressions, functions and identifiers, are all important, but if we only focus on details, we might miss the big picture. We might not see the forest because we’re looking so intently at the trees. Let’s step back and learn about M’s modus operandi.
Without an awareness of M’s methodology, we might spend our days writing M code perplexed by certain behaviors, puzzled by why M seemingly doesn’t let us do certain things we might want to do and perspiring as we waste effort trying to mimic code patterns from other languages that are needless in M.
In the ideal world, all .Net asynchronous methods support cancellation tokens: When invoking a method, simply pass it a cancellation token. Then, at the appropriate time, cancel the token and the asynchronous operation terminates.
Alas! We don’t live in the ideal world. Not every method we might asynchronously invoke works with cancellation tokens. When faced with an asynchronous operation we want to be able to cancel that doesn’t support cancellation tokens, one option is to implement our own cancellation logic by registering a callback with the token. When the token is cancelled, callbacks registered with it are executed.
Introducing callback into the picture raises questions around if/when, where and how those callbacks are executed. For example:
- Will the callback be invoked via the synchronization context that was current when it was registered?
- If multiple callbacks are registered, are they run synchronously or in parallel?
- If a callback raises an exception, can that exception be caught?
In the case of .Net’s Task Parallel Library and its
CancellationTokenSource, the answers to these questions revolve around when cancellation occurs and how it is triggered.
Up until now, we’ve used variables without talking much about their specifics. Since we’re operating under the assumption that you have at least a little experience with another programming or scripting language, how we’ve used variables has probably made sense.
However, there might be differences—perhaps even significant differences—between how variables work in other languages you’ve used with and how they behave in the Power Query M language. Let’s spend some time exploring variables and related concepts to clear up any lurking confusion and position you to take full advantage of what M offers.
We’ll start with a brief recap of the main unit where we define variables: the
let expression. Then, we’ll talk about how variables (and other things) are identified. Related to identifiers is scope, so we’ll cover that, too. Next time, we’ll expand our understanding of how variables work by learning about M’s paradigm.
Not only can you define and invoke functions (as we covered in part 2), you can also pass them around. The ability to pass a function around without invoking it gives lots of flexibility.
Sounds complex? Yes, in words, but not necessarily in practice. If you’ve touched the Power Query M language, you’ve probably already passed functions around—just perhaps without realizing about it.
Then there’s the mysterious
each. It shows up a lot in code generated by the query editor. What does it actually mean or do? Turns out, it’s a handy shortcut that can simplify code you write.
We have ground to cover. Let’s get going!
If you read part 1 in this series, you may have picked up a theme about expressions that produce values: A simple statement, like
1, is an expression that produces a value. let is also an expression that produces a value.
Guess what? A function is an expression that ultimately produces a value. Unlike the expressions we looked at last time, a function only produces this value when it’s invoked.
Usually, parameters are passed to a function when it’s invoked. The function can reference these inputs as it computes what it will return. Continue reading
Last night, I had the privilege of sharing a lightning talk at the OKC SQL Server User Group (OKCSQL). This 17-minute presentation touches on four scenarios where the way columns are returned by
SELECT might not match what you expect.
When you write
SELECT statements, you probably have expectations like:
SELECT * returns all columns in the referenced object(s).
- Each column is returned in a separate column.
- Each column returned in visible in the result set.
However—at least in the world of Microsoft SQL Server/Transact SQL (T-SQL)—these assumptions aren’t always true.
Sooner or later, you may find yourself working directly with the Power Query M formula language. Tools like Microsoft Excel’s Get & Transform Data and Microsoft Power BI provide point-and-click interfaces allowing you to build data transformation sequences which behind-the-scenes are implemented in Power Query M. However, these query editors can’t do everything you might need. Sometimes direct editing and authoring of M is required.
Search the Internet and you’ll find many examples showing how to use this language to solve one problem or another but little is out there describing the syntax and paradigm of the language itself. A limited knowledge of these details may be insignificant when simply copying and pasting samples and editing variable and column names. Move beyond this to weaving and writing your own solutions directly in Power Query M and a solid understanding of the language’s syntax, rules and capabilities becomes most helpful.
Object-oriented is a methodology that uses a language feature known as objects as its fundamental unit of expression. The acronym SOLID identifies foundational principles differentiating between an object-oriented design and a design that simply uses objects.
#ObjectOriented uses #Objects but using #Objects isn’t always #ObjectOriented.
– #Programming #Architecture #ObjectOrientedDesign – @bgribaudo (original Tweet)
Recently, I read “Uncle Bob” (Robert C.) Martin’s exposition of the SOLID in his book Agile Software Development, Principles, Patterns, and Practices (Pearson, 2002, ISBN: 978-0135974445). Below is a combination of notes from his explanation and thoughts of my own that came as I contemplated what Uncle Bob wrote. Continue reading
Microsoft SQL Server 2017’s support for Python had me curious. Would this integration make it practical to use Python in non-data analytic scenarios where a general-purpose programming language—like Python—is more suitable than set-based T-SQL?
The answer to this question hinges around the integration’s design. Presumably, the new Python integration is designed to work well for data analysis, as this seems to be the prime impetus motivating it. An integration that excels in one scenario, like the assemble data -> process & analyze -> return results flow common in data analytics, may or may not be a good fit for other use cases.
To help us evaluate when and where SQL Server’s Python support may be helpful for non-data analytics applications, let’s compare it—or, more specifically, let’s compare the underlying external script execution environment that powers it—with two integrations that have been included with SQL Server for some time: Common Language Runtime (CLR) and xp_cmdshell.