Category Archives: Query Optimization

Query folding workaround for Azure Devops and Power BI

Query folding is one of the most powerful tools in Power Query and Power BI. It is the automatic process of pushing down filters and other transformations back to the data source. This can dramatically improve performance for your queries.

Unfortunately, OData is not guaranteed to support query folding. According to the Power BI documentation on incremental refresh.

Most data sources that support SQL queries support query folding. However, data sources like flat files, blobs, web, and OData feeds typically do not. In cases where the filter is not supported by the datasource back-end, it cannot be pushed down. In such cases, the mashup engine compensates and applies the filter locally, which may require retrieving the full dataset from the data source.

I recently did some tests on this for the OData source for Azure Devops. When I tested with the sample Northwind database, query folding was working. I was able to see with Fiddler that my date filter was getting pushed back down.

clip_image001

However, when I tried to the same with Azure Devops, none of my filters where getting pushed down to the source. As a workaround, I was able to put my filters in the URL. So to filter based on date, I used the following url:
https://analytics.dev.azure.com/eugene1234/TestingOdata/_odata/v2.0/WorkItems?$select=WorkItemId,WorkItemType,Title,State,ChangedDate&$filter=ChangedDate%20gt%202018-12-30T23:59:59.99Z

clip_image001[5]

In this case, I was manually specifying the date filter in the URL. But it should be possible to use M code to dynamically generate the URL. Another option might be to create a custom data connector for oData that supports query folding.

Slides for PASS BI Virtual Group

Today I’m going to be presenting on DAX for the PASS BI Virtual Group. The focus is on all the hard mental concepts of DAX. If I could sum up the talk in one picture, it would be this:

That red area is where I banged my head when learning DAX. The learning curve shoots up wildly in the middle of learning the technology, instead of a slow gentle curve. This presentation covers the middle parts that are key to understanding DAX.

Here are the slides for the presentation: Introduction-to-DAX-2017-03-30

Here is the recording:


Here is the talk by Marco Russo I mentioned in my presentation: Optimizing Multi-Billion Row Tables in Tabular