• Create a Power Query custom data connector in 10 minutes

    Getting set up

    When I heard about custom data connectors for Power Query, I had assumed there would be a lot of work involved. While there is definitely quite a bit of work in implementing advanced features like query folding,  creating your very first connector is simple.

    So, first you need Visual studio installed and the Power Query SDK installed as well. Once you do that, you will see Power Query as an option when creating a new project. Visual studio will also have support for .pq or Power Query files.

    image

    Once you create a new data connector project, you are presented with two main Power Query files. The first one, is simply a test query you can run on demand to test your connector.

    image

    The other file is your data connector. It has a bit of boilerplate to specify the types of credentials it accepts and publishing details such as beta status. Otherwise there is just a little bit of code defining the actual functionality. In this case we are defining the Contents function, which acts as a hello world:

    image

    If we run it as is, our test query will be run and we’ll see the results in a testing program.

    image

    Adding a function

    So now, what if we want to add some more functionality? Say maybe a function to square numbers. First, we’ll add a SquareNumbers.Squared function to the main file:

    shared SquareNumbers.Squared = (x as number) =>
    let
    y = x * x
    in
    y;

    Then we update the sample query to call out function:

    let
    result = SquareNumbers.Square(7)
    in
    result

    And it works as expected:

    image

    Exporting the connector

    Once you have the connector working the way that you want,  run a release build in visual studio. This will create a .mez in the bin/Release folder of your solution. Copy that file to the [Documents]\Power BI Desktop\Custom Connectors folder. You will likely have to create that folder.

    Whenever you open Power BI Desktop, it will recognize the connector but won’t let you use it because of security settings.

    image

    To get around this, go into the options for Power BI Desktop and then security. Under security, select “Allow any extension to load without validation or warning.” Then Restart Power BI Desktop.

    image

    Now we can see it is available in our list of connectors.

    image

    By default it will call the Contents function:

    image

    But we can easily modify the M code to call our squared function as well.

    image

    Which will give us the output we expect.

    image

    What next?

    If you are interested in going deeper with Custom Data Connectors, such as adding a navigation view or  query folding, check out the TripPin tutorials.

  • T-SQL Tuesday #111 – What is your why?

    For this T-SQL Tuesday, Andy Leonard asks us, “What is your why?”, why do you do what you do?

    Many of the answers from others are heartfelt and admirable. Mine are not. Most fall under the category of either dumb or dumb luck.

    Starcraft and Scholastic

    My very first taste of programming was back in first grade or so with LOGO. I thought it was a lot of fun but never did any significant after that. I first got my second taste of programming with Starcraft custom maps. The year was 1999. I know this because my username was eugene11.  I’ll let you do the math. Starcraft had a scripting logic that was basically a simple if/then system.

    But the day I became a true programmer was when looking in the back of a scholastic catalog, I found Interplay’s Learn to Program BASIC! The description promised to teach me how to make video games.
    Interplay's Learn to Program Basic - Junior High Edition (Windows 95/Mac OS Required)

    It was pretty easy to follow along with the lessons, although I didn’t truly understand what I was doing.  I could make small modifications to the games they provided but didn’t understand half of the logic therein.

    In high school I would make video games on my graphing calculator, because I was bored. I also participated in a game jam called PyWeek.

    Brothers and ex-girlfriends

    When I went to college, I figured I’d either be a programmer or a professor. I went to Penn State beaver because it was close, cheap and my brother graduated there, so I already knew one of the professors.

    In my senior year, I needed to take an elective in my major. I took Business intelligence, because the other option didn’t work with my girlfriend’s schedule. Remember when I said some of the reasons were dumb?

    Later on I asked the professor for a job. He worked full time as the head of Business Intelligence at Bayer Material Science. He hired me and I worked there for a year. Ultimately, though, it wasn’t a good fit.

    My next job was labeled .net/SQL developer. But after I was accepted I realized it was 90% SQL and suddenly I was a DBA and the new BI department. This part falls under the dumb luck, and it was lucky. I was in a role that I could grow into and over the years ended up working as a BI consultant within the company.

    While much of the path to BI was an accident, I’ve stayed with it in part because of the strong community and the friendships I’ve made. I really wonder if I’d find a community as strong and giving as #sqlfamily in a different field. But also, if I’m being honest, because the opportunity cost is high and this is a well-paying profession.

    Mom and my new adventures

    My career for past 7 years has largely been an accident. I’ve always loved computers and thinking, but I can hardly say I chose this field. But back in September, I quit my job to work for myself. I did this to work on my physical health, my mental health, and have more control over my career.

    And honestly, why not? Worst case I have a gap year on my resume, and I go back to get a regular job, right?

    I’ve written about the lessons learned so far and more recently, I’ve discovered another why. Working for myself allows me the flexibility to take care of my mom. And so why not becomes much more of a why.