Monthly Archives: February 2019

Create a Power Query custom data connector in 10 minutes

Miscellaneous Tech Explorations

Getting set up

When I heard about custom data connectors for Power Query, I had assumed there would be a lot of work involved. While there is definitely quite a bit of work in implementing advanced features like query folding,  creating your very first connector is simple.

So, first you need Visual studio installed and the Power Query SDK installed as well. Once you do that, you will see Power Query as an option when creating a new project. Visual studio will also have support for .pq or Power Query files.

image

Once you create a new data connector project, you are presented with two main Power Query files. The first one, is simply a test query you can run on demand to test your connector.

image

The other file is your data connector. It has a bit of boilerplate to specify the types of credentials it accepts and publishing details such as beta status. Otherwise there is just a little bit of code defining the actual functionality. In this case we are defining the Contents function, which acts as a hello world:

image

If we run it as is, our test query will be run and we’ll see the results in a testing program.

image

Adding a function

So now, what if we want to add some more functionality? Say maybe a function to square numbers. First, we’ll add a SquareNumbers.Squared function to the main file:

shared SquareNumbers.Squared = (x as number) =>
let
y = x * x
in
y;

Then we update the sample query to call out function:

let
result = SquareNumbers.Square(7)
in
result

And it works as expected:

image

Exporting the connector

Once you have the connector working the way that you want,  run a release build in visual studio. This will create a .mez in the bin/Release folder of your solution. Copy that file to the [Documents]\Power BI Desktop\Custom Connectors folder. You will likely have to create that folder.

Whenever you open Power BI Desktop, it will recognize the connector but won’t let you use it because of security settings.

image

To get around this, go into the options for Power BI Desktop and then security. Under security, select “Allow any extension to load without validation or warning.” Then Restart Power BI Desktop.

image

Now we can see it is available in our list of connectors.

image

By default it will call the Contents function:

image

But we can easily modify the M code to call our squared function as well.

image

Which will give us the output we expect.

image

What next?

If you are interested in going deeper with Custom Data Connectors, such as adding a navigation view or  query folding, check out the TripPin tutorials.

T-SQL Tuesday #111 – What is your why?

SQL Server

For this T-SQL Tuesday, Andy Leonard asks us, “What is your why?”, why do you do what you do?

Many of the answers from others are heartfelt and admirable. Mine are not. Most fall under the category of either dumb or dumb luck.

Starcraft and Scholastic

My very first taste of programming was back in first grade or so with LOGO. I thought it was a lot of fun but never did any significant after that. I first got my second taste of programming with Starcraft custom maps. The year was 1999. I know this because my username was eugene11.  I’ll let you do the math. Starcraft had a scripting logic that was basically a simple if/then system.

But the day I became a true programmer was when looking in the back of a scholastic catalog, I found Interplay’s Learn to Program BASIC! The description promised to teach me how to make video games.
Interplay's Learn to Program Basic - Junior High Edition (Windows 95/Mac OS Required)

It was pretty easy to follow along with the lessons, although I didn’t truly understand what I was doing.  I could make small modifications to the games they provided but didn’t understand half of the logic therein.

In high school I would make video games on my graphing calculator, because I was bored. I also participated in a game jam called PyWeek.

Brothers and ex-girlfriends

When I went to college, I figured I’d either be a programmer or a professor. I went to Penn State beaver because it was close, cheap and my brother graduated there, so I already knew one of the professors.

In my senior year, I needed to take an elective in my major. I took Business intelligence, because the other option didn’t work with my girlfriend’s schedule. Remember when I said some of the reasons were dumb?

Later on I asked the professor for a job. He worked full time as the head of Business Intelligence at Bayer Material Science. He hired me and I worked there for a year. Ultimately, though, it wasn’t a good fit.

My next job was labeled .net/SQL developer. But after I was accepted I realized it was 90% SQL and suddenly I was a DBA and the new BI department. This part falls under the dumb luck, and it was lucky. I was in a role that I could grow into and over the years ended up working as a BI consultant within the company.

While much of the path to BI was an accident, I’ve stayed with it in part because of the strong community and the friendships I’ve made. I really wonder if I’d find a community as strong and giving as #sqlfamily in a different field. But also, if I’m being honest, because the opportunity cost is high and this is a well-paying profession.

Mom and my new adventures

My career for past 7 years has largely been an accident. I’ve always loved computers and thinking, but I can hardly say I chose this field. But back in September, I quit my job to work for myself. I did this to work on my physical health, my mental health, and have more control over my career.

And honestly, why not? Worst case I have a gap year on my resume, and I go back to get a regular job, right?

I’ve written about the lessons learned so far and more recently, I’ve discovered another why. Working for myself allows me the flexibility to take care of my mom. And so why not becomes much more of a why.

The real reason to become self-employed: being a caretaker.

Self-Employment Insights

Normally, I’d spend a few hours writing some eloquent 1,000 word blog post, with a dozen sub-headers and very mild puns. Today is not one of those days, as much as I want to make sure I do the topic justice. I’d rather get this written than push it out.

My rickety raft

If you are looking to become an independent consultant for the money, welll I’d advise against it. I wrote before about lessons learned and while you can make a lot of money, it’s a big slog and a big stress. I would compare it to trying to build a big wooden ship, from a rickety raft while you are using the raft.

Only now, about 5 months in, are things stable enough that I can relax. Our finances are solid for the next 3 months and we’ve finally gotten some walls on our raft. It’s still ricketey, but it’s going to take more than any single wave to topple us. It will likely take the rest of the year to get things completely stable.

If I was looking to make more money, the smart move probably would have been to take a job with “Senior” in the title and get a 20k pay bump.

The real benefit

There are a handful of reasons why I made this leap. I was tired of feeling overworked, I wanted to give this a shot, I wanted to work less hours, I wanted to travel more and give more presentations, etc. But a least a third of it, was knowing that in 5 years, or 10 years stuff was going to hit the fan with my mom.

I’m currently the primary caretaker for my mom. She lives independently and gets a lot of services from a Medicare replacement program, called Life Beaver County. I usually describe it as adult daycare meets medical center. They clean her place once a week and have nurses make sure she takes her medicine. For a while, my responsibilities were just grocery shopping every week and occasionally taking her out to go shopping.

But in the back of my mind I knew that as some point those needs were going to escalate. I knew at some point down the road there was going to be a year where she wasn’t well enough to live totally independently but not ill enough to go into assisted living. And very recently, I’ve gotten a, ahem, new commute.

My new commute

Things had escalated recently to the point where I knew that if my mom didn’t get more care, her physical and mental health were going to deteriorate. And so, somewhat reluctantly, I decided to start driving my mom to Life Beaver County every morning.

I am utterly blessed to be able to do so. While it’s a pain to spend 60-90 minutes every morning waking up my mom, getting her ready, taking her in; I am truly lucky to have it as an option. Right now I’m optimistic that her physical and mental health are going to greatly improve and that this could be the difference between being in a home in 10 years instead of 2.

Working for yourself is an utter pain. It requires a whole new set of skills as difficult as learning to be a manager, in my mind. But it also brings some options that just aren’t available with most jobs.