All posts by Eugene Meidinger

My thoughts on diversity in tech conferences

Today there has been some discussion on Twitter about diversity in tech conferences. I’m not going to link to the discussion directly, because this isn’t about the specific conference that spurred the conversation. I’m not here to name and shame anyone.

I volunteered for 3 years as program manager for PASS Summit, so I will be speaking from experience. I have written before that diversity is important, and I think the bar for that is raising for tech conferences. So where should that bar be?

What is your target goal?

First, I think every single tech conference needs some kind of target goal for diversity. It doesn’t have to be a a hard numerical goal and it doesn’t have to be a quota, but diversity should be somewhere in the thought process, every step along the way. If you are not intentional about this, you will trend towards the default, which is a very homogenous speaker pool.

So what kinds of goals are there? In my mind I see 4 easily defined ones:

  1. No goal. No consideration given to diversity. This is unacceptable in 2021.
  2. Attendee demographics. Aiming for parity with the diversity of your audience
  3. Speaker demographics. Aiming for parity with your speaker pool
  4. Regional/Global demographics. Aiming for parity with the general population.

I personally believe the bar today should be set at #2 & #3, whichever is more difficult. In my experience, there are concrete steps you can do to improve the diversity of your speaker pool, such as encouraging folks to submit, or setting aside a certain number of invitation slots. If the your selected speakers aren’t at least as representative as your submitted speakers, then you aren’t trying hard enough.

When I was working on PASS Summit, diversity was a secondary concern, but it was a consistent one, and we took steps when there were issues. We didn’t have any hard targets, but if we felt there was an imbalance, we would go out of our way to juggle the schedule or intentionally invite speakers. There was plenty of room for improvement, however, especially with decisions outside of the selection process.

Where does diversity matter most?

It is not enough to say that your total pool of selected speakers has diversity. Diversity is most important where there is an implied endorsement or there is money involved. You don’t get to shine a spotlight and then ignore your responsibility.

If a conference is placing emphasis on a subset of speakers, the bar for diversity is even higher there for two reasons. First, there is where a conference has the most discretion. It is much easier to get it right with a dozen speakers than a hundred. And if they are promoting a specific set of speakers, they have much more control over who they are promoting and why.

The other reason is any time you elevate a subset of speakers, you are making a statement, even if it’s an unintentional one. Let’s say that hypothetically your conference had 25% females speakers, but not a single female precon speaker. That would send strong implied message, because precons are lucrative, highly coveted, and a strong endorsement of the speaker. These kinds of implied messages can be immensely discouraging.

For us at PASS, this applied to two areas main areas. First was the televised sessions, because these sessions had a broader reach and there was an implied endorsement of quality. It also applied to precon trainings. If folks saw the same old faces for precons, we would get roasted, and rightfully so. Lastly, it was also an issue in some of the marketing of the early invited speakers, and sometimes that presented a tension between marketing process and the selection process.

One other area where it matters are panels. In the year 2021, there is no reason to ever have an all male panel, ever, ever, ever. On a panel of 4-5 speakers, it does not take that much work to for you to find at least one female speaker. If you can’t, you should cancel the panel and look into the deeper issues with your selection process. Even if it is completely accidental, a “manel” comes across as lazy at best.

What can conference organizers do?

Generally speaking, conference organizers have a wide variety of tools to improve diversity if they are willing to get creative and especially if they have a budget to work with. PASS was notoriously stingy, which forced us to depend on the former.

Ideally, it should start during the call for speakers. If the speaker pool seems lopsided, the conference organizers should be taking steps to encourage new and diverse speakers. Any conference integrated into the community is likely going to have connections that can amplify those messages. Even better, if a conference is willing to pay speakers, that opens up a much broader pool instead of the most privileged folks. In some cases, conferences can also set aside a certain number of slots for invited speakers and use those to improve diversity.

During the selection process, a lot of it comes down to mindfulness. A conference should be spot checking things all along the way, even better if they can get folks from outside the team to help. Again, having a certain number of reserved slots and being willing to invite and pay people can go a long way here.

Finally, there are indirect steps the conference can take. First, what are you doing to encourage new or low-profile speakers? Things like feedback on abstracts or speaker mentoring can increase the breadth of speakers available to you. As a new speaker in 2017, I felt discouraged about submitting to big conferences and I’m sure under-represented folks feel the same way.

Additionally you can take steps like having a clear code of conduct, having sessions on diversity, and providing support for underrepresented groups. Signs that a conference is safe and fair to underrepresented groups will encourage more of those folks to submit.

Summary

Improving diversity in a conference is hard work, and the average call for speakers is often an unbalanced starting point. However, conference organizers have a number of tools available to them, especially for large paid conferences.

In 2021, there is never an excuse for an all male panel, and any elevated subset of speakers, such as paid precon speakers, should come under heightened scrutiny. Every tech conference should at least be aiming to match the demographics of their submitted speakers and their attendees. To me these all feel like a reasonable starting point.

My experiences as a PASS program manager: improving transparency is hard

Right now there is an election going for PASS Board, and an item that came up multiple times during the AMA was transparency, and the lack there of. In fact Jen Stirrup recently wrote a blog post about PASS special board minutes that are ominous, but too vague to be actionable. I think there is a really important quote from her post:

For transparency, you need to walk the talk. PASS should be more transparent, and all the candidates have it on their ticket.

So let’s try to walk the talk. I’m going to write about my experiences trying to improve transparency at PASS, why I think it’s been so difficult, and the problems any new board members are going to face.  I feel that most of the issues are structural and not the fault of any one person. Some of them are likely cultural and leads to vicious cycles.

Finally, I’m under NDA so I will be talking in generalities. This is probably some level of irony, like rain on your wedding day, but it is what it is. I think some level of NDA is necessary to do this job, but I’d love to see some loosening of NDAs, as well as written guidance on what’s okay to talk about.

We are all volunteers

The first impediment is that we are all volunteers here. The board are volunteers, the program managers are volunteers. As PMs we meet weekly for an hour, sometimes multiple times per week when we are working on the selections and schedule.

I think it would be reasonable to estimate that take 75-100 hours of time to do just the job of managing the program. That is all time that I could be billing or working on my courses. From speaking with a former board member, being on the board probably takes 10x that amount of time, which is huge.

So, just doing the job itself takes quite a bit of time, anything beyond that is extra time and effort. Improving transparency is extra time and effort. That is the reality of the situation. I would love to have a monthly blog post on our process, in practice I’ve barely written two.

I think it’s reasonable to feel that certain goals or values like this should be first-class citizens. It’s natural though, to get into the position and get focused on going the work, on meeting the deadlines, etc. We get distracted by the short term tasks.

I can only assume that it’s the same way for the board members. Anyone running for the first time is going to have the herculean task of trying to change organizational inertia, while dealing with an existential crisis.

People will be shitty on Twitter

First, let me say that people have good reason to be upset and frustrated with PASS and PASS HQ. It’s heartbreaking to read how Andy Mallon’s efforts to promote the LGBT community have been stymied again and again. I’ve seen decreasing support for SQL Saturday’s and UGs, and I’ve seen plenty of poor communication. All valid frustrations.

Second, let me say that how people communicate that frustration and anger is unrelated to how valid those feelings are. That’s a lesson I’ve been taking away from the discussion on race earlier this year. Someone can be snarky, sarcastic or downright mean and still be right.

As a quick aside, hat’s off to three people who I’ve seen communicate criticism without forgetting there are humans on the other side: Erin Stellato, Meagan Longoria, and Monica Rathbun. We should all be more like these folks.

That being said, people being shitty is going to mean it’s more work to be transparent. This isn’t blaming them, but just trying to acknowledge the fact that as human being, it’s easier to not say anything. It’s easier to not write this blog post and take the extra effort, than risk the hot takes and the sarcasm. I want people to like me.

Thankfully, I have not personally been the subjects of any attacks or direct criticism. When I first joined the team 3 years ago, I didn’t want to tell anyone that I was on the team, because I assumed all of the criticism would get directed at me. Mercifully this has not happened, and I’ve slowly been more confident in acknowledging the work I do.

Ill-will and poor transparency breed more of the same.

I definitely think there is a culture with PASS of closed by default instead of open by default. It seems like nearly everything needs an NDA. I can speculate how we got there, but that doesn’t really change the fact. And the bigger issue is that it’s inertial and self-reinforcing. Ill-will and poor transparency breed more of the same.

What do I mean by this? Well, at my last job we had a merger and there were tons of rumors and gossip flying around. When you don’t say anything, people assume the worst. People assume ulterior motives.

This is human nature. If you don’t communicate your selection process for precons, people will assume favoritism.  If you don’t communicate that diversity is a goal in the selection process, people will reasonably assume it’s not a goal at all.

This is frustrating because very often Hanlon’s law applies: “Never attribute to malice that which is adequately explained by stupidity.”. So many of the mistakes I’ve made as a program manager have been because of my own stupidity. Something I forgot to check, something I didn’t think would be an issue, etc.

This is exacerbated when you’ve burnt bridges in the community or made mis-steps in the past. Now malice is assumed more quickly, quite reasonably. Now it’s not enough to improve transparency, but you have to do more on top of that to slowly rebuild trust. I’ve heard it said that trust is built in drops and lost in buckets.

Poor transparency leads to ungrounded decisions

Another way this becomes self-enforcing is that we touch base with the community less, we sanity check our ideas less. Communication takes work and the default assumption is people will complain no matter what you do.  But this leads to catastrophic errors.

When you are the person selecting the schedule, it’s easy forget what it feels like to be a nervous speaker, begging for advice on how to get selected. You know how the sausage gets made, and you don’t empathize enough with how people might feel about speakers being announced in waves. And suddenly what you though was non-controversial blows up because you aren’t in touch with your community.

And by you, I mean me.

Summary

This is a long post and I could go for longer. The thing I want to convey is that fixing transparency isn’t some light switch, it’s swimming up-hill. It’s extra work on top of a voluntary position. I like some of the ideas proposed by board candidates, but I think it’s going to be a monumental effort no matter what.

To do it, you need a thick enough skin to ignore what everyone says, and you need enough heart and empathy to listen closely to what everyone says. I’ve found this to be a real struggle.

T-SQL Tuesday #131: Power BI ELI5

T-SQL Tuesday Logo

This month’s T-SQL Tuesday asks us to explain a concept like the audience like they are 5 years old. I’ve decided to literally do that.

Do you like jelly beans? I like jelly beans. Imagine if you had a big, big bowl of jelly beans.

assorted flavors of jelly beans

How many jelly beans do you have? Can you tell by looking? I think you would have to count and it would take a very long time. Well, computers are good at counting! We could teach a computer to count our jelly beans and tell us how many Jelly beans we have. Using a program called Power BI, the computer can tell us we have 800 jelly beans! That’s a lot of jelly beans.
card visual

Do you have a favorite flavor of jelly beans? I really like the pink ones that taste like bubble gum.

multicolored candies on white ceramic bowl

What if we could tell the computer to count by flavor? We could then see how many we have total and how many by flavor.

Card visual and bar chart

Wait a minute! We have a bunch of cinnamon jelly beans, ew. I can’t stand cinnamon jelly beans. I will probably give them to my mom. I want to hide the cinnamon jelly beans, to see how many tasty ones I have. We can add a filter to look at specific flavors.

Card visual, bard chart and filter visual

We can even add categories of things, so we don’t have to remember which flavors are good and which flavors are bad.

Card visual, bard chart and filter visual, grouped by flavor
Oh good, we have 700 tasty flavors and 100 gross flavors. Okay, I want you to imagine one more thing. Imagine instead of one bowl of jelly beans, you had 100 jelly bean dispensers and you owned a whole jelly bean store!

candy dispenser lot

Well now we *really* need a computer to count everything. We want to know when a dispenser is empty. We want to know which flavors are most popular. Some people actually like cinnamon jelly beans. Gross! But if you own a store, you have to buy and sell the flavors people want.

Power BI is a program that helps business owners count everything going on in their stores. It helps them count everything but also let’s them look at little details like a single flavor or a single dispenser. Power BI helps store owners make more money and sell people’s favorite candy!

Lessons learned from being self-employed, 2 years in

Two years ago I quit my job and started working for myself. After two years, I think most people would look at it as a big success, I’m now finally able to make a stable living doing what I want. As we enter a recession, I know a lot of people would be grateful to be in my position.

But I’m also vaguely dissatisfied, because the past two years have have been an unfocused 24 months, running around putting out fires and trying to pay the bills. After 4 years of work, Pluralsight revenue pays a reasonable salary and I can take a peaceful vacation. When I come back from my cabin trip next week, I’m hoping to have a better idea of what the next 4 years will look like.

Four years from idea to full-time income

Four years ago, a friend convinced me to think bigger and in March 2016 I applied to be a Pluralsight author. I have never spoken at any large conference, I didn’t have any impressive credentials, and I didn’t expect to be accepted. But it turns out the hard part isn’t getting your foot in the door, but getting your audition approved. Imagine doing a recorded lightning talk that has to convey a technical concept but also tells a story.

It took about a year to go from starting the audition to getting my first course being published. A lot of this was delays from underestimating just how difficult it would be to make my first course, taking 140 hours plus some outsourced editing. All of this was during nights and weekends while working a full time job.

Chart showing7 courses released over 7 years, with revenue slowly increasing during that time.

The chart above is the 3 years it took to go from first course to full-time income. And while I can’t share exact numbers I’ve got two dotted lines there. The first one, in grey, is the bare minimum I would need to make, in addition to my husband’s income, to cover our minimum expenses. This includes paying $900 per month for our own health insurance.

The green dotted line is what I’d need to make to cover our expenses and live comfortably here in the Greater Pittsburgh Area. Additionally each orange dot is a new course being published. I’ve also labeled when I went part time, when I quit my job, when the Power BI learning path came out (with 5 of my courses in it), and the Free April promotion that brought in a lot of new users.

Two years of juggling and struggling

I think one of the key takeaways from the chart above is it took nearly a year of working for myself before my courses could cover by base expenses, and another 8 months before I could finally breathe and not worry where my next paycheck was coming from. During that time, I was trying to do consulting jobs to pay the bills, while completing the next course.

The whole of is chart is punctuated by moments of luck and risk. I was lucky that my second course, on DAX, was enough of a hit that I was about to go part time. It was a big risk quitting my job, since it took a year to get somewhere stable. Both times I hit one of those financial goals were because of an external event. The first was the release of the Power BI learning path, which increased viewership. The second was the FreeApril promotion, which brought in a lot of new viewers, many of which wanted to learn about Power BI.

Success comes from rhythm and routine

The reason I feel dissatisfied is that success comes from rhythm and routine. When you work for yourself, you are the only person rowing. You are the only person blogging, speaking, marketing, etc. And either you find a weekly routine that keeps you going through the difficult times or you will struggle and have moments of no momentum.

In retrospect, trying to do consulting and content creation at the same time has been a great way to diversify my income. As we enter a recession, I see the consulting work drying up and at least one friend going back into regular employment. But it has been challenging balancing the two. Consulting work is all hurry up and wait. Content creation is a slow and steady focus. The two always seem to be in conflict.

I never quite hit a solid rhythm and with Coronavirus happening, all of my extracurricular activities have stopped. No newsletter, no blogging, no presenting. I think this has happened to a lot of people, and I admire the folks who have been putting out weekly content during a global pandemic. When I come back from my vacation, my biggest priority is starting up a new rhythm.

Would I do it again?

I would absolutely do it all over. I wish I had been smarter about some things, but the hard part is over. I don’t need to worry about being fired, or losing my job during a recession. I’ve been able to visit my mom three days a week without worrying about an angry boss or work conflicts.

So am I going to keep consulting? Yes! I think that being a full-time author runs the risk of losing touch with the technology and where things are going. Consulting forces you to be grounded. I think doing a blend is great when you can do it, and awful when you are still figuring it out.

Five ways to reduce impostor syndrome

T-SQL-Tuesday-Logo
This T-SQL Tuesday is about Impostor Syndrome. I think that the term gets used in a broader context than the original meaning. From Wikipedia:

Imposter Syndrome is a psychological pattern in which one doubts one’s accomplishments and has a persistent internalized fear of being exposed as a “fraud”. Despite external evidence of their competence, those experiencing this phenomenon remain convinced that they are frauds, and do not deserve all they have achieved.

To me, it’s a syndrome when there is a persistent fear of being outed as a fraud. But I think there is a broader and more diffuse feeling of one’s skills falling short of what is expected or what other people perceive. I think a lot of people mean this when they say impostor syndrome. Heck, it might even apply to a broader sense of “winging it” or figuring it out as you go.

In this blog post I’m going to list concrete steps that have helped me fight this sense that I might be a fraud or don’t know what I’m doing. I personally believe it’s a tractable problem.

Pursue a growth mindset

Do you feel that you are born with a certain amount of intelligence and talent, or do you feel that it is something you can grow? When I was young, I felt that you were born with a certain IQ or a certain amount of smarts. The problem with this mindset is when you encounter evidence contrary.

Let’s say you screw something up at work. If you believe that you have a fixed amount of intelligence or skills, then you will see this a reflective of you as a person. If you have a growth mindset you are less likely to see this as reflective of you as a whole.

Build a peer network

So much of imposter syndrome can be broken down into two things: a disconnect with reality and a sense of invalidation. The disconnect is that, unless you are an actual fraud, your thoughts and feelings are not lining up with reality. In cognitive behavioral therapy, these things are called cognitive distortions.

The invalidation is the sense that this isn’t normal. That you are alone or unusual in the struggles you are having. That you should be feeling x or doing Y.

In both cases, having a peer network can ground you. Hearing other peoples experiences can be validating and reassuring. You’d be surprised how many of us are winging it. You’d be surprised how many of us have been handed a task and need to figure it out on the fly. For example, I am probably going to do a course on Azure Event Hubs and I know absolutely nothing about them.

Focus on just-in-time learning

In consulting, if you never say “I don’t know”, then you are full of crap. You can’t know everything. If you merely say “I don’t know”, then you are a fool. A good consultant says “I don’t know, but I can find out!”. That is one of the keys of consulting.

So much of my confidence in working as a consultant is knowing that I have a strong peer network I can reach out to. So much is knowing I have a strategy for just in time learning.

One way to do this is to practice presenting and blogging as a way of practicing your research skills. Another is to work on home labs. Multiple times I’ve had to learn something new like mirroring or availability groups for a customer job. By doing those home labs I was able to build confidence for the project as well as confidence for future learnings.

Present, write, and teach

A lot of this vague unease comes from never achieving mastery in a specific area. From those lingering unknown unknowns. One way to rout out those unknown unknowns is to create and share content on the subject.

Doing so will force you to think through the subject, to think through what questions people might ask. The first dozen presentations might be a bit rough. But as you continue, you’ll have a better sense of what it means to really know a topic.

Do paid consulting

I realize this simply isn’t an option for most people. Consulting can be stressful and anxiety inducing, especially if you are struggling with impostor syndrome. But it’s amazing how good it feels to charge for a project and by the end of it have a happy customer.

I still giggle when I tell people my hourly rate, and I keep expecting to get pushback about being too expensive. But it doesn’t happen. It turns out that other people value my time and skills more than I do.

Summary

My suggestions boil down to three things:

  1. Build confidence in your ability to learn on-demand
  2. Experience mastery through content creation
  3. Find people who value your skills enough to pay for them

Where do people get stuck with Power BI?

I was recently talking to a potential customer about getting started in Power BI. A question they asked me was essentially “What are the difficult parts of Power BI, how do these engagements usually go?”. This was a really interesting question and for me a lot of customer engagements fall under two broad categories.

The lost and confused. A lot of potential customers are just starting to look into Power BI. They cans see what it can do, they understand what it is for, but they have questions. Generally these questions are all about governance and implementation. How do we deploy Power BI? How do we license Power BI? How do we secure it?

This is a reasonable challenge because Power BI has a lot of nooks and crannies. It has a bunch of ways to deploy it and a bunch of ways to license it, and it seems like that add a new one every 6 months. For these customers they generally just seem to be concerned about the unknown unknowns out there. Usually after that first hour, that seems to be all the help they need.

The out of control. These customers have made an investment in Power BI. These customers have started creating reports and seeing result. The problem here is scaling their reports. M and DAX have goofy learning curves. Both start out easy, but get incredibly difficult when you try to do more complex things. Maybe reading CSV files was easy, but now you need to work with a web api now. Maybe your report was fast when it was 60 MB. But now your measure or more complex and you have 250 MB of data and it is slow. These customers need help applying good data modeling and optimizing those models.

What convinced me that diversity is important?

T-SQL Tuesday Logo
For this month’s T-SQL Tuesday, we are to write about a time we changed our minds on something. For me, something  I think about often is that 12 years ago I didn’t value diversity. Today I do value diversity, and if we want to persuade others, we have to figure out what changed.

Where was I 12 years ago?

Twelve years ago, I was going to college and I experienced a bit of a culture shock. The county I’ve lived my entire life in, Beaver County, is 92.5% white and 6% black. The local college I went to, Penn State Beaver, was decidedly more diverse from a racial standpoint.

I remember quite clearly thinking at that point that there had only been 3 times in my life when I had seen that contrast: college, travel stations, and visiting Washington D.C. The rest of my life was in this monoculture.

And when I say that I didn’t value diversity at the time, it wasn’t that I was against it or had negative feelings. It was just that I didn’t see a lack of diversity as a problem. I had a vague awareness that my IT classes were heavily white and heavily male, but that didn’t register as an issue.

Looking back, I think I figured that if different people had different natural talents or natural interests, then what was the harm in that? Part of the reason I was in IT was the things I was bad at. I was bad at people and interacting with them.

Bound up in the previous paragraph were a whole host of assumptions and naïve ideas. The idea that life was a meritocracy. The idea that people freely chose their profession, without discrimination.

What is your ethical structure?

On subjects like this, we often talk past each other. Sometimes in bad faith, but often because we are starting from different ethical precepts. In college I took an Ethics in Computer Science course and learned of two big terms: Utilitarian Ethics and Deontological Ethics.

Essentially, are things right or wrong because of the consequences or because of something inherent to the action? I think one example was “Is it wrong to clean your toilet with the American flag?”. Many people would say yes, but they would have trouble pointing to how anyone was visibly harmed by the act.

I mention this because I think a number of people that are supporting diversity do so from a strongly deontological perspective. So if less than 50% of CEOs are women, then that is in its very nature unjust or unfair. Regardless of the specific consequences.

I think this is a valid and reasonable viewpoint. But I also don’t think that this alone would have changed my mind. Generally I was more swayed by empathy for individuals and seeing concrete negative consequences.

What changed my mind?

I think there were a number of things that all changed my mind in little bits and pieces.

Monoculture leads to failure

Something that planted the seeds early on for me were stories of how having a monoculture of life experiences lead to failure. How HP webcams couldn’t follow black people. How Apple stores had glass staircases. How websites often don’t support names with umlauts or other markings because of assumptions about how names work.

I also learned about groupthink and the often exaggerated Asch conformity experiments. So often these failures occurred either because everyone thought the same or had the same background. Or people were afraid to speak up and stand out.

Life is not a meritocracy

For all of my schooling, I did fairly well. And there was a fairly stable correlation between how good I was at a given task and the grades I got. It seemed to me like school was a meritocracy and I expected work to be the same. I was woefully wrong.

I got a job at Bayer Material Science after college and after 8 months I was fired. I deserved to be fired and wasn’t particularly good at my job. Still, as someone used to getting A’s this was pretty shocking to me.

I learned a number of things during that process. I learned that how well we excel in certain areas can depend on a number of secondary factors, like our ability to work with others, to communicate, etc.

I learned that my skills at being a student didn’t necessarily translate into being a good worker, and I could infer the reverse must be true. There were undoubtedly people who flunked out of school but where crushing it in the workplace.

Violence, discrimination and bias are common

As far as I’m aware, I’ve never been racially or sexually discriminated against. If I have then the person did a terrible job of it, because I didn’t notice. Every police ticket I’ve gotten has been valid. I rarely have to worry about my safety when walking alone at night.

Again, in college I had some vague awareness of this. I understood the idea that my female friends might appreciate someone walking them to their car when it is dark. But I didn’t appreciate just how constant it was for some people. Heck, if I’m being honest, the #MeToo movement was shocking for me to see how many people close to me had been sexually assaulted.

It was helpful too to realize that bias can be invisible, implicit or even well-intentioned. I remember reading about implicit bias in a book and being frustrated about the implicit bias I had in regards to African Americans. I kept failing the test.

I remember meeting an internal salesperson at my last job and saying “Oh so you work in staffing?”, because I had this idea that all sales people were pushy and that women had better people skills, so clearly this person must be a staffing salesperson, not hardware or IT Services. That was 5 years ago and I still feel like an ass to this day.

Representation matters

So I’m going to talk about a type of diversity that may sound strange, but is very meaningful to me. I’m a Type 1 diabetic. And when I was diagnosed, in many ways I felt like my life was over. There were so many things I wasn’t going to be able to do. For the entirety of my last job, I told very few people because I feared I would get fired.

But one of the best things I did was subscribe to Diabetes Forecast magazine. Because in every issue they had a story of someone with diabetes doing something badass. Like racing, or mountain climbing or even winning a beauty pageant. These people weren’t letting diabetes stop them from living their lives.

For this same reason, Scott Hanselman is a hero of mine. He is very openly a diabetic and will present to hundreds of people without letting his diabetes stop him. If he has to check his blood sugar on stage or drink some orange juice, then so be it.

I say all of this to say that representation matters. My life was impacted because I saw myself in successful people. I saw myself in others.

It doesn’t take a colossal leap of empathy to imagine how I might feel if roles were reversed, if all the speakers at a tech conference were female or non-white. If that happened enough, I would quickly internalize the idea that I would never succeed, that I would never make it, that I would never get up on that stage.

If you optimize for individuals, you optimize for assholes

When I see a backlash against people trying to re-balance the scales, a lot of that backlash makes perfect sense under a certain set of assumptions or axioms.

If you believe that 1) the world is or should be a meritocracy and 2) we have an objective way to measure “skill” and 3) we should focus solely on the merits of the individual, then it’s reasonable to conclude that picking anyone but the “best” for a job is an injustice.

And while I think I believed all of those things in college, given my experiences since then, I no longer believe those things.

I’ve learned that often people fail for reasons completely outside of their control. I’ve learned that technical prowess is a minimal part of my job, and there are a plethora of other factors that make me good at my job. I’ve learned that when you narrowly look only at the individual, there are negative consequences both inside the team and outside of the team.

When you optimize for individuals, you optimize for assholes. We’ve all heard the story of person X who delivers results but is inherently toxic and drives others away. Throwing other people under the bus can be a great way to get results in the short term. We have to look beyond the individual.

I think any person in a place of power such as a manager or conference organizer, needs to look not just at the talent or popularity of the individuals, but the mix as a whole and the benefits diversity brings. We can disagree on to what degree, but it needs to be done.

Summary

I won’t pretend I have all the answers here. I still haven’t figured out when being pro-diversity swerves into being pro-tokenism, for example. I haven’t figured out that line where in trying to promote a group you accidently reduce them to a name, an identifier, a statistic.

But I can say that these are the things that have helped me value and appreciate diversity. And I hope we can all promote it, even if in small ways.

Query folding workaround for Azure Devops and Power BI

Query folding is one of the most powerful tools in Power Query and Power BI. It is the automatic process of pushing down filters and other transformations back to the data source. This can dramatically improve performance for your queries.

Unfortunately, OData is not guaranteed to support query folding. According to the Power BI documentation on incremental refresh.

Most data sources that support SQL queries support query folding. However, data sources like flat files, blobs, web, and OData feeds typically do not. In cases where the filter is not supported by the datasource back-end, it cannot be pushed down. In such cases, the mashup engine compensates and applies the filter locally, which may require retrieving the full dataset from the data source.

I recently did some tests on this for the OData source for Azure Devops. When I tested with the sample Northwind database, query folding was working. I was able to see with Fiddler that my date filter was getting pushed back down.

clip_image001

However, when I tried to the same with Azure Devops, none of my filters where getting pushed down to the source. As a workaround, I was able to put my filters in the URL. So to filter based on date, I used the following url:
https://analytics.dev.azure.com/eugene1234/TestingOdata/_odata/v2.0/WorkItems?$select=WorkItemId,WorkItemType,Title,State,ChangedDate&$filter=ChangedDate%20gt%202018-12-30T23:59:59.99Z

clip_image001[5]

In this case, I was manually specifying the date filter in the URL. But it should be possible to use M code to dynamically generate the URL. Another option might be to create a custom data connector for oData that supports query folding.

A comprehensive guide to Power BI performance tuning

Promotion: Use code PERFBLOG to save 10% off my course. Module 1 is free.

While it’s easy to make reports in Power BI, it can be a pain to troubleshoot and tune them. Properly performance tuning Power BI reports requires identifying the bottleneck and using a handful of external applications. This article covers how to narrow down the performance problem, as well as general best practices. Once you understand the fundamentals it gets a lot easier.

Which part is slow?

As a report developer, it can be frustrating a report developer, knowing that something is slow, but not being able to put your finger on it. In my mind, there are 4 main areas where there might be a slowdown:

  1. Data refresh
  2. Model calculations
  3. Visualization rendering
  4. Everything else

Identifying which one of these is the problem is the first step to improving performance. In most cases, if a report is slow it’s an issue with step 2, your data model.

Tuning the data refresh

The refresh is rarely the problem because the user never sees the refresh process if you are using Import mode. So usually you are going to see a slow refresh when you are authoring the report or if a scheduled refresh fails. Still, it’s important to tune your data refresh to avoid timeouts and minimize how much data your are loading.

Verify that query folding is working

If you are querying a relational database, especially SQL Server, then you want to make sure that query folding is being applied. Query folding is when M code in PowerQuery is pushed down to the source system, often via a SQL query. One simple way to confirm that query folding is working is to right click on a step sand select View Native Query. This will show you the SQL query that will be run against the database. If you have admin privileges on the server, you can also use extended events to monitor the server for queries. If you are doing this, you should limited the monitoring to the Data Mashup engine, which is the application name for Power Query.

Some transformation steps can break query folding, making any steps after them unfoldable. Finding out which steps break folding is a matter of trial and error. But simple transformations, such as filtering rows and removing columns, should be applied as early

Minimize the data you are loading

If you don’t need certain columns, then remove them. If you don’t need certain rows of data, then filter them out. This can improve performance when refreshing the data AND when modelling it later on, which is a win-win.

If your Power BI file is more than 100MB, there is a good chance you are going to see a slowdown due to the data size. Once it gets bigger than that it is important to either work on your DAX code, or look into an alternative querying/hosting method such as DirectQuery or Power BI Premium.

Consider performing joins in DAX, not in M

If you need to establish a relationship purely for filtering reasons, such as using a dimension table, then consider creating the relationship in DAX instead of in PowerQuery. DAX is blazing fast at applying filters, whereas Power Query can be very slow at applying a join, especially if that join is not being folded down to the SQL level. That being said, sometimes the data makes more sense as a flattened table for usability purposes.

Review your applied steps

Because Power Query is such a graphical tool, it can be easy to make changes and then forget about them. I had one customer who would often sort his data and then accidently leave the step in there. This was terrible for performance.

Make use of SQL indexes

If your data is in a relational database, then you want to make sure there are indexes to support your queries. If you are using just a few columns it may be possible to create a covering query that covers all of the columns you need. Work with your DBA to see what your options are. If you are using SQL Server, you can use execution plans to identify if the query is using an index. You will need the SHOW_PLAN permission on the server and you will need to use the View Native Query functionally to get the underlying SQL query

Tuning the model calculations

When someone says that a Power BI report is slow, it is usually an issue with the DAX modelling. Unfortunately, that fact isn’t obvious to the user and it can look like the visuals themselves are slow. Thankfully, there is a tool to identify the difference: the Power BI Performance Analyzer.

Use the Power BI Performance Analyzer

If your report is slow, the very first thing you should do is run the Power BI performance analyzer. This will give you detailed measurements of which visuals are slow as well as how much of that time is spent running DAX and how much is spent rendering the visual. Additionally, this tool gives you the actual DAX code being run behinds the scenes, which you can run manually with DAX Studio. When I am dealing with a tricky performance problem, this is my go to tool.

Confirm that the storage engine is being used

When using DAX studio, you can see how much time is spent with the storage engine versus the formula engine. The storage engine can only handle simple operations, but is multi-threaded and very fast. Whenever possible, it’s better to make use of the storage engine.

Remove data you don’t need

Because of the way the data is stored in Power BI, the more columns you have the worse compression and performance you have. Additionally, unnecessary rows can slow things down as well. Two years of data is almost always going to be faster than 10 years of the same data.

Additionally, avoid columns with a lot of unique values such as primary keys. The more repeated values in a column, the better the compression because of run-length encoding. Unique columns can actually take up more space when encoded for Power BI than the source data did.

Avoid iterator functions

Iterator functions will calculate a result row by agonizing row, which is not ideal for a columnar data store like DAX. There are two ways to identify iterator functions. The aggregation functions generally end in an X: SUMX, MAXX, CONCATENATEX, etc. Additionally, many iterators take in a table as the first parameter and then an expression as the second parameter. Iterators with simple logic are generally fine, and sometimes are secretly converted to more efficient forms.

Use a star schema

Using a star schema, a transaction table in the center surrounded by lookup tables, has a number of benefits. It encourages filtering based on the lookup tables and aggregating based on the transaction table. The two things DAX is best at is filtering and aggregating. A star schema also keeps the relationships simple and easy to understand.

Visualization Rendering

Sometimes the issue isn’t necessarily the data model but the visuals. I’ve seen this when a user tries to put >20 different things on a page or has a table with thousands of rows.

Lean towards aggregation

The DAX engine, Vertipaq, is really good at two things: filtering and aggregations. This means it’s ideal for high level reporting like KPIs and traditional dashboards. This also means it is not good at very detail-heavy and granular reporting. If you have a table with 10,000 rows and complex measures being calculated for each row, it’s going to be really slow. If you need to be able to show detailed information, take advantage of drill-though pages or report tooltips to pre-filter the data that is being shown.

Filter what is being shown

Unless you are using simple aggregations, it’s not advisable to show all of the data at once. One way to deal with this is to apply report or page filters to limit how many rows are being rendered at any one time. Another options is to use drill-through pages and report tool-tips to implicitly filter the data being shown.

Limit how many visualizations are on screen. The part of Power BI that renders the visualization is single-threaded and can be slow sometimes. Whenever possible, try to not have more than 20 elements on screen. Even simple lines and boxes can slow down rendering a little bit.

Everything else

Review your query method. This article assumes that you are using the import method to pull in your data. However, DirectQuery or live connections might be much faster depending on the size of your data.

Sometimes a report can seem slow when you are developing it. Make sure that there aren’t any applications on your PC that are consuming all of your resources and making Power BI Desktop seem slow.

Resources for Freelancers – Free and Paid

In my experience , working for yourself provides a unique set of challenges and required skills. You need to learn how to work on your business, not just in your business. You have to become your marketing, sales and HR departments. Without useful resources, I undoubtedly would have failed as a freelancer. Below is a list of resources I would recommend to anyone getting started as a freelancer.

Blog posts

As I’ve made the leap to working for myself, I’ve written a set of “lessons learned” posts.

Podcasts

Podcasts are my most valuable resource, because they provide a slow drip of insight and expertise. Listening to a podcast provides a regular insight on how other think about this stuff, without a large commitment.

  • Business of Authority. This is my #1 recommendation in podcasts. Rochelle and John are focused and understandable. This isn’t some 2 hour rambling podcast. They provide expertise from very different types of businesses. Every week I enjoy listening to this.
  • Ditching Hourly. John continually beats the drum of “hourly work is bad”. Even if you never switch to a flat-rate model, this podcast will help you think of yourself as a business, not as a technician.
  • Startups for the Rest of Us. While the topic is startups, the actual focus is small businesses. If you are a business, this podcast will have something for you.
  • The Freelancer’s Show. More of a casual chat format, this show regularly has lived experiences from developers working for themselves.
  • Finish your Damn Course. If you are a freelancer, you should consider offering products, not just services. Janelle interviews a wide variety of guest who are making training courses on wildly different subjects.
  • Creative Class. This podcast is generally only active when the Creative Class course is open for enrollment. I love the easy style Kaleigh and Paul have when talking about freelancing.
  • Building a Story Brand. Marketing isn’t always fun or intuitive work, but it is important. Donald Miller explains marketing in wonderful down to earth terms.

Books

Sometimes you need to read a book to get a core idea to stick. Many of these books are also available as audio books, so even if you are busy you can listen to them while exercising or cleaning.

    • Consulting
      • Getting Started in Consulting. If you travel, you probably have a checklist of everything to pack. This book is a big checklist of everything for starting your consulting business.
      • The Secrets of Consulting. Consulting is the art of telling people they are wrong without them having to admit it. This is a fun read on all the weirdness of being a consultant.

  • Running a business
    • The E-Myth. So many of us go freelance because we are great technicians. But we also have to be managers and entrepreneurs, and much as we might hate it. This book covers that through a great parable.Company of One. I absolutely love Paul’s approach to running your own company. This is a great book for thinking about the unique advantage of running a company of one.
    • Building a Story Brand. If you buy one book on marketing, buy this. It is the most intuitive, plain-English explanation of what marketing is that I have ever seen.
    • Built to Sell. While it’s almost impossible to sell a one person business, it’s important to think about it anyway. Much of your work can be automated or outsources, but you have to work hard to identify which parts.
  • Productivity
    • Getting Things Done. The best book I ever read on productivity. A series of simple guidelines for defining and organizing your work.
    • Atomic Habits. One of the hardest parts of working for myself was rebuilding structure and routine. This is a great book on the subject, very practical.
    • The Phoenix Project. While technically a book about IT management, there is insight for everyone. I found this book tremendous in helping me think of work as a flow and not just concrete tasks.
  • Communication

Courses

I’ve only taken one course for freelancing and the was the Creative Class course by Kaleigh and Paul Jarvis. If you are brand new to freelancing, I would consider this but definitely listen to their podcast before you buy. The content is easy to follow and enjoyable, but quite introductory. If you’ve been doing this a year or more, you likely won’t get as much value out of this course.

Software

There are tons of pieces of software out there for running a business, but here are some I have had experience with.

  • Toggl. I use toggle to track my time spent. Having a weekly timesheet makes it easy to track billable hours and review what my focus is.
  • Trello. If you need to collaborate or track todos, Trello is a great free tool.
  • Quickbooks. If you are running a business, then you need to track your books and send invoices.
  • Mailchimp. I send my weekly newsletter using Mailchimp. It’s simple to use and fairly cheap to start out.
  • Buffer. whenever I have a presentation or a course coming up, I like to schedule social media posts. For that, I use buffer.
  • Emergent Task Planner. Not actually software, but I find this note pad to be useful for planning and tracking my day.

Summary

When you work for yourself, you need to manage the entire pipeline Marketing –> Sales –> Delivery. You need to think of yourself as a business, and you need to have a plan for growing your business even if you always stay a “Company of One”. You need to rebuild structure and routine and find a way to focus on your work.

Consulting is a unique job. Freelancing is a unique job. With the right resources you can succeed at both.