All posts by Eugene Meidinger

Why DAX is a PITA: part 1

DAX Functions
  • So, I think that DAX is a pain in the butt to use and to learn. I talk about that in my intro to DAX presentation, but I think it boils down to the fact that you need a bunch of mental concepts to have a proper mental model, to simulate what DAX will do. This is very deceptive, because it looks like Excel formulas on steroids, but conceptually it’s very different.

Here is the problem with DAX, in a nutshell:

image

This example below is a perfect example of that sharp rise in learning curve, and dealing with foreign concepts like calculated columns, measures, applied filters, and evaluation contexts.

So, one of the things I’m hoping to catalog are example where DAX is a giant pain if you don’t know what you are doing. People make it look really simple and smooth, and that can be frustrating sometimes. Let’s see more failures!

How do I GROUPBY in DAX?

John Hohengarten asked me a question recently on the SQL Community Slack. He said:

I need to sum an amount column, grouped by a column
Measure 1 :=
GROUPBY (
det,
det[nbr],
    “Total AR Amt Paid calc”, SUM ( det[amt] )
)
I’m getting a syntax error

So automatically, something seemed off to me. Measures are designed to return a single value, given the filter context that’s applied to them. That means you almost always need some aggregate function at an outer level. But based on the name, you wouldn’t necessarily expect GROUPBY to return a single value. It would return values for each grouping instance.

If we take a look at the definition for GROUPBY(), we see it returns a table, which makes sense. But if you are new to DAX, this is really unintuitive because DAX works primarily in columns and tables. This is a really hard mental shift, coming from SQL or Excel.

 What do you really want?

None of this made any sense to me. Why would you try to put a GROUPBY in a measure? That’s like trying to return an entire table for a KPI on a dashboard. It just doesn’t make sense. So I asked John what he was trying to do.

He sent me an image of some data he was working with. On the far left is the document id and on the far right is the transaction amount.
Pasted image at 2017_07_06 09_28 AM

He wanted to add another column on the right, that summed up all of the amounts for transactions with the same document. In SQL, you’d probably do this using a Window function with a SUM aggregate, like here.

 Calculated columns versus measures

This highlights another piece of DAX that is unintuitive. You have two ways of adding business logic: calculated columns and measures. The both use DAX, both look similar and are added in slightly different spots.

But semantically and technically, they are very different beasts. Calculated columns are ways of extending the table with new columns. They are very similar to persisted, computed columns in SQL. And they don’t care at all about your filters or front-end, because the data is defined at time of creation or time of refresh. Everything in a calculated column is determined long before you are interacting with them.

Measures on the other hand, are very different. They are kind of like custom aggregate functions, like if you could define your own version of SUM. But to carry the analogy, it would be like if you had a version of SUM that could manipulate the filters you applied in your WHERE clause. It gets weird.

My point is, if you don’t grok the difference between calculated columns and measures, you will never be able to work your way around the problem. You will be forced to grope and stumble, like someone crawling in the dark.

Filter context versus row context

So in this case we’ve determined we actually want to extend the table with a column, not create a free-floating measure. Now we run headlong into our next conceptual problem: evaluation contexts.

In DAX there are two types of evaluation contexts: row contexts and filter contexts. I won’t go too deep here, but they define what a formulas can “see” at any given time, and in DAX there are many ways to manipulate these contexts. This is how a lot of the time intelligence stuff works in DAX.

In this case, because we are dealing with a calculated column, we have only a row context, not filter context. Essentially, the formula can only see stuff in the same row. Additionally, if we use an aggregate like SUM, it only cares about the filter context. But the filter context comes from user interaction. Because this data is defined way before that, there is no filter context.

This is another area, where if you don’t understand these concepts you are SOL. Again, for the newbie, DAX is a pain.

 What’s the solution?

So what is the ultimate solution to his problem? There are probably better ways to do it, but here is a simple solution I figured out.

SUM =
CALCULATE (
    SUM ( Source_data[Amount] ),
    ALL ( Source_data ),
Source_data[Document] = EARLIER ( Source_data[Document] )
)

Walking through it, The CALULATE is used to turn our row context, into a filter context. Then it manipulates that filter context so SUM “sees” only a certain set of rows.

The first manipulation is to run ALL against the table, to undo any filters applied to it. In this case, the only filter is our converted row context. (confused yet?)

The next manipulation is to use EARLIER (which is horribly named) to get the value from the earlier row context. In this case we are filtering ALL the rows, to all of them that have the same document. Then, finally we apply the SUM, which “sees” the newly filtered rows.

Here is what we get as a result:

image

 How do we verify that?

A fourth pain with DAX is that it’s very hard to look at intermediate stages of a process, like you can with SQL or Excel formulas, but in this case we have a way. If we convert our SUM to a CONCATENATEX, we can output all the inputs as a comma separated list. This gives us a slightly better idea of what’s going on.

image (1)

 What’s the point?

My point is, that DAX, despite it’s conciseness and richness is hard to start using. Even basic tasks can require complex concepts, and that was a big frustration point for me. You can’t just google GROUPBY and understand what’s going on.

Again, check out my presentation I did for the PASS BI virtual group. I tried to cover all the annoying parts that people new to DAX will run into. That and buy a book! you’ll need it.

Wrangling GotoWebinar Stats with Power Query: Part one

Project Logs

So, this week I gave my first presentation to image

Ugh.

Power Query to the rescue

Normally this would be a giant pain to work with. When it comes to data quality, this is quite the image

Excel is going to make some assumptions about what is part of the table. This is convenient for our needs, but we’ll have to find a work around when we want to scale to multiple excel files.

image

We can’t tell it we have headers, because it’s going to think that first row is a header. We’ll deal with that later. Once we click OK, we are taken to the Power Query / Power Pivot window.

image

I made a mistake

Hmm, so it looks like I made a mistake. I hope my honesty won’t lose me any image

Trying again

Let’s take a different approach. I’m going to open a blank excel workbook and pull the data into there. Okay, so let’s go to manage under the Power Pivot tab.

image

Next, we are going to click “Get External Data From Other Sources”

image

Then I’m going to scroll to the bottom and select Excel File.

image

Once selected, I only have the whole first sheet as an option. If I had table objects or named ranges, that would be different.

image

Hmmm, I still can’t find a way to edit the Power Query. Fiddlesticks!

Normally, in Power BI it would be right here:

image

Trying to do this in Excel is quite the image

Okay, let’s try opening that Excel file. Ah, much better. Now I want to click Edit at the bottom right.

image

Cleaning the Data

So, First thing we need to do is get rid of all of the non-header rows at the top.

image

To do that, I just select Remove Rows –> Remove Top Rows.

image

Then I specify I want to get rid of the top 7 rows.

image

Next, I want to turn the actual header row into a header.

image

Okay, so now it looks like a real table.

image

Comma Delimited BS

Okay, so now we need to parse out the times someone was watching. The problem is that some people were in and out. Their entries are comma delimited. Ugh.

image

Okay, let’s split them up. I’m going to select Split Column –> By Delimiter

image

Unfortunately, splitting by column a) splits into more columns and b) you have to specify how many.

image

Thankfully, we can select those new columns and unpivot them.

image

Perfect. Now we have a row for every time a person as watching.

image

String parsing

Okay, so now we just need to parse out the dates. First, we are going to split on the dash, and then the parenthesis.

image

This is starting to look good.

image

Now we just need to get rid of the timezone and convert it to a datetime. First we need to select Replace Values.

image

image

Lastly, we select the data type we want.

image

What’s next?

Now that are data is cleaned up, we’ll join to sessions table and do some simple data modeling. But that’s for the next blog post.

Keeping up with Technology: a Guide to Drinking from the Firehose

Miscellaneous Tech Explorations

I often ask people with more experience than me “How do you stay relevant in our field?”. I joke that I live in perennial fear of being replaced someday, by a 22 year-old with no family commitments. Really, it’s a joke. Really, it’s a fear.

So, let’s talk about how to keep up with technology.

 First, we must grieve.

So here’s the secret to keeping up with technology. You can’t.

There are too many technologies, too many features, too many updates:

  • SQL Server 2017 is coming out this year.
  • Big data technologies are so numerous as to be indistinguishable from Pokemon.
  • PowerBI ships features on a weekly basis.
  • Worst of all, you’ve got NoSQL which is literally the mathematical dual of SQL. It’s everything that SQL isn’t, by definition!

It’s all too much.

This realization is likely to be expressed in the 5 stages of grief.

  • Denial. Other people can keep up, why can’t I? I’m learning tons of stuff all the time! This is easy.
  • Anger. Why they heck are they making releases every month!? When did Microsoft go agile? I thought SQL Server versions every 2 years was bad enough.
  • Bargaining.  Okay, maybe if I stick to core SQL stuff, I’ll be fine. Hadoop seems like a fad. And Azure is never going to be popular with my  company.
  • Depression. This is impossible. I’m going to lose my job when I’m 40 years old and arthritis starts kicking in.
  • Acceptance. There’s more happening than I can ever learn, but Hacker News doesn’t define my success as an IT professional.

This post isn’t about doing the impossible. It’s about making the most of your resources. That’s something that you can do.

Are you asking the right questions?

I would like to propose that it’s not about keep up with technology at all. This is a second-order goal, a proxy of sorts. Really, there are two questions at the heart of things:

  1. How do I keep my job?
  2. How do I keep my friends?

This is why technology causes so much angst. We want to learn enough that we can put food on the table; and we want to do it in short enough time that it doesn’t destroy our personal lives.

Next, we must think.

In order to get ahold of the the problem, let’s use some analogies: investments and radioactive decay.

How learning is like investing

Do you have a retirement account? If so, do you invest primarily in stocks or bonds? Why?

If you are at all young, the you invest primarily in stocks. That’s because stocks have a higher rate of growth than bond. You are trying to outrun inflation, where the value of your money is steadily decreasing. This is like your current knowledge becoming outdated. All that vb6 coding knowledge is like a pile of cash in your mattress. When I was a kid, $20 was a lot of money.

The next question  is, do you invest in just one stock, like Apple? No, you diversify your portfolio. High growth stocks go up, on average. Some however, tank. High growth means high volatility. A good example is Apache Flex. It used to be a really promising application platform, until Steve Jobs killed Flash.

So we’ve got two risks to our learning portfolio: Losing value in our existing knowledge, and making the wrong choices for our new knowledge. These different risks have different mitigation strategies.

Specialization and generalization

These comparisons to bonds and stocks relates to the challenges of specialization versus generalization. We specializes to pay the bills. We generalize to keep our jobs.

Specialization is how we get paid. The reason anyone pays us is because we have skills or knowledge that is not quickly acquired. The reason consultants like David Klee make gobs of money is that they have taken a subject, like virtualizing SQL, and have gotten really  good at it. To get paid more than minimum wage you are going to need some level of specialization.

Generalization is how we get paid in 10 years. You need to be specialized to get paid right now, it’s a short term investment. However, that investment decays, just like the value of money in your mattress. To get paid a decade from now, you need to broaden your horizons. If you are a data person, it might mean learning R and python. It might mean learning Azure. Heck, it might mean PowerShell and Docker.

The tension here is that you need both. You need paid now AND in 10 years. Kevin Feasel talks about trying to find that right balance. I’m not here to tell you what that balance is. What’s important is that each has different constraints. Specialization requires focus. Generalization requires time. More on that later.

How learning is like radioactive decay

Let’s take another approach.

Our knowledge has a limited shelf-life. Allen White said, in the SQL Data Partners podcast, that you have to retool yourself every 5 years. In college, I remember joking that half of what you knew would be useless in 5 years.

Well, what if we took that literally? How would we model that? How would we think about that?

In nuclear physics, there is the idea of a half-life.  You have radioactive material that decays in half every X units of time. Why not apply this concept to IT? The half-life in this example then would be 5 years.

So the next question is this: if half of what you learn is either irrelevant or forgotten in 5 years, how much do you need to learn in a given year, to keep steady? Let’s ask our friends at Wolphram Alpha.

image

If x is our rate of decay and our half-life is 5 years, we can work backwards from that and solve for x. In this case, x equals 87%.

So, what that means is that if today you know 100 relevant things, then a year from now you’ll only know 87 relevant things. That’s like going from a solid A+ to a weak B+. You just lost a whole grade!  Not good.

Below is a curve showing what this model looks like over 10 years.

image

Changing the math

Well, this isn’t great. What if I want to know 120 things? How can we do that? One option is to learn more things.

Learn more things

If normally we have to learn 13 things each year, then we have to learn another 20 things on top of that. Or, if you are patient, you can learn an extra 4 things each year, and eventually  you will get there.

image

Slow the decay

Another option is to change the rate of decay. Instead of learning more things each year, what if we didn’t have as much decay? If we can slow that rate of decay just a little bit, to a half-life a 6 years instead of 5 years, we’ll have the same effect. That means that instead of brute-forcing things, we might be able to be smarter.

What does this all mean?

So that gives a path forward. We need to either

  1. Increase the number of relevant things you can learn each year
  2. Or, decrease the rate of decay for relevant knowledge

These are the only three knobs we have. Either learn more stuff, learn the right stuff, or learn stuff that lasts longer. Let’s investigate all three.

How do we fix this?

Learning more things

So one solution to our dilemma is to brute force it. Let’s just learn more things and hope that they are the right ones. To do that, we can look at our inputs and constraints.

What are the constraints that affect our learning? There are 3 big ones I can think of:

  1. Time
  2. Energy
  3. Money

If we can increase any one of these, then we might be able to increase how much we can learn in a week.

Time

So, lets say we decide to go the simple route and go for volume.  You’ve got a 168 hours in a week. You can’t make any more hours, and if you try to use all of them in a week, you are going to need some amphetamines.

So, if we can’t create more hours, how can we make more time? One option would be to cut out other activities. If you are willing to quit watching TV or playing video games, that frees up more time for learning. You could use a service like toggl.com to do a time audit and see where all of your time goes.

There is a limit to going down that path, however. You need to sleep. You need to have a social life. You need to have some fun. Like we said, you want to keep your job and keep your friends.

Multitasking

Another option is to multi-task. There is a lot of learning you can do that while doing other things. Things like podcasts are more about exposure than mastery. You can still get value out of them, even while mildly distracted. There are a number of times you could listen to a podcast:

  • Commuting
  • Exercising
  • Washing dishes

This is a way to take back time you are spending on other things without giving up all your free time.

5 minute learning

Another way of reusing your time is taking advantage of those weird breaks in time. Those breaks that are too small to get anything meaningful done. The 15 minutes at the doctor’s office. The 5 minutes waiting for a meeting.

Feed readers are a great way to take advantage of these weird units of time. With Feedly, I’m able to to read a blog article during the 5 minutes I’m waiting. This is way more useful than playing candy crush or reading twitter.

Focus/energy

Okay, so let’s say you’ve managed to find more time in the day. Unfortunately, not all learning takes the same amount of energy. Reading Twitter is mindless. Configuring a homelab requires focus. Listening to a podcast is mindless. Making a presentation takes focus.

One way of getting more focus is to prioritize the harder learning for when you naturally have focus. For some people this is early morning. For many people this is the weekend. I know that after a long day of work, I’m wiped out.

Part of that is learning to take care of yourself. Eat healthy. Exercise. Get sleep. Focus is so easy to destroy by poor lifestyle.

Finally, if you schedule time consistently and push everything out, you can have more focus. I also find having a separate office/learning space helps with this too. Cut out the distractions. Install StayFocusd for Chrome to block timewasters.

Money

If you are anything like me, money is your most plentiful resource of all 3. When I was kid, I had no money at all, but tons of time. When I was in college, I had pretty much no money, and a good bit less time. Now that I have been working for a while, that equation has flipped. I’ve got plenty of money, but no time to use it.

If you are an average DBA, you make plenty of money. According to the Bureau of Labor Statistics, the median salary for DBA’s is $85,000. The median for all occupations is $37,000. Think about that, that’s like 2.5x as much.

Now you may say “Hey, I don’t live in San Francisco.” or “Hey, I just got started.” Let’s take those factors into account. The market here in Pittsburgh is terrible, salary-wise. Even if you are just getting started in the field, you are probably making at least  40K.

image

Let’s do a little math on this. If you make 40K per year, you make roughly  $20 per hour. that means your time is worth $20 per hour. If you can spend $5-10 to save an hour of your time, do it.

If a $50 book saves your 5 hours of Googling, buy it. If a Pluralsight subscription saves you 30 hours per year of frustration, buy the subscription. Don’t be afraid to spend money to save yourself time and energy.

Consider budgeting your money

If you don’t feel like you have a lot of money, I would deeply urge you to start budgeting. I used to look at my checking account to see if I had money. Then if there was money there, I would spend it. This would work until my car insurance bill would come in, and suddenly I needed $600. And just as suddenly, I didn’t have nearly as much money as I thought

Get a budgeting program. There’s some free one’s out there, but I use You Need a Budget.

Learning the right things

So we covered how to learn more things. An alternative is to learn the right things. If you learn the right thing, you’ve have less wasted effort.

Lean on curation

Sturgeon’s law says that 90% of anything is crap. That’s certainly true of learning materials. Not only is there a lot of bad training out there, but there’s a lot of content in general that just isn’t relevant. It’s cool that someone wrote an OS in Rust, but it’s probably not relevant to your job in any significant way.

If you are time poor or focus poor, you need a gatekeeper so the flood of possibilities doesn’t overwhelm you. This means depending on curation.

That curator could be you, first off. One of the reason I suggest a feed reader over Twitter is that you are able to heavily curate the content that hits your eyes. You are the one picking the blogs to read instead of the pachinko ball machine of social media and fate. You are the one deciding what’s relevant to you.

Another options is to depend on professionals in the field that you trust. Find people to follow that consistently have good material or recommendations.

The third option is to pay money. Sure, there’s some paid crap out there. But general it does act as a quality filter. Things like written books and video libraries are far less likely to be total crap, because those things have editors and investors and such. Someone was being paid to make sure it was worth making. Additionally, the author worked hard to condense the knowledge into a concise format. Be willing to pay money.

Having a plan

Okay, so now you are investing your time and energy into quality materials, but are you learning the right things? Are you learning the things that are right for you? You need to have a career plan. You need 1 and 3 year goals. Otherwise your career plan will be a spread out mess.

Figure out where you want to focus, figure out where you want to go. Pick a specialization. More of what you learn will be relevant if you know what that is.

Learning things that last longer

Okay, so we are learning more things and more of the right things. But we still have the problem that our knowledge is decaying whether from atrophy or irrelevance. This is the last step in the funnel and the last thing we can optimize.

Improving retention

One way to keep ahold of knowledge for longer is to lean on active learning. If something goes in through your eyes and ears and back out your fingers and mouth, you will retain it for longer.

If you want to specialize and go deeper, you need to do things like blogging, presenting,coding, etc. Reading isn’t enough. Writing isn’t enough. if you truly want to learn, you have to do.

Reddit and hacker news are good for exposure to a topic, but exposure fades quickly. Use exposure to get a lay of the land, then go deep once you find the right things to learn. If you focus on mastery, you are going to learn things in a way that last longer.

Avoiding planned obsolescence

Another issue is that certain areas of knowledge just have a faster rate of decay. Some of these are facts tied to a specific version of a technology. Another thing are unproven technologies that might turn out to just be fads. Big data is still in this phase, where there are so many technologies that will be gone forever in 5 years.

People change far more slowly than technologies, so skills relating to them last far longer. Soft skills never go out of style. Communications skills never go out of style. If you learn how to write an abstract today, that knowledge will still be relevant 40 years from now.

The fundamentals last longer too. The core basics of computer science are far more establish than this weeks JS framework. If you have solid underpinnings, you can take your knowledge of C++ to Go, for example. If you understand how a ACID and CAP theorem, The new consistency models of Cosmos DB will make sense. Under understanding of one layer beneath will allow you to jump to new technologies far quicker.

Putting it all together

To summarize what we talked about:

  • Take advantage of multitasking to get back more time. Listen to podcasts while you drive or exercise.
    • Invest in a pod catcher. I use itunes and an iPod nano.
    • Invest in a feed reader. I use Feedly.
  • Take care of yourself, physically. Diet, sleep and exercise will all improve focus.
  • Schedule time to maintain focus. Treat focus as your most precious resource. Don’t browse twitter when you are firing on all cylinders.
  • Be willing to spend money on curation. There is a good chance that you have more money than time or focus. Be willing to spend money to avoid wasting time.
  • Understand exposure versus mastery. Spend your time on exposure and your focus on mastery.
  • Pick a specialization. Have a plan on what technologies you need to go deep on. You won’t get there by accident.
  • Learn things that last. Learn the fundamentals. Learn internals. Learn people skills. Do home labs.

How to write a good abstract for GroupBy.org

Public Speaking & Community Involvement

Recently, I’ve been going through a lot of the presentations for GroupBy.org. I’m trying to provide as much feedback as I can, because I think good feedback is hard work. A lot of the existing comments are along the lines of “This looks cool!”, which does not provide much direction. As a presenter myself, I’m a big fan of receiving actionable criticism. It’s the only way I will grow as a presenter.

This post is going to cover some general guidance for making a fantastic abstract. It’s targeted at the GroupBy.org site, but much of the advice is broadly applicable

You should write like you fight

When you edit your abstract, you should be relentless, you should be merciless. Every sentence should dance. Every sentence should sing. Every word owes you rent, and you are here to collect. You, my friend, have neither time nor patience for any freeloaders. If anything does not enhance your message, ditch it. This ain’t a charity, kids.

You should write like your life is on the line.

You should write like you fight. This is not a joke. This is not hyperbole. Because someday, your life, your ability to provide for your family, will depend on your ability to communicate clearly. Someday one of these things will happen to you:

  1. Your company will get bought, and you will have to explain “what do you do around here?”
  2. You will be out of a job and need to write an amazing cover letter
  3. You will need to summarize what you’ve done this year and why you deserve that raise
  4. Something will go horribly wrong at work, and it will be your job to write the retrospective
  5. Your coworker will cross a line, and you will need to stand up for yourself in a polite, professional way.

Are you ready for that day? If you are writing half-hearted abstracts today, you are probably half-hearted emails and cover letters. If you aren’t writing like your life it on the line today, then you aren’t preparing for when it actually is. We practice when the stakes are low, so we are ready for the day that they aren’t.

What martial arts teaches us about good writing

I do martial arts every week. Not because I’m a particularly aggressive or athletic person. I do it because it helped me lose weight and because it keeps me healthy. I lost 70lbs in large part to martial arts, so I think I owe it some respect and deference. In a very real sense, it has changed my life forever.

In my school, everything we do is preparing us for a fight that hopefully never happens. The goal isn’t to get into these scenarios, but to be prepared should the worst ever happens. Learning how to give a solid punch doesn’t make me cocky and reckless, it makes me humble and cautious. This is because I know how quickly a fight can fall apart; I know how much a punch to the face hurts.

In martial arts, specifically, we practice moves hundreds and thousands of times. We refine and we focus until it’s reflex. Because trust me, when you are scared and under pressure, all of your form goes to crap. When I participated in my only tournament, I got hit really hard on the chin. Hard enough that my ears were ringing and they had to check if I was okay. It was because I was scared. It was because under pressure, I forgot all of my form. It was because good form wasn’t reflexive for me.

Good form should be reflexive

We practice things over and over, when the stakes are low, so we don’t have to think when the stakes are deadly. I can’t say this enough times. This is how lives are saved.

When you write an abstract, you are practicing for when it really matters. In martial arts, if you don’t practice keeping your fists up when there is no danger, you’ll get sucker punched when there is. Trust me, I know from experience.

Good form comes from intentional, relentless practice. Edit, edit, edit. Please, for your own sake.

How to write a great abstract

So what makes a good abstract? What makes good form? I think there are a number of fundamental things that people regularly miss.

Dear presenter, why do I care?

Your first sentence should tell me why I care. Why do I want to attend your presentation? Don’t assume that just because you think it’s important that I agree with you. You have to persuade me. You have to explicitly communicate how it benefits me.

Your whole abstract should hang on this premise. If a sentence does not in some way help answer this question, cut it. If it isn’t abundantly clear, rework it.

So how do I communicate this? There are a number of ways:

  • Give them a headache. Tell them what problems they have.
  • Give them a solution. Tell them how this talk will solve those problems.
  • Tell them how they will grow. People want an immediate payoff. Explain how they will be better for watching your presentation.
  • Don’t assume it’s important. Show me why your topic is important.

If you can answer why people should care, you will be a step ahead.

Figure out your audience. Narrow it.

Who is your audience? Who cares about your topic?

Did you figure it out? Great, now narrow it. Audience statements are often too broad to begin with. Ask yourself, “Who would be really excited to watch your presentation?”. Make them your target audience. Don’t feel that you have to cater to everyone.

Curiosity is almost a terrible audience goal. Find people who have a need and fill it.

Additionally, who isn’t your audience? There should be people who you don’t want to attend your presentation. This concept is often more helpful than knowing your target audience. Don’t be afraid to exclude people. A broader target leads to a muddled message.

In the agile world, there is idea of personas. Use them. Let’s say that Susan is a fictional person who really wants to watch your presentation.

  1. Who is she?
  2. Why is she super pumped about your presentation?
  3. How do you communicate this to her, efficiently?
  4. What things does she already know? What new things is she going to learn?
  5. Who isn’t she?

If you can paint a vivid picture of this person, your abstract will be better for it. Even better, your presentation will improve too.

Get your level right

Relating to the item above, figure out what level your presentation is. Is it for total newbs? Then make sure you have a lot of introductory content. Be very clear about what you are assuming they already know. Write it out on a piece of paper. Don’t assume. Don’t assume.

Is this more of a 300-level practical presentation? Well then “curiosity” had better be nowhere in your target audience. Make sure you elaborate the detailed content that you will cover. Make it clear that they will take something practical away from this.

I’ve seen a number of abstracts that try to split the difference and just muddy the waters. Pick an audience and stick with it. Decisions aren’t decisions until you give something up. You have to make a choice. Who are you targeting? Be clear about this and your viewers will thank you for it.

Put the bottom line up front

Get to the point right away. Explain your general thesis in the first couple of sentences. Explain why the reader cares in the first few sentences. You can include all the detail later on. This is a matter of being respectful to your reader, who is a busy person. Don’t waste their time.

In journalism, this is called inverted pyramid style. The professionals use this method. You should too.

To summarize, If you can’t sell me in a tweet, you’ve lost me as a reader. Keep it tight. You can add the details later.

Make your prose scannable

People don’t read the internet like a book. People scan. They’ve done eyetracking studies where they literally watch people’s eye movement. Keep your prose tight and short. Use technical writing techniques.

  1. Keep your sentences short. Break up run-on sentences. Avoid sentences more than 15 words, like the plague.
  2. Use bullet lists, where possible. These are GREAT for scanning text.
  3. Use multiple paragraphs. You have the space, use multiple paragraphs for multiple purposes.
  4. Have a structure. I personally like the 3 paragraph structure of
    1. “Why do I care?”
    2. “What will we cover?”
    3. “What will I take away from this?”
  5. Use action verbs. Avoid is, was, became, etc.
  6. Use a little formatting. Unlike most events, you have full control of your formatting. Use it.

If you can’t see the flow of your text from 10 feet away, reconsider how your have structured your text. Blobs don’t scan well.

I got sucker-punched via email this week, but I was ready

So at the beginning of this week, I got thrown under the bus. In large part, it was my own fault. I had, in fact, missed deadlines that delayed a colleague’s work. That part was true.

The part that was the sucker punch was that he contradicted with an earlier conversation. When the author and I spoke, it sounded like his deliverables were being blocked by time constraints as well. I was late, but I offered to rush Friday morning so he could get his weekend part done on time. He indicated that there were other factors holding things up, that there was no point in rushing.

So now I’m in a situation where I thought everything was fine, but instead my boss is getting a surprise email, Monday morning. An email indicating that the project is being delayed a week, solely because of me. One hour before the manager’s meeting. Ugh.

So my boss sends me a one sentence email. “Did we know about this?”

The advantage of being ready

So now I have half an hour to lay out a timeline of events, and give my side of things. And because I practice my writing, I was able to write this:

Even with a heavy blur and shrink you can see the structure. It’s got the bottom line up front. Everything you really need to know is in the first 4 sentences.

Again, I want to be clear. It was my fault for missing deadlines. It was my fault for not communicating that to my boss. I’m not some victim here.

But I did receive a surprise, and I was ready for it. Because I practice my form daily. And you should too. Write good abstracts. Write good emails. Practice, practice, practice.

Keep your fists up folks, it’ll guard your chin. Otherwise it’ll hurt like hell, and your ears will be ringing. I know from personal experience.

 Summary

Write your abstracts like your life depends on communicating clearly and efficiently. Determine a targeted audience and tell them why they should care. Keep it tight, keep it scannable. Edit, edit, edit. Practice, practice, practice.

Good luck!

Things I like about the changes to PASS Summit submission process

Public Speaking & Community Involvement

So, PASS Summit is making changes to how the curriculum is managed. There’s been some conversation about these changes and whether they are good are not. I thought I would enter the conversation and add my thoughts.

Before that though, I want to thank the board for the work that they do and working on these changes. I think that being on the board is difficult work. It seems to me that no matter what changes they make, there will be criticism. Often, the criticism is the loudest of all the commentary. Sure, I agree there is more they could do to involve community feedback; however, the fact that they have a survey and community townhall is evidence that they are listening and value our opinion.

So, I’m going to try to focus on the positives I see, instead of criticizing. Overall, I think these changes are a good move. I wrote last PASS that I think the data platform is broadening. And while I think that the idea of a data profession presents challenges, I think it’s reflective of trends going on:

  1. The devops-ification of the data platform
  2. A broadening of the surface area of  “data platform”
  3. Virtualization, Moore’s law and Cloud

These are all forces that are causing more technologies to fall under a broader and broader grouping. They are also causing a blurring what used to be more distinct roles. And as a result, PASS Summit is responding to that change.

Having a broader, more nuanced set of topics requires more control. If  you look at companies like Pluralsight or Lynda, they have people whose job is to manage a curriculum. And part of that job is aggressively targeting specific topics to avoid gaps.  It also means doing market analysis to see what education people want and need, not just what people want to present on. These goals makes sense for Summit too.

Part of targeting specific topics, is acknowledging that there are top-level experts in those content areas. If you want something on DAX, reach out to Marco Russo. If you want something on U-SQL, reach out to Michael Rys. Dealing with the changes in the platform require a different approach. In some ways, it would be a shame to to not try to get access to the top leaders of specific topic areas.

I admit that it’s reasonable to worry about Summit becoming some sort of elite club of presenters. Part of the solution is transparency, transparency, transparency. The other part of the solution is answering the question “How do I present at Summit?”

There’s a joke about industries with no entry level jobs. How do those types of industries exist? How do they not just die out? But I don’t think entry level is the issue here. There are tons of opportunities to speak at user groups. SQL Saturdays are a clear stepping stone after that. The next step is…less clear. Do you blog? Do you speak at virtual user groups? Do you go to 20 SQL Saturdays a year?

No seriously, I don’t know. Can someone tell me? You wanna know why I started presenting at virtual user groups? Because I asked someone how to get to Summit, and they suggested that as a stepping stone. I think it’s important to give “new” speakers like me a path for reaching the top.

It’s straightforward to get your white belt in speaking, even your green belt. But it’s a lot less clear how to get your brown or black belt. That’s an area I’d love to see more people talk about. A good example is Brent Ozar’s Career Internals.

That being said, I’m happy with the minimum bar of experience for applying. The current minimum is 3 presentations. I honestly think it could be 5 and it’d be fine. This is a good balance between requiring a base level, while not discouraging new speakers. Three is so low, you could trip over it. That’s two user groups and a SQL Saturday. I can promise you that when I had given only 3 presentations, I was in no way qualified to present to PASS.

One final thing I’m really happy about is that they are going to give guidance on the content they want, and they are going to ask people to specify a general content area they are focusing. I think this is a smart move. The current system just wasn’t sustainable.

The old system seemed to encourage just throwing as much stuff against the wall as you could and see if anything stuck. I don’t think we want to incentivize people submitting more than 3-4 sessions. It puts an undue burden on the people reviewing the submissions. I honestly think it’d be fine to have an explicit limit on submissions, but I expect that would make some waves.

Overall, I think they are making moves in the right direction and I like it.

Slides for PASS BI Virtual Group

Public Speaking & Community Involvement, Query Optimization

Today I’m going to be presenting on DAX for the PASS BI Virtual Group. The focus is on all the hard mental concepts of DAX. If I could sum up the talk in one picture, it would be this:

That red area is where I banged my head when learning DAX. The learning curve shoots up wildly in the middle of learning the technology, instead of a slow gentle curve. This presentation covers the middle parts that are key to understanding DAX.

Here are the slides for the presentation: Introduction-to-DAX-2017-03-30

Here is the recording:


Here is the talk by Marco Russo I mentioned in my presentation: Optimizing Multi-Billion Row Tables in Tabular

The challenge with being a “data professional”

SQL Server

During PASS Summit, I wrote a post about the broadening data platform. I talked about the term Data Professional, and how I feel how it describes the changes going on the in SQL space. Here’s the problem: It’s a terrible guide. It’s a great description, it’s a wonderful attitude, it’s an ambitious goal; but it’s a terrible guide.

Being a data professional means being a jack of all trades. It means being a renaissance man (or woman). It’s means a career plan that looks like this:

And then you end up with Buck Woody telling you you are trying to do too much, cut it out kid.


So that’s the problem. Sometimes broadening your horizons is really a mask for being scared of commitment. Sometimes it’s a mask for being scared of an ever-changing future. You have to bet on a horse, you can’t bet on them all. Being a data “professional” is great in theory, but in practice it turns into majoring in the “universe” (see XKCD).

Major in the Universe

I’m not saying don’t learn Docker or Powershell. If you don’t learn those things, Kevin Feasel will warn you about becoming homeless. And who wants that.

What I am saying is that if someone asks you “Where do you want to be in 3 years?”, “everywhere” is not an answer. If someone asks you “What are you going to learn this week?”, “everything” is not an answer. So yes, generalize your skill set, who knows what you’ll be doing in 5 years. But right now you need a focus, it’s the only way to become an expert at anything.

Ultimately, I think it comes down to two quotes:

If you don’t know where you are going, any road will get you there.

-George Harrison, paraphrasing Lewis Carrol

and

Two roads diverged in a yellow wood,
And sorry I could not travel both
And be one traveler,

[…..]

I took the one less traveled by,
And that has made all the difference.
-Robert Frost, The Road Not Taken

For me, I’m looking into Data Science. The problem is I’m not sure what Data science actually is! What I know for sure is it involves R and pirate jokes. We’ll cover that in next week’s blog post.

PASS Summit–Quick thoughts

Public Speaking & Community Involvement

Last week I was at PASS Summit and it was an incredible time. I went in feeling deflated about my career and came out excited and energized. It was wonderful seeing the potential of the Microsoft Data Platform and where things are going. It was also great to meet a lot of people and hang out with my SQL Saturday friends.

Overall, there was one theme I saw over and over again. You could call it technological diversity. You could call it fragmentation. You could call it accelerating growth. What it really comes down to is more technologies, more platforms and more things to learn.

You could see it in the keynotes multiple times. SQL Server is now on Linux. Polybase connects to Mongo. NoSQL support is available via DocumentDB. PASS is rebranding and focusing on the Microsoft Data Platform, not just SQL Server. Clearly the surface area of a data professional is expanding.

I like that term a lot, Data Professional. At first, it sounds meaningless and generic. But it’s one of the few things that aptly describes what I do. Because being able to be just a DBA or a database developer is going to get rarer. Now you need to know a bit about Azure. Now you need to know a bit about Excel and PowerPivot. Now you need know the difference between Pokemon and Big Data.

All of this has inspired me to work on my first professional development session; Drinking From the Firehose: a Guide to Keeping Up With Technology. I’ve submitted it to SQL Saturday Cleveland. I also might get to present it for the Professional Development VC. I’m hoping if I can put some of these concepts into words, I can get a better handle on them myself.