Category Archives: General Technical Topics

TSQL Tuesday: Who is on my server?

tsqltuesday

Here is my entry for this month’s T-SQL Tuesday.

I once had to some auditing for a customer and it was a complicated, multi-stage process. We had to be able to demonstrate who had admin access and what kind of activity was going on, on the server. But before we could do any of that, we first had to identify who was actually logging on.

Triggers to the rescue

So what are the different options for telling who is logging on to a a SQL server? 5 options come to mind:

  1. Configure login auditing.
  2. Login Trace
  3. Login Extended Event Session
  4. SQL Audit
  5. Server Trigger

So going through each one of them:

Configuring login auditing really isn’t a good solution. What you are doing is changing the base settings to log successful logins in addition to failed logins. The problem is that these events are written to the SQL Server event log, which isn’t convenient to parse.

Well what about using a trace? Well I’ve always been told that traces are expensive in terms of performance so I shied away from using one of those. In retrospect, I doubt it would have been too expensive since it’s only tracing logins. If anyone knows, let me know!

The next option is to use Extended Events, which often have better performance. Unfortunately, this server was SQL Server 2008 R2 and there was no GUI support for extended events. So that wasn’t ideal.

What about SQL Audit? Underneath the hood, SQL Audit is just Extended Events. That being said, there is at least some GUI component to it. For 2008R2, it required Extended Edition. while that wasn’t an issue for us, it seemed like overkill.

So what’s the last option? Creating a server level trigger. This was simple to implement and easy to dump the data into a SQL table for reporting purposes.

Proceed with caution

So, what’s the downside. Wellllllll. What happens if you have an error in your code? If you hit an error, then you can’t login. At all. Anyone.

There are ways to resolve this issue, but it requires shutting down the SQL Server and taking an outage to fix it. Suffice it to say, I spent a looooot of time testing before I pushed this out to production.

Overall, triggers provided a simple solution to a simple problem. But the solution required a good dose of caution.

TSQL Tuesday #100: Industry changes and Meidinger’s Law

MJTuesday

Meidinger’s law

To celebrate the 100th TSQL Tuesday, this month’s image

Maybe a bit image

And what about image

We are all imposters

If you are going to change jobs every two years, then that’s how long you have to learn before it’s on to the next thing.

I’ve had my freakout about the data platform constantly broadening. Mindy Curnutt has a great podcast episode about imposter syndrome. We are all worried about the pace of change.

And I think Meidinger’s law is our saving grace in some sort of way. Things keep changing, we have to keep learning. But how much we actually have to learn is constrained by that 22 year old drinking red-bull with no family obligations.

Throwing Darts

Oh yeah, we were supposed to be predicting the future. Well I think the fact that we are all going to be replaced by that 22 year old some day gives us a hint at the future.

I think more and more things are going to be abstracted away. We’ve seen it with virtualization. We’ve seen it with the cloud. These abstractions mean the new guy can learn new, more important things. He doesn’t have to be intimately familiar with RAID 5. He doesn’t have to have the OSI model memorized.

I think we are seeing it now with data science and machine learning. So much of those areas require a Phd and years and years of study. But things like Azure machine learning and Azure cognitive services are going to get easier and easier. So easy that even the new guy can do it.

T-SQL Tuesday #99: I’m secretly a LARPer

For the 99th T-SQL Tuesday, I’m going to talk about something I do that’s completely unrelated to work. Something I’ve never told any of you about.

MJTuesday

What I don’t want you to know

There are 3 things in my life that could cost me a job, things that I fear an employer ever finding out about. I suffer from 3 big problems in life: depression, diabetes, and LARP. That’s right, I’m… a LARPer.

13305171_10156981353440430_7657738536820278554_o

I’m not kidding about the fear.  People get weirded out easily, and it’s easier to say no to an applicant than to say yes. I really worry about someone finding this post and turning me down.

I once heard a story about a potential staffed employee. Everything was looking good until the client looked up the employee’s Facebook and found out that they were a furry. After that point, the client was no longer interested in staffing that employee.

It’s unfortunate that harmless personal hobbies present such a risk, but here we are.

How it started

When I first heard about it, I was skeptical. In my head, all I knew was the stereotype of the uber-nerd shouting MAGIC MISSILE!
missile GIF

My then fiancée and a couple friends of ours had started going, and she wanted me to come along. I told her quite clearly, “Listen, LARP is so dorky that if I go with YOU, you’ll end up breaking up with ME. No way.”

Well it turned out that my excuse had a limited shelf-life. A couple days after we are married, my new wife reminds me that I’m very much against us getting divorced. So now I have no excuse not to go, because we aren’t breaking up any time soon. We go to LARP a week after we are married.

What is it?

I could tell you it’s Dungeons and Dragons in the woods, but that would be doing it a disservice. Instead, let me tell you a couple stories from Memorial Day weekend, 2016. It was 8 months later, and the start of our honeymoon.

That time I tried to commandeer a flying ship

So my character is a diplomat / minstrel / healer. Which means, if possible, I’d prefer to solve things with words instead of violence. And one of my abilities is to use my charisma to persuade people, convince them we are friends, etc.

So, me and my group are informed by a Non-Player Character that some bandits have taken over his grounded airship, and he needs us to get it back.

13329549_10156981353945430_2087717705991759007_o

We approach the ship, and as we get on the ramp the bandits warn us to stay back. I tell my group, “Let me handle this.”

So I walk up the ramp, alone and in mild danger.
I tell the bandits, “Hey, I’m here to help. That guy you stole this ship from, he’s trying to find adventurers to kill you.”
“We didn’t steal it!”, they reply.
“Fine, the former owner is looking for you.”, I say as I keep walking up the ramp.
”Stay back!” they shout. Then I throw two spell packets at them. Spell packets are basically birdseed wrapped in cloth. This allows for ranged attacks without hurting us or the wild life.

Now the two bandits at the front are charmed and best friends with me. They are trying to convince the rest of the bandits that I’m legit. That they know me from…somewhere?

“Now, here’s how I’m going to help you.” I say as I lean forward, at the top of the ramp. As my foot hits the ground, I hear a loud squeak.

Crap.

Hoooooold!”, we shout. We have to pause the game because something important just happened. I stepped on a trap. An obvious trap. A trap so obvious that the person setting them up expected everyone to see it. Nope, not me. I’m too busy with my silver tongue.

“10 fire damage in a 10 foot radius. That counts as an aggressive action and breaks charisma.”

Crap. I guess we aren’t BFFs anymore.13320412_10156981353955430_161362360380645738_o

I try to re-charisma the one bandit and convince her that she set off the trap. She apologizes profusely, but the rest of the bandits are not convinced and a melee ensues.

Well, at least I tried. Now we have to kill them all. This is what happens when you travel with murderhobos.

Thankfully, all of the dead bandits are very forgiving.

13323354_10156981354860430_1308299216675088895_o

That time my hat was a cauldron, and it saved the swamp

“Keshan, do you know how to featherfall?” I’m asked.
“Sure, I just learned it.” I reply.
“Okay, we need you to climb this tree [metaphorically] with these magic flowers, and then jump out of the tree.”, Angus says.
“No problem.”

I jump and do a twirl and pretend I’m falling out of a tree. Now they can make some magic potion that’s going to cure the swamp of some ancient curse.

Only there is  a problem. “Where is the cauldron?” says the game marshal, “You can’t make a potion without a cauldron.”

Mind you, we are preparing for a town fight, a battle between all of the players and all of the NPCs. As far as our characters are concerned, we could be attacked at any moment.

So I shout, “There’s no time! Zanrick, turn my hat into a cauldron.” Zanrick here is a gnome, and they are a crafty folk. And they have an ability called improvise, which means they can make something into something else if they B.S. well enough.

13329428_10156981353075430_2886636220585355019_o

More shouts go out, “We need an amour patch!”. We end up with 3 of them. Technically they are mason jar lids, but in game they are used to repair your armor after a fight. And suddenly, my hat is a cauldron brewing a magic potion to save the swamp. Pretty cool, right?
13320412_10156981356775430_7653977813504254818_o

The best part of all of it? The game marshal was so delighted by my silliness, that my hat now counts as plate armor in-game. She was just expecting one of us to go to the kitchen and grab a pot. Well, that simply wouldn’t do.

To hell with what everyone thinks

There was a recent conversation on Twitter about how when you are 40, you learn to worry less about what other people think, and you learn to do what you love. I’m not quite 30, and I worry what people will think of me, but damnit I love LARP.

I mentioned depression at the beginning of this post. I usually don’t like to talk about it, because I don’t want to seem like I’m fishing for sympathy. But it’s worth mentioning that one of the causes of depression is ruminating or obsessing over negative thoughts.

In my life there have been only 4 things that have truly gotten me out of my own head and allowed me to stop worrying about things:

  1. Video game programming competitions
  2. Board games
  3. First-person shooters
  4. LARP

That’s it. Those are the only things that allow me to escape for a little while.

But even more than that, LARP is an excuse to turn off my cell phone. It’s an excuse to go outside and walk around. It’s a situation where I’m forced to be hyper-social and meet new friends. It’s this beautiful mathematical dual to everything I do in IT.

I do it to get away. I do it get a break. I do it to make friends.

As I get older, all of those things get harder and harder to do. I may take my work home with me, but there’s at least one place in this world where my work can’t follow me.

One last thing

One last thing, that I should probably mention. So, I might have published a book, in-game. Because why the hell not.

IMG_20180210_182915940

The Mockingbird is a project I did for fun, collecting stories and lore from other players.

And so I’ll end with the introduction from my book. And if you ever come LARP with me, I’ll sell you a copy, for only two and a half gold.

IMG_20180210_182933274

Building a DBA Salary Calculator, Part 0: Initial findings

I’m planning on building a salary calculator based on the data from played around with some of the numbers earlier. This time I’m planning on going a lot deeper.

I want your help and feedback! I want to know what would make a calculator most useful to you. Feel free to poke holes in my methodology and tell me how a real data scientist would handle this project.

In this post, I’m going to outline some initial findings as well how I’m planning to approach this project. All of the information below is based on a narrow subset:

  • USA, full postal code
  • DBA job
  • Between $15,000 and $165,000

Regarding zip codes, some people only entered a portion of their zip for privacy sake. In the final analysis, I plan on taking into account the ~200 US individuals who did that.

Initial findings

The data isn’t very predictive

So I’m using something called a multiple linear regression to make a formula to predict your salary based on specific variables. Unfortunately, the highest Coefficient of Determination (or R2) I’ve been able to get is 0.37. Which means, as far as I understand it, that at most the model explains 37% of the variation.

Additionally the spread on the results isn’t great either. The standard deviation, a measure of spread, is about $25,000 on the original subset of data. Which means we’d expect 68% to be within +/- $25,000 of the average and 95% to be within +/- $50,000 of the average. So what happens when we apply our model?

When we apply the model we get something called residuals, which are basically the difference between what we predicted and what the actual salary was. The standard deviation on those residuals is $20,000. Which means that our confidence range is going to be +/- 20-40k. That to me doesn’t seem like a great range.

There are a few strong indicators

Let’s take a look at what we get when we do a multiple regression with the Excel Analysis ToolPak addin:

image

The two biggest factors by far seem to be how long you’ve worked and and where you live. In fact, we can explain 30% of the variance using those two variables:

image

The two other variables that are very strong are whether you telecommute and whether you are independent. When we add those, our adjusted R2 goes up to 33%.

Then after that we have a handful of variables that have a less than 5% chance of being erroneous:

  • Gender. It’s still a bit early to jump to conclusions, but it looks like being female might cost you $6,000 per year. This is after controlling for years of experience, education, hours worked, and if this is your first job. Gender could still be tied to other factors like a gap in your career or if you negotiate pay raises.
  • First Job. “First job” I identified as having identical values for years of experience and years in this job. If you haven’t changed jobs, it could be costing you $4,000, which lines up with my personal experience.
  • Hours worked per week. This is basically what you would expect.
  • Education. This is the number of years of education you received outside of high school.
  • Build Scripts and automation. One of the tasks people could check was if they are automating their work. Out of all the tasks people could list, this seems to have the biggest impact.

There are some interesting correlations

Part of doing a multiple regression is making sure your variables aren’t too strongly correlated or “collinear”. As part of this, is possible to find some interesting correlations.

  • If you are on-call, you are less likely to have post-secondary education. You are also probably overworked and learning PowerShell (no surprise there).
  • Certifications correlate negatively with being a dev-dba instead of a production dba.
  • If this is your first job, you are less likely to be working more than 40 hours per week. Maybe that $4,000 paycut is worth it Winking smile
  • Independents also work less hours per week. So maybe your second job should be going independent.
  • If you telecommute, you might make $2,000 more per year for every day of the week you telecommute; but you are going to be working more hours as well.

Plans moving forward

So here is the current outline for this blog series:

  1. Identifying features (variables)
  2. Data cleanup
  3. Extracting features
  4. Removing collinear features
  5. Performing multiple regression
  6. Coding a calculator in Javascript
  7. Reimplementing everything in R

So let me know what you think. I plan on making all of the data and code freely available on github.

T-SQL Tuesday #98: Learning Troubleshooting from Games

 

TSQLTues

This week’s T-SQL Tuesday is about a time that you solved a difficult technical problem. Unfortunately my brain doesn’t store events that way, so I can’t think of any good stories. Instead, I want to talk briefly about how games have made me a better programmer and a better troubleshooter.

Map-making in TFC

TFC Boxart.png

The first game I want to talk about is Team Fortress Classic. It’s a team based shooter from the late 90’s. I used to play this game all the time. But I did even more than that, I would make custom levels to play on with other people.

Mapmaking for TFC, was generally a simple process. You would create simple polyhedrons and then apply textures/patterns to them. Then you would place non-terrain objects, called entities, inside of your terrain. Everything is pretty straightforward…until you get a leak.

A leak is when the outside of the level is accessible to the inside of the level. Imagine you are building a spaceship or a submarine, if you have a leak it just won’t work. The challenge is that the level editor won’t tell you where you have a leak1. So how do you solve it?

In my case, you encase half the level in solid rock, so to speak. I would just make a big cube and cover up half of the level. If the level compiled, I knew my leak was somewhere in that half. Then I just kept repeating with smaller and smaller cubes.

I do the same thing all the time in my professional life. I’ll comment whole swathes of code. I’ll jump to half-way to the data pipeline to see where the error starts. TFC taught me to keep cutting the problem in half until I find it.

Guessing the secrets of the universe with Zendo

Image result

Undoubtedly proof that I was destined to be a programmer, one of my favorite board games ever is Zendo. It was actually one of my nicknames in college. It’s got a silly theme about discerning if something has the Buddha nature. In reality, it boils down to one player making up a rule, and everyone else trying to determine what the rule is.

If it sounds easy, I dare you to play something similar over at the New York Times. Chances are you are going to get it wrong.

The biggest thing Zendo taught me, was fighting against confirmation bias. It taught me to ask “What would prove my theory wrong”. Good troubleshooting involves guessing a cause, determining a test that will give you new information, and then running that test.

That test might be running a simpler version of a query that’s failing. It might mean adding a breakpoint to your code and inspecting variables.

Learning how to think systematically about this sort of thing has been tremendously useful.

Learning outside of programming

Troubleshooting is very often a set of skills and approaches that don’t need to do anything with technology per-se. I think looking at how we can get these skills in other places, like games can be very useful.

Speaking of other sources, there are two book I can recommend wholeheartedly. The first is How to Solve It which is about how to solve mathematical problems, but it provide a number of ways to break down a problem or approach it from different angles. The second is called Conceptual Blockbusting. It focuses on the nebulous issue of how we think about problem solving. It’s very much a book about thinking and I definitely enjoyed it.

Footnotes

1 Only after writing this blog post did I find an article explaining out to get the level editor to tell you exactly were the leak is. Sigh.

Keeping up with Technology: a Guide to Drinking from the Firehose

I often ask people with more experience than me “How do you stay relevant in our field?”. I joke that I live in perennial fear of being replaced someday, by a 22 year-old with no family commitments. Really, it’s a joke. Really, it’s a fear.

So, let’s talk about how to keep up with technology.

 First, we must grieve.

So here’s the secret to keeping up with technology. You can’t.

There are too many technologies, too many features, too many updates:

  • SQL Server 2017 is coming out this year.
  • Big data technologies are so numerous as to be indistinguishable from Pokemon.
  • PowerBI ships features on a weekly basis.
  • Worst of all, you’ve got NoSQL which is literally the mathematical dual of SQL. It’s everything that SQL isn’t, by definition!

It’s all too much.

This realization is likely to be expressed in the 5 stages of grief.

  • Denial. Other people can keep up, why can’t I? I’m learning tons of stuff all the time! This is easy.
  • Anger. Why they heck are they making releases every month!? When did Microsoft go agile? I thought SQL Server versions every 2 years was bad enough.
  • Bargaining.  Okay, maybe if I stick to core SQL stuff, I’ll be fine. Hadoop seems like a fad. And Azure is never going to be popular with my  company.
  • Depression. This is impossible. I’m going to lose my job when I’m 40 years old and arthritis starts kicking in.
  • Acceptance. There’s more happening than I can ever learn, but Hacker News doesn’t define my success as an IT professional.

This post isn’t about doing the impossible. It’s about making the most of your resources. That’s something that you can do.

Are you asking the right questions?

I would like to propose that it’s not about keep up with technology at all. This is a second-order goal, a proxy of sorts. Really, there are two questions at the heart of things:

  1. How do I keep my job?
  2. How do I keep my friends?

This is why technology causes so much angst. We want to learn enough that we can put food on the table; and we want to do it in short enough time that it doesn’t destroy our personal lives.

Next, we must think.

In order to get ahold of the the problem, let’s use some analogies: investments and radioactive decay.

How learning is like investing

Do you have a retirement account? If so, do you invest primarily in stocks or bonds? Why?

If you are at all young, the you invest primarily in stocks. That’s because stocks have a higher rate of growth than bond. You are trying to outrun inflation, where the value of your money is steadily decreasing. This is like your current knowledge becoming outdated. All that vb6 coding knowledge is like a pile of cash in your mattress. When I was a kid, $20 was a lot of money.

The next question  is, do you invest in just one stock, like Apple? No, you diversify your portfolio. High growth stocks go up, on average. Some however, tank. High growth means high volatility. A good example is Apache Flex. It used to be a really promising application platform, until Steve Jobs killed Flash.

So we’ve got two risks to our learning portfolio: Losing value in our existing knowledge, and making the wrong choices for our new knowledge. These different risks have different mitigation strategies.

Specialization and generalization

These comparisons to bonds and stocks relates to the challenges of specialization versus generalization. We specializes to pay the bills. We generalize to keep our jobs.

Specialization is how we get paid. The reason anyone pays us is because we have skills or knowledge that is not quickly acquired. The reason consultants like David Klee make gobs of money is that they have taken a subject, like virtualizing SQL, and have gotten really  good at it. To get paid more than minimum wage you are going to need some level of specialization.

Generalization is how we get paid in 10 years. You need to be specialized to get paid right now, it’s a short term investment. However, that investment decays, just like the value of money in your mattress. To get paid a decade from now, you need to broaden your horizons. If you are a data person, it might mean learning R and python. It might mean learning Azure. Heck, it might mean PowerShell and Docker.

The tension here is that you need both. You need paid now AND in 10 years. Kevin Feasel talks about trying to find that right balance. I’m not here to tell you what that balance is. What’s important is that each has different constraints. Specialization requires focus. Generalization requires time. More on that later.

How learning is like radioactive decay

Let’s take another approach.

Our knowledge has a limited shelf-life. Allen White said, in the SQL Data Partners podcast, that you have to retool yourself every 5 years. In college, I remember joking that half of what you knew would be useless in 5 years.

Well, what if we took that literally? How would we model that? How would we think about that?

In nuclear physics, there is the idea of a half-life.  You have radioactive material that decays in half every X units of time. Why not apply this concept to IT? The half-life in this example then would be 5 years.

So the next question is this: if half of what you learn is either irrelevant or forgotten in 5 years, how much do you need to learn in a given year, to keep steady? Let’s ask our friends at Wolphram Alpha.

image

If x is our rate of decay and our half-life is 5 years, we can work backwards from that and solve for x. In this case, x equals 87%.

So, what that means is that if today you know 100 relevant things, then a year from now you’ll only know 87 relevant things. That’s like going from a solid A+ to a weak B+. You just lost a whole grade!  Not good.

Below is a curve showing what this model looks like over 10 years.

image

Changing the math

Well, this isn’t great. What if I want to know 120 things? How can we do that? One option is to learn more things.

Learn more things

If normally we have to learn 13 things each year, then we have to learn another 20 things on top of that. Or, if you are patient, you can learn an extra 4 things each year, and eventually  you will get there.

image

Slow the decay

Another option is to change the rate of decay. Instead of learning more things each year, what if we didn’t have as much decay? If we can slow that rate of decay just a little bit, to a half-life a 6 years instead of 5 years, we’ll have the same effect. That means that instead of brute-forcing things, we might be able to be smarter.

What does this all mean?

So that gives a path forward. We need to either

  1. Increase the number of relevant things you can learn each year
  2. Or, decrease the rate of decay for relevant knowledge

These are the only three knobs we have. Either learn more stuff, learn the right stuff, or learn stuff that lasts longer. Let’s investigate all three.

How do we fix this?

Learning more things

So one solution to our dilemma is to brute force it. Let’s just learn more things and hope that they are the right ones. To do that, we can look at our inputs and constraints.

What are the constraints that affect our learning? There are 3 big ones I can think of:

  1. Time
  2. Energy
  3. Money

If we can increase any one of these, then we might be able to increase how much we can learn in a week.

Time

So, lets say we decide to go the simple route and go for volume.  You’ve got a 168 hours in a week. You can’t make any more hours, and if you try to use all of them in a week, you are going to need some amphetamines.

So, if we can’t create more hours, how can we make more time? One option would be to cut out other activities. If you are willing to quit watching TV or playing video games, that frees up more time for learning. You could use a service like toggl.com to do a time audit and see where all of your time goes.

There is a limit to going down that path, however. You need to sleep. You need to have a social life. You need to have some fun. Like we said, you want to keep your job and keep your friends.

Multitasking

Another option is to multi-task. There is a lot of learning you can do that while doing other things. Things like podcasts are more about exposure than mastery. You can still get value out of them, even while mildly distracted. There are a number of times you could listen to a podcast:

  • Commuting
  • Exercising
  • Washing dishes

This is a way to take back time you are spending on other things without giving up all your free time.

5 minute learning

Another way of reusing your time is taking advantage of those weird breaks in time. Those breaks that are too small to get anything meaningful done. The 15 minutes at the doctor’s office. The 5 minutes waiting for a meeting.

Feed readers are a great way to take advantage of these weird units of time. With Feedly, I’m able to to read a blog article during the 5 minutes I’m waiting. This is way more useful than playing candy crush or reading twitter.

Focus/energy

Okay, so let’s say you’ve managed to find more time in the day. Unfortunately, not all learning takes the same amount of energy. Reading Twitter is mindless. Configuring a homelab requires focus. Listening to a podcast is mindless. Making a presentation takes focus.

One way of getting more focus is to prioritize the harder learning for when you naturally have focus. For some people this is early morning. For many people this is the weekend. I know that after a long day of work, I’m wiped out.

Part of that is learning to take care of yourself. Eat healthy. Exercise. Get sleep. Focus is so easy to destroy by poor lifestyle.

Finally, if you schedule time consistently and push everything out, you can have more focus. I also find having a separate office/learning space helps with this too. Cut out the distractions. Install StayFocusd for Chrome to block timewasters.

Money

If you are anything like me, money is your most plentiful resource of all 3. When I was kid, I had no money at all, but tons of time. When I was in college, I had pretty much no money, and a good bit less time. Now that I have been working for a while, that equation has flipped. I’ve got plenty of money, but no time to use it.

If you are an average DBA, you make plenty of money. According to the Bureau of Labor Statistics, the median salary for DBA’s is $85,000. The median for all occupations is $37,000. Think about that, that’s like 2.5x as much.

Now you may say “Hey, I don’t live in San Francisco.” or “Hey, I just got started.” Let’s take those factors into account. The market here in Pittsburgh is terrible, salary-wise. Even if you are just getting started in the field, you are probably making at least  40K.

image

Let’s do a little math on this. If you make 40K per year, you make roughly  $20 per hour. that means your time is worth $20 per hour. If you can spend $5-10 to save an hour of your time, do it.

If a $50 book saves your 5 hours of Googling, buy it. If a Pluralsight subscription saves you 30 hours per year of frustration, buy the subscription. Don’t be afraid to spend money to save yourself time and energy.

Consider budgeting your money

If you don’t feel like you have a lot of money, I would deeply urge you to start budgeting. I used to look at my checking account to see if I had money. Then if there was money there, I would spend it. This would work until my car insurance bill would come in, and suddenly I needed $600. And just as suddenly, I didn’t have nearly as much money as I thought

Get a budgeting program. There’s some free one’s out there, but I use You Need a Budget.

Learning the right things

So we covered how to learn more things. An alternative is to learn the right things. If you learn the right thing, you’ve have less wasted effort.

Lean on curation

Sturgeon’s law says that 90% of anything is crap. That’s certainly true of learning materials. Not only is there a lot of bad training out there, but there’s a lot of content in general that just isn’t relevant. It’s cool that someone wrote an OS in Rust, but it’s probably not relevant to your job in any significant way.

If you are time poor or focus poor, you need a gatekeeper so the flood of possibilities doesn’t overwhelm you. This means depending on curation.

That curator could be you, first off. One of the reason I suggest a feed reader over Twitter is that you are able to heavily curate the content that hits your eyes. You are the one picking the blogs to read instead of the pachinko ball machine of social media and fate. You are the one deciding what’s relevant to you.

Another options is to depend on professionals in the field that you trust. Find people to follow that consistently have good material or recommendations.

The third option is to pay money. Sure, there’s some paid crap out there. But general it does act as a quality filter. Things like written books and video libraries are far less likely to be total crap, because those things have editors and investors and such. Someone was being paid to make sure it was worth making. Additionally, the author worked hard to condense the knowledge into a concise format. Be willing to pay money.

Having a plan

Okay, so now you are investing your time and energy into quality materials, but are you learning the right things? Are you learning the things that are right for you? You need to have a career plan. You need 1 and 3 year goals. Otherwise your career plan will be a spread out mess.

Figure out where you want to focus, figure out where you want to go. Pick a specialization. More of what you learn will be relevant if you know what that is.

Learning things that last longer

Okay, so we are learning more things and more of the right things. But we still have the problem that our knowledge is decaying whether from atrophy or irrelevance. This is the last step in the funnel and the last thing we can optimize.

Improving retention

One way to keep ahold of knowledge for longer is to lean on active learning. If something goes in through your eyes and ears and back out your fingers and mouth, you will retain it for longer.

If you want to specialize and go deeper, you need to do things like blogging, presenting,coding, etc. Reading isn’t enough. Writing isn’t enough. if you truly want to learn, you have to do.

Reddit and hacker news are good for exposure to a topic, but exposure fades quickly. Use exposure to get a lay of the land, then go deep once you find the right things to learn. If you focus on mastery, you are going to learn things in a way that last longer.

Avoiding planned obsolescence

Another issue is that certain areas of knowledge just have a faster rate of decay. Some of these are facts tied to a specific version of a technology. Another thing are unproven technologies that might turn out to just be fads. Big data is still in this phase, where there are so many technologies that will be gone forever in 5 years.

People change far more slowly than technologies, so skills relating to them last far longer. Soft skills never go out of style. Communications skills never go out of style. If you learn how to write an abstract today, that knowledge will still be relevant 40 years from now.

The fundamentals last longer too. The core basics of computer science are far more establish than this weeks JS framework. If you have solid underpinnings, you can take your knowledge of C++ to Go, for example. If you understand how a ACID and CAP theorem, The new consistency models of Cosmos DB will make sense. Under understanding of one layer beneath will allow you to jump to new technologies far quicker.

Putting it all together

To summarize what we talked about:

  • Take advantage of multitasking to get back more time. Listen to podcasts while you drive or exercise.
    • Invest in a pod catcher. I use itunes and an iPod nano.
    • Invest in a feed reader. I use Feedly.
  • Take care of yourself, physically. Diet, sleep and exercise will all improve focus.
  • Schedule time to maintain focus. Treat focus as your most precious resource. Don’t browse twitter when you are firing on all cylinders.
  • Be willing to spend money on curation. There is a good chance that you have more money than time or focus. Be willing to spend money to avoid wasting time.
  • Understand exposure versus mastery. Spend your time on exposure and your focus on mastery.
  • Pick a specialization. Have a plan on what technologies you need to go deep on. You won’t get there by accident.
  • Learn things that last. Learn the fundamentals. Learn internals. Learn people skills. Do home labs.

The challenge with being a “data professional”

During PASS Summit, I wrote a post about the broadening data platform. I talked about the term Data Professional, and how I feel how it describes the changes going on the in SQL space. Here’s the problem: It’s a terrible guide. It’s a great description, it’s a wonderful attitude, it’s an ambitious goal; but it’s a terrible guide.

Being a data professional means being a jack of all trades. It means being a renaissance man (or woman). It’s means a career plan that looks like this:

And then you end up with Buck Woody telling you you are trying to do too much, cut it out kid.


So that’s the problem. Sometimes broadening your horizons is really a mask for being scared of commitment. Sometimes it’s a mask for being scared of an ever-changing future. You have to bet on a horse, you can’t bet on them all. Being a data “professional” is great in theory, but in practice it turns into majoring in the “universe” (see XKCD).

Major in the Universe

I’m not saying don’t learn Docker or Powershell. If you don’t learn those things, Kevin Feasel will warn you about becoming homeless. And who wants that.

What I am saying is that if someone asks you “Where do you want to be in 3 years?”, “everywhere” is not an answer. If someone asks you “What are you going to learn this week?”, “everything” is not an answer. So yes, generalize your skill set, who knows what you’ll be doing in 5 years. But right now you need a focus, it’s the only way to become an expert at anything.

Ultimately, I think it comes down to two quotes:

If you don’t know where you are going, any road will get you there.

-George Harrison, paraphrasing Lewis Carrol

and

Two roads diverged in a yellow wood,
And sorry I could not travel both
And be one traveler,

[…..]

I took the one less traveled by,
And that has made all the difference.
-Robert Frost, The Road Not Taken

For me, I’m looking into Data Science. The problem is I’m not sure what Data science actually is! What I know for sure is it involves R and pirate jokes. We’ll cover that in next week’s blog post.

Starting a todo app: part 2, technologies

Last week, I wrote about my desire to make a todo app. this week, I’m going to talk about the technologies I’m using and what I think so far.

Currently, I’m building this app on the following technologies:

  • C#
  • Asp.net
  • Azure
  • DocumentDB
  • Xamarin

Notice any pattern? I am a giant Microsoft shill. My career focuses on Microsoft SQL server and Power BI, so It’s in my best interest to learn surrounding technologies.

Of course Microsoft keeps doing weird things like including Git in Visual Studio, or putting SQL Server on Linux. So the surrounding technologies get a big broader these days.

C#

I really like C#, but I don’t feel like I “get it” just yet. I think it’s clean and understandable as a language, but it feels like 80% of the language is learning about random libraries and classes. And if you don’t know that image

I can’t say I have a good idea when to choose between one or the other within a project. It seems like 3 very different modes of working and routing.

Azure

Azure is to today what virtualization was 5 years ago. The cloud is just going to keep getting bigger and bigger and bigger. If you aren’t learning about it now, you are going to be very remiss about it 5 years from now.

Unfortunately most of us don’t have a need for cloud hosting. You probably don’t have a need for your own personal Azure Active Directory or Hadoop Cluster. So a lot of times, that means doing projects and labs to learn it preemptively. If you are waiting until it comes up at work, it’s going to be too late.

DocumentDB

DocumentDB is… interesting. Brent Ozar mentioned that he is using NoSQL for a new project. He picked Amazon DynamoDB over DocumentDB. I’m picking DocumentDB as a way to learn Azure.

So, why go the NoSQL route? Partly it’s something new to learn, but there are a couple of advantages. I think Document databases when you want to store lists of things or hierarchical data. In my app, I have lists for times of day, task categories, and tools/contexts. I want to be able to do filtering on these things, which can be clunky in SQL.

Things I like

Some other things I like so far are the dynamic schema, dynamic serialization, and TTL. Normally I appreciate have a well-defined structure to my data. However, this is a design that’s constantly evolving and changing. It’s hard to plan it up front. With Document DB, I can add a field to my model class in C#, set a good default, and that’s it. Now I have a due date field

The other thing I like so far is the TTL feature. I can mark completed tasks with a ttl of a week. Then, if I don’t touch them for  a week, they will self-destruct. That is a really cool feature.

Things I don’t like

One of my big pet peeves with DocumentDB is that collections are billing containers not logical containers. This to me just seems silly. If I want to have different types of objects, such as summary data, I have to store them all in the same place or pay 2-4x as much. That approach feels really messy.

Another thing that’s an issues is case sensitivity.  Because it’s a dynamic schema, if I get a column name wrong, it won’t tell me I screwed up. So I have to remember if my columns start with a capital or not. Also, if I do a contains I have to get the case right for that as well. This is more of a mindset shift than an actual issue with Document DB.

Finally, because it’s JSON there isn’t a date standard. C# does a great job of serializing and deserializing things. However, if I want to store the dates in a human writable way, I have to find another option. For all of my pure dates I’ve been using an integer for now. So September 11th would be 20160911. It requires an extra step to convert or add days to a date, but at least it’s easy to edit. Additionally, because it’s an int, it allows for range filtering.

Xamarin

Xamarin technology is pretty exciting. I absolutely love the idea of being able to write C# and deploy it to a bunch of platforms. It’s especially exciting because my phone is a great place to run a todo app.

That being said, if understanding C# is difficult, then understanding C# that’s a thin wrapper on Java is even worse. I am finding the learning curve on Xamarin to be a bit difficult.

Starting a personal todo app

Todo apps and gamificiation

Todo apps have been done to death. If you go on Google Play, there’s at least 200 of them. There’s even a website that uses todo app tutorials to compare Javascript frameworks.

Still, I’m a huge fan of David Allen and Getting Things Done. I love the idea of tracking all your “open loops” and the next action on everything. I love the idea of context aware lists that take into account location, time and energy.

I’m also a huge fan of gamifying your life. If video game developers can use psychological tricks to get me to play more video games, why can’t I use the same concepts to get me to be healthy and accomplish more.

A great example of this is Beeminder. It’s this really cool application where you set a goal and track the goal on a daily basis, sometimes automatically. If you don’t meet it, they take your money, and then restart with higher stakes. I love it. If there was a single startup idea I wish I had thought of, this would be it.

The problem with Beeminder is you either need to pick just a few simple goals, or you need to guarantee you have an even workload. If your day-to-day has a lot of ups and downs, it can be stressful.

Finally, I like the idea of externalizing my executive function. I seem to run into the problem of having to many things I want do to, have trouble choosing, and do none as a result. It’s very frustrating. If I can outsource that analysis paralysis, I think I would get more done.

The idea for the app

This is actually the second time I’ve tried to make a todo app. The first time had more of a gamification feel and I called it EugeneQuest. I figured if I was going to be paying someone to motivate me to do things, why not pay myself. It worked out decently well, but I mostly broke my addiction to Magic The Gathering by accident.

Despite how that went, it’s still really important to me to have personal projects. I think personal projects are the best way to learn and I’m desperate to learn C# and Azure. So I’m trying again, but focusing more on filtering tasks by time and energy. So far it seems promising.

Next week I’ll talk about the technology I’m using and what I’ve learned so far.

Jumping back into gamedev with Unity

My motivation

As I’ve been getting older, I’ve found a simple trick for introspection: look at yourself a decade ago and ask that person what makes him happy. The core parts of you, the essential parts of you, probably haven’t changed much. For me it was 3 things: boardgames, video game programming, and first-person shooters. Those are the only things that would regularly get me out of my own head.

Other things, however, have changed in the last 10 years. First, for some reason I seem to have a 1/5th of the leisure time that I used to have as a kid. Second, someone now pays me to program 8 hours a day. Suddenly, the idea of spending 2 hours every night working on video games seems onerous, burdensome. It’s just not as fun when you spend every day programming stuff.

But, as I said, your core self rarely changes. Just because you don’t have the time or energy to be passionate doesn’t change what gives you joy. Growing up is kinda like being a pokemon and going through evolutions. You are still some Blastoise

Adulthood. It’s like having cannons coming out of your shell.

Since I got married, I’ve started playing a boardgame every night with my wife. It’s really been beautiful to watch. It’s great seeing a hobby you thought was lost forever, come back to life. It reminds me of the signs at the dormant

Your passions. Not dead, just dormant.

Now that boardgaming has come back to life for me, next up is reviving game development. Of course, since it’s been 7-10 years since I’ve really been into video game programming, my analogy is more like the

Jumping means my game is inherently better than Stanley Parable

It’s not a lot, but I can already see a lot of the power and appeal of the tooling. Additionally, the tutorials are top notch and very easy to follow. I really expected working with 3d to be a lot more intimidating.

Overall thoughts

Even after working with Unity for less than an hour, I have a few first impressions.

It’s easy to get started

The first tutorial takes just over an hour to complete. At that point you’ve created the minimal definition of a 3d game. It would probably have taken me a week to do all of that from scratch, hand writing OpenGL calls.

It’s also got a very quick debug cycle. You code something, test it, make a tweak, test it again. It’s literally as simple as hitting play on the scene. Additionally, much of the details are accessible as GUI controls. This is especially cool when it’s your own custom scripts. All public variables on your scripts become editable. You can even drag and drop, if the variable is a game object. For example, for the camera controller we drag and drop the player object.

image

It seems really well polished

I don’t know if there is any one thing I can point to, but overall a couple of things stick out:

  • The UI seems fairly easy to navigate.
  • There are a ton of features regarding components or physics.
  • I can see right away how I could use the power of C# instead of being limited.
  • It makes adding behavior to an object super simple. Just a few lines of code and your object is controllable.

It just seems like a quality piece of software. I’ll be interested to see how it looks after 100 hours isntead of just 1.

It’s different than what I’m used to

Unity does things very differently than I have in the past. Using components over inheritance is new to me. I can see how it makes it really easy to add behavior or functionality to entities.

Another thing is there seems to be a heavy focus on encapsulation and separation of concerns. Components have functions for the event cycle: start, update, fixedUpdate, lateUpdate, onDestroy, etc. The components know only as much as they need to, and you have to work to go outside of that. I’m used to one giant loop of logic and processing behavior manually.

My way or the highway

One of the downsides of using a framework or engine is that there is usually a “right” way to do things. The downside of having guard rails is when you want to go out of them. Working with Unity, I definitely get a vibe that there is a “Unity way” of doing things. Not in any way that is a problem at first. But I immediately have a few questions in my mind:

  • How do you determine which entities get processed first in an event? Do you have any control?
  • How do you handle mass interactions between entities?
  • What if you want to use a different physics engine?
  • What if I don’t want to use scenes?
  • What if my whole game is procedurally generated?

It’ll be interesting to see if it presents a problem or not.

Final thoughts

Overall, I’m very excited to be learning Unity and see where it takes me. It’s used by tons of indies and has a lot of potential.

css.php