Full-stack cloud infrastructure engineering

Recently, I was honored to be a guest on a Datanauts podcast in which we discussed public cloud maturity. I was deliberately doctrinaire in that episode to make the point that the skills an infrastructure engineer needs to be effective in the cloud are most definitely not what many infrastructure engineers in today’s enterprises think of as their wheelhouse. I think my assertions came as a bit of a surprise to the very knowledgeable Datanauts hosts, Chris Wahl and Ethan Banks.

In case you haven’t heard the podcast, I suggested (in no particular order):

• That infrastructure engineers need to learn to love to code;
• Data center folks need to learn development tools including source code management (basically, git) as well as development environments; and
• The “bare metal” API (excuse the mixed metaphor) in the cloud is JSON.

As Mark Betz writes, the bottom line is that “…console actions don’t capture intent, can’t be version controlled and are in general evil.” One should never implement a production resource in AWS or Azure that was defined by using the console. It’s OK to use the console to check the results of a deployment from code. But using a “wizard” or a UI to create a production deployment is dubious at best and dangerous at worst for reasons anyone who has ever had to take over a wizard-based piece of infrastructure knows all too well. Active Directory, anyone?

Listening to two subsequent episodes of Datanauts, one on Azure recovery and the other on AWS DNS, I was struck by how both guests assumed that infrastructure as code (IaaC) is the only way to work in the cloud. IIRC, one of the guests even said something on the order of, “You’d better have these resources defined in code.” These guys get up in the morning thinking of cloud resources purely as IaaC.

Having worked in enterprise IT since punch cards, I realize how tough a prescription this is for many. I had the advantage of being an application developer for a part of my career. As a result, I’ve always loved code (these days, especially PowerShell code).

But what can a network engineer who’s spent a decade or more racking routers do to become an IaaC expert? What about the VMware admin who’s life has been centered around vCenter (sorry)? How to explain to that AD administrator that she will declare via code a service principal for a multi-tenant app in Azure AD?

I’ve got a couple of recommendations — but first, it all comes down to a willingness to change. If you are an IT infrastructure engineer in an enterprise data center today, you already know something’s coming. You can either see it as an existential threat to your career or you can migrate (pun intended) your skill set along with the VMs, networks, databases and storage to the cloud. You have to want to change and, possibly for the first time in a while, be just a little uncomfortable with things that are, at first, hard to digest.

Here are some suggestions that help transform you into tomorrow’s full-stack cloud infrastructure engineer:

• Learn to code. There’s no shame in using an app meant for children, like Swift Playgrounds, to get the basic idea. I won’t tell if you don’t. If you’ve got the basics, learn some basic Python — it’s everyone’s favorite language today.
• Get out of your silo. In the cloud, there are no hard lines between networks and databases; storage and identity; disaster recovery and security.
• Talk with end users. The days of you being hidden behind a self-service ServiceNow portal are over. When you can create a thousand VMs in a few lines of code, you’ll have more time to meet with the business users, understand their objectives and build a secure, performant infrastructure for them in days, if not minutes. When you don’t have to do anything other than declaring in JSON what you want, you can deliver all your expertise and, finally, get the visibility infrastructure engineering deserves.
• Don’t duplicate the data center. This is crucial. For the purposes of argument, accept for a moment that the cloud transition is inevitable. (Remember those who said PCs would never overtake mainframes in the data center? Nobody today will ever admit he or she said that — and in five or 10 years nobody will remember they doubted the cloud would become the main enterprise computing idiom either.) If that’s the case, why would you ever create an “extension” of your data center’s infrastructure in a cloud provider? Is it so perfect that it should be translated — literally subnet by subnet, database by database — into a 100% software-defined environment? Of course not. You know in your bones this isn’t a good idea. But it also does not mean you forget everything you knew about good practice in network, compute, storage, identity and database. It does mean you should start with a clean page cloud infrastructure and, after you have thought through the cloud-native design, figure out how to integrate an existing enterprise resource. It’s a simple matter of not continuing to do what you’ve always done.

So, where are we with this shift to full-stack cloud infrastructure engineering? I’m hopeful, especially when I hear people like the Datanauts guests who just assume a code-driven cloud. It’s clear that an exciting transition — and an enormous professional opportunity — is underway.