Python for macOS

This is how I set up my mac to use different versions of python with virtualenvwrapper. virtualenvwrapper enables virtual environments with the friendly workon <project> command instead of source <path/to/venv>/bin/activate.

Prerequisites

We need homebrew. Install homebrew (I use the defaults) before proceeding.

Base Pythons

We now install both python3 and python2. These take over the default python2 and python3 binaries from macOS.

# python3
brew install python

# python2 
brew install python2

Use the homebrew versions of python2/3 for dependencies within homebrew. Don't use the homebrew versions of python for applications you develop. Homebrew updates packages like python quickly and causes headaches with my applications that need a pinned version of python2 or python3.

App-specific Pythons

For applications, install pyenv to manage pinned versions of python. Once again, reach for homebrew:

brew install pyenv

Now install the different versions of python you need for your code.

pyenv install 3.6.9
pyenv install 2.7.19

Now activate all the pythons. Enabling all versions of python removes the requirement of using pyenv to switch between current versions of python.

pyenv global system 3.6.9 2.7.19

Now you access each version of python with a version suffix – e.g., python3, python3.7, python3.6, python2, etc.  

virtualenvwrapper + pyenv

I love the workon <project> functionality enabled with virtualenvwrapper. virtualenvwrapper doesn’t work with pyenv naturally because the virtualenvwrapper package pins to the current version of python. This is problematic because pyenv swaps out the python path that virtualenv (and workon) expects. To have this work with pyenv use pyenv-virtualenvwrapper.

brew install pyenv-virtualenvwrapper

Now add the following to your bash shell's ~/.bashrc or ~/.profile:

if which pyenv > /dev/null; then eval "$(pyenv init -)"; fi
pyenv virtualenvwrapper_lazy

App Setup

To use the correct version of python with a virtualenv when setting up a new repository, use the following pattern:

cd <repo>
mkvirtualenv -p python3.6.8 -a $(pwd) myproject

Now you have workon myproject available in your shell that activates the correct version of python.

(Untitled)

I'm slowing down with these coffee thoughts.

Articles I write tagged coffee thoughts serve several purposes. First, I want to write because I find that it helps me clear my internal thinking. I thought I'd write more because I paid for ghost pro, but this site had two posts for the first half of this year. Second, I want to overcome my obsessive behavior in perfecting my writing. Perfectionism is also the reason this site only had two posts. I have plenty of drafts that are still unpublished because I don't think the writing is good enough.

So here's where coffee thoughts come in. I spend every morning writing as much as possible and then post it when my oversized coffee runs out. But, I didn't realize I would have an even harder struggle figuring out what to write.

I wake up, and I stare at the blank post, and nothing comes out of my head. Typically, my mind travels at light speed after I sip my coffee. I have a million thoughts going on in my head. I think about life. I think about work. I even think about thinking! My brain is crazy.

But when I stare at this editor, my mind is blank. And it's incredibly frustrating!

The title of this post is what happens in the ghost editor when you start typing in the body.

And I'm posting this. Because my coffee is empty.

Using My Product is a Terrible Idea For You

If you asked me what products I built in the past five years, I'd answer with cloud infrastructure products (you know, databases, compute clusters, etc). If you then asked me if you could use my cloud infrastructure products, I would gladly say yes because they're awesome. If you then asked me how to use it, I would show you. And you would spit your coffee out in disgust. Probably in my face. Because to use my product, you'd have to know as much about cloud infrastructure as I did.  

/shrug

I started my career building  web applications. I built features that I thought were awesome (cool javascript animations, beautiful navigation, well designed forms, etc). Then over the years I focused on what users were actually doing through metrics and user interviews. I used user stories to build new features. I measured user interactions to build better features. I reveled in making the difficult simple for a user. And I loved building products in this way.

Unfortunately, I had lost the user-centric focus while I moved to operations and infrastructure engineering. As I learned how to use configuration management systems, built continuous integration and deployment systems, learned git branching models, prepared deployment artifacts, and coded my infrastructure, I somehow lost the focus on the end user. Frankly, I didn't know an end user existed.

Instead, I focused on provisioning and maintaining cloud infrastructure. I think the tools in this domain are difficult to use. I think the complexity of gluing tools together (like Chef and EC2, or SaltStack with Terraform) required so much knowledge that building a sensible user interface wasn't even at the top of my mind. Learning how to build an EC2 instance in the correct subnet with the correct security group was hard. I didn't even know what a security group was until I realized I couldn't SSH into my EC2 instance. Making this easy to use for a developer? No way, I had other problems to solve.

And to be honest, the products I've built in this way are terrible. I definitely deserve some coffee in my face.

I also believe this is how many infrastructure teams at small startups build cloud infrastructure (assuming they're not using Heroku). If I had to join another startup in an infrastructure team again, I think I would definitely focus on building for the users instead of myself.

How I would do this and what are those users? That's for future coffee thought because I'm now out of coffee.  

A solution full of holes

I came to infrastructure engineering (or DevOps or SRE or Systems engineering) because I was a software engineer solving similar problems on a continuous loop. This lead me to uncover the events after code on my laptop works. I started by asking a simple question. How do I run my web application for users? The answer to this question describes the domain of an infrastructure engineer. And, in this post, I describe a solution to this question.

To keep the solution simple, let's assume my application is a simple go application. I've compiled the application into a linux binary and it's ready to go. This assumption is a discussion for another post, but also belongs under the infrastructure engineering domain.

The first task I need to do is acquire a linux server. There are many ways to acquire a server. For instance, you could use cloud services like AWS or a hosting provider like Linode. But, there are two important features that I need from this server:

  1. I can SSH to the server
  2. My users can access my application via their web browsers

The actual process of acquiring a server is too complicated to describe in this post, so let's assume I now have my server. Let's also assume that I can SSH into the server from my laptop user the username myuser. Finally, let's assume my server's public hostname is example.com.

Now that I have my server, I will create a directory on the server where I can upload my application. Here's how I do this.

ssh myuser@example.com mkdir myapp

I need to upload my application to the server. Since my server has SSH access, I can run the scp command to upload my application's code onto the server. I can run the following from my laptop terminal:

cd path/to/my/app
scp -r myappbinary server-username@server-address:myapp

Now that my application code is on my server, I can start the application.

ssh myuser@example.com
cd myapp
nohup ./myappbinary &
exit

At this point, the myappbinary is running on the server and I can access it via https://example.com.

I know. This is way too simple. But it will allow me to break down the reasons infrastructure engineering exists from this simple example.

And now I'm out of coffee.

Castles, Moats, and Coffee

Early in my software engineering career, I worked with experienced developers and I noticed I outpaced them in projects. I finished tasks faster. I learned codebases faster. I committed code faster. I wrote faster (read: shitty) unit tests. I was bright eyed and fearless. I (falsely) identified as the best programmer of the bunch. But, I always wondered why the more experienced developers were so freakin' slow??

I am now in my 15th year. I have stories to tell and scars to show. And yes, I develop software slower than my former self. I am now the freakin' slow developer according to my 15 year younger self.

So why am I a freakin' slow developer?

I ask more questions now than I ever did when I was younger. I want to understand the problem. I want to understand the users, the current system, past solutions, technical debt, team goals, company goals, timing, and most importantly the reason the problem should be solved. These are buried facts about a problem and require a shovel and elbow grease to uncover. And, I find surfacing these facts a more difficult task than the technical solution. My younger self didn't even know he could ask questions.

On the technical level, I also have a greater understanding of the cost to maintain software. Early in my career, I never had to maintain software because, well, I never had experience. I never experienced the difficulty of maintaining large, old, tech debt filled codebases. Now, I maintain and build solutions entirely in old codebases. These codebases make a boat load of money unlike the codebases earlier in my career. They have technical debt and requires skill to navigate and pay off those debts. The stakes are higher and more rewarding. So yes, I am slower now with these codebases. But, my solutions also make money. I don't think my younger self ever made the connection that there's more to software than committing and deploying code.

Finally, I'm a freakin' slow developer because I've burned myself on my own poorly written, unmaintainable, and buggy code. I've seen what my dumpster fire of inexperience can do – burn, baby, burn. I understand that my poor architectural decisions create difficulties for problem solving in the future. I know that hacking a solution to fix an incident means I need to remind myself to sit down and build a better solution to prevent that incident from happening again. Lastly, I support my code with on-call rotations because unsupported code is useless code. Also, I don't ever want to deploy code that can wake me up in the middle of the night. I'm not a masochist, so fuck that.

So yes. I'm a freakin's slow developer and I love it.

If I were to meet my younger self today, I'd allow him to build a castle with his code. But I'll build a giant moat with no bridges around his castle and let him burn and starve inside while I sip on my coffee.