Since the dawn of the shell scripting, developers have been using shell scripts and shortcuts to accomplish the quick, tedious things in everyday programming. Everything from starting a server, to creating a properly formatted diff, to navigating and searching directories are quick, small functions that can be encapsulated by shell scripts or functions.

The case was the same at Zillow. There’s no point spending time performing complicated, tedious workflows over and over again. So developers started writing scripts to make some of Zillow’s workflows easier.

Now developers’ workflows were faster, but there was only one problem. Developers were writing the same shell scripts over again! Without a central methodology to share these scripts, multiple developers would implement similar functionality in different ways, and without standard naming conventions, a developer couldn’t work quickly on another’s machine when doing something like pair programming or debugging a problem.

Our first attempt to resolve this was with a wiki page that had a giant list of shortcuts that developer can pick and choose to include:

However, this methodology was problematic as well:

• Someone using it doesn’t know all the commands available to them, or  necessarily what they do.
• Developers don’t always find the wiki page.
• Updating required checking the page and reintegrating the commands every time.

These were minor problems, but they resulted in developer machines and the state between them being inconsistent, and useful commands not revealed or missing.

Enter sub. Sub solved a lot of the problems we were having:

• Namespaced commands and automatically generated help docs for discover-ability.
• By using a git repository to store the commands, there’s a common place where developers can simply perform a pull, and you’re done!

So we promptly took the whole list of shell commands on the wiki page and converted them into sub commands. We modified the sub library to allow modification of environment variables in the current shell (something that we would like to contribute back into the main sub at some point), and pushed the whole finished set into a git repository. Oh, and we dubbed the command ‘zb’ (short for Zillow-Bootstrap):

In the beginning, we had 10 very simple commands. However, there’s a lot of complex workflows at Zillow that developers are happy to automate, and contribute those back when the mechanism is as easy as contributing to a git repository or starting a pull request. There’s too many commands to go over them all, but here’s some highlights:

• zb-search-repos : searches through our massive git hosts to find repositories
• zb-jira-create-tickets: reads a markdown file, and generates stories, tasks, and subtasks in Jira based off of the organization
• zb-pre : our one line magic bullet to start the main zillow server locally
• zb-createmavenproject: creates a maven project, in the zillow standard
• zb-se: configures the shell to work in a specific test environment

Remember, these are five out of the 50 commands that are available in zb. Many developers use these and several other zb commands every day.

One of the interesting things about zb is the fact that contributions to zb directly over the half year since it’s been implemented has exceeded 40 commands from about 10 developers, while contributions to the wiki page over two years were roughly 20, mainly from two developers. It seems that when there’s a direct place to put commands that you know are being shared in a reproducible manner and the upgrade path is simple, that’s when collaboration starts to really skyrocket.

Lessons Learned

So what lessons have we learned? Well, a few:

• An investment in even small internal things like shell scripts can have a large payoff. We no longer spend time re-automating even the smallest task, thanks to an easy share path.
• If you provide an easy, clear channel to provide tools, the developers will be more than happy to contribute their own to the channel.

Resources

You can find Zillow’s own branch of sub here:

https://github.com/zillow/sub