If you’ve been following trends in software development, you’ve probably heard about continuous delivery and continuous deployment. Depending on your background, you may be a practitioner or they may sound like buzzwords. Although somewhat new to the software industry, they’re actually based on ideas that have been around for decades. In fact, Toyota was one of the first companies to put the ideas that underpin continuous delivery and deployment into action. In particular, Toyota focused on reducing inventory. Not only is inventory an unproductive asset, but it’s expensive to store, can go obsolete, get damaged, and needs someone to manage it. This motivated Toyota to strive to reduce inventory including work in progress. Not only did this make them more efficient but it allowed them to be more nimble to changing requirements and consumer demand. Today these practices are emulated at other car manufacturers and in other industries.

So how does this relate to a team that does research and writes code? We don’t have warehouses full of cars and parts to build them; we have software. We can think of software that hasn’t shipped to customers as our inventory. Why is it important to reduce our inventory? Here’s an example to illustrate: let’s assume it takes a year for us to add and ship a new feature in our software. If we imagine that our new feature is worth $10 million dollars a year in revenue, then we are missing out on that entire amount each year by not shipping.  Now let’s assume that we ship half of the feature in 6 months. By shipping sooner, we could see $2.5 million in revenue for the first year. After two years, we’d have $12.5 million instead of $10 million, which is an increase of 25%. That’s big and for most companies that makes shipping software faster a worthwhile goal.

The benefits seems clear, so how do we start practicing continuous delivery and deployment? The first step is to identify the path to shipping to your users. Once you know the path, you can start to identify the parts of the process that slow down the entire process. Steps that limit the speed of releases are on what’s known as the critical path. It’s important to distinguish what steps are on the critical path because only reducing the time to complete those steps will actually decrease the amount of time required for a release.

When we did this exercise to identify parts of the Zestimate release process, we saw that testing and validation required a significant amount of time. To reduce this time, we decided to invest heavily in test automation. We did this by building infrastructure to automate the running of tests. Now, whenever we push code to our internal repository, a build is automatically created and it runs unit tests. Immediately afterward, the code is packaged as a versioned artifact and uploaded to an artifact repository. If you aren’t familiar with artifacts, they are the output of builds and contain anything that gets reused in the future. For example, they can be Python eggs, Java JARs, ZIP files, and many other formats.

Once we have artifacts, we can automate deployment. Once again, this is where Zillow already has created infrastructure. We’ve adapted the current deployment tools for use with Amazon Web Services and EMR (Elastic MapReduce). This gives us the ability to automatically test our code in an environment that mirrors production. We use smaller datasets that are samples of the data we use to calculate Zestimates for about 100 millions homes. With smaller amounts of data, we’re able to run integration and regression tests more quickly and get feedback to our data scientists and engineers faster. Once a change has passed all of the tests, we are confident in the quality of that change and can deploy it to production. Here is a diagram of that flow:

In summary, we’re investing in our release pipeline to streamline releases with continuous delivery and deployment. As a result, our team can move faster, try new ideas more easily, and get feedback sooner. We no longer have research and development sitting in inventory while it waits to be released. We can ship software as it’s ready and provide value to our users right away.