Menu

In the case the Zestimate, it’s a lot.  Lloyd’s blog post Thursday announced that we officially came out of beta in concert with expanding our database and improving our accuracy.  I wanted to offer a little more perspective on what goes into your Zestimate now.

When we first launched in 2006, we set out to do something no one else was doing – putting a value on millions of homes in America, for free.   But not just any value.  We wanted  a value that we could update often (so consumers would always have a fresh valuation) and one that we could compute historically for all homes (so that consumers would have a sense for how prices changed over time).  So, we spent some time looking at how others had approached this problem in the past, decided that none of these approaches would suffice for the accuracy, speed and scope of the problem we were trying to solve, and set about inventing a new way to do daily home valuations on millions of homes.

To get started, we borrowed heavily from the fields of machine learning, artificial intelligence, automated knowledge discovery and statistics, and from applications of these fields to the areas of genomic research, pattern detection for national security problems, computational biology, and automated equity trading.  The result was an algorithm that was able to take records of recent sales and incorporate billions of unique home details (which varied in how these were defined and recorded by each county) in order to compute current Zestimate valuations with a pretty reasonable rate of accuracy.   And do this all every night, for 40 million homes.

After we launched, we got the benefit of real-time input as we were constantly adding new information and data sources along with sales transactions.  This helped expand our database and identify issues within markets.  As we learned more about market variances, we knew we needed a more sophisticated algorithm that could digest the amount of intelligence we were gathering.  In addition, we realized that some of our data sources were more limited than others and we wanted to offer owners and real estate agents the ability to update home facts since they have the most intimate knowledge of the property.   We launched the ability to edit home facts in December of 2006 and soon after began constructing our new algorithm.

In the early stages of this construction task, our statisticians and analysts team spent extensive time looking at where the initial algorithm was challenged.  For example, in California, our models didn’t always fully counteract the effects of Proposition 13 on tax assessments.   In New York City, many homes don’t have the number of bedrooms and bathrooms listed in public data.  In Chicago, you often only get number of bathrooms and square footage.

After the statisticians sketched out a prototype of a new algorithm designed to remedy or work around the many market variances, our talented software engineers contributed their own ideas of how to improve the algorithm, worked tirelessly to implement the whole thing, developed solutions to fill gaps that emerged during implementation and, in general, made the final product substantially better than where it started.

Concurrent to the effort on the valuation front, our data team  was furiously working on bringing new data sources online with the aim of getting millions more homes into our data base of homes and to permit Zestimate valuations on  homes already in our database but for which we were unable to value previously because of lack of sales data.

The result of these combined efforts over the last year is what you find today in your Zestimate, which depending on where you live, is up to 30% more accurate than before.  How’d we do this?  Our new algorithm has 20 times more statistical models than our original one, running approximately 334,000 models every day.  For our release last week, this involved calculating Zestimates over the last 12 years on our entire database of 80 million homes – all told, we churned 4 terabytes of data using 67 million statistical models to calculate 13 billion Zestimates.    That’s a whole lot of Zestimates.  So many, in fact, that we turned to our friends up the hill here in Seattle to use 500 computing nodes at Amazon Web Services in order to compute all of the historical Zestimates.   And, thanks to the efforts of the small cadre of people that bring new data into Zillow, we’ve got these better Zestimates on 14 million more Zestimates than before plus data on still another 10 million more homes.

The significant increase in models has allowed us to get much more granular and take into account more of those market variances when calculating a Zestimate – including looking at data at a more local level.   As Lloyd pointed out, we are also now factoring in edited home facts, and in our current Zestimates, we’ve included 16 million new data points provided to us by users that have supported the accuracy gain.  What’s more, the new Zillow algorithm is self-learning so as we incorporate more data – it becomes smarter.  The result is a more robust and accurate database that, in turn, helps consumers become smarter about real estate.

All of this is not to say that we don’t still get it wrong sometimes.  We certainly do, and sometimes in a big way.  It is to say, however, that today we’ve taken a big step forward in accuracy and we’re only going to be taking more, smaller, quicker steps in the future.  This couldn’t have been done without the hard work and unrelenting drive for perfection by the team at Zillow.  There are a lot of bloodshot and tired eyes in getting this live for the world to see…and it was well worth it.  And, there’s more to come.

Dr. Stan Humphries is a real estate economist and real estate expert for Zillow. Stan is in charge of the data and analytics team at Zillow, which develops housing market data for most major metropolitan statistical areas in the U.S., and provides economic research for current real estate market conditions. He helped create the algorithms for the popular Zestimate® home value and the Zillow Home Value Index (ZHVI).

About the Author

Stan is Zillow's Chief Economist. To learn more about Stan, click here.

  • Jim Patterson

    In regard to your article on the new algorithm being used by Zillow, if there is smoke there is fire. It is evident by all the questions raised since the introduction of your new algoritm that there is a problem. My experience is that our home, 16012 N. Cerro Alto, Fountain Hills, AZ 85268 had a zestimate of $1,000,000 + for the past several years. Now it is down to $746,000 and and the owners estimate is listed at $1,271,794 based on the many update and improvements we made to our home. All the homes on this section of our street have been listed at $1,000,000+. In addition, all the homes that have sold over the past few years have sold for more than the Zillow estimate at the time. Our home is slighty smaller than our neighbers, anad that has been shown in the estimates. However, our home suffered the biggest lost under your new algorithm, and that doesn’t make sense. We have more property that all but one and all were buuilt at at about the same time. In addition, our tax evaluations are all is the same ballpark.

    In addition, the original zestimate was in line with the published yearly sales value increases for our area. Now your estimate is even $50k below the tax assessment, which is a lower figure that the actual value.

    Also, the fact that you changed historical estimates make me look like a nut because my owners estimate, based on the additions I added to the estimate at that time,is twice the Zillow estimate.

    Since Zillow has had such an impact with so many home owners, I would hope that they review the current algorithm and restore their credibility.

  • http://www.mattamyhomes.com/corporate alexander

    Interesting post.

    Certainly something to consider, including the above comment.

  • http://www.lonestarfinancing.com/mortgage/texas/austin/ Austin Mortgage Broker

    I read something similar in a mortgage blog today and hope Zillow can overcome these issues in the short term.

  • Oshun

    People obviously do not bother to read anything…Zillow NEVER said they were providing accurate MARKET VALUES of residential houses. In fact, an accurate market value is totally impossible to predict since the market value of a house is the price point at which a seller will sell and a buyer will buy! They never said use their site to value your residence…The site has a lot of data that you can use ALONG with other tools to analyze market value trends in local areas without bothering a real estate agent for COMPS. I’m sorry, how is a computer supposed to know that your house has hardwood floors, gold door knobs, and crystal chandeliers? It’s not! If you dont like zillow, go bug a real estate agent or appraiser. I have followed residences in California, that I purchased almost 10 years ago and can tell you that the historical sales data and tax data alone provides a great view that I did not have before zillow!

  • http://www.lakeshoredreams.com Minnesota & Wisconsin Lake Property

    i found this article very informative specailly comments of Jim are worth reading

  • Pingback: The Odysseus Medal competition — Voting for the People’s Choice Award is open | BloodhoundBlog: Real estate marketing and technology blog | Realtors and real estate, mortgages, lending, investments

  • http://www.PinnacleHouses.com Vincent Spoto

    The most unsettling issue with your algorithms is that there is no confidence score given with the estimate. One simply has no idea if the value given is likely to be on target or off by ten, twenty percent or more.

    Since you looked into pattern detection you certainly realize that many of those algorithms determine a result and a confidence score of the result.

    Most of your users are looking for the value for a specific property. Without knowing the confidence of the value given your estimate can never be relied upon. Perhaps after your initial estimate the confidence value can be increased as more data is added to your knowledge base by the user.

  • http://www.zillow.com/profile/DavidG David Gibbons

    Vincent –

    Good feedback; it’s a topic we’ve debated at length. Confidence is actually communicated with every home’s Zestimate value; it is simply embedded in the value range. Homes where our confidence in Zestimate accuracy is high have a MUCH narrower value range (relative to their Zestimate values) than those where confidence is low. We have in fact usability tested this and we found that interpreting a pure “confidence score” is a challenge for most people yet almost everyone can accurately interpret a value range – and the range obviously communicates more information than just confidence.

    As for your next suggestion; we’re already there. If owners post additional information about their homes, we will consider that data when we recalculate Zestimates. And the Zestimate value range will also reduce in response to the added certainty provided the owners’ additional data.

  • http://Zillow.com Michael Nocera

    Tring to list my house with Zillow.com . I am a single family home owner . Tring to sell my home . I do not have a company name . I am retire and live in the Villages Fl. 32162 . How can I list my property with Zillow .com
    ? . Could you help me .
    Michael Nocera

  • http://www.zillow.com David G from Zillow.com

    Michael –

    It’s both free and easy for you to post your home for sale on Zillow. Just go to this page and click on “Owners start here”
    http://www.zillow.com/postings/Postings.htm?s_cid=fsb-site-topnavpost

  • http://www.georgiamls.com/agentsite/search/propertyDetail.cfm?SiteID=DUFFYRHONDA&LN=2318533 Deborah Curlette

    Hello Drew,

    I have completed an Excel chart which includes information such as sq footage, amount of property taxes paid, bedrooms and baths, whether the house has a backyard or not, whether the house is attached to another one or not, whether the house is on a slab or has a basement. All houses in my sub were built in a one year period. I realize you use a number of different zestimate models. However, facts and numbers should also count for something. My information came from public information, including yours. If you email me drdebc@bellsouth.net, I will be happy to send as an attachment, the excel chart that includes this research.

  • Master Shake

    Let’s face it: a property is “worth” whatever a buyer is willing to pay for it. No matter how many sophisticated statistical models and algorithms the Zillow geniuses throw at real estate valuation, it all boils down to a prediction problem… and prediction is subject to serious limitations. Prediction is hard to do, eapecially the future. A comparatively easy problem, stock market trends, has never been solved by statisticians, economists and computer scientists so don’t expect much from Zillow.

    It appears that the Zestimate is just a massive data mining operation. Since the methods are completely opaque, it’s hard to test them. I assume they’re primarily based on some sort of supervised learning method. Stan Humphries seems to think that using more complex and sophisticated methods will result in better estimates but they’re unlikely to ever be any good (especially at the margins). I’m sure Dr. Humphries is well aware of the variance-bias tradeoff and the underlying assumptions these models are built on. To be accurate, the methods would somehow have to model human psychology and behavior, socio-economics, micro and macro-economics, weather, gentrification, the price of raw materials and energy, speculative behavior and mass-hysteria, greed, and the host of other factors that (directly or indirectly) enter into the equation(s). It not all about what the neighbor was stupid, or smart, enough to pay for his house or the number of bathrooms and square feet. Besides, complex machine-learning methods tend to be no better at prediction than simple ones. I’d be shocked if the Zestimate did better than a simple linear regression on all these data. At least the linear model would be more robust and stable.

    The one saving grace would be if the Zestimate became part of the valuation process itself. In this case, appraisals would be based on the Zestimates, which would then, magically, become much more accurate. Did somebody say feedback loop?

  • Master Shake

    Oshun writes: “People obviously do not bother to read anything…Zillow NEVER said they were providing accurate MARKET VALUES of residential houses.”

    In fact Zillow DOES claim that it’s providing a MARKET VALUE: “The Zestimate (pronounced ZEST-ti-met, rhymes with estimate) home valuation is Zillow’s estimated market value, computed using a proprietary formula.”

    http://www.zillow.com/howto/Zestimate.htm

    I guess the point of contention is the (purposely vague?) definition of “accurate”.

    Frankly, I don’t see how any amalgam of statistical formulae and algorithms could adequately valuate property. Economists can’t even do it in aggregate. As the joke goes: “Economists have forecasted 9 out of the last 5 recessions.”

    I actually admire what the eggheads at Zillow are doing. It’s fascinating research that should, perhaps, be relegated to academia and obscure economics journals.

  • Donna

    A lot of people over at the zillow message boards are up in arms about a very recent drop in home values.
    I checked mine and OMG.. It dropped 123k from a year ago. That is over 25 % of the value of my home that I purchased in 2005 with every penny of savings i could muster.
    I checked around all of the other websites and either they get their bad information at the same place or crap!…. They’re all correctly estimating my home at this new low price.

    This might be something with the banks and the term we are all hearing on tv these past few weeks… “The new economy” Americans may well be looking at a lower standard of living.. Lower savings, lower chance of getting into your first home or a new home loan. Lower salary, lower job function.. you name it. Our “new economy” is lower!

    My retirement age will be significantly higher and my standard of living at that time will be significantly “lower”, as will my childrens educational choices.

    SO.. all of the sudden, Im depressed. Someone tell me my thoughts are way off track … please.

  • Uno Bloom

    The biggest problem with Zillow is that algorithms can NEVER accurately estimate the intangible value of school systems, location, and all the other unquantifiable elements that can affect price. Zillow states that their estimates are a starting point. They do not want to accept the fact that the majority of people will look at the Zillow zestimate and view it as the starting price, and not the starting point of valuation. Traditional pricing in virtually all markets start high and go down, not the other way around. The old adage in sales is, “you can always go down in price but you can never go up.” Zillow is collecting advertising income at the expense of home sellers by forcing the seller to justify a higher sticker price.

    I cannot block or erase the zestimate on my house even though it is over $100,000 less than the true estimated value of my house. That is bordering on price fixing and extortion. I don’t see how Zillow can legally fix a home’s price utilizing a created perceptual authority on their behalf based on algorithms and other mathematical formulas when it has no specific knowledge regarding the intangibles that make up individual markets divided by school district lines,county lines, city limits, location, etc.
    The ability to reveal or block zestimates should be left in the hands of the homeowner who researches and intimately knows their area market. Zillow’s owner estimates do not allow the potential home owner/seller to value the intangibles, i.e. to sell the sizzle that goes with the steak in any given market. Zillow may be facing a number of lawsuits due to fixing grossly inaccurate price points that will cause sellers to never have contact with buyers who may have been interested in a particular property but fail contact the owner due to the large price difference between Zillow and Owner estimates.

  • Mike

    Your anal, uno bloom, why don’t you go try block all other forms of human progress because they cause you a minor inconvenience. Go ahead bring it to a real life judge so he can tell you how stupid you are.

    If zillow winds up causing a market effect that lowers the prices of homes, thats a good thing for buyers, considering the ridiculous appreciation we have had. If I was buying a home who cares if I pay 100,000 less. The value of the area you live in is based on another set of human perceptions , which may or may not be true. Maybe they should be challenged.

    Go find something else to worry about whiner. Like the 2 billion people on earth that live in shacks and mud, straw huts.

    I hope Zillow crashes the real estate market. At least they put more work into their site, they you did just sitting back and watching people bid your home up.

    GO ZILLOW!!!!!!!!!!!!!!!

    POP THE HOUSING BUBBLE!!!!!!!!

  • John

    The houses in our neighborhood are so completely skewed (up or down), with many being completely opposite of what they should be, based on features, upgrades, how well they are maintained, etc. This website is so unreliable and annoying, it’s simply not worth using.

  • tom

    I looked up my home on your site, and though you show the right home, you list as 1000 sq ft 2bdrm, 1 ba. it is 1800 sq ft. 4 bdrm, 2ba, with a detached garage and granny unit. I would like to have the info corrected. How do I go about it?

  • http://auto-loan.xetisa.ru/ BekKerlatte

    great domain name for blog like this)))
    ————————
    sponsor: http://auto-car-loan.xetisa.ru/

  • Chuck Jolley

    “This website is so unreliable and annoying, it’s simply not worth using.”

    Except of course that once your house is listed you can never remove it.

You also might like...

Carrousel-Option1_2014_Q3_Zillow_b_01

Even With More Homes for Sale, Low-Priced Homes Tough to Find

11-18-14 954AM

30-Year Fixed Mortgage Rate Falls Further, Below 4% for Seventh Straight Week

11-11-14 945AM

30-Year Fixed Mortgage Rate Firmly Below 4 Percent for the Sixth Week in a Row

11-04-14 951AM

30-Year Fixed Mortgage Rates Spike on News of Stimulus Program Ending, Flatten Out by Week’s End

Subscribe for Zillow Blog updates

We will not rent, share or spam your account, ever. Please read and review our privacy policy.

You can also stay updated by following us below

instagram googleplus pinterest