- Find a Real Estate Professional
- Realtors®
- Mortgage Lenders
- Home Improvement Pros
- Other Real Estate Services
- Review an Agent, Lender or Pro
- Marketing on Zillow
- Real Estate Agent Advertising
- Join the Professional Directory
- Popular
- Real Estate Market Reports
- More
Best Answer
- ConnieK_Oklahoma
- Contributions:2899
so here's a chart of our accuracy. (per my small...crude sample)
excuse the chart, too lazy to make it all pretty and as detailed as Pasa's charts.:)
so if you are a "glass half empty" person you could say that 36% of the time we are WAY off and 8% of the time too low.
if you are a glass half full then you could say that 55% of the time we are within the acceptable range for pricing.
excuse the chart, too lazy to make it all pretty and as detailed as Pasa's charts.:)
so if you are a "glass half empty" person you could say that 36% of the time we are WAY off and 8% of the time too low.
if you are a glass half full then you could say that 55% of the time we are within the acceptable range for pricing.
- ConnieK_ppm
- Contributions:97
I will provide more detail later. Just might be many days. Right now I have more to do than can be done. Won't forget
- Pasadenan
- Contributions:21466
I would like the kurtosis and skewness too...
Zillow's tolerance data indicates a positive kurtosis, but doesn't provide enough data to easily calculate it. Zillow skips the skew data, but I'm pretty sure that there is a skew in either case, but that Zillow's will be opposite that of listing CMA's.
Zillow's tolerance Standard Deviation runs around 13%, depending on the region.
It is interesting that the CMA's run about 8% high in addition to the 10% standard deviation though. So this means that generally the CMA's are no better than Zillow's estimates.
Zillow's tolerance data indicates a positive kurtosis, but doesn't provide enough data to easily calculate it. Zillow skips the skew data, but I'm pretty sure that there is a skew in either case, but that Zillow's will be opposite that of listing CMA's.
Zillow's tolerance Standard Deviation runs around 13%, depending on the region.
It is interesting that the CMA's run about 8% high in addition to the 10% standard deviation though. So this means that generally the CMA's are no better than Zillow's estimates.
- ConnieK_Oklahoma
- Contributions:2899
don't have time to do groupings...to see breakdown but ran a quick query on it. this is sales price/original list price. There was a LOT of garbage to throw out. There were 30 or so listings that clearly had one too many zeros in the original list- on those i used the last list price but ONLY if it was exactly 10% or 1% of original price- if there was any other number different other than too many zeros I threw it out.
houses under 30k thrown out. Pricing was not too bad
Average sales price was 92% of original list.
min was 24% max was 210% st dev was 10%
i think there are still bad entries represented. this was 17408 records.
i like the chart with the percentage of each grouping better, but since the average was 92% it looks like the trend is still acceptable, maybe a chart later. hectic week ahead.
houses under 30k thrown out. Pricing was not too bad
Average sales price was 92% of original list.
min was 24% max was 210% st dev was 10%
i think there are still bad entries represented. this was 17408 records.
i like the chart with the percentage of each grouping better, but since the average was 92% it looks like the trend is still acceptable, maybe a chart later. hectic week ahead.
- Mack McCoy
- Contributions:1117
Thanks, Pasadenan; I'll keep that in mind when squinting doesn't help!
- Pasadenan
- Contributions:21466
You don't have to squint... just open the image at the full uploaded resolution.
The web address is:
http://photos1.zillow.com/is/image/i0/i10/i8216/IS-9w7lv8ce89p.jpg
(Zillow won't let it be posted as a "link" as they automatically add in www. before "zillow", which makes the link not work). So you have to copy and past it to the browser address bar.
The web address is:
http://photos1.zillow.com/is/image/i0/i10/i8216/IS-9w7lv8ce89p.jpg
(Zillow won't let it be posted as a "link" as they automatically add in www. before "zillow", which makes the link not work). So you have to copy and past it to the browser address bar.
- Mack McCoy
- Contributions:1117
The MLS is not going to review CMAs, you can take that to the bank.
To answer the question, I would ignore the selling agent's CMA. If I thought those comps were of value, I would have used them in my own CMA, and if they hit the market or sold subsequent to my CMA, I would have already advised my client because I provide my sellers with market updates.
I'm squinting at Connie's chart and see that she was 10% or more off the sales price 15% of the time; Zillow claims that their median margin of error for the Seattle market is 9.1%, so I'd have to give Connie four and a half stars!
To answer the question, I would ignore the selling agent's CMA. If I thought those comps were of value, I would have used them in my own CMA, and if they hit the market or sold subsequent to my CMA, I would have already advised my client because I provide my sellers with market updates.
I'm squinting at Connie's chart and see that she was 10% or more off the sales price 15% of the time; Zillow claims that their median margin of error for the Seattle market is 9.1%, so I'd have to give Connie four and a half stars!
- ConnieK_ppm
- Contributions:97
I will repost after looking at the data better and tossing out garbage My thinking was that the set was so large the garbage wouldn't matter but that is likely not true. I grabbed all sold from 1/1/2011 to 12/29/2011 all active and all pending. They were in txt files it I imported them all to SQL server. So that it would be "easier" to do functions. However. Spitting back to excel is usually where I end up. I grabbed a LOT of data because I have many curiosities
It was over 25k records but each download was limited to 1500 I grabbed a lot of fields that nobody else will care about Wish I had included list agent and sell agent and ther offices. But it redoing it Fields are all about the property and list details
It was over 25k records but each download was limited to 1500 I grabbed a lot of fields that nobody else will care about Wish I had included list agent and sell agent and ther offices. But it redoing it Fields are all about the property and list details
- Pasadenan
- Contributions:21466
Still helpful as it is easy to convert...
But what I still want is sold price to INITIAL list price ratio distribution. We already know it will be closer to sold price after the "mark downs".
Then, the other question; does it follow a normal distribution curve? If so, what is the mean percentage difference from sold price, and the percent standard deviation of the mean? And is there some Skewness? And is there some Kurtosis? These should be easy to quantify if one uses the Excel frequency function with uniform "bins", and one just uses the average, stdev, skew, and kurt functions on the frequency data. Actually, one can just calculate these functions from the raw data without even using the frequency bins, but it would be nice to see a histogram with the curve fit shown on the same diagram.
So, the way I read the chart (with the legend correction stated), 62.8% of the homes sold for between 84% of final list price and full final list price. So up to 16% "off". That is an awfully wide bin for determining % standard deviation.
And 9.7% sold between 101% and 105% of final list price. Again, not real useful in determining the standard deviation of the mean.
But what I still want is sold price to INITIAL list price ratio distribution. We already know it will be closer to sold price after the "mark downs".
Then, the other question; does it follow a normal distribution curve? If so, what is the mean percentage difference from sold price, and the percent standard deviation of the mean? And is there some Skewness? And is there some Kurtosis? These should be easy to quantify if one uses the Excel frequency function with uniform "bins", and one just uses the average, stdev, skew, and kurt functions on the frequency data. Actually, one can just calculate these functions from the raw data without even using the frequency bins, but it would be nice to see a histogram with the curve fit shown on the same diagram.
So, the way I read the chart (with the legend correction stated), 62.8% of the homes sold for between 84% of final list price and full final list price. So up to 16% "off". That is an awfully wide bin for determining % standard deviation.
And 9.7% sold between 101% and 105% of final list price. Again, not real useful in determining the standard deviation of the mean.
- ConnieK_Oklahoma
- Contributions:2899
well....I just finished and realized...I screwed up and did the math backward. here's the pretty chart anyway. this is what percent is list price to final sales (almost meaningless). redoing the percentage is quick- it's one forumla- but it's the goofy way I wrote the script to group them that I thought was a one time thing and now I don't trust myself to edit it and redo another chart -maybe tomorrow after some sleep. at least the piece of pie for 100% would be the same. this is entire market, entire year.
- ConnieK_ppm
- Contributions:97
getting some numbers together now. suppose I'll log in as old profile to post it. (and change name back so references in old thread make sense).
- ConnieK_ppm
- Contributions:97
new profile...incase it wasn't obvious already.
- ConnieK_Oklahoma
- Contributions:2899
well i missed the weekend and didn't do it but will gather the additional info this week. I suspect this is already tracked and reported. I would think it would make great material for one of the many classes that you can attend for getting some fancy credentials behind your name.
I will go get some info later this week. Right now I'm working on killing this profile- deleting zip codes, changing avatar, changing email address...it's my way of deleting reviews. Oh...and you're welcome. I don't see I said that.
I will go get some info later this week. Right now I'm working on killing this profile- deleting zip codes, changing avatar, changing email address...it's my way of deleting reviews. Oh...and you're welcome. I don't see I said that.
- SoCal_Engr
- Contributions:5666
Many thanks to Connie for the effort. Seems that it would be child's play for NAR to turn a statistical eye to their MLS and perform the same. Any bets on whether or not they've already done so, and decided not to publish the information? (Yeah, a bit conspiracy-theorist of me, but...)
- ConnieK_Oklahoma
- Contributions:2899
I'll have a little time this weekend- will add another chart (don't let me forget) will do one with last list pirce vs sold price and whatever else seems of value at the moment.
-without REOs with REOs etc
another zip code
some other random sampling of housess..(think on this one a while.)
-without REOs with REOs etc
another zip code
some other random sampling of housess..(think on this one a while.)
- Pasadenan
- Contributions:21466
"would you have every REA run a Fibonacci Sequence on every sale in their market, and present some range to the clientele? You think Sellers would choose the top of the range and Buyers would lean toward the bottom? What would it help, or have helped?" -
Well, I'm sure you realize a Fibonacci Sequence doesn't relate to the issue; but if an agent is to represent me, I would want both a Standard Deviation of the mean of the difference of sold price to CMA estimate, as well as the mean difference, and whether high or low. And no, a seller will not always want high for listing nor low for offering. They might like the low for offering to submit with the offer indicating the comparables chosen; but one can sell faster if they price right to begin with. And it is much easier to set a pricing and offering strategy if one has a better clue what the real present market value is.
And since individual agents don't really do enough CMA's to have a good statistical tolerance range in a year, I would like to see the local MLS do statistical analysis on the CMA's or initial listing prices and publish the actual distribution of the deviance to sold price. I'm still thinking a Kurt function may help. But as the distribution is clearly not symmetrical, and spread out in the opposite direction as a log-normal distribution, it appears a different function should be used to curve fit the deviance from market price data.
With the statistical approach used by Zillow, it appears theirs may be closer to symmetrical, especially as their estimates are not used for setting listing prices. But having looked at Zestimate to Listing price differences for my city, the upper priced housing still tends to be estimated low and the lower priced housing still tends to be estimated high, thus I still believe there is some asymmetry in the tolerance data, and I really would like Zillow to provide more statistics on that for better curve fitting.
Well, I'm sure you realize a Fibonacci Sequence doesn't relate to the issue; but if an agent is to represent me, I would want both a Standard Deviation of the mean of the difference of sold price to CMA estimate, as well as the mean difference, and whether high or low. And no, a seller will not always want high for listing nor low for offering. They might like the low for offering to submit with the offer indicating the comparables chosen; but one can sell faster if they price right to begin with. And it is much easier to set a pricing and offering strategy if one has a better clue what the real present market value is.
And since individual agents don't really do enough CMA's to have a good statistical tolerance range in a year, I would like to see the local MLS do statistical analysis on the CMA's or initial listing prices and publish the actual distribution of the deviance to sold price. I'm still thinking a Kurt function may help. But as the distribution is clearly not symmetrical, and spread out in the opposite direction as a log-normal distribution, it appears a different function should be used to curve fit the deviance from market price data.
With the statistical approach used by Zillow, it appears theirs may be closer to symmetrical, especially as their estimates are not used for setting listing prices. But having looked at Zestimate to Listing price differences for my city, the upper priced housing still tends to be estimated low and the lower priced housing still tends to be estimated high, thus I still believe there is some asymmetry in the tolerance data, and I really would like Zillow to provide more statistics on that for better curve fitting.
- Hamp Yonce, "Zilluminati"
- Contributions:3463
Connie, great point about REO's. What percentage of your sample would you guess is REO? What percentage is true organic sales, undistressed?
- wetdawgs
- Contributions:26834
Belated thanks for pulling those numbers together Connie. They are very useful!
- ConnieK_Oklahoma
- Contributions:2899
i think it would be a more accurate representation if all REO listings were simply tossed out, only because those are priced by some opinion external to the listing agents CMA (at least in my experience) I know with the ones I have listed they determine the price before i turn in the CMA
- sunnyview
- Contributions:25139
Great chart Connie!
- SoCal_Engr
- Contributions:5666
@ Connie...
Thanks for the info and chart. It's actually in the range of expectations. Better yet, it's refreshing to see someone who's willing to turn a critical eye on their own efforts.
As for the Java, I've been doing it for way too long - more than happy to help if needed. Really not that bad. And, as with everything else, you can write "good code" and "bad code" with Java. Just no need to explicitly tear down memory, although you can force garbage collection if needed.
Cheers
Thanks for the info and chart. It's actually in the range of expectations. Better yet, it's refreshing to see someone who's willing to turn a critical eye on their own efforts.
As for the Java, I've been doing it for way too long - more than happy to help if needed. Really not that bad. And, as with everything else, you can write "good code" and "bad code" with Java. Just no need to explicitly tear down memory, although you can force garbage collection if needed.
Cheers
- Hamp Yonce, "Zilluminati"
- Contributions:3463
I'd like to see the same type of data from five, or six, years ago, and then, from 10 years ago. The art, or science, of predicting "fair market value" hasn't gotten any easier, lately.
That was good of you to do that work, Connie!
Does volatility make stats more valuable, or less valuable? I'd argue less, but I'm borderline mentally deficient.
Pasadenan, in a perfect world, would you have every REA run a Fibonacci Sequence on every sale in their market, and present some range to the clientele? You think Sellers would choose the top of the range and Buyers would lean toward the bottom? What would it help, or have helped?
That was good of you to do that work, Connie!
Does volatility make stats more valuable, or less valuable? I'd argue less, but I'm borderline mentally deficient.
Pasadenan, in a perfect world, would you have every REA run a Fibonacci Sequence on every sale in their market, and present some range to the clientele? You think Sellers would choose the top of the range and Buyers would lean toward the bottom? What would it help, or have helped?
- ConnieK_Oklahoma
- Contributions:2899
Maybe next week. I am having to learn java this week and while it is not so hard I sucessfully avoided it tor years and been stuck in my ways so there is a lot of time set aside for whining which leaves little time for this. But. I do plan to look at that
- Pasadenan
- Contributions:21466
Thanks for the Chart Connie!
Very informative. About where one might "guess", but something like this needs some empirical data and not just guesses.
Yes, I recognize it is only one case study and may not be transferable to other areas; but at least it is a starting point.
I would be very interested to see the revised numbers if one used the original list prices, as final list price is already substantially influenced by market response. (When you have the time...)
I also would be interested in any case study of selling agent CMA's, as I believe the distribution would be substantially different, with a lot more of them being "low".
It would also be nice if Zillow would publish separate tolerance of "high" verses "low", as it is not at all clear that the distribution follows a normal distribution pattern, and the few data points they give for their tolerance range indicates it is more peaked than a typical normal distribution as well as more spread out.
Very informative. About where one might "guess", but something like this needs some empirical data and not just guesses.
Yes, I recognize it is only one case study and may not be transferable to other areas; but at least it is a starting point.
I would be very interested to see the revised numbers if one used the original list prices, as final list price is already substantially influenced by market response. (When you have the time...)
I also would be interested in any case study of selling agent CMA's, as I believe the distribution would be substantially different, with a lot more of them being "low".
It would also be nice if Zillow would publish separate tolerance of "high" verses "low", as it is not at all clear that the distribution follows a normal distribution pattern, and the few data points they give for their tolerance range indicates it is more peaked than a typical normal distribution as well as more spread out.
- ConnieK_Oklahoma
- Contributions:2899
so I grabbed a little data from one zip code, everything sold in the last year (in the MLS)
Here's the details.
this zip code had 331 sales in the last year (defined as 12 months from today)
28 of the 331 listings sold for MORE than the orginal sales price with the highest sales price being 118% of the original list price. (i threw out ones that were clearly typo's like list price of $100 instead of $100,000.) - there were a total of 3 to throw out, they are not in the 331 number.
for those that sold above original list, the average was 104% the median was 102% of original list price. 6 of these were REOs
for those that sold AT or below ORIGINAL list price, the average was 94% of original sales price. with a median of 96%
there were 50 of the 331 listings that sold at 100%
there were 120 that sold LOWER than 95%
so...it seems our pricing as a group isn't all that better than zestimates.
i didn't compare original to last list price...too lazy.
i also didn't count how many were REOs in the under 95% group. and this was just one zip code. and didnt look for DOM trends, new construction, or other special details like type of financing. so it's not meant to be a full story with the data.
but it's good enough for me to say....as a collective we aren't all that much better at zestimating than zillow is. We're not that bad just not perfect either. it's too small of a sample of data to claim any degree of accuracy or explain it...that's not my point as much as to just say...zestimate + or - doesn't seem to be too far out of line of real world attempts at finding values either.
I am speaking as a collective to collective comparison and not any one house or person. there are areas where zestimates seem to be horribly bad or incredibly inflated-just like there are people that list for insanely off the mark prices (low or high)
Here's the details.
this zip code had 331 sales in the last year (defined as 12 months from today)
28 of the 331 listings sold for MORE than the orginal sales price with the highest sales price being 118% of the original list price. (i threw out ones that were clearly typo's like list price of $100 instead of $100,000.) - there were a total of 3 to throw out, they are not in the 331 number.
for those that sold above original list, the average was 104% the median was 102% of original list price. 6 of these were REOs
for those that sold AT or below ORIGINAL list price, the average was 94% of original sales price. with a median of 96%
there were 50 of the 331 listings that sold at 100%
there were 120 that sold LOWER than 95%
so...it seems our pricing as a group isn't all that better than zestimates.
i didn't compare original to last list price...too lazy.
i also didn't count how many were REOs in the under 95% group. and this was just one zip code. and didnt look for DOM trends, new construction, or other special details like type of financing. so it's not meant to be a full story with the data.
but it's good enough for me to say....as a collective we aren't all that much better at zestimating than zillow is. We're not that bad just not perfect either. it's too small of a sample of data to claim any degree of accuracy or explain it...that's not my point as much as to just say...zestimate + or - doesn't seem to be too far out of line of real world attempts at finding values either.
I am speaking as a collective to collective comparison and not any one house or person. there are areas where zestimates seem to be horribly bad or incredibly inflated-just like there are people that list for insanely off the mark prices (low or high)
- Rita A. Walker, "Rita Walker"
- Contributions:277
I have never had a selling agent ask for documents to support my list price.
On the other hand, I have had occasion to request the comps used to support the sellers price, when negotiating on behalf of my buyer.
CMAs can be worked to give most any answer you want, as the agent (or an appraiser) running the CMA has a .5 mile radius within which to find comps.
If the agent works within the subdivision or within surrounding subdivisions with comparable homes, the figures should be most accurate.
On the other hand, I have had occasion to request the comps used to support the sellers price, when negotiating on behalf of my buyer.
CMAs can be worked to give most any answer you want, as the agent (or an appraiser) running the CMA has a .5 mile radius within which to find comps.
If the agent works within the subdivision or within surrounding subdivisions with comparable homes, the figures should be most accurate.
- SoCal_Engr
- Contributions:5666
@ Hpvanc...
Appreciate the input. But, unfortunately, this is one topic where I truly believe only REA input will/should carry any weight.
I've had my own personal experience with CMAs, primarily on the selling side. However, my data points are very few compared with what REAs should have - and could likely be very skewed. The same should hold true for any other consumer - too few data points.
Of course, we could attempt to pool input from a large group of consumers. However, I'm fairly certain that the issue of only those with a strong viewpoint would respond - and that would color the data points as well.
Appreciate the input. But, unfortunately, this is one topic where I truly believe only REA input will/should carry any weight.
I've had my own personal experience with CMAs, primarily on the selling side. However, my data points are very few compared with what REAs should have - and could likely be very skewed. The same should hold true for any other consumer - too few data points.
Of course, we could attempt to pool input from a large group of consumers. However, I'm fairly certain that the issue of only those with a strong viewpoint would respond - and that would color the data points as well.
- hpvanc
- Contributions:2579
I have been hoping we would see something on this topic as well. From the consumer perspective here are my observations:
I know several people in different parts of the country that have apparently been able to get CMA's in their effort to sell that weren't close enough to reality to even get showings. All of them deny that they had any agent give them one that was substantially lower, could have been they did not bother to ask for one from an agent that did not tell them everything they wanted to hear in the initial conversation. All of these people still need to move for family/career/economic reasons and will eventually have to accept reality (sooner would be better than later but at least no one has divorced over it yet).
On the buyer side, it has been 2 or 3 years now, but I bought at the same time as several of my local co-workers. CMA's we were given by our agents were all over the board on properties we viewed based on eventual sales price (if they have sold). The only person in this group that claims (I haven't seen them to check them for accuracy, but do actually trust the person making the claim) accuracy of the CMA's is the one that picked an agent that is a full time electrical engineer (unfortunately licensed in the wrong state for me to use them).
I know several people in different parts of the country that have apparently been able to get CMA's in their effort to sell that weren't close enough to reality to even get showings. All of them deny that they had any agent give them one that was substantially lower, could have been they did not bother to ask for one from an agent that did not tell them everything they wanted to hear in the initial conversation. All of these people still need to move for family/career/economic reasons and will eventually have to accept reality (sooner would be better than later but at least no one has divorced over it yet).
On the buyer side, it has been 2 or 3 years now, but I bought at the same time as several of my local co-workers. CMA's we were given by our agents were all over the board on properties we viewed based on eventual sales price (if they have sold). The only person in this group that claims (I haven't seen them to check them for accuracy, but do actually trust the person making the claim) accuracy of the CMA's is the one that picked an agent that is a full time electrical engineer (unfortunately licensed in the wrong state for me to use them).
- Hamp Yonce, "Zilluminati"
- Contributions:3463
I'm also still in the market for a grammar agent, or grammar algorithm.
There hard to find!
There hard to find!
- wetdawgs
- Contributions:26834
CMAs are opinions, Zestimates are opinions, appraisals are opinions. However, they all do have numbers attached so all could have accuracy calculated. Between you, me and crickets (and all other readers), I can read between the lines.
Awesome lunar eclipse this morning!
Awesome lunar eclipse this morning!
Question for REAs regarding list/offer price and CMAs...
I'm curious about REA experience in the spread of CMA values, real world vice AVM.
How often, as the listing REA, did you have other REAs bring in buyers and support an offer with a CMA >5% below the asking price (assuming it was based on a CMA and not the owner's opinion).
How often, as the buy-side REA, have you run a CMA and determined that the asking price is >5% greater than what you believe it should be?
Stating a discriminatory preference in an advertisement for housing is illegal. If you think this content is discriminatory or otherwise inappropriate and feel it should be removed from Zillow, please let us know by completing the information above.
We will review this content. Thanks for helping make the site more useful to everyone. To learn more, read Zillow's Good Neighbor Policy.