patrick.net

OK Patrick, I finally played around with the data a little. It isn't going to reveal anything new, but it is sort of fun to fart around with the data & see what can be determined (or more importantly, what CAN'T be determined) from it.

So, first of all it is a good idea to look at the distribution of the data sets and get a feel for them. At first glance they might look like moderately-skewed normal distributions with some really high outliers. Well, that isn't the case, and they are VERY much log-normally distributed. The top of the plots shows a bunch of stuff (confidence interval of mean, quantile bars, some other stuff we won't worry about), but most important are the clouds of black dots. Those are the outliers...as-calculated for a normally distributed data set. This is not one, so they aren't the actual outliers.

So, I took the log10 of price & rent values and plotted the distribution of those, since they are log-normally distributed. When you do that, suddenly your data set will give a far better resemblance to a normal distribution. This is confirmed by the fact that a normal distribution line is fit to the data and that it fits very well. The goodness of fit part might seem a little confusing or contradictory, and that seems to be an issue in this software (SAS JMP 9.0). Anyway, if the "Prob>D" value is small & has a "*" next to it, then the fit line is very well suited to the data.

Now we can select the outliers and exclude them from the analysis. Since the two data sets do overlap, it makes things a little messy and with the new mean/stdev it looks like there are still outliers. Those are ignored at this point since we don't want to go mixing variables up during sorting.

So, now it is time to plot something that probably look familiar in this thread. Rent vs. Price. Is there a statistically significant link between them?

NO! Here is a brief explanation of what you are looking at.

- The magenta ellipse contains 95% of the data points.
- The blue ellipse contains 50% of the data points. These are just to give a feel for where the points mostly lie since there is this giant cloud.

- The red stuff pertains to a linear fit to the data.
- The green stuff pertains to a log(price) fit to the data.

- The darker, narrow shaded regions by the lines are the 95% confidence-of-fit intervals. These basically say, "For the true population of sales prices and rents, we are confident that the fit line will be within this region 95% of the time. The other 5% of the time, we don't know where it will be."
- The lighter, wide shaded regions are the 95% confidence intervals for the correlation between price and rent. These basically say, "For a given sales price, we are confident that the true population of rent costs lie within this colored band 95% of the time. The other 5% of the time, we don't know what they will be."

Discussion:
Being that the CIs for the price-rent correlation are gigantic, we can basically infer that there is no really solid link between prices and rent nationwide, at least given a sampling of the averages of both. I added the log(price) fit line in there because my intuition tells me that something like that is at play. People are willing to may more per month as "owners" than they are as "renters." We see this all over the Bay Area. Low-tier properties rent for cash flow-positive amounts, while higher-end areas rent for a lot less than it would cost to buy with a mortgage. Then again, that is super location-specific, and perhaps other places don't follow this. According to the limited data set, it varies wildly across the US since no fit line really has any significance here. There just isn't a correlation between price & rent, across the US, if we are using the averages of both as the inputs.

Now, I would like to re-run this using the MEDIAN of both to see if that can tease any correlation out. Averages are flawed for these types of analysis for reasons that we have all read on here before.

If nothing else, just look at the r^2 values. They are pitifully low, which is really all the information anyone needs to know that the fit is really meaningless! There isn't a fit, just a line drawn over a cloud of uncorrelated data!

So, Patrick:
a) Can we get the same data, but with the medians of both?
b) Would it be possible to have an additional column with the 2-letter state code corresponding to the zip code?
c) Do you have some monstrous database with data for every rent listing and recorded sale that you pull this from?

EDIT:

Also, the log-distribution of prices & rents makes perfect sense. People are generally seeking low rents and low house prices, and most market activity falls around these "affordable" levels. There is VERY little available below this since people are seeking the low-end to begin with. Then you see the long tail tapering off into the higher & higher price points. That also makes perfect sense since there are a number of people out there that can afford more expensive stuff. As prices go up, the number of participants goes down, and you see the tapering of prices and rents as they increase. I guess I am a huge nerd, but I love it when data agrees with intuition, and particularly when it seems to explain human behavior.

Also, trying to plot price, rent or price/rent versus zip code isn't useful at all. Zip codes are nominal data, while price and rent are continuous. They can be analysed, and I tossed them into JMP, which treats nominal-X axis + continuous-Y axis data as a One-Way analysis. Nothing useful popped out of that, which is sort of to be expected. It "looks" like there is a bunch of expensive real estate on the west coast, but no statistically significant fits could be applied to the data set to back that up.

I took statistics in German in Munich and didn't really absorb much.

bmwman91 says

So, Patrick:
a) Can we get the same data, but with the medians of both?
b) Would it be possible to have an additional column with the 2-letter state code corresponding to the zip code?
c) Do you have some monstrous database with data for every rent listing and recorded sale that you pull this from?

I don't know an easy way to get medians from my database.

I could pretty easily put the 2-letter state code because I can just join with my zip code table.

Yes, I have a monstrous database with data for every rent and asking price from Craigslist for about a year.

SFace says

Then there is still the issue of median rent and median sale is not the same thing.

This is true. Really, it seems like the only proper data set to work with would be one comprised of raw sales prices and rent values. Sampling methodology is a huge factor, and unless EVERY single sale & rental are taken, all sorts of sampling-related issues could mask trends and cause false ones to appear.

I have the x64 version of JMP & 16GB of RAM...I probably COULD work on the entire set of raw data. My gut feeling is that there is going to be some sort of correlation between price and rent without the masking effects of averaging or median filtering. Oh yeah, confirmation bias...about that, lol.

I could pretty easily put the 2-letter state code because I can just join with my zip code table.

Yes, I have a monstrous database with data for every rent and asking price from Craigslist for about a year.

Shoot me an email if you want to see what we can glean from the data. I know that this is sort of Patrick-proprietary stuff since you make a living off the site, so maybe we could make something interesting to share with readers without giving everyone all of your data.

JMP can access databases and stuff, so we wouldn't necessarily need to email monstrous text files.

Another complicating factor is that I don't actually have sales prices, just asking prices. Wish I did.

Asking rents are less of a problem, because the landlord usually gets exactly what he's asking.

bmwman91 says

Shoot me an email if you want to see what we can glean from the data. I know that this is sort of Patrick-proprietary stuff since you make a living off the site, so maybe we could make something interesting to share with readers without giving everyone all of your data.

OK, I'll mail you.

In my area I would LOVE to have the number of homes, apparts, commer, and land held by the same persons and groups. And how many of the individuals that hold rentals are also members of groups that hold rentals ..... and their financial history (forclosure, BR, ShortSales, ect) .... just a dream, I know.

Personally I think that this sort of information would be very useful. The other graph that I think would be great is rent vs the tenant is likely/not likely to destroy your investment property/not pay the rent graph.

Or

Price vs rental rate as it's no point buying a investment if you can't rent it out.