patrick.net

 
  forgot password?   register

#housing #investing #politics more»
736,951 comments in 75,807 posts by 10,913 registered users, 2 online now: FortWayne, Strategist

new post

Aggregating Real Estate Searches

By Patrick   2012 Aug 27, 7:44am   2,673 views   13 comments   watch (0)   quote      

While each real estate listing site hates all the others with a passion, and deems it a violation of their "Terms of Use" to copy and aggregate their listings, none of them can stop me from linking to all of their sites from one page.

And why would they even want to? If I drive traffic to their site from Patrick.net, they'll probably be happy.

So... I put up a little service on the Patrick.net home page that takes a zip code and a number of bedrooms, and generates search links for the major listing sites.

This saves buyers time, because they don't have to manually enter their search zip code and BR on each site, but only once instead.

The next improvement would be to rank the listing sites so that the best one for any particular zip code is on top. And to come up with a much more complete list of listing sites.

What else could it do to help buyers navigate the thicket of listings and get a good deal?

#housing

Comments 1-13 of 13     Last »

1   Patrick   1851/1851 = 100% civil   2012 Aug 27, 8:09am  ↑ like   ↓ dislike   quote    

I see that I have to find the best real estate search sites for each zip code, and they will probably be very different. There are more than 33,000 zip codes, so it's going to be a lot of work. Any advice on automating it a bit?

Also, how can I let users vote on which RE search sites are the best? Maybe put up/down arrows next to the links, or just track clicks on those links?

2   PockyClipsNow   60/60 = 100% civil   2012 Aug 27, 8:31am  ↑ like   ↓ dislike   quote    

the MLS sites go not on zip, nor county, but based on MLS. Each MLS can overlap into multiple county,zip, probably not state though.

Then you have overlapping craiglist 'counties'.

Many sites ignore whole areas of the country (redfin).

In some small areas you almost have to use the MLS sites search engine - a few wont even let realtor.com display thier listings (even though THATS THIER SITE, they wanna hog listings/traffic).

Then there is FSBO - CL, local newspaper sites,plus ebay, plus 100 other 'sell it here' sites.

To get max value of your time, start in large metro areas in CA as that will iron out any issues - CA is so large - we have all the above issues. Some entire states have only one MLS, one zip, one county, and maybe only one local newspaper.

I am imaginging this works like Kayak.com ( a nice site, oddly weird in frames due to similar legal issues I suppose - its like stuck in 1995 internet)

3   Patrick   1851/1851 = 100% civil   2012 Aug 27, 8:53am  ↑ like   ↓ dislike   quote    

PockyClipsNow says

the MLS sites go not on zip, nor county, but based on MLS. Each MLS can overlap into multiple county,zip, probably not state though.

I like zip though, since it's so brain-dead simple. The question becomes how to map the various sites to the zip codes that they cover.

PockyClipsNow says

Then you have overlapping craiglist 'counties'.

Yes, CL is a problem. Not sure how to map zip codes to their neighborhoods yet.

PockyClipsNow says

I am imaginging this works like Kayak.com ( a nice site, oddly weird in frames due to similar legal issues I suppose...)

Yes, exactly. The legal restrictions of the Terms of Use of travel sites and real estate sites all prohibit direct aggregation.

4   PockyClipsNow   60/60 = 100% civil   2012 Aug 27, 8:57am  ↑ like   ↓ dislike   quote    

Its a good idea for sure.

I guess you could start by just coding to search ALL search sites that you can, THEN maybe only show pop ups that have >0 results. Order by importance the sites with the most results - you probably cant show more than 20 results - maybe you could.

Thus maybe little or no filters. Might need to do a pre search on the server, then only launch popup window/frame where result >0.

5   PockyClipsNow   60/60 = 100% civil   2012 Aug 27, 8:58am  ↑ like   ↓ dislike   quote    

To map CL counties to zip codes you simply need a zip code table with a 'CL County' field/url to use per zip.

6   Patrick   1851/1851 = 100% civil   2012 Aug 27, 9:00am  ↑ like   ↓ dislike   quote    

But CL doesn't even use counties, do they?

They just seem to have their own weird abbreviations for everyplace.

7   PockyClipsNow   60/60 = 100% civil   2012 Aug 27, 9:35am  ↑ like   ↓ dislike   quote    

yes im calling them 'craigslist counties'. they are not counties or even groups of counties.
there are over 400 I think.

You will simply have to map one zip code to a CL search area manually.

8   Patrick   1851/1851 = 100% civil   2012 Aug 27, 10:53am  ↑ like   ↓ dislike   quote    

400 is OK, I can do that. But I suspect there are more than that.

Added a new feature: now if you do a search, it should set a cookie that remembers your search and fills in that zip code and number of BR automatically the next time you load the page.

9   LarryPatrickMaloney   5/5 = 100% civil   2012 Aug 29, 4:34pm  ↑ like   ↓ dislike   quote    

Good patrick!

One of the annoying thing about surfing real estate in CraigsList, are all the MobileHome postings. If you can filter that out, I would use it everyday.

10   Patrick   1851/1851 = 100% civil   2012 Aug 30, 2:33am  ↑ like   ↓ dislike   quote    

Thanks for the tip!

Craigslist has been actively suing websites that do any kind of filtering or mapping of Craigslist results, so I think I'm limited to creating only the search links that Craigslist uses, but I used to filter out mobile homes and could perhaps make some kind of software that lets you do the filtering in your own browser.

PockyClipsNow says

yes im calling them 'craigslist counties'. they are not counties or even groups of counties.
there are over 400 I think.

There were 934 of them!

Got a decent mapping of zip codes to Craigslist areas working now! I don't know of anywhere else you can do that, so that's some value to search added already.

11   FunTime     2012 Aug 30, 5:12am  ↑ like   ↓ dislike   quote    

I typed in my zip code, 94107, and got weird results at craigslist, homes in Portland OR, Florida, Austin TX, maybe that's just how craigslist works?

Also, got a no results at forsalebyowner, which might just be the accurate result.

12   PockyClipsNow   60/60 = 100% civil   2012 Aug 30, 5:21am  ↑ like   ↓ dislike   quote    

CL trys hard to make sure no one aggregates them. Really its an awsome site. Craig single handedly destroyed the newspaper classified industry. forever. Those guys had a monopoly for like 500 years. pretty amazing yet simple.

13   Patrick   1851/1851 = 100% civil   2012 Aug 30, 6:38am  ↑ like   ↓ dislike   quote    

FunTime says

I typed in my zip code, 94107, and got weird results at craigslist, homes in Portland OR, Florida, Austin TX, maybe that's just how craigslist works?

I tried it myself. My site generates this link:

http://sfbay.craigslist.org/search/rea/sfc?ref=patrick.net&bedrooms=3

Yes, that search does indeed show some houses that are not in the Bay Area at all. It seems to be a flaw in Craigslist search, because they let anyone list any house on whichever part of Craigslist they want. I suspect brokers list far-away houses there just because there are lots of rich people in the Bay Area.

What can I do about that, if anything?

FunTime says

Also, got a no results at forsalebyowner, which might just be the accurate result.

Ideally, I would not even show forsalebyowner.com in that case. I need to figure out which sites have the most results for which zip codes. Can I generate some good quality metric for each listing site without violating their terms of use? Maybe I can let users like you vote on the quality of results. How about up and down arrows?

Comments 1-13 of 13     Last »

users   about   suggestions   contact  
topics   random post   best comments   comment jail  
patrick's 40 proposals  
10 reasons it's a terrible time to buy  
8 groups who lie about the housing market  
37 bogus arguments about housing  
get a free bumper sticker:

top   bottom   home