« previous   housing   next »

Aggregating Real Estate Searches


By Patrick   Follow   Mon, 27 Aug 2012, 7:44am PDT   2,015 views   13 comments   Watch (0)   Share   Quote   Permalink   Like (1)   Dislike  

While each real estate listing site hates all the others with a passion, and deems it a violation of their "Terms of Use" to copy and aggregate their listings, none of them can stop me from linking to all of their sites from one page.

And why would they even want to? If I drive traffic to their site from Patrick.net, they'll probably be happy.

So... I put up a little service on the Patrick.net home page that takes a zip code and a number of bedrooms, and generates search links for the major listing sites.

This saves buyers time, because they don't have to manually enter their search zip code and BR on each site, but only once instead.

The next improvement would be to rank the listing sites so that the best one for any particular zip code is on top. And to come up with a much more complete list of listing sites.

What else could it do to help buyers navigate the thicket of listings and get a good deal?

Comments 1-13 of 13     Last »

Patrick   befriend   ignore   Mon, 27 Aug 2012, 8:09am PDT   Share   Quote   Like   Dislike     Comment 1

I see that I have to find the best real estate search sites for each zip code, and they will probably be very different. There are more than 33,000 zip codes, so it's going to be a lot of work. Any advice on automating it a bit?

Also, how can I let users vote on which RE search sites are the best? Maybe put up/down arrows next to the links, or just track clicks on those links?

PockyClipsNow   befriend   ignore   Mon, 27 Aug 2012, 8:31am PDT   Share   Quote   Like   Dislike     Comment 2

the MLS sites go not on zip, nor county, but based on MLS. Each MLS can overlap into multiple county,zip, probably not state though.

Then you have overlapping craiglist 'counties'.

Many sites ignore whole areas of the country (redfin).

In some small areas you almost have to use the MLS sites search engine - a few wont even let realtor.com display thier listings (even though THATS THIER SITE, they wanna hog listings/traffic).

Then there is FSBO - CL, local newspaper sites,plus ebay, plus 100 other 'sell it here' sites.

To get max value of your time, start in large metro areas in CA as that will iron out any issues - CA is so large - we have all the above issues. Some entire states have only one MLS, one zip, one county, and maybe only one local newspaper.

I am imaginging this works like Kayak.com ( a nice site, oddly weird in frames due to similar legal issues I suppose - its like stuck in 1995 internet)

Patrick   befriend   ignore   Mon, 27 Aug 2012, 8:53am PDT   Share   Quote   Like   Dislike     Comment 3

PockyClipsNow says

the MLS sites go not on zip, nor county, but based on MLS. Each MLS can overlap into multiple county,zip, probably not state though.

I like zip though, since it's so brain-dead simple. The question becomes how to map the various sites to the zip codes that they cover.

PockyClipsNow says

Then you have overlapping craiglist 'counties'.

Yes, CL is a problem. Not sure how to map zip codes to their neighborhoods yet.

PockyClipsNow says

I am imaginging this works like Kayak.com ( a nice site, oddly weird in frames due to similar legal issues I suppose...)

Yes, exactly. The legal restrictions of the Terms of Use of travel sites and real estate sites all prohibit direct aggregation.

PockyClipsNow   befriend   ignore   Mon, 27 Aug 2012, 8:57am PDT   Share   Quote   Like   Dislike     Comment 4

Its a good idea for sure.

I guess you could start by just coding to search ALL search sites that you can, THEN maybe only show pop ups that have >0 results. Order by importance the sites with the most results - you probably cant show more than 20 results - maybe you could.

Thus maybe little or no filters. Might need to do a pre search on the server, then only launch popup window/frame where result >0.

PockyClipsNow   befriend   ignore   Mon, 27 Aug 2012, 8:58am PDT   Share   Quote   Like   Dislike     Comment 5

To map CL counties to zip codes you simply need a zip code table with a 'CL County' field/url to use per zip.

Patrick   befriend   ignore   Mon, 27 Aug 2012, 9:00am PDT   Share   Quote   Like   Dislike     Comment 6

But CL doesn't even use counties, do they?

They just seem to have their own weird abbreviations for everyplace.

PockyClipsNow   befriend   ignore   Mon, 27 Aug 2012, 9:35am PDT   Share   Quote   Like   Dislike     Comment 7

yes im calling them 'craigslist counties'. they are not counties or even groups of counties.
there are over 400 I think.

You will simply have to map one zip code to a CL search area manually.

Patrick   befriend   ignore   Mon, 27 Aug 2012, 10:53am PDT   Share   Quote   Like   Dislike     Comment 8

400 is OK, I can do that. But I suspect there are more than that.

Added a new feature: now if you do a search, it should set a cookie that remembers your search and fills in that zip code and number of BR automatically the next time you load the page.

LarryPatrickMaloney   befriend   ignore   Wed, 29 Aug 2012, 4:34pm PDT   Share   Quote   Like   Dislike     Comment 9

Good patrick!

One of the annoying thing about surfing real estate in CraigsList, are all the MobileHome postings. If you can filter that out, I would use it everyday.

Patrick   befriend   ignore   Thu, 30 Aug 2012, 2:33am PDT   Share   Quote   Like   Dislike     Comment 10

Thanks for the tip!

Craigslist has been actively suing websites that do any kind of filtering or mapping of Craigslist results, so I think I'm limited to creating only the search links that Craigslist uses, but I used to filter out mobile homes and could perhaps make some kind of software that lets you do the filtering in your own browser.

PockyClipsNow says

yes im calling them 'craigslist counties'. they are not counties or even groups of counties.
there are over 400 I think.

There were 934 of them!

Got a decent mapping of zip codes to Craigslist areas working now! I don't know of anywhere else you can do that, so that's some value to search added already.

FunTime   befriend   ignore   Thu, 30 Aug 2012, 5:12am PDT   Share   Quote   Like   Dislike     Comment 11

I typed in my zip code, 94107, and got weird results at craigslist, homes in Portland OR, Florida, Austin TX, maybe that's just how craigslist works?

Also, got a no results at forsalebyowner, which might just be the accurate result.

PockyClipsNow   befriend   ignore   Thu, 30 Aug 2012, 5:21am PDT   Share   Quote   Like   Dislike     Comment 12

CL trys hard to make sure no one aggregates them. Really its an awsome site. Craig single handedly destroyed the newspaper classified industry. forever. Those guys had a monopoly for like 500 years. pretty amazing yet simple.

Patrick   befriend   ignore   Thu, 30 Aug 2012, 6:38am PDT   Share   Quote   Like   Dislike     Comment 13

FunTime says

I typed in my zip code, 94107, and got weird results at craigslist, homes in Portland OR, Florida, Austin TX, maybe that's just how craigslist works?

I tried it myself. My site generates this link:

http://sfbay.craigslist.org/search/rea/sfc?ref=patrick.net&bedrooms=3

Yes, that search does indeed show some houses that are not in the Bay Area at all. It seems to be a flaw in Craigslist search, because they let anyone list any house on whichever part of Craigslist they want. I suspect brokers list far-away houses there just because there are lots of rich people in the Bay Area.

What can I do about that, if anything?

FunTime says

Also, got a no results at forsalebyowner, which might just be the accurate result.

Ideally, I would not even show forsalebyowner.com in that case. I need to figure out which sites have the most results for which zip codes. Can I generate some good quality metric for each listing site without violating their terms of use? Maybe I can let users like you vote on the quality of results. How about up and down arrows?

Email (Required, will not be visible)

Username (Just pick a name if you're new)

Watch comments by email

home   top   share   link sharer   users   register   best of   about   questions or suggestions? write p@patrick.net