Aggregating Real Estate Searches


By Patrick   Follow   Mon, 27 Aug 2012, 2:44pm   1,760 views   13 comments
In Menlo Park CA 94025   Watch (0)   Share   Quote   Permalink   Like (1)   Dislike  

While each real estate listing site hates all the others with a passion, and deems it a violation of their "Terms of Use" to copy and aggregate their listings, none of them can stop me from linking to all of their sites from one page.

And why would they even want to? If I drive traffic to their site from Patrick.net, they'll probably be happy.

So... I put up a little service on the Patrick.net home page that takes a zip code and a number of bedrooms, and generates search links for the major listing sites.

This saves buyers time, because they don't have to manually enter their search zip code and BR on each site, but only once instead.

The next improvement would be to rank the listing sites so that the best one for any particular zip code is on top. And to come up with a much more complete list of listing sites.

What else could it do to help buyers navigate the thicket of listings and get a good deal?

Viewing Comments 1-13 of 13     Last »     See most liked comments

  1. Patrick


    Follow
    Befriend (55)
    5,652 threads
    6,325 comments
    male
    Menlo Park, CA
    Premium

    1   3:09pm Mon 27 Aug 2012   Share   Quote   Permalink   Like   Dislike  

    I see that I have to find the best real estate search sites for each zip code, and they will probably be very different. There are more than 33,000 zip codes, so it's going to be a lot of work. Any advice on automating it a bit?

    Also, how can I let users vote on which RE search sites are the best? Maybe put up/down arrows next to the links, or just track clicks on those links?

  2. PockyClipsNow


    Follow
    Befriend
    19 threads
    1,582 comments
    Los Angeles, CA

    2   3:31pm Mon 27 Aug 2012   Share   Quote   Permalink   Like   Dislike  

    the MLS sites go not on zip, nor county, but based on MLS. Each MLS can overlap into multiple county,zip, probably not state though.

    Then you have overlapping craiglist 'counties'.

    Many sites ignore whole areas of the country (redfin).

    In some small areas you almost have to use the MLS sites search engine - a few wont even let realtor.com display thier listings (even though THATS THIER SITE, they wanna hog listings/traffic).

    Then there is FSBO - CL, local newspaper sites,plus ebay, plus 100 other 'sell it here' sites.

    To get max value of your time, start in large metro areas in CA as that will iron out any issues - CA is so large - we have all the above issues. Some entire states have only one MLS, one zip, one county, and maybe only one local newspaper.

    I am imaginging this works like Kayak.com ( a nice site, oddly weird in frames due to similar legal issues I suppose - its like stuck in 1995 internet)

  3. Patrick


    Follow
    Befriend (55)
    5,652 threads
    6,325 comments
    male
    Menlo Park, CA
    Premium

    3   3:53pm Mon 27 Aug 2012   Share   Quote   Permalink   Like   Dislike  

    PockyClipsNow says

    the MLS sites go not on zip, nor county, but based on MLS. Each MLS can overlap into multiple county,zip, probably not state though.

    I like zip though, since it's so brain-dead simple. The question becomes how to map the various sites to the zip codes that they cover.

    PockyClipsNow says

    Then you have overlapping craiglist 'counties'.

    Yes, CL is a problem. Not sure how to map zip codes to their neighborhoods yet.

    PockyClipsNow says

    I am imaginging this works like Kayak.com ( a nice site, oddly weird in frames due to similar legal issues I suppose...)

    Yes, exactly. The legal restrictions of the Terms of Use of travel sites and real estate sites all prohibit direct aggregation.

  4. PockyClipsNow


    Follow
    Befriend
    19 threads
    1,582 comments
    Los Angeles, CA

    4   3:57pm Mon 27 Aug 2012   Share   Quote   Permalink   Like   Dislike  

    Its a good idea for sure.

    I guess you could start by just coding to search ALL search sites that you can, THEN maybe only show pop ups that have >0 results. Order by importance the sites with the most results - you probably cant show more than 20 results - maybe you could.

    Thus maybe little or no filters. Might need to do a pre search on the server, then only launch popup window/frame where result >0.

  5. PockyClipsNow


    Follow
    Befriend
    19 threads
    1,582 comments
    Los Angeles, CA

    5   3:58pm Mon 27 Aug 2012   Share   Quote   Permalink   Like   Dislike  

    To map CL counties to zip codes you simply need a zip code table with a 'CL County' field/url to use per zip.

  6. Patrick


    Follow
    Befriend (55)
    5,652 threads
    6,325 comments
    male
    Menlo Park, CA
    Premium

    6   4:00pm Mon 27 Aug 2012   Share   Quote   Permalink   Like   Dislike  

    But CL doesn't even use counties, do they?

    They just seem to have their own weird abbreviations for everyplace.

  7. PockyClipsNow


    Follow
    Befriend
    19 threads
    1,582 comments
    Los Angeles, CA

    7   4:35pm Mon 27 Aug 2012   Share   Quote   Permalink   Like   Dislike  

    yes im calling them 'craigslist counties'. they are not counties or even groups of counties.
    there are over 400 I think.

    You will simply have to map one zip code to a CL search area manually.

  8. Patrick


    Follow
    Befriend (55)
    5,652 threads
    6,325 comments
    male
    Menlo Park, CA
    Premium

    8   5:53pm Mon 27 Aug 2012   Share   Quote   Permalink   Like   Dislike  

    400 is OK, I can do that. But I suspect there are more than that.

    Added a new feature: now if you do a search, it should set a cookie that remembers your search and fills in that zip code and number of BR automatically the next time you load the page.

  9. LarryPatrickMaloney


    Follow
    Befriend (13)
    68 threads
    168 comments
    Mountain View, CA
    LarryPatrickMaloney's website

    9   11:34pm Wed 29 Aug 2012   Share   Quote   Permalink   Like   Dislike  

    Good patrick!

    One of the annoying thing about surfing real estate in CraigsList, are all the MobileHome postings. If you can filter that out, I would use it everyday.

  10. Patrick


    Follow
    Befriend (55)
    5,652 threads
    6,325 comments
    male
    Menlo Park, CA
    Premium

    10   9:33am Thu 30 Aug 2012   Share   Quote   Permalink   Like   Dislike  

    Thanks for the tip!

    Craigslist has been actively suing websites that do any kind of filtering or mapping of Craigslist results, so I think I'm limited to creating only the search links that Craigslist uses, but I used to filter out mobile homes and could perhaps make some kind of software that lets you do the filtering in your own browser.

    PockyClipsNow says

    yes im calling them 'craigslist counties'. they are not counties or even groups of counties.
    there are over 400 I think.

    There were 934 of them!

    Got a decent mapping of zip codes to Craigslist areas working now! I don't know of anywhere else you can do that, so that's some value to search added already.

  11. FunTime


    Follow
    Befriend (2)
    63 threads
    1,180 comments
    San Francisco, CA

    11   12:12pm Thu 30 Aug 2012   Share   Quote   Permalink   Like   Dislike  

    I typed in my zip code, 94107, and got weird results at craigslist, homes in Portland OR, Florida, Austin TX, maybe that's just how craigslist works?

    Also, got a no results at forsalebyowner, which might just be the accurate result.

  12. PockyClipsNow


    Follow
    Befriend
    19 threads
    1,582 comments
    Los Angeles, CA

    12   12:21pm Thu 30 Aug 2012   Share   Quote   Permalink   Like   Dislike  

    CL trys hard to make sure no one aggregates them. Really its an awsome site. Craig single handedly destroyed the newspaper classified industry. forever. Those guys had a monopoly for like 500 years. pretty amazing yet simple.

  13. Patrick


    Follow
    Befriend (55)
    5,652 threads
    6,325 comments
    male
    Menlo Park, CA
    Premium

    13   1:38pm Thu 30 Aug 2012   Share   Quote   Permalink   Like   Dislike  

    FunTime says

    I typed in my zip code, 94107, and got weird results at craigslist, homes in Portland OR, Florida, Austin TX, maybe that's just how craigslist works?

    I tried it myself. My site generates this link:

    http://sfbay.craigslist.org/search/rea/sfc?ref=patrick.net&bedrooms=3

    Yes, that search does indeed show some houses that are not in the Bay Area at all. It seems to be a flaw in Craigslist search, because they let anyone list any house on whichever part of Craigslist they want. I suspect brokers list far-away houses there just because there are lots of rich people in the Bay Area.

    What can I do about that, if anything?

    FunTime says

    Also, got a no results at forsalebyowner, which might just be the accurate result.

    Ideally, I would not even show forsalebyowner.com in that case. I need to figure out which sites have the most results for which zip codes. Can I generate some good quality metric for each listing site without violating their terms of use? Maybe I can let users like you vote on the quality of results. How about up and down arrows?

Premium member Patrick is moderator of this thread.

Email

Username

Watch comments by email
Home   Tips and Tricks   Questions or suggestions? Mail p@patrick.net   Thank you for your kind donations

Page took 76 milliseconds to create.