This is a general question - not necessarily pinned to anything Mac related.
How does one go about extracting data from a web site, designed for humans running searches, to programmatically capture information and process it?
Let me give you an example.
On http://www.realty.com, I can search for houses. Let's say I want write an application that will find all houses in the Houston area, in Katy, that are selling for < $100,000 and have 3 bedroom and 2 baths. Then, once I fond all those, I want to dig deeper.
If I go to that website, I click on Texas on a map of the US, then pick "Houston Area", then pick Katy from a drop down list. Then, a new window opens and I can specify price, rooms, baths, and then "search now". The screen is then populated with data about price, square feet, blah blah blah.
And, then let's say I'm only interested in homes with a price per square foot under a certain threshold.
How would you go about writing a ruby script or java application that would do this automatically? Or, based on this type of interface, could you even do that?
Todd
How does one go about extracting data from a web site, designed for humans running searches, to programmatically capture information and process it?
Let me give you an example.
On http://www.realty.com, I can search for houses. Let's say I want write an application that will find all houses in the Houston area, in Katy, that are selling for < $100,000 and have 3 bedroom and 2 baths. Then, once I fond all those, I want to dig deeper.
If I go to that website, I click on Texas on a map of the US, then pick "Houston Area", then pick Katy from a drop down list. Then, a new window opens and I can specify price, rooms, baths, and then "search now". The screen is then populated with data about price, square feet, blah blah blah.
And, then let's say I'm only interested in homes with a price per square foot under a certain threshold.
How would you go about writing a ruby script or java application that would do this automatically? Or, based on this type of interface, could you even do that?
Todd
