As a preface, I don't know how to code. I worked with C++ in high school, minimally. I'm just trying to find the most efficient way to get my project done.

So I'm trying to get every fast food restaurant in America and Canada into a MySQL database. I can't do it myself, so I need to contract or hire someone to do it for me. I've found that Google Earth is a viable place to get this information.

What programming language would be good for this? I was thinking C++ or C#, but I'm an idiot.

Please help! Any advice and most flaming would probably be useful.

6/17/2008 1:55:46 PM

So, what are you looking to use this for. I dont see a reason why you would need to scrape it all from google earth specifically. There are much easier places to get this information from.

PM me if you dont want to broadcast your plans. I have this database myself already.

6/17/2008 2:58:55 PM

this thread sounds a lot like: message_topic.aspx?topic=522074

6/17/2008 3:44:26 PM

better off scraping yellowpages.com

and it's not hard, but it is illegal if this is for a commercial application

6/17/2008 4:12:51 PM

i didn't realize things could be illegal on the internets

6/17/2008 4:19:25 PM

can't be illegal because there is no way to enforce the terms (if it exist in the terms), and there is no way to find out if someone did scrape the data.

6/17/2008 4:21:52 PM

they are better off with their data being open anyways.

6/17/2008 4:25:39 PM

^^ yes there is, and yes there is.

It's against TOS to scrape data for commercial use on damn near any site out there. And it's trivially easy to find out if someone is scraping you, as well as verifying the data theft.

^Open data != free for commercial use. Google has an open data system with google maps. You cannot use the DATA for your own commercial ventures, you have to use their API and map controls on top of the data.

6/17/2008 5:07:34 PM

I didn't think yellowpages had an API

Internet rules clearly state that old stodgy content owners are susceptible to the fullest abuse nerds can come up with and long standing precedents include web scraping for their sweet sweet datas

[Edited on June 17, 2008 at 5:38 PM. Reason : true story]

6/17/2008 5:36:49 PM

Scraping for data manipulation may be okay. Scraping for duplication will get you sued. They don't have to have an API. They have a closed data system (their choice), but my point is, even with open data systems, you CANNOT directly duplicate the data for commercial use. At least not for free, you can license the data repository information from most places (Hence why companies like Navtec exist)

Internet nerd or not, it's not about whether you can do it (you can), it's about whether you will lose the lawsuit from doing it (you will).

[Edited on June 17, 2008 at 5:42 PM. Reason : .]

6/17/2008 5:41:00 PM

Ah. I figured that would be too easy. Very well then.

I suppose I will have to pay for this information.

[Edited on June 17, 2008 at 6:34 PM. Reason : That's cool. Good thing I posted this.]

6/17/2008 6:33:44 PM

chicken.

bock bock bock bock BOCK!!!!

6/17/2008 8:01:25 PM

hahaha

6/17/2008 8:53:18 PM