PHP web scraper

system · January 31, 2012, 9:35pm

Hi guys,

I’ve created an awesome PHP web scraper that works like a charm! The problem is however, when I try to scrape the site which is seen in my code, I get nothing. I believe that it has something to do with how the page is displayed in frames and how the data is presented after a query is made. Is there any way to scrape this page sucessfully?

The PHP scraper that I am using:
[php]

</head>
<body>

<?php $url = 'http://www.tides.gc.ca/cgi-bin/tide-shc.cgi?queryType=showFrameset&zone=10&language=english&region=1&stnnum=7735'; $output = file_get_contents($url); echo $output; ?>

</body>

[/php]

Thanks guys!

ErnieAlex · February 1, 2012, 4:14pm

Well, hackers use web scraper to steal code and is illegal to use if you are capturing private websites.

Now that I have said that, first, you are scrapping webpages. They contain data such as quotes, double-quotes and other punctuation. When echo’d to the screen, these special codes cause problems in browsers.

Best thing is to echo them into a frame as text or just into a which may show the url.
Most likely you will have to save the $output as a text file and then use it to parse whatever you are looking for inside it. If you are just trying to display webpages, use a browser.

Guess it depends on what you want to do with the data. Here is a link to a tutorial on web scraping.
( *** Please remember not all scrapes are legal ***)
http://www.oooff.com/php-scripts/basic-php-scrape-tutorial/basic-php-scraping.php (Look at the last part of this 3 part article for dealing with the data…) Good luck