How to Post Data and Fetch Remote Pages from PHP Scripts

http://phpstarter.net/2008/12/how-to-post-data-and-fetch-remote-pages-from-php-scripts/

Published: December 18th, 2008 by: Andrew

                    There are times where a PHP script needs to fetch the HTML from a remote page, or even post some data to a remote location.  Learning how to use cURL or fsockopen() can be time consuming and unnecessary.  There is a nice PHP class called Snoopy that can take care of all these functions very easily.

<!-- -->

---

First, head on over to the project page and download a copy of the script.  Once you extract the archive, you will only need Snoopy.class.php.  Upload it to your favorite web host, and create a blank PHP script to get started with some examples.

Fetching a Page

Fetching data from a remote URL is very useful to have if you don’t want to mess with cURL or fsockopen.  Snoopy takes care of all that for you.

view sourceprint?

01.<?php

02./* load the snoopy class and initialize the object */

03.require('../includes/Snoopy.class.php');

04.$snoopy= new Snoopy();

05.

06./* load the page and print the results */

07.$snoopy->fetch('http://snoopy.sourceforge.net/');

08.echo'<pre>' . htmlspecialchars($snoopy->results) .'</pre>';

09.?>

Fetching a web page may be of little use, but we can also use this to retrieve remote XML or CSV files.  Here is a more practical example.

Run This Example

view sourceprint?

01.<?php

02./* load the snoopy class and initialize the object */

03.require('../includes/Snoopy.class.php');

04.$snoopy= new Snoopy();

05.

06./* fetch the data */

07.$snoopy->fetch('http://www.weather.gov/xml/current_obs/KLOT.xml');

08.

09./* parse the XML data */

10.$xml= new SimpleXMLElement($snoopy->results);

11.

12./* output some of the elements */

13.echo'<h3>Some Parsed Information</h3>';

14.echo'<img src="' . $xml->icon_url_base .'/' . $xml->icon_url_name .'" /><br />';

15.echo'<b>Reporting Station:</b> ' . $xml->location . '<br />';

16.echo'<b>Observation Time:</b> ' . $xml->observation_time_rfc822 .'<br />';

17.echo'<b>Temp:</b> ' . $xml->temperature_string .'<br />';

18.echo'<b>Wind:</b> ' . $xml->wind_string .'<br />';

19.

20.

21.echo'<hr /><h3>Raw XML</h3>';

22.echo'<pre>' . htmlspecialchars($snoopy->results) .'</pre>';

23.?>

Sending Post & Cookie Data

Some APIs, like PayPal’s Instant Payment Notification tool, require data to be sent to their servers in post form.  Snoopy makes it as easy as putting the data in an array and sending it to the URL.

Here is the script that sends the some post data and a couple cookies:

view sourceprint?

01.<?php

02./* load the snoopy class and initialize the object */

03.require('../includes/Snoopy.class.php');

04.$snoopy= new Snoopy();

05.

06./* set some values */

07.$p_data['color'] ='Red';

08.$p_data['fruit'] ='apple';

09.

10.$snoopy->cookies['vegetable'] = 'carrot';

11.$snoopy->cookies['something'] = 'value';

12.

13./* submit the data and get the result */

14.$snoopy->submit('http://phpstarter.net/samples/118/data_dump.php',$p_data);

15.

16./* output the results */

17.echo'<pre>' . htmlspecialchars($snoopy->results) .'</pre>';

18.?>

And here is the result data on the server side:

view sourceprint?

01./* $_POST */

02.array(2) {

03. ["color"]=>

04. string(3)"Red"

05. ["fruit"]=>

06. string(5)"apple"

07.}

08./* $_COOKIE */

09.array(2) {

10. ["vegetable"]=>

11. string(6)"carrot"

12. ["something"]=>

13. string(5)"value"

14.}

Fetching Links

When you need to crawl a page for links, Snoopy has taken care of the trouble to parse the links from the page.

view sourceprint?

01.<?php

02./* load the snoopy class and initialize the object */

03.require('../includes/Snoopy.class.php');

04.$snoopy= new Snoopy();

05.

06./* load the page and print the results */

07.$snoopy->fetchlinks('http://google.com/');

08.echo'<pre>' . var_export($snoopy->results, true) .'</pre>';

09.?>

Script output:

view sourceprint?

01.array(

02. 0 =>'http://images.google.com/imghp?hl=en&;tab=wi',

03. 1 =>'http://maps.google.com/maps?hl=en&;tab=wl',

04. 2 =>'http://news.google.com/nwshp?hl=en&;tab=wn',

05. 3 =>'http://www.google.com/prdhp?hl=en&;tab=wf',

06. 4 =>'http://mail.google.com/mail/?hl=en&;tab=wm',

07. 5 =>'http://www.google.com/intl/en/options/',

08. 6 =>'http://video.google.com/?hl=en&;tab=wv',

09. 7 =>'http://groups.google.com/grphp?hl=en&;tab=wg',

10. 8 =>'http://books.google.com/bkshp?hl=en&;tab=wp',

11. 9 =>'http://scholar.google.com/schhp?hl=en&;tab=ws',

12. 10 =>'http://finance.google.com/finance?hl=en&;tab=we',

13. 11 =>'http://blogsearch.google.com/?hl=en&;tab=wb',

14. 12 =>'http://www.youtube.com/?hl=en&;tab=w1',

15. 13 =>'http://www.google.com/calendar/render?hl=en&;tab=wc',

16. 14 =>'http://picasaweb.google.com/home?hl=en&;tab=wq',

17. 15 =>'http://docs.google.com/?hl=en&;tab=wo',

18. 16 =>'http://www.google.com/reader/view/?hl=en&;tab=wy',

19. 17 =>'http://sites.google.com/?hl=en&;tab=w3',

20. 18 =>'http://www.google.com/intl/en/options/',

21. 19 =>'http://www.google.com/url?sa=p&;pref=ig&pval=3&q=http://www.google.com/ig%3Fhl%3Den%26source%3Diglk&;usg=AFQjCNFA18XPfgb7dKnXfKz7x7g1GDH1tg',

22. 20 =>'http://www.google.com/intl/en/ads/',

23. 21 =>'http://www.google.com/services/',

24. 22 =>'http://www.google.com/intl/en/about.html',

25. 23 =>'http://www.google.com/intl/en/privacy.html',

26. 24 =>'http://www.google.com/advanced_search?hl=en',

27. 25 =>'http://www.google.com/preferences?hl=en',

28. 26 =>'http://www.google.com/language_tools?hl=en',

29.)

More Options

We can set various other options and values such as:

• User name/password for basic HTTP authentication
• Proxy hosts & ports
• Raw headers
• Maximum redirects

view sourceprint?

01.<?php

02./* don't forget to include the file */

03.$snoopy= new Snoopy;

04.

05./* make the request through a proxy server */

06.$snoopy->proxy_host ="my.proxy.host";

07.$snoopy->proxy_port ="8080";

08.

09./* change the user agent or refer URL */

10.$snoopy->agent ="(compatible; MSIE 4.01; MSN 2.5; AOL 4.0; Windows 98)";

11.$snoopy->referer ="http://www.microsnot.com/";

12.

13./* set some cookies */

14.$snoopy->cookies["SessionID"] = 238472834723489l;

15.$snoopy->cookies["favoriteColor"] = "RED";

16.

17./* set some raw headers */

18.$snoopy->rawheaders["Pragma"] = "no-cache";

19.

20./* set some redirect options */

21.$snoopy->maxredirs = 2;

22.$snoopy->offsiteok = false;/* allow a redirect to different domain */

23.

24./* set user/pass for basic HTTP authentication */

25.$snoopy->user ="me";

26.$snoopy->pass ="p@ssw0rd";

27.?>

There are even more options not covered here.  To see what those options are, see the Snoopy.class.php file and the readme.txt file contained in the downloaded package.

No related posts.