Fetching a Web Page From Your PHP Code

Nov 4, 2008 | Tags: PHP, HTTP | del.icio.us del.icio.us | digg Digg

To fetch a web page from your PHP application, you could use curl functions or simply open the page with fopen(). But these have some limitations. Your server needs to have PHP with curl enabled, or the PHP configuration should allow scripts to open URLs with fopen().

If that is not the case, you can still fetch a web page by opening a socket connection to the remote host and make HTTP request. It looks something like this:

Listing 1: fetch_page.php

  1. <?php
  2. function fetch_page($url)
  3. {
  4.     /* get hostname and path */
  5.     $host = parse_url($url, PHP_URL_HOST);
  6.     $path = parse_url($url, PHP_URL_PATH);
  7.  
  8.     if (empty($path)) {
  9.         $path = "/";
  10.     }
  11.    
  12.     /* Build HTTP 1.0 request header. Defined in RFC 1945 */
  13.     $headers = "GET $path HTTP/1.0\r\n"
  14.              . "User-Agent: myHttpTool/1.0\r\n\r\n";
  15.    
  16.     /* open socket connection to remote host on port 80 */
  17.     $fp = fsockopen($host, 80, $errno, $errmsg, 30);
  18.    
  19.     if (!$fp) {
  20.         /* ...some error handling... */
  21.         return false;
  22.     }
  23.    
  24.     /* send request headers */
  25.     fwrite($fp, $headers);
  26.    
  27.     /* read response */
  28.     while(!feof($fp)) {
  29.         $resp .= fgets($fp, 4096);
  30.     }
  31.     fclose($fp);
  32.    
  33.     /* separate header and body */
  34.     $neck = strpos($resp, "\r\n\r\n");
  35.     $head = substr($resp, 0, $neck);
  36.     $body = substr($resp, $neck+4);
  37.    
  38.     /* omit parsing response headers */
  39.    
  40.     /* return page contents */
  41.     return($body);
  42. }
  43. ?>

With the function above you should be able to fetch a page without any dependencies. However, it lacks many important features. You may want to add some looping for page redirects and add support for HTTP 1.1. For a more complex HTTP client, see my EasyWebFetch class.

EasyWebFetch is a simple class for web fetching from your application. This class is an alternative if your server doesn't have PHP with curl enabled, or the PHP configuration doesn't allow opening URLs with fopen(). This class fetch a web page by opening socket connection to remote host, so it has no dependencies and should work on any server configuration.

Features:

  • No dependencies
  • Support for HTTP 1.1
  • Support redirects
  • Support proxies
  • Support HEAD and GET methods. This might be useful for link checker applications

Related Articles

The Downloads

3 Comments

Abbas Khan on Aug 27, 2008:

Thanks Nasharuddin. Thats really a cool program of yours.
Looking into curl screwed my head ! LOL

Binny V A on Oct 23, 2008:

I wrote a similar program for this - try
http://www.bin-co.com/php/scripts/load/

blaaze on Dec 14, 2008:

(1/3)can anyone help me here actually im looking for a script that need to satisfy these things 1.it must be in php language 2.it must work like a robot, when initiated it must run automatically on server side only

Leave a comment

Name (required)
Email (will not be published) (required)
Website

Characters left = 1000

Tags

Correlation  HTTP  Image  Networking  OpenCV  PHP  Projects  Regex  SEO  Security  WebService 

Newsletter

Send me your new post on:
OpenCV
PHP
Projects 		Networking
Regex
to my email:
No, I won't spam your email.

Recent Posts

  1. PHP Script for Converting Relative to Absolute URL
  2. Streaming OpenCV Videos Over the Network
  3. Password Protected Images with PHP
  4. OpenCV Region of Interest (ROI)
  5. Web Based Surveillance System with OpenCV, PHP and Javascript

Popular Posts

  1. OpenCV Eye Detection
  2. OpenCV Examples Part 1
  3. OpenCV Face Detection
  4. OpenCV Eye Tracking
  5. Display Video from Webcam
  6. Phase Correlation in OpenCV
  7. Simple File Server and Downloader Script
  8. Building Dynamic SQL String from Associative Array
  9. Template Matching with OpenCV
  10. Fetching a Web Page From Your PHP Code

About the Author

avatar Cool PHP programmer writing cool PHP scripts. Feel free to contact
Tel. +62 31 8662872
+62 856 338 6017
ICQ 489571630
Skype dede_bl4ckheart
Yahoo dede_bl4ckheart
Google nashruddin.amin