Tuesday, April 8, 2008

PHP: Curl to fetch html page from HTTPS


Share at Facebook

If you need to use HTTPS protocol with your curl, then here is the help for you. I'm giving you an example of Php code that will show you how to fetch the HTML content from HTTPS URL. You just need to enable the option for SSL certificate verification to curl. Thats all.

Below is the PHP/curl code that just browse the yahoo mail's login page which is a HTTPS link. When the content returned, it will print it on browser, which will appear as if it is Yahoo login page :)

## HTTPS url that you are targeting.
$url = "https://login.yahoo.com/config/login?";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_USERAGENT, 'Opera/9.23 (Windows NT 5.1; U; en)');
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);

## Below two option will enable the HTTPS option.
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
$result = curl_exec($ch);
echo $result;
You can set other options timeout, Post values for Post method, etc. By the way, If you don't know how to configure Curl with PHP then follow the link http://icfun.blogspot.com/2008/04/configure-php-curl-with-apache.html




9 comments:

onassar said...

thanks for the tip. worked perfectly.

Anonymous said...

The world really is indeed an amazing place (every very rare now and then)
I spent 2 years developing a real estate datamining system that must update daily.
Today, the main site I scrape base data from trned on SSL.
Your https example was exactly what I needed and totally saved me!

Anonymous said...

thank you!

WarpY said...

You have done me a world of good. I'd kiss you now if were next to you!

Thank you very much!

WarpY said...

Thank you very much!!!
This has helped me a lot!

Geremy said...

Excellent example. I wanted to take it further and add cookie and header parsing, this served as a great reference for achieving this next step: http://elame.com/damnthatsannoying/?p=7

José B said...

Should you not be setting CURLOPT_SSL_VERIFYPEER to TRUE? Otherwise you are just disabling SSL verification.

Demon said...

yes, i am just disabling the ssl verification.

php curl https said...

also you can check if bot ends talks on same ssl version.