Question

31

Crawling Google Search with PHP

rated 0 times [ 35] [ 4] / answers: 1 / hits: 16551 / 12 Years ago, sun, january 27, 2013, 12:00:00

I trying to get my head around how to fetch Google search results with PHP or JavaScript. I know it has been possible before but now I can't find a way.

I am trying to duplicate (somewhat) the functionality of

http://www.getupdated.se/sokmotoroptimering/seo-verktyg/kolla-ranking/

But really the core issue I want to solve is just to get the search result via PHP or JavaScript,the rest i can figure out.

Fetching the results using file_get_contents() or cURL doesn't seem to work.

Example:

$ch = curl_init();

$timeout = 5;

curl_setopt($ch, CURLOPT_URL, 'http://www.google.se/#hl=sv&q=dogs');

curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);

$result = curl_exec($ch);

curl_close($ch);

echo '<pre>';

var_dump($result);

echo '</pre>';

Results:

string(219) 302 Moved The document has moved here.

So, with some Googling i found http://code.google.com/apis/customsearch/v1/overview.html but that seems to only work for generating a custom search for one or more websites.
It seem to require a Custom Search Engine cx-parameter passed.

So anyway, any idea?

Answers

Only authorized users can answer the question. Please sign in first, or register a free account.

tristab

Add To Favorites

Follow

Total Points: 735

Total Questions: 106

Total Answers: 96

Location: Grenada

Member since Sun, Dec 20, 2020

3 Years ago

tristab questions

1 How to use the new Webpack devServer configuration?

Sat, Sep 25, 21, 00:00, 3 Years ago

1 Discord.JS get member from user ID

Sun, Jan 31, 21, 00:00, 3 Years ago

1 React: Passing Props not working. What am I missing?

Wed, Dec 2, 20, 00:00, 4 Years ago

1 How can i extract vowels in a Javascript String?

Fri, Oct 23, 20, 00:00, 4 Years ago

1 Jest encountered an unexpected token when testing a Vue single file component

Sat, May 23, 20, 00:00, 4 Years ago

View All

answered 12 Years ago chazw · Accepted Answer

I did it earlier. Generate the html contents by making https://www.google.co.in/search?hl=en&output=search&q=india http request, now parse specific tags using the htmldom php library. You can parse the content of result page using PHP SIMPLE HTML DOM

DEMO : Below code will give you title of all the result :

<?php



include(simple_html_dom.php);



$html = file_get_html('http://www.google.co.in/search?hl=en&output=search&q=india');



$i = 0;

foreach($html->find('li[class=g]') as $element) {

    foreach($element->find('h3[class=r]') as $h3) 

    {

        $title[$i] = '<h1>'.$h3->plaintext.'</h1>' ;

    }

       $i++;

}

print_r($title);



?>