Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
1.3k views
in Technique[技术] by (71.8m points)

php proDOM parsing error

I am using the following code for parsing dom document but at the end I get the error "google.ac" is null or not an object line 402 char 1

What I guess, line 402 contains tag and a lot of ";", How can I fix this?

<?php

//$ch = curl_init("http://images.google.com/images?q=books&tbm=isch/");


// create a new cURL resource
$ch = curl_init();

// set URL and other appropriate options
curl_setopt($ch, CURLOPT_URL, "http://images.google.com/images?q=books&tbm=isch/");
curl_setopt($ch, CURLOPT_HEADER, 0);

// grab URL and pass it to the browser
$data = curl_exec($ch);

curl_close($ch); 

$dom = new DOMDocument();
       $dom->loadHTML($data);
    //@$dom->saveHTMLFile('newfolder/abc.html')

     $dom->loadHTML('$data');

    // find all ul

    $list = $dom->getElementsByTagName('ul'); 
    // get few  list items 

    $rows = $list->item(30)->getElementsByTagName('li'); 
    // get anchors from the table   

    $links = $list->item(30)->getElementsByTagName('a'); 

    foreach ($links as $link) { 
        echo "<fieldset>"; 
        $links = $link->getElementsByAttribute('imgurl');

    $dom->saveXML($links);
                }
?>
See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

There are a few issues with the code:

  1. You should add the CURL option - CURLOPT_RETURNTRANSFER - in order to capture the output. By default the output is displayed on the browser. Like this: curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);. In the code above, $data will always be TRUE or FALSE (http://www.php.net/manual/en/function.curl-exec.php)

  2. $dom->loadHTML('$data'); is not correct and not required

  3. The method of reading 'li' and 'a' tags might not be correct because $list->item(30) will always point to the 30th element

Anyways, coming to the fixes. I'm not sure if you checked the HTML returned by the CURL request but it seems different from what we discussed in the original post. In other words, the HTML returned by CURL does not contain the required <ul> and <li> elements. It instead contains <td> and <a> elements.

Add-on: I'm not very sure why do HTML for the same page is different when it is seen from the browser and when read from PHP. But here is a reasoning that I think might fit. The page uses JavaScript code that renders some HTML code dynamically on page load. This dynamic HTML can be seen when viewed from the browser but not from PHP. Hence, I assume the <ul> and <li> tags are dynamically generated. Anyways, that isn't of our concern for now.

Therefore, you should modify your code to parse the <a> elements and then read the image URLs. This code snippet might help:

<?php
$ch = curl_init(); // create a new cURL resource

// set URL and other appropriate options
curl_setopt($ch, CURLOPT_URL, "http://images.google.com/images?q=books&tbm=isch/");
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);

$data = curl_exec($ch); // grab URL and pass it to the browser
curl_close($ch); 

$dom = new DOMDocument();
@$dom->loadHTML($data); // avoid warnings

$listA = $dom->getElementsByTagName('a'); // read all <a> elements
foreach ($listA as $itemA) { // loop through each <a> element
    if ($itemA->hasAttribute('href')) { // check if it has an 'href' attribute
        $href = $itemA->getAttribute('href'); // read the value of 'href'
        if (preg_match('/^/imgres?/', $href)) { // check that 'href' should begin with "/imgres?"
            $qryString = substr($href, strpos($href, '?') + 1);
            parse_str($qryString, $arrHref); // read the query parameters from 'href' URI
            echo '<br>' . $arrHref['imgurl'] . '<br>';
        }
    }
}

I hope above makes sense. But please note that the above parsing might fail if Google modifies their HTML.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...