php实现的简单的csdn博客文章抓取(续:添加用户名搜索提示)

来源：互联网发布：天下3鬼墨捏脸数据编辑：程序博客网时间：2024/05/19 17:51

上篇blog文章实现了根据指定的csdn用户名列出其所有的blog文章的功能，不过这个其实没多大实际用处，我们必须得精确知道感兴趣用户的用户名才能列出他的blog文章，所以闲来想弄一个类似于google suggest的输入提示功能。

要实现搜索提示，当然手头得有csdn用户列表可供使用，毫无疑问这个用户列表要靠自己去crawl，于是写了个极其简单的爬取php脚本，如下：

<?php//////require_once("config.php");require_once("database.php");require_once("crawl_func.php");/////fetch url contentfunction get_url_content($url){ if(extension_loaded('curl')) { $ch = curl_init($url); curl_setopt($ch, CURLOPT_HEADER, 0); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $content = curl_exec($ch); curl_close($ch); } else { $content = file_get_contents($url); } //!$content && die("get $url content error."); !$content && $content = "zfq"; return $content; }/////set_time_limit(0);header("Content-Type:text/html; charset=utf-8");////zfqsql_open();addUrlToDb($START_CRAWL_URL);////$url_counter = 0;////crawl....print "start to crawl ... <br />/n";while($crawlurl = getUrlToCrawl()){////hackusleep ($CRAWL_THREAD_SLEEP_TIME);print "handle URL : ".$crawlurl."<br />/n";////$content = get_url_content($crawlurl);$reg_pattern = '/(http:////(blog|hi).csdn.net//([a-zA-Z0-9_-]+)//?)/"/';if(preg_match_all($reg_pattern, $content, $matches)){$tempcrawurls = $matches[1];$tempblogname = $matches[3];$num = count($matches[0]);for($i = 0; $i < $num; $i++){addUrlToDb($tempcrawurls[$i]);addBlognameToDb($tempblogname[$i]);}}$reg_pattern = '/(http:////(forum|topic).csdn.net//[^/"]+)/"/';if(preg_match_all($reg_pattern, $content, $matches)){$exclude_pattern = '/User/';$tempcrawurls = $matches[1];$num = count($matches[0]);for($i = 0; $i < $num; $i++){preg_match($exclude_pattern, $tempcrawurls[$i], $exclude_match);if($exclude_match){continue;}addUrlToDb($tempcrawurls[$i]);}}////$url_counter++;updateUrlToDb($crawlurl);}?>

crawl的过程比较慢，截止到发帖时间已经爬取了3、4小时的时间，爬取到的用户总数已经达到2w多，已经分析的url数目仅1000多条，待分析和爬取的url目前为止是6w条左右，这个数目还会继续增加，当然爬取得到的用户数目和待爬取的url数目越往后关系越不大。

crawl得到所需的csdn用户列表后，剩下的工作就是怎样实现搜索提示的功能，这部分的实现参考了别人的代码，在此就不作介绍，只给出相关代码和效果图

////////index.htm

<html><head><meta http-equiv=Content-Type content="text/html;charset=utf-8"><title>for list csdn blog entry</title><mce:script type="text/javascript" src="jquery-1.2.1.pack.js" mce_src="jquery-1.2.1.pack.js"></mce:script><mce:script type="text/javascript"></mce:script><mce:style type="text/css"></mce:style><style type="text/css" mce_bogus="1">body {font-family: Helvetica;font-size: 11px;color: #000000;}h3 {margin: 0px;padding: 0px;}.suggestionsBox {position: relative;left: 300px;margin: 10px 0px 0px 0px;width: 200px;background-color: #212427;-moz-border-radius: 7px;-webkit-border-radius: 7px;border: 2px solid #000;color: #00ff00;}.suggestionList {margin: 0px;padding: 0px;}.suggestionList li {margin: 0px 0px 3px 0px;padding: 3px;cursor: pointer;}.suggestionList li:hover {background-color: #1f20ff;}</style></head><h1 align="center">CSDN BLOG ENTRY LIST</h1><form method="GET" action="get_csdnblog.php"><br /><br /><br /><table cellSpacing=0 cellPadding=0 width=400 align="center"><tr><td align="center"><input maxLength=256 size=50 name="blogname" id="blogname" type="text" onkeyup="lookup(this.value);" onblur="fill();"> <input type=submit value=" List "></td></tr></table><div class="suggestionsBox" id="suggestions" style="display: none;" mce_style="display: none;"><img src="upArrow.png" mce_src="upArrow.png" style="position: relative; top: -12px; left: 30px;" alt="upArrow" /><div class="suggestionList" id="autoSuggestionsList"> </div></div></form></body></html>

/////////bloginputhint.php

<?php// PHP5 Implementation - uses MySQLi.// mysqli('localhost', 'yourUsername', 'yourPassword', 'yourDatabase');$db = new mysqli('localhost', 'root' ,'841025', 'csdnblog');if(!$db) {// Show error if we cannot connect.echo 'ERROR: Could not connect to the database.';} else {// Is there a posted query string?if(isset($_POST['queryString'])) {$queryString = $db->real_escape_string($_POST['queryString']);// Is the string length greater than 0?if(strlen($queryString) >0) {// Run the query: We use LIKE '$queryString%'// The percentage sign is a wild-card, in my example of countries it works like this...// $queryString = 'Uni';// Returned data = 'United States, United Kindom';// YOU NEED TO ALTER THE QUERY TO MATCH YOUR DATABASE.// eg: SELECT yourColumnName FROM yourTable WHERE yourColumnName LIKE '$queryString%' LIMIT 10$query = $db->query("SELECT user FROM blogname WHERE user LIKE '$queryString%' LIMIT 10");if($query) {// While there are results loop through them - fetching an Object (i like PHP5 btw!).while ($result = $query ->fetch_object()) {// Format the results, im using <li> for the list, you can change it.// The onClick function fills the textbox with the result.// YOU MUST CHANGE: $result->value to $result->your_colum echo '<li onClick="fill(/''.$result->user.'/');">'.$result->user.'</li>'; }} else {echo 'ERROR: There was a problem with the query.';}} else {// Dont do anything.} // There is a queryString.} else {echo 'There should be no direct access to this script!';}}?>

效果图：