《c# 抓取Web网页数据分析》由会员分享，可在线阅读，更多相关《c# 抓取Web网页数据分析（2页珍藏版）》请在金锄头文库上搜索。
1、c#抓取Web网页数据分析详细出处参考：http:/ 把分析的结构，即整理好的数据写入数据库。那么我们的思路就是：1、发送 HttpRequest 请求。2、接收HttpResponse返回的结果。得到特定页面的html源文件。3、取出包含数据的那一部分源码。4、根据html源码生成HtmlDocument，循环取出数据。5、写入数据库。程序如下：复制代码代码如下：根据Url地址得到网页的html源码private string GetWebContent(string Url) string strResult=”;try HttpWebRequest request = (HttpWebR
2、equest)WebRequest.Create(Url);声明一个 HttpWebRequest 请求request.Timeout = 30000;设置连接超时时间request.Headers.Set(Pragma”, no-cache);HttpWebResponse response = (HttpWebResponse)request.GetResponse();Stream streamReceive = response.GetResponseStream();Encoding encoding = Encoding.GetEncoding(GB2312);StreamRead
3、er streamReader = new StreamReader(streamReceive, encoding);strResult = streamReader.ReadToEnd();catchMessageBox.Show(出错”)；return strResult;为了使用HttpWebRequest和HttpWebResponse，需填名字空间引用using System.Net;以下是程序具体实现过程：private void button1_Click(object sender, EventArgs e) 要抓取的URL地址string Url = http:/ strW
4、ebContent = GetWebContent(Url);richTextBoxl.Text = strWebContent;取出和数据有关的那段源码int iBodyStart = strWebContent.IndexOf(body”, 0);int iStart = strWebContent.IndexOf(歌曲 TOP500, iBodyStart);int iTableStart = strWebContent.IndexOf(table”, iStart);int iTableEnd = strWebContent.IndexOf(”, iTableStart);string
5、 strWeb = strWebContent.Substring(iTableStart, iTableEnd - iTableStart + 8);生成 HtmlDocumentWebBrowser webb = new WebBrowser();webb.Navigate(about:blank);HtmlDocument htmldoc = webb.Document.OpenNew(true);htmldoc.Write(strWeb);HtmlElementCollection htmlTR = htmldoc.GetElementsByTagName(TR);foreach (H
6、tmlElement tr in htmlTR) string strID = tr.GetElementsByTagName(TD)0.InnerText;string strName = SplitName(tr.GetElementsByTagName(TD)1.InnerText, MusicName);string strSinger = SplitName(tr.GetElementsByTagName(TD)1.InnerText, Singer);strID = strID.Replace(.”, );插入 DataTableAddLine(strID, strName, st
7、rSinger,0);string strID1 = tr.GetElementsByTagName(TD)2.InnerText;string strName1 = SplitName(tr.GetElementsByTagName(TD)3.InnerText, MusicName);string strSinger1 = SplitName(tr.GetElementsByTagName(TD)3.InnerText, Singer);插入 DataTablestrID1 = strID1.Replace(.”, );AddLine(strID1, strName1, strSinger
8、1,0);string strID2 = tr.GetElementsByTagName(TD)4.InnerText;string strName2 = SplitName(tr.GetElementsByTagName(TD)5.InnerText, MusicName);string strSinger2 = SplitName(tr.GetElementsByTagName(TD)5.InnerText, Singer);插入 DataTablestrID2 = strID2.Replace(.”, );AddLine(strID2, strName2, strSinger2,0);插入数据库InsertData(dt);dataGridView1.DataSource = dt.DefaultView;详细出处参考：http:/
