HttpWebRequest获取网页源代码时自动识别网页编码

来源：百度文库 编辑：神马文学网 时间：2024/04/29 07:58:49
HttpWebRequest获取网页源代码时自动识别网页编码，通过读取页面中的charset和读取http头中的编码信息获取页面的编码，基本可以正确获取网页编码static string GetEncoding(string url)
        {
            HttpWebRequest request = null;
            HttpWebResponse response = null;
            StreamReader reader = null;
            try
            {
                request = (HttpWebRequest)WebRequest.Create(url);
                request.Timeout = 20000;
                request.AllowAutoRedirect = false;                response = (HttpWebResponse)request.GetResponse();
                if (response.StatusCode == HttpStatusCode.OK && response.ContentLength < 1024 * 1024)
                {
                    if (response.ContentEncoding != null && response.ContentEncoding.Equals("gzip", StringComparison.InvariantCultureIgnoreCase))
                        reader = new StreamReader(new GZipStream(response.GetResponseStream(), CompressionMode.Decompress));
                    else
                        reader = new StreamReader(response.GetResponseStream(), Encoding.ASCII);                    string html = reader.ReadToEnd();                    Regex reg_charset = new Regex(@"charset\b\s*=\s*(?[^""]*)");
                    if (reg_charset.IsMatch(html))
                    {
                        return reg_charset.Match(html).Groups["charset"].Value;
                    }
                    else if (response.CharacterSet != string.Empty)
                    {
                        return response.CharacterSet;
                    }
                    else
                        return Encoding.Default.BodyName;
                }
            }
            catch
            {
            }
            finally
            {                if (response != null)
                {
                    response.Close();
                    response = null;
                }
                if (reader != null)
                    reader.Close();                if (request != null)
                    request = null;            }            return Encoding.Default.BodyName;
        }        ///
        /// 获取源代码
        ///
        ///
        ///
        static string GetHtml(string url, Encoding encoding)
        {
            HttpWebRequest request = null;
            HttpWebResponse response = null;
            StreamReader reader = null;
            try
            {
                request = (HttpWebRequest)WebRequest.Create(url);
                request.Timeout = 20000;
                request.AllowAutoRedirect = false;                response = (HttpWebResponse)request.GetResponse();
                if (response.StatusCode == HttpStatusCode.OK && response.ContentLength < 1024 * 1024)
                {
                    if (response.ContentEncoding != null && response.ContentEncoding.Equals("gzip", StringComparison.InvariantCultureIgnoreCase))
                        reader = new StreamReader(new GZipStream(response.GetResponseStream(), CompressionMode.Decompress), encoding);
                    else
                        reader = new StreamReader(response.GetResponseStream(), encoding);
                    string html = reader.ReadToEnd();                    return html;
                }
            }
            catch
            {
            }
            finally
            {                if (response != null)
                {
                    response.Close();
                    response = null;
                }
                if (reader != null)
                    reader.Close();                if (request != null)
                    request = null;            }            return string.Empty;
        }本文来自CSDN博客，转载请标明出处：http://blog.csdn.net/vrix/archive/2009/10/03/4629036.aspx
HttpWebRequest获取网页源代码时自动识别网页编码 网页源代码查看器-站长帮手网 javascript实现禁止查看网页源代码 破解所谓的“网页源代码加密” javascript实现禁止查看网页源代码12121 javascript实现禁止查看网页源代码341 网页固定位置挂件的源代码 **破解所谓的“网页源代码加密” 用Excel获取网页内容的方法 用Excel获取网页内容的方法 用Excel获取网页内容的方法 用Excel获取网页内容的方法 获取网页里的flash地址 HTML代码基础知识：如何查看一个网页的HTML源代码？ 让你的网页实现禁止查看源代码 博客网页常用几款【播放器】HTML编码 VC知识库文章 - 如何获取网页密码框中的密码 如何获取网页里的flash地址_ 双色球开奖时间如何获取网页里的flash地址 C# WebBrowser 抓图获取网页验证码 | 真有意思 用Boost.regex库进行网页分析源代码（批量下载篇） 破解动态地址的RayFile网页播放器源代码 - RayFile | Fs2You - 爱... 网页编码转换软件 UTF-8转GBK或者GBK转UTF-8 java调用jruby获取网页内容（JDK1.5） - coldtest的专栏 - CSD...