神马文学网.net 2.0的URL重写导致的Bug及...
来源：百度文库 编辑：神马文学网 时间：2024/04/30 17:17:46
最近在服务器的日志中出现许多Web事件的警告信息。时间间隔非常规律，一分钟出现一至两次，其错误信息摘取关键部分如下：
事件类型: 警告
事件来源: ASP.NET 2.0.50727.0
事件种类: Web 事件
事件 ID: 1309
日期: 2008-1-13
事件: 14:43:41
用户: N/A
描述:
事件代码: 3005
事件消息: 发生了未处理的异常。
事件时间: 2008-1-13 14:43:41
事件时间(UTC): 2008-1-13 6:43:41
事件 ID: f94c4a45b696449db6df2a3d571059e3
事件序列: 110527
事件匹配项: 9121
事件详细信息代码: 0
异常信息:
异常类型: HttpException
异常消息: 无法使用前导 .. 在顶级目录上退出。
请求信息:
请求 URL: http://dormforce.net/Music/*****.aspx?id=7085&id2=633358175566250000
请求路径: /Music/*****.aspx
用户主机地址: 66.249.66.38
用户:
是否已经过身份验证: False
身份验证类型:
线程帐户名: NT AUTHORITY\NETWORK SERVICE
线程信息:
线程 ID: 1
线程帐户名: NT AUTHORITY\NETWORK SERVICE
是否正在模拟: False
堆栈跟踪: 在 System.Web.Util.UrlPath.ReduceVirtualPath(String path)
在 System.Web.Util.UrlPath.Reduce(String path)
在 System.Web.Util.UrlPath.Combine(String appPath, String basepath, String relative)
在 System.Web.HttpResponse.ApplyAppPathModifier(String virtualPath)
在 System.Web.UI.HtmlControls.HtmlForm.GetActionAttribute()
在 System.Web.UI.HtmlControls.HtmlForm.RenderAttributes(HtmlTextWriter writer)
在 System.Web.UI.HtmlControls.HtmlControl.RenderBeginTag(HtmlTextWriter writer)
在 System.Web.UI.HtmlControls.HtmlForm.Render(HtmlTextWriter output)
在 System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)
在 System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter)
在 System.Web.UI.HtmlControls.HtmlForm.RenderControl(HtmlTextWriter writer)
在 System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)
在 System.Web.UI.Control.RenderChildren(HtmlTextWriter writer)
在 System.Web.UI.Page.Render(HtmlTextWriter writer)
在 System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)
在 System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter)
在 System.Web.UI.Control.RenderControl(HtmlTextWriter writer)
在 System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
最初以为是源代码中的问题，打开源码仔细分析没有找到逻辑上的错误，不得其解。查找主机的IP地址，发现居然是GoogleBot 搜寻引擎机器人，后来在网上看到一篇文章，Get GoogleBot to crash your .NET 2.0 site
才知道这是由于ASP.NET 2.0 本身的一个bug造成的。
解决方法不复杂，在根目录建立一个App_Browsers文件夹，在里面写一个以.browser扩展名的文件，并对里面的文件进行相应的设置，对于现在遇到的这个问题，我的设置如下：
其实这个Bug不解决，GoogleBot就搜不到某些页面了。（貌似也没什么关系-_-）
下面把原文也引用一下：（注意一下，原文中的App_browser少了一个s)
Get GoogleBot to crash your .NET 2.0 site
If you’re developing in ASP.NET 2.0 and you’re using url rewriting, you should proceed with caution. Especially if you value your ranking in search engines. I’m posting this as a follow up and with reference to the original find on this post. The issue came about in a thread on the Community Server 2.0 forums. I was fast to post a solution to the problem, but obviously, it’s more about working around the issue than actually solving the root cause.
Url rewriting is mostly used to make URL’s more friendly to read. Also, if you have migrated from one site to another and you want accommodate people still linking to old urls, rewriting is a common practice. Nonetheless, if you use this on a public internet website, it won’t be long until you see the following exceptions listed in your Event Log:
Exception type: HttpException
Exception message: Cannot use a leading .. to exit above the top directory.
...
Stack trace: at System.Web.Util.UrlPath.ReduceVirtualPath(String path)
...
For any site of significance, and www.codes-sources.com is most certainly one of them, this exception was logged more than 2000 times … every hour. Now, that’s something to notice, right? And the effect on your ranking in the search engines? Well, within a few days, your site is either kicked out completely from the index or the index contains nothing more than the url of your site without content. Nobody looking for content you may have on your site will be directed to it. Worried? Read on.
My personal instinct would be: it’s something I did wrong. So one takes a long time trying to figure out what that is. But in fact, it’s a bug in a .NET component that’s not easy to trace and reproduce. If you don’t check the event logs every once in a while, it can surely be missed. Let’s take a look at what’s going on:
A first note to people trying to reproduce the issue, the bug does not appear using Cassini (the built-in web server in Visual Studio 2005). You need to have a running IIS 6 web server on Windows 2003. Doesn’t matter if it’s in a VPC or on an actual server.
If you’re using url rewriting in .NET 2.0, you have the Context.RewritePath method at your disposal. Here’s a sample project for testing.
1. First you create a page, say page.aspx
2. In this page, you can put whatever you want; it doesn’t really matter. For example:
<%=Request("ID")%>
3. Then you add your rewriting HttpModule, with the following implementation:
Public Class Rewriter
Implements System.Web.IHttpModule
Public Sub Dispose() Implements System.Web.IHttpModule.Dispose
End Sub
Public Sub Init(ByVal context As System.Web.HttpApplication) Implements System.Web.IHttpModule.Init
AddHandler context.BeginRequest, AddressOf Me.HandleBeginRequest
End Sub
Private Sub HandleBeginRequest(ByVal [source] As [Object], ByVal e As EventArgs)
Dim app As System.Web.HttpApplication = CType([source], System.Web.HttpApplication)
app.Context.RewritePath(“~/page.aspx?ID=1”, False) ' sidenote, same effect when using “/page.aspx?ID=1”
End Sub
End Class
As you can see, it’s a simple example rewriting all urls to page.aspx?ID=1. It’s does not serve a specific function, other than show the problem at hand. Now, add the HttpModule in the Web.Config file.
With Fiddler (available at www.fiddlertool.com), you can create web requests and analyze the result in very good detail. It’s especially useful in this case, as you can create a request specific for certain user-agents. So download the tool and setup your ASP.NET 2.0 site on an IIS 6.0 environment. One thing to note as well, is that this site needs to be running under its own hostheader, not as a virtual directory.
Once installed, you take your web browser and go to
http://localsitename/default.aspx
The page default.aspx will be rewritten as page.aspx?ID=1 and everything works just fine.
Now, open up Fiddler and create the following request:
Accept: */*
Accept-Encoding: gzip, x-gzip
User-Agent: Mozilla/4.0
Set the url to
http://localsitename/default.aspx
and hit Execute. You should get status code 200, meaning OK. Now set the url to
http://localsitename/justafolder/default.aspx
and after you hit OK, again, you will get a 200 code. No problems so far.
Now, change the request to
User-Agent: Mozilla/5.0
instead of User-Agent: Mozilla/4.0
Hit Execute and bang… error 500, indicating an application error.
Here’s a list of user-agent entries that will result in an error:
Mozilla/1.0
Mozilla/2.0
Mozilla/5.0
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
Yahoo-Blogs/v3.9 (compatible; Mozilla 4.0; MSIE 5.5; http://help.yahoo.com/help/us/ysearch/crawling/crawling-02.html )
Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://sp.ask.com/docs/about/tech_crawling.html)
Mozilla/5.0 (compatible; BecomeBot/3.0; MSIE 6.0 compatible; +http://www.become.com/site_owners.html)
Mozilla/5.0 (compatible; Konqueror/.... (Tous les users agent de Konqueror que j'ai testés plantent)
Etc...
Some funny details:
Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 <= no error
Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.1) <= error 500!
Ok, so let’s try to explain what happens. If you call RewritePath with the rebaseClientPath parameter set to “True”, the virtual path is reset. So why set it to False? Well, the setting of rebaseClientPath affects the action-tag of a form.
If I have an url http://mysite/myfolder/mypage.aspx which is rewritten to http://mysite/page.aspx?id=mypage, the form tag will we set as follows.
With rebaseClientPath set to true:
But with rebaseClientPath set to false:
In case of a postback, the action in the latter situation (with rebaseClientPath set to false) is correct. Not in the first instance, because there is no page.aspx in the subfolder /myfolder.
Now, a workaround would be to manually set the UrlPostback to the correct location, but the ramafications are significant, and may affect the execution of the javascript manipulation the UrlPostback on a number of browsers.
What’s really troublesome is that it affects only the production version of a website. It’s not visible during development (usually you don’t let searchengines index your development and test environment, right?). Also, it happens only under IIS 6.0, using ASP.NET 2.0. And the only variable in this case is the user agent with which the site is accessed.
Now we know what the issue is, how do we resolve it? Well, doing it yourself is not easy and without risk, but here goes.
It has to do with the capabilities of the user-agent. When the site is hit by a user-agent having Mozilla/5.0 in the string, ASP.NET will be using System.Web.UI.Html32TextWriter. I’ve traced the bug with Intercept Studio and can confirm this. If you use another user-agent, for example Mozilla/4.0, System.Web.UI.HtmlTextWriter will be used. This is called adaptive rendering, which can lead to weird behavior.
So the problem lies in Html32TextWriter, but it’s unclear where this goes wrong exactly. The exception is thrown in System.Web.Util.UrlPath.ReduceVirtualPath(), but by the time the stack reaches this method, we’re already six method calls away from the last usage of Html32TextWriter (being used in System.Web.UI.HtmlControls.HtmlForm.RenderAttributes). If you have time to walk the stack with Reflector, go ahead. There are about 23 method calls in all.
You may want to wait for a fix to come from Microsoft, but if you can’t wait that long, there’s a hack solution to try for yourself. Say for instance you want to fix the issue with the Yahoo searchbot, Yahoo! Slurp (you would need to apply this to all user-agents in a similar fashion.)
Since Visual Studio 2005 we have the capability to create .browser files. These .browser files contain a definition of the capabilities of browsers. In your web project, add a folder called “App_Browser” and create a new file (i.e.: yahooslurp.browser). In this file, you put
Now, restart your website and redo the tests with Fiddler, setting the user-agent to
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
Voilà, no more 500 error message. Repeat the steps for the different user-agents out there.
There may be other possibilities to fix this problem, but this one seems to be the most straightforward to implement and doesn’t require you to recompile the code. With solutions like CommunityServer, you simply don’t have this option anyway. It will suffice to add the App_Browser folder and the various .browser files to the root folder of the website and it will work.
At the end of the original post by Nix, there are some closing remarks. For example, if you create the site as a virtual directory under a website root, the problem does not appear. Changing the user-agent in a header, as suggested by Poppyto, might work, but the consequences are uncertain. Also, you would need to recompile.
Now, you might think, oh well, just add the appropriate .browser files and we're done. But that’s a bit short sighted. You never know when Google or any other search-engine decides to change the user-agent string. Your first notification would come from the exception messages in your event log. I manage about 7 big Windows 2003 servers in our datacenter. Perhaps not much, but I know what issues can come along every once in a while). Both as a developer and a system administrator, I would rather prevent issues, than handle them afterwards.
If you want to debate the issue, please go to the thread at the Community Server forums.
PS: Why did I write this post? Well, the entire explanation is written in French, and not everyone can read this. Also, Nix and I know each other and he’s been helpful on a number of occasions, so if there’s anything I can do in return, like explaining the issue in English, I’m more than happy to help. Finally, this problem needs a proper solution, and the sites and servers I host can be just as much affected. I was fortunate not to have too much url rewriting in my sites, but that doesn’t affect the scope of the issue.