Lang Code Project

Some times we need to determine the referrer URL. For example, an online advertising company wants to display special information based on which page its audience is looking at. A common way is checking the referrer URL of the current scripts page then embedded in the publisher’s pages.

However, different browser may have different behavior while sending the referrer URL in HTTP request header. Most of browsers except IE will escape the path and query part to % form, and this is what the standard required. IE will convert the query part to UTF-8 encoding before sending it, while other browsers will just leave it as the current encoding using for display. For Firefox, it will be a little bit more complicated, since it can be configured by the about:config page.

Here are some examples:

• search.php?q=one two, will be sent as it is by IE, and be sent as search.php?q=one%20two by others.
• search.php?q=中文, will be sent as it is by IE, however, the Chinese charactes are encoding to UTF-8 no matter what encoding the page is. For other browser, it will be sent as search.php?q=%E4%B8%AD%E6%96%87 if search.php is displayed using UTF-8 or as search.php?q=%D6%D0%CE%C4 if search.php is displayed using GB2312.

I will try to give in depth explanation on this issue in the coming articles. Stay tune!

This entry is filed under Lang Code, Spider. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.