 |
ASP Kitchen |
|
ASP Kitchen |
Ensuring Your Site is Search Engine FriendlyThe ASPAlliance is full of useful articles about how to build your website using ASP and ASP.NET. But even though you may have the most technically brilliant website around, it's going to be of limited use if the website isn't able to be crawled by the web robots that Google and the other search engines use to find content to add to their search catalogs. This article highlights some of the (surprisingly common!) problems that affect search engine robots. The problems can be divided into two groups:
If you want to test your website for many issues, then an application I wrote called The Website Utility can be used. It contains a built-in web crawler which can be used to test whether web robots can successfully navigate a website. It also reveals common website search engine optimization issues. Show Stoppers!These are serious issues that will prevent a web crawling robot from indexing some or in a worst case scenario, all of a website's pages. Here are a few commonly encountered problems: Invalid HTMLWeb crawling robots may have difficulties navigating websites. These issues can be hard to spot, since most web browsers are tolerant of sloppy HTML. To resolve possible errors, try using an HTML validator such as the W3C's HTML validation service. You should pay particular care to ensuring that the website's internal hyperlinks are using valid HTML. For example, this is a valid link: <a href="default.asp" title="Home">Home Page</a> Whereas this link may not be recognised by a web crawling robot: <a href=default.asp title=Home>Home Page</a> Robots.txt or Robots Meta Tag IssuesMost web robots, including those used by search engines to seek out new content for inclusion in their directories will obey both a web server's robots.txt file as well as the 'robots' meta tag that may be present in individual pages. If a website, or portions of that website cannot appear to be submitted successfully to search engines it is possible that the search engine is not accepting that website due to it being marked as non-retrievable in either the robots.txt file or in individual files through the use of the 'robots' meta tag. JavaScript NavigationWeb crawling robots are getting more sophisticated, but they will almost certainly have problems following links that are only accessible from JavaScript. This is common on websites that have JavaScript-based drop-down menus as their main means of site navigation. It is, therefore, a good idea to ensure that the website is fully navigable without having to rely on JavaScript. Needless to say, providing alternative means of navigation will also assist human visitors using web browsers that do not support JavaScript, or those that have disabled JavaScript in their browsers. As well as JavaScript, this issue also applies to links that are only in widgets such as Java Applets and Macromedia Flash animations. Compulsory CookiesAlthough it is perfectly acceptable to use cookies on a website, care should be taken to avoid compulsory cookies (i.e. the website won't work if a specific cookie isn't set). The easiest way to check this is to browse the website with cookies disabled in the web browser. Some web crawling robots are also able to support session cookies, such as those used by ASP and ASP.NET. Search Engine Optimisation DeficienciesMissing Page TitlesMissing page titles is a common search engine optimisation deficiency. Since many search engines place significant weighting on a page's title, it is essential to ensure all pages have a title. Duplicated Page TitlesAlongside missing page titles, duplicate page titles is another common issue. The report will show each duplicated title and which pages share the specific title. A surprising number of websites use the same page title every single page in the site, which is in most instances an obvious waste of potential site optimisation for search engines. Missing/Blank Description and Keywords Meta TagsAlthough most search engines now give much less weighting to the content of description and keywords meta tags than they once did, it is still a good idea to include them. In particular, the description meta tag is often used as a page's summary in search engine results pages. It is also worth including them if your website has its own search facility based on Microsoft's Index Server or The Website Utility's ASP Search Engine or JavaScript Search Engine. Insufficient Indexable ContentSearch engines are still mostly based on the retrieval and indexing of text content on web pages. Consequently, it is important to ensure there is plenty of text on . Although file size is often an indication of the amount of indexable text on a page, this may also indicate that the page contains a lot of non-indexable content (e.g. HTML tags and JavaScript). Testing Your WebsitesA good way of testing to ensure your website is search engine friendly is to crawl it using a web crawler. The Website Utility is a Windows application that can be used to test the crawlability of websites. It contains a built-in web crawler which can be used to test whether web robots can successfully navigate a website. It also reveals common website search engine optimization issues and errors such as broken links. Finally, remember that testing a website for errors and search engine optimisation issues will almost certainly result in higher levels of satisfaction from the owner - useful if your livelihood depends on happy customers and repeat business! Useful Development Tools
Author detailsBrett Burridge has worked as a web developer since 1997 and has developed web applications for a range of corporations, start up busiensses and educational establishments. Brett is presently employed as an Internet developer and technical writer through his own company, Winnersh Triangle Web Solutions Limited. The company produces a number of innovative products, including a range of software documentation tools, which include the ASP Documentation Tool™, the .NET Documentation Tool for VB.NET and C#, and the SQL Server Documentation Tool. Other products include The Website Utility, which functions as a website error checker, search engine optimizer and ASP/ASP.NET search engine builder application. As well as the ASPAlliance, Brett has written articles for Ariadne.ac.uk, ASPToday, the software documentation portal www.softwaredocumentation.info, and has contributed recipes to the ASP.NET Developer's Cookbook. links Outside web development, Brett is interested in travelling (here are my travel logs from New York, Hong Kong and Tokyo), digital photography (here's my photo gallery), tropical fishkeeping and collecting contemporary works of art by artists such as Doug Hyde. Contact Brett by emailing
Article history"Ensuring Your Site is Search Engine Friendly" published on ASPAlliance.com on 22 September 2004. ASP Kitchen: Classic ASP Articles: Ensuring Your Site is Search Engine Friendly |
|
||||||||||||||||||||||||
| © page content copyright Brett Burridge 1998 - 2008. | |||||||||||||||||||||||||
