Sitemap – Complete Guide
What is a Sitemap and why do you need it?
Sitemap is a hierarchical list of all the webpages in a website listed in a XML format. The webpages in this list are accessible to crawlers and users. All the major search engines like Google, Yahoo, Bing and Ask support the XML Sitemaps. With this Sitemap guide we have tried to list all aspects you need to know for building and using a Sitemap.
The listing of the webpages can be in 2 formats
- Sitemap – Top Down View of website
- Site Index – Arranged in an alphabetical order
Sitemaps can improve the SEO of your website by making sure that all pages are found by the search engines. It makes easy for a search engine to keep track of all pages and crawl them properly. It is also helpful for sites built using Adobe Flash or any other non-HTML languages. Although, sitemaps do not guarantee that all pages will be crawled and if crawled, it doesn’t guarantee that they will be indexed. Search engines give more value to websites which are free of broken or unreachable links; ‘dirt‘ as they call them.
In an interview a few months ago with Stone Temple Consulting, Bing’s Duane Forrester said this:
“Your Sitemaps need to be clean. We have a 1% allowance for dirt in a Sitemap. Examples of dirt are if we click on a URL and we see a redirect, a 404 or a 500 code. If we see more than a 1% level of dirt, we begin losing trust in the Sitemap”.
Refer our previous post to know more on How to Fix Crawl Errors
How to build and maintain a good sitemap?
Sitemaps for search engines are written in XML format. The sitemap.xml file is placed in the root directory of the web server and you can access it by visiting www.yoursite.com/sitemap.xml . You can have a look at the sitemap of any website by navigating to their sitemap address which is usually similar to http://www.searchenabler.com/sitemap.xml
Let’s have a look at the tags which are used in the Sitemap (as defined by www.sitemaps.org in their documentation):
- Format for Sitemap.xml:
- Sitemap format for images:
- Tools to generate Sitemap.xml:
The ‘urlset‘, ‘url‘ and ‘loc‘ are compulsory tags while other tags are optional and you can use them if you want but we recommend you to use those optional tags.
If there are images in a webpage that you want to get indexed, use this format with special image tags.
Sitemap is best handled if generated automatically, whenever there is a change in your site. We do similarly by using plugin over our site which is built using wordpress.
WordPress sitemap plugin: http://www.arnebrachhold.de/projects/wordpress-plugins/google-xml-sitemaps-generator/
If you are not using any standard framework or don’t want to use plugins, you can try tools like
Steps to validate, upload and maintain Sitemaps:
- Validating sitemap
- Submitting sitemap
- Submit sitemap to Google Webmaster
- Submit sitemap to Bing Webmaster
- Mention the location of Sitemap file in robots.txt
- Updating the Sitemap using HTTP request
- Maintain Sitemap using SearchEnabler SEO Platform
Also mention the location of Sitemap in the robots.txt file. Read more about ‘What is robots.txt – A quick Tutorial‘ for more information.
Use the format displayed below to mention the sitemap location in robots.txt file.
When a Sitemap is registered, you can Ping request if a sitemap is updated without having to do the complete process again using Webmasters Tool.
The format for the HTTP request is:
Replace the ‘example‘ with your website’s actual URL.
You can always check the Sitemap errors using SearchEnabler SEO Platform which constantly scans the website and displays errors so that you can correct them.
The above steps will make your Sitemaps error free and keep them updated so that search engines can scan them properly. The whole process will gradually improve the SEO of the website.