sitemap
This article is a stub. You can help the IndieWeb wiki by expanding it with relevant information.
A sitemap is a structured list of pages on a website that helps people and software discover content. Sitemaps can be human-readable HTML pages designed for visitors, machine-readable XML files intended for search engines, or both.
Why
It can be easier to find the content you're looking for on a sitemap, since they are typically text-heavy pages and do not require navigating through a site which may be loading lots of Javascript or image content.
Types of Sitemaps
Websites commonly provide both human-readable HTML sitemaps for visitors and XML sitemaps for search engines and other automated tools.
Relationship to Robots.txt
XML sitemaps are commonly referenced from a site's robots txt file so that search engines can discover them more easily.
IndieWeb Examples
Zachary Kai
https://zacharykai.net/sitemap.xml (XML)
Anthony Ciccarello
Anthony Ciccarello has a human readable sitemap since 2024-10-25.
Zinzy Waleson Geene
Zinzy Waleson has a human-readable sitemap on https://zinzy.website/sitemap (HTML)
Thamara Kandabada
Thamara Kandabada has a sitemap on https://thamara.co.uk/sitemap-0.xml (XML)
Peter's Path
Peter's Path has a sitemap at https://peterspath.net/sitemap.xml (XML).
Joe Crawford
Joe Crawford has a
/sitemapslash page at artlung.com/sitemap and also includes a button to open a dialog with it on most of the pages of his site. He also includes sidefile XML sitemaps for search engines, and linked from robots.txt.- He has shared PHP code to generate sitemap.xml files from plain text files: [1] [2] [3]
Trevor Morris
trovster has a
/sitemap slash page at www.trovster.com/sitemap. He also includes a XML sitemaps for search engines. Both of these are maintained from a global navigation configuration, so they are kept in sync.
Add yourself!
Add yourself here⦠(see this for more details)
Past Examples
Bret Comnes
Bret Comnes, archived example (XML)
Organization Examples
Examples of corporate or other organization sites with sitemaps.
- https://www.apple.com/sitemap/
- https://www.ibm.com/sitemap/
- http://www.microsoft.com/en-us/sitemap.aspx
- https://www.un.org/en/site-index
- https://www.unodc.org/unodc/en/site-map.html
Subsites
Some sites have subsites, e.g. at subdomains, which have their own sitemaps.
Past Examples
- Google used to have a sitemap at http://www.google.com/sitemap.html however that now redirects to http://www.google.com/about/products/ which still resembles a sitemap but it's not clear if that's its intent (judging by the renaming)
- The W3C had a sitemap at http://www.w3.org/Consortium/siteindex.html but that page was redirected to the homepage in 2023. (archived version)
- Yahoo subsites had their own sitemaps. Note the inconsistent paths.
- Yahoo news had a sitemap at https://my.news.yahoo.com/sitemap/ until 2014 (archive)
- Yahoo finance had a sitemap at http://finance.yahoo.com/sm until 2017 (archive)
Brainstorming
sitemap microformat
See http://microformats.org/wiki/sitemap for the effort to develop a sitemap microformat to markup visible sitemap pages for semantic crawling and indexing, eliminating any need for a separate XML sitemap sidefile.
sitemap graphic
The [Internet Archive] has a sitemap graphic which shows the pages of a site by path segment.
Example: ciccarello.me sitemap
XML sitemap
There is also an XML sitemap file format:
However, this XML sitemap file is a sidefile that is subject to the same risks as feed files. Because it exists separately from the primary website content, it can become outdated, incomplete, or inconsistent if it is not generated automatically from the same source as the rest of the site. If it is maintained separately from the website, it may omit pages, include stale URLs, or otherwise become inconsistent with the site's actual content.
XML sitemap resources
Specifications, documentation and tools for creating and using XML sitemaps.
- https://developers.google.com/search/docs/crawling-indexing/sitemaps/overview β Google Search Central introduction to XML sitemaps.
- https://seotesting.com/free-seo-tools/xml-sitemap-generator/ β Free XML Sitemap Generator for creating sitemap.xml files.
- https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap β Google's guide to building and submitting XML sitemaps.
- https://www.bing.com/webmasters/help/sitemaps-3b5cf6ed β Microsoft Bing Webmaster documentation for XML sitemaps.
- https://digital.gov/resources/introduction-xml-sitemaps/ β U.S. General Services Administration guide to XML sitemaps.
Why XML sitemap
XML sitemap indexing
Google Webmaster Tools suggests[4] creating an XML file in the sitemaps.org format and submitting it with the claim that their crawler can "more intelligently crawl your site"[5]. Despite that, they also say that using a sitemap "doesn't guarantee you that all the web pages listed in your sitemap can be crawled or indexed".
XML sitemap unnecessary
On 2014-12-18, 
Aaron Parecki discovered that his sitemap.xml file had been empty, with no idea of how long it had been empty. Despite that, his content continues to be indexed by Google.
For many websites, especially smaller sites with strong internal linking, XML sitemaps provide relatively little benefit. While they can help search engines discover content more efficiently, they are not a substitute for a well-structured website.
