>>17
Sitemaps standard: http://sitemaps.org/

It's used by Google for finding every link in a site along with how often they are updated... a nice, centralised listing. In theory it replaces robots.txt although they recommend using both (at least until everyone uses the new one anyway.)

>>19
It's true that sitemaps can be large, but the number of hits a person makes on your site can also be large -- remember, the OP was suggesting that every single file on your site gets this new header... that means all images, all javascripts, all stylesheets... this will add up as they visit it more and more, whereas you only need to fetch the sitemap once, and you only need to fetch the favicon as often as the sitemap file tells you to (it might say "this favicon refreshes once every year.")