2 graham Feb 07, 2005 00:07

That's great! Thanks Graham! :D
The crawlers are able to interpret the php url parameters such as "?disp=arcdir" as a valid site address?
And if so, I notice the HTML 'head' section for the archive directory has a robots tag with "noindex, follow" specified.
The way I read it, the follow would indicate that they'll be crawled, but the noindex means that they won't be indexed? Shouldn't I change that to "index, follow" if I want my old blogs to be indexed by the search engine bot?
Tags in the document head refer to the specific page - not anything beyond that page.
If you have links to your archives on your front page, then they'll be crawled.