1 yasserhamed Feb 06, 2005 23:44
3 yasserhamed Feb 07, 2005 00:43
That's great! Thanks Graham! :D
4 stk Feb 10, 2005 01:36
The crawlers are able to interpret the php url parameters such as "?disp=arcdir" as a valid site address?
And if so, I notice the HTML 'head' section for the archive directory has a robots tag with "noindex, follow" specified.
The way I read it, the follow would indicate that they'll be crawled, but the noindex means that they won't be indexed? Shouldn't I change that to "index, follow" if I want my old blogs to be indexed by the search engine bot?
5 edb Feb 10, 2005 01:41
Tags in the document head refer to the specific page - not anything beyond that page.
If you have links to your archives on your front page, then they'll be crawled.