Search Engines Optimization (SEO)

1 ralphy Jun 13, 2005 18:09

I noticed every page on [url=http://blog.lesperlesduchat.com]my blog[/url] was first indexed by search engines. Then, search engines applied filters removing "noise". However, they appear to remove the blog articles URLs instead of removing a month or a year summaries.

Search engines users want to find articles about a particular subject containing specific keywords. Returning a whole year page containing all the searched keywords doesn't make sense. Thus, indexing a page containing several blog articles is a nonsense.

So, I made up a simple hack to insert into your <head> section of a skin:

<?php

$uri = $_SERVER[ 'REQUEST_URI' ];

$IsHomePage = preg_match( "/^\/[^\/\?]+$/", $uri );
$IsSingle   = preg_match( "/^\/[^\/\?]+.*[0-9]{1,4}(\/[0-9]{1,2}){2,2}\/[^?&]+$/", $uri );
$IsPage     = preg_match( "/^\/[^\/\?]+\?.*page=[0-9]+.*$/", $uri );

if( $IsHomePage || $IsSingle || $IsPage )
{
    // Index this page (and follow it either)
    echo( '<meta name="robots" content="index,follow"/>'."\n" );
}
else
{
    // Do not index this page (however, still follow it)
    echo( '<meta name="robots" content="noindex,follow"/>'."\n" );
}

?>

There are probably cleaner ways to implement this... a more b2evolution-oriented way. Anyway, in addition to that, my "All" blog contains only articles' titles and links to the site's blogs' full articles. The aim is to index each article only once on search engines indexes.

Since I made this change about two weeks ago, the traffic coming from search engines (mainly Google) increased by +91,6% (while direct access increased by +35,0% and external links by +28,3%) compared to last month figures.

2 isaac Jun 14, 2005 00:37

There are probably cleaner ways to implement this... a more b2evolution-oriented way.

Yep, it sure can.

I've approached the problem in a slightly different way: my goal is to only index non-changing content. So, I don't want to index the main page, but archives (monthly, etc.) can be indexed (since they don't change.)

While there will be some overlap (the yearly archive contains the monthly archive, etc.), this ensures that there aren't any cases where the search engine indexes something that then goes away. (I always feel bad when I see search hits that go to the main page where someone was searching for something that I blogged about last year.)

<meta name="robots" content="<?php if( $m || $w >= 0 && !isempty($w) || $disp=='single' ) echo 'all'; else echo 'noindex, follow'; ?>" />

Explanation:
If it's an archive page ($m and/or $w aren't empty) or a permalink ($disp is 'single') then set the robots meta to ALL (which is the same as "index,follow".) Otherwise, don't index the page, but do follow the links.

3 kwa Jul 13, 2005 19:55

Most search engines display pages containing the requested keywords. I don't think a search engine user is satisfyied finding the requested keywords on a page displaying dozens of articles published on a given year, month or week. However, chances are higher two or three (unrelated) keywords appear on your blog's year summary than on a single article. I'm afraid displaying a whole year summary won't satisfy that new visitor.

Moreover, most search engines use a PageRank-related system to rank pages. When a page is linked by many other pages, it improves its position in search enines results. When you include a list of months into your skin, the months pages have better PageRanks, since every blog's post link them, while few pages link to each post. If you let search engines to index months and years summaries, chances are higher they direct users to those summaries instead of direct visitors to specific articles.

Furthermore, week, month and year summary pages are probably the most CPU and bandwith expensive pages to serve.

So, I finally limited the indexing of my blogs' pages to single articles adding the following line to the head section of my main blog's skin (somewhere between the <head> and </head> (X)HTML tags):

<meta name="robots" content="<?php if( $disp != 'single' ) echo( 'no' ); ?>index,follow"/>

According to my site's AwStats statistics, my site served 116,601 pages and 13.71 GB in June 2005. The november 2005 summary is the most viewed "entry page" with 7,641 entries, 7,466 exits and 12,377 page views. That page alone used 2.3 GB of bandwith, probably 1.4 GB was useless, since visitors leaved it to go to another site. Two other summaries appear into the top ten of entry pages with a loss of about 1.6 GB bandwith in June (12% of the total June bandwith).

Since I also noticed RSS and ATOM feeds appear into my blog's top 10 entry pages, I used the [url=http://forums.b2evolution.net/viewtopic.php?t=4672]Simple Cache Hack[/url] to reduce my server's CPU usage on those pages.

4 kwa Jul 14, 2005 21:38

In order to redirect external visitors from your blog's month and year summary pages to the blog's homepage, add the following code at the beginning of your skins:

// Avoid useless heavy CPU and bandwith external links
if( !empty( $m ) && strlen( $m ) <= 6 && !empty( $_SERVER[ 'HTTP_REFERER' ] ) )
{
	// A large selection of posts has been requested and the
	// visitor comes from an external URI

	// Is it an external URI?
	global $blog, $BlogCache;
	$current_Blog = $BlogCache->get_by_ID( $blog );
	$blogurl = $current_Blog->gen_blogurl();
	$pos = stristr( $_SERVER[ 'HTTP_REFERER' ], $blogurl );
	if( $pos === FALSE || $pos > 8 )
	{
		// A year or month summary have been requested coming
		// from an external site.

		// Since this is a heavy page to display and often
		// useless for externally driven visitors, we redirect
		// them to the blog's homepage.
		header( 'Location: ' . $blogurl );
		die();
	}
}

The above code must appear before any display, since it is an HTTP header information redirecting command. It should also preceed any cache system.

A good location for the above code is just after the following line:

if( !defined('DB_USER') ) die( 'Please, do not access this page directly.' );

Please also notice using an .htaccess file to redirect those visitors is quicker, since the Apache web server does not have to execute any PHP code before redirecting. However, most people have probably no enough rights on their server to use a .htaccess file.

5 malakajoe Jul 25, 2005 16:52

isaac wrote:

There are probably cleaner ways to implement this... a more b2evolution-oriented way.

Yep, it sure can.

I've approached the problem in a slightly different way: my goal is to only index non-changing content. So, I don't want to index the main page, but archives (monthly, etc.) can be indexed (since they don't change.)

While there will be some overlap (the yearly archive contains the monthly archive, etc.), this ensures that there aren't any cases where the search engine indexes something that then goes away. (I always feel bad when I see search hits that go to the main page where someone was searching for something that I blogged about last year.)
<meta name="robots" content="<?php if( $m || $w >= 0 && !isempty($w) || $disp=='single' ) echo 'all'; else echo 'noindex, follow'; ?>" />
Explanation:
If it's an archive page ($m and/or $w aren't empty) or a permalink ($disp is 'single') then set the robots meta to ALL (which is the same as "index,follow".) Otherwise, don't index the page, but do follow the links.

Any way to have the main page and permalinks indexed, and not the other stuff?

6 tejaaa Jul 26, 2005 08:43

kwa wrote:

<meta name="robots" content="<?php if( $disp != 'single' ) echo( 'no' ); ?>index,follow"/>

My Google listing have dropped down terribly while going high on other search engines. What can I do to bring it on the top back? For example, I used to write "Tejas Patel" in Google and my current blog used to show up as the first entry before (www.itejas.com) but now it does not even come up on the listing of first 25 whereas my old blog still shows up first even though I have updated that for a long time.

Being a non developer, can you please explain me what the above code does? I have included this on the main template of my blog (blog no. 2)for now, will it make any difference?

Thanks,

Pingguru :)

7 kwa Jul 26, 2005 15:37

tejaaa wrote:

kwa wrote:
<meta name="robots" content="<?php if( $disp != 'single' ) echo( 'no' ); ?>index,follow"/>
My Google listing have dropped down terribly while going high on other search engines. What can I do to bring it on the top back? For example, I used to write "Tejas Patel" in Google and my current blog used to show up as the first entry before (www.itejas.com) but now it does not even come up on the listing of first 25 whereas my old blog still shows up first even though I have updated that for a long time.

Being a non developer, can you please explain me what the above code does? I have included this on the main template of my blog (blog no. 2)for now, will it make any difference?

The following code is intended for search engines robots:

<meta name="robots" content="<?php if( $disp != 'single' ) echo( 'no' ); ?>index,follow"/>

For each single page article (the "permalinks" links), the following (X)HTML tag is seen by your visitors (including search engines robots):

<meta name="robots" content="index,follow"/>

This tag instructs the search engines robots to index the page, and to follow the links appearing on this page to discover new pages (they are added to the list of pages to check in future web crawls).

For any other page (year summary, month summary and so on including your blog's homepage) the code discussed here produces the following (X)HTML tag is produced:

<meta name="robots" content="noindex,follow"/>

That instructs the search engines robots to avoid indexing the page. That means the page is not intended to appear as a result of a web search. However, that page may still appear as "existing" without caching its content by some robots or until the index is updated once upon time. This tag also invites the robots to follow the links appearing on that page to discover new pages (they are added to the list of pages to check in future web crawls).

When you search a keyword on a web search engine, the results are sorted using a "scoring" system. Each page has a given "score" associated for each keyword or set of keywords. Google includes a page's PageRank to a given page score. The PageRank (you can discover using the Google Toolbar) is a number ranged from 0 (very "confidential" page) to 10 (extremly popular page). The PageRank grows when the number of referrers to the page increases. To cumpute the PageRank, each page "gives" its own "score" divided by the number of links appearing on it to each of the linked pages. Having a link to your blog from a page with a PageRank of 8 makes your own PageRank increasing (and the position of your page in the Google search results becomes more interesting) more than from a page with a PageRank of 1. However, it is more interesting to have a link to your page from another page with a PageRank of 5 with 3 other links than from another page with a PageRank of 6 with 3,000 links.

That scoring system based on the number of pages linking your page makes some people to write billions of satellite pages or link farms mainly made of links to the pages one's want to see their PageRank increasing, making them having better position in search engines results. Other people create dozens of copies of the same sites on miscellaneous URLs hoping that might increase their traffic.

To fight this behaviour intended to mislead search engines robots, the scoring system includes some negative parameters. If you make copies of the same contents on several pages, your pages have lower scores than those same pages if they would have original content. The code discussed above is intended to avoid search engines from adding this penality, since each post is indexed only once.

Now, the code discussed here prevents your homepage from being indexed (since only single posts are):

<meta name="robots" content="<?php if( $disp != 'single' ) echo( 'no' ); ?>index,follow"/>

Searching your blog's title should follow search engine users to your most referenced article and not the homepage itself. But since few pages link other posts, looking for your blog's title may lead people on other sites with better ranked pages.

However, since your posts should have no more penalty for duplicated content, haven't your blog experienced more visits from search engines on meaningfull keywords?

Anyway, to index the homepage of your blog in addition to permalink pages, modify the initial code discussed here with that new version:

<meta name="robots" content="<?php if( ( $disp!='posts' && $disp!='single' ) || ( $disp=='posts' && ( $paged>1 || $cat!='' || $m!=0 || ( is_array( $catsel ) && count( $catsel )>0 ) || $w >=0 || $s != '' ) ) ) echo( 'no' ); ?>index,follow"/>

The main issue is going to be search engines are going to prefer proposing your homepage for recent posts instead of direct links. That's annoying if those posts have moved to other pages (since the homepage changes each time you publish a new post). Google updates its index about 1 to 2 days after crawling the page (see [url=http://www.google.com/addurl/?continue=/addurl]Add you URL to Google[/url] for manual submissions or see [url=http://forums.b2evolution.net/viewtopic.php?t=4466]Code to generate 'Google Sitemaps'[/url] for automatic Google pages submission). However, other search engines (including AOL) have lower reactivity (they indexed pages are sometimes not updated for month, making them obsolete for some news sites).

To avoid that, I use the  tag into my posts to separate the introduction from the rest of my post. The only way to read the whole post is to follow the permalink (or the "Read more" link automatically added). That avoids the main page from include some outdated keywords.

If not, adding the homepage again would help that. Since every page of your blog link your homepage, its PageRank (its default Google "score") should be high. Since the homepage links to other pages of your blog, the other pages' scores should increase a bit and so on (the PageRank scoring system is iterative).

Another way is to link posts between themselves. When you talk about a given subject, insert links to similar posts appearing in your own blog. Take into account the keywords appearing in the link itself is used as a keyword associated with a given page. Type "[url=http://www.google.com/search?hl=en&q=leave]leave[/url]" in [url=http://www.google.com/]Google[/url]. The first pages include [url=http://www.yahoo.com/]Yahoo![/url], [url=http://disney.go.com/]Disney[/url] and [url=http://www.google.com]Google[/url]. None of those pages has the "leave" keyword appearing on it. So, why do they appear there? Because most adult-oriented sites display disclaimer messages like "This is an adult-oriented site. You must have 18 to enter. Enter / [url=http://disney.go.com/]Leave[/url]. The [url=http://disney.go.com/]Leave[/url] link follow to [url=http://disney.go.com/]Disney[/url], [url=http://www.google.com]Google[/url], [url=http://www.yahoo.com/]Yahoo![/url] or anywhere else... I have added many links with relevant keywords to the most popular posts of my own blog to increase those posts traffic, including the trafic from search engines.

8 kwa Jul 26, 2005 15:40

To sum up, if you want to index your blog's homepage and each post's parmalink page, add the following code to your skin, between the <head> and </head> (X)HTML tags (if a previous <meta name="robots" ...> already appears in your skin, replace it by the following one):

<meta name="robots" content="<?php if( ( $disp!='posts' && $disp!='single' ) || ( $disp=='posts' && ( $paged>1 || $cat!='' || $m!=0 || ( is_array( $catsel ) && count( $catsel )>0 ) || $w>=0 || $s!='' ) ) ) echo( 'no' ); ?>index,follow"/>

Any feedback is welcome.

9 tejaaa Jul 27, 2005 01:30

Thanks Kwa. I will go through your posts and try to understand it all :) . Your taking time to explain is appreicated.

Tejaaa

10 horschdi Jul 27, 2005 10:54

ralphy wrote:

I noticed every page on [url=http://blog.lesperlesduchat.com]my blog[/url] was first indexed by search engines. Then, search engines applied filters removing "noise". However, they appear to remove the blog articles URLs instead of removing a month or a year summaries.

Search engines users want to find articles about a particular subject containing specific keywords. Returning a whole year page containing all the searched keywords doesn't make sense. Thus, indexing a page containing several blog articles is a nonsense.

So, I made up a simple hack to insert into your <head> section of a skin:
<?php

$uri = $_SERVER[ 'REQUEST_URI' ];

$IsHomePage = preg_match( "/^\/[^\/\?]+$/", $uri );
$IsSingle   = preg_match( "/^\/[^\/\?]+.*[0-9]{1,4}(\/[0-9]{1,2}){2,2}\/[^?&]+$/", $uri );
$IsPage     = preg_match( "/^\/[^\/\?]+\?.*page=[0-9]+.*$/", $uri );

if( $IsHomePage || $IsSingle || $IsPage )
{
    // Index this page (and follow it either)
    echo( '<meta name="robots" content="index,follow"/>'."\n" );
}
else
{
    // Do not index this page (however, still follow it)
    echo( '<meta name="robots" content="noindex,follow"/>'."\n" );
}

?>
There are probably cleaner ways to implement this... a more b2evolution-oriented way. Anyway, in addition to that, my "All" blog contains only articles' titles and links to the site's blogs' full articles. The aim is to index each article only once on search engines indexes.

Since I made this change about two weeks ago, the traffic coming from search engines (mainly Google) increased by +91,6% (while direct access increased by +35,0% and external links by +28,3%) compared to last month figures.

I take this code an put it in /templates(Vorlagen)/<Custom Skin> also /b2evolution/skins/custom/_main.php after the tag

<head>

thats right?

Result = no result.

Gives a plugin for SEO?

11 kwa Jul 27, 2005 14:18

horschdi wrote:

I take this code an put it in /templates(Vorlagen)/<Custom Skin> also /b2evolution/skins/custom/_main.php after the tag
<head>
thats right?

Yes, you're right. However, it is more interesting to insert the new version of the code (instead of the old one) as discussed above:

<meta name="robots" content="<?php if( ( $disp!='posts' && $disp!='single' ) || ( $disp=='posts' && ( $paged>1 || $cat!='' || $m!=0 || ( is_array( $catsel ) && count( $catsel )>0 ) || $w >=0 || $s != '' ) ) ) echo( 'no' ); ?>index,follow"/>

This new version invites the search engines robots to index the homepage (this is new) as well as the single posts pages (permalinks) of the blog using the skin you put the above code in with a much cleaner way (more [url=http://b2evolution.net]b2evolution[/url]-friendly way).

horschdi wrote:

Result = no result.
Gives a plugin for SEO?

The code discussed here doesn't give any plug-in. It's just a skin update, adding search engines-specific tag instructing them to index some pages (homepage and permalinks) and not indexing the other ones.

You can notice the change by watching at the (X)HTML source code of the viewed page: it should add:

<meta name="robots" content="index,follow"/>

for your blog's homepage and permalinks and:

<meta name="robots" content="noindex,follow"/>

for all other pages.

Search engines need time to update their indexes after a page has been crawled. To manually insert a given page to the crawling list, you can use these forms:

[url=http://www.google.com/addurl/]Google Add URL[/url]
[url=http://search.msn.com/docs/submit.aspx]MSN Search URL Submission[/url]
[url=http://submit.search.yahoo.com/]Yahoo! Submit Your Site[/url] (you must create a free Yahoo! profile first)[/list:u]

http://www.google.com]Google[/url

https://www.google.com/webmasters/sitemaps/docs/en/about.html]Google

http://forums.b2evolution.net/viewtopic.php?t=4466]Code

12 nomad Jul 27, 2005 17:50

I'm going to put online my b2evo-blog soon. I'm about to transfer older posts. So here some questions:

With your hack, the search engines will only index the single posts that are linked to from the current front-page? If so, how can I make sure that all my single posts are getting indexed?

How can I modify this hack so that my weekly archive is getting indexed?

As I'm eager to learn more PHP

$cat!=''

Is it here I could make some categories being indexed (by adding their numbers?

$paged>1

What can I modify here?

Thanks for your help!

EDIT: OK, I found out how to index the weekly archive

<meta name="robots" content="<?php if( ( $disp!='posts' && $disp!='single' ) || ( $disp=='posts' && ( $paged>1 || $cat!='' ||  ( is_array( $catsel ) && count( $catsel )>0 ) ||  $s != '' ) ) ) echo( 'no' ); ?>index,follow"/>

AH okay so easy :D index categories but not weekly archive

<meta name="robots" content="<?php if( ( $disp!='posts' && $disp!='single' ) || ( $disp=='posts' && ( $paged>1 ||  $m!=0 || ( is_array( $catsel ) && count( $catsel )>0 ) || $w >=0 || $s != '' ) ) ) echo( 'no' ); ?>index,follow"/>

13 kwa Jul 28, 2005 00:12

nomad wrote:

[...]
How can I modify this hack so that my weekly archive is getting indexed?
[...]
Is it here I could make some categories being indexed (by adding their numbers?
[...]
EDIT: OK, I found out how to index the weekly archive
[...]
AH okay so easy :D index categories but not weekly archive
[...]

As you can see, it's pretty easy. The $w variable contains -1 or the week identifier while the $m variable contains 0 or the current year, year/month or year/month/day reference. The $paged variable contains the page number when you follow the Next Page and Previous Page links on the bottom of your blog.

After your modifications, check every kind of link in your blog by displaying the source code and making sure the desired pages are well indexed! Remember:

<meta name="robots" content="index,follow"/>

tells a search engine robot to index the page containing that tag while:

<meta name="robots" content="noindex,follow"/>

asks it to not index a page.

On my own site, I have a specific homepage with the list of the latest 30 posts displayed with the title and a link to the permalink of the post. That helps search engines robots to update the most recent posts. However, when a few updates would be enough every day, I see up to 400 pageviews a day by robots, mainly those from the most popular search engines. So, don't care: search engines robots won't forget you. They just tend to suppress from their indexes duplicated or contentless pages.

Please notice I've volountarily left the follow parameter inviting the robot to add contained links to its list of pages to crawl later. That means even if a page is not indexed, its links are processed. That means even if you don't index all the pages, your blog is going to be fully indexed (I mean, the permalinks/single posts are going to be indexed), since the not indexed but followed pages contain links to them anyway.

Since [url=http://www.google.com]Google[/url] is the most popular search engine (about 70-90% of my site's search engines visitors come from [url=http://www.google.com]Google[/url] or its localized versions), implementing the [url=http://forums.b2evolution.net/viewtopic.php?t=4466]Code to generate 'Google Sitemaps'[/url] making pages submissions automatic may be interesting.

You can also use the manual submission forms:

[url=http://www.google.com/addurl/]Add your URL to Google[/url]
[url=http://search.msn.com/docs/submit.aspx]MSN Search URL Submission[/url]

http://submit.search.yahoo.com/]Yahoo

14 nomad Jul 29, 2005 00:45

Thanks a lot for your explanations! :D

15 stephenr Aug 10, 2005 09:25

Excellent information Kwa. It was just what I was looking for. You da man.

16 richard_gekko Aug 22, 2005 13:23

kwa wrote:

To sum up, if you want to index your blog's homepage and each post's parmalink page, add the following code to your skin, between the <head> and </head> (X)HTML tags (if a previous <meta name="robots" ...> already appears in your skin, replace it by the following one):
<meta name="robots" content="<?php if( ( $disp!='posts' && $disp!='single' ) || ( $disp=='posts' && ( $paged>1 || $cat!='' || $m!=0 || ( is_array( $catsel ) && count( $catsel )>0 ) || $w>=0 || $s!='' ) ) ) echo( 'no' ); ?>index,follow"/>

Ok that’s right!
But…
How can I disallow indexing by search engines of other blogs homepages?
For example:

I want to index
Mysite.com/blog/index.php
Mysite.com/blog/index.php/2005/08/22/today_post
Mysite.com/blog/index.php/blog_2/2005/08/21/another_post
Mysite.com/blog/index.php/blog_3/2005/08/20/the_first_post

I disallow indexing
Mysite.com/blog/index.php?cat=1
Mysite.com/blog/index.php?cat=2
Mysite.com/blog/index.php/blog_2
Mysite.com/blog/index.php/blog_3
Mysite.com/blog/index.php/2005
Mysite.com/blog/index.php/2005/08/
Mysite.com/blog/index.php/2005/08/22
Mysite.com/blog/index.php/2005/08/21
Mysite.com/blog/index.php/2005/08/20

Thanks

17 kwa Aug 22, 2005 19:34

Richard Gekko wrote:

How can I disallow indexing by search engines of other blogs homepages?
For example:

I want to index
Mysite.com/blog/index.php
Mysite.com/blog/index.php/2005/08/22/today_post
Mysite.com/blog/index.php/blog_2/2005/08/21/another_post
Mysite.com/blog/index.php/blog_3/2005/08/20/the_first_post

When using the proposed code:

<meta name="robots" content="<?php if( ( $disp!='posts' && $disp!='single' ) || ( $disp=='posts' && ( $paged>1 || $cat!='' || $m!=0 || ( is_array( $catsel ) && count( $catsel )>0 ) || $w>=0 || $s!='' ) ) ) echo( 'no' ); ?>index,follow"/>

the above links are going to be indexed, since:

Mysite.com/blog/index.php appears to be a blog homepage and the above code asks search engines to index the homepage.
Mysite.com/blog/index.php/2005/08/22/today_post and so on are single post pages and the above code asks search engines to index single post (or "permalinks") pages.[/list:u]

Richard Gekko wrote:

I disallow indexing
Mysite.com/blog/index.php?cat=1
Mysite.com/blog/index.php?cat=2
Mysite.com/blog/index.php/blog_2
Mysite.com/blog/index.php/blog_3
Mysite.com/blog/index.php/2005
Mysite.com/blog/index.php/2005/08/
Mysite.com/blog/index.php/2005/08/22
Mysite.com/blog/index.php/2005/08/21
Mysite.com/blog/index.php/2005/08/20

$cat!=''

( $cat==1 || $cat==2 )

<meta name="robots" content="<?php if( ( $disp!='posts' && $disp!='single' ) || ( $disp=='posts' && ( $paged>1 || ( $cat==1 || $cat==2 ) || $m!=0 || ( is_array( $catsel ) && count( $catsel )>0 ) || $w>=0 || $s!='' ) ) ) echo( 'no' ); ?>index,follow"/>

Mysite.com/blog/index.php/blog_2

( $blog == 2 || $blog == 3 )

<meta name="robots" content="<?php if( ( $disp!='posts' && $disp!='single' ) || ( $disp=='posts' && ( $paged>1 || $cat!='' || $m!=0 || ( is_array( $catsel ) && count( $catsel )>0 ) || $w>=0 || $s!='' ) ) || ( $blog == 2 || $blog == 3 ) ) echo( 'no' ); ?>index,follow"/>

http://b2evolution.net]b2evolution[/url

backoffice

Blogs

18 edb Aug 22, 2005 20:25

kwa wrote:

Finally, I believe indexing anything else than single posts and a blog's homepage is nonsense.

Aw come on! It's fun to let search engines index your profile page. Lots of people probably want to search for [url=http://www.google.com/search?hl=en&q=%22You+are+not+logged+in%22&btnG=Google+Search]"You are not logged in"[/url]

Hey thanks for all the info in this thread. I'm going to have to put some effort into this on my blog. Like, I deny indexing profile and arcdir and stats, but I'm pretty sure I use the hack that lets a profile be seen by the public. Perhaps I would want that indexed? Heck maybe I'm using the hack but don't have a link to my profile!

This would make a nice plugin for the future: a way to handle indexing and following from one simple screen. Hmm...

19 richard_gekko Aug 22, 2005 22:35

<meta name="robots" content="<?php if( ( $disp!='posts' && $disp!='single' ) || ( $disp=='posts' && ( $paged>1 || $cat!='' || $m!=0 || ( is_array( $catsel ) && count( $catsel )>0 ) || $w>=0 || $s!='' ) ) || ( $blog == 2 || $blog == 3 ) ) echo( 'no' ); ?>index,follow" />

But in this way also Mysite.com/blog/index.php/blog_2/2005/08/21/another_post and Mysite.com/blog/index.php/blog_3/2005/08/20/the_first_post pages will not be indexed because this code write noindex,follow.

possible solutions?

I'm very sorry for my terrible english :'(

20 kwa Aug 23, 2005 00:35

Richard Gekko wrote:

<meta name="robots" content="<?php if( ( $disp!='posts' && $disp!='single' ) || ( $disp=='posts' && ( $paged>1 || $cat!='' || $m!=0 || ( is_array( $catsel ) && count( $catsel )>0 ) || $w>=0 || $s!='' ) ) || ( $blog == 2 || $blog == 3 ) ) echo( 'no' ); ?>index,follow" />
But in this way also Mysite.com/blog/index.php/blog_2/2005/08/21/another_post and Mysite.com/blog/index.php/blog_3/2005/08/20/the_first_post pages will not be indexed because this code write noindex,follow.

possible solutions?

I'm very sorry for my terrible english :'(

If I follow you, you would like to make your main blog's homepage indexed, but not the other blogs' homepages.

Try this:


<meta name="robots" content="<?php
$IsSinglePost = ( $disp == 'single' );
$IsHomePage = ( $disp=='posts' && ( $paged>1 || $cat!='' || $m!=0 || ( is_array( $catsel ) && count( $catsel )>0 ) || $w>=0 || $s!='' ) );
if( !$IsSinglePost || ( $IsHomePage && ( blog == 2 || blog == 3 ) ) )
{
    // This is NOT a single post page
    // OR
    // this is a homepage of blogs ID #2 or #3
    // SO DON'T INDEX THIS PAGE!
    echo( 'no' );
}
?>index,follow" />

I hope this works as I believe it does. ;)

21 kwa Aug 23, 2005 00:36

EdB wrote:

This would make a nice plugin for the future: a way to handle indexing and following from one simple screen. Hmm...

You are probably right. However, AFAIK, there is no "skin modifier" plug-in [url=http://b2evolution.net]b2evolution[/url] right know.

It would also be interesting to change a page description and/or its keywords regarding of its contents. They are static for a whole blog in the current (0.9.12) version.

22 richard_gekko Aug 23, 2005 09:40

kwa wrote:

If I follow you, you would like to make your main blog's homepage indexed, but not the other blogs' homepages.

Yes, it' s all I want to do.

kwa wrote:

Try this:


<meta name="robots" content="<?php
$IsSinglePost = ( $disp == 'single' );
$IsHomePage = ( $disp=='posts' && ( $paged>1 || $cat!='' || $m!=0 || ( is_array( $catsel ) && count( $catsel )>0 ) || $w>=0 || $s!='' ) );
if( !$IsSinglePost || ( $IsHomePage && ( blog == 2 || blog == 3 ) ) )
{
    // This is NOT a single post page
    // OR
    // this is a homepage of blogs ID #2 or #3
    // SO DON'T INDEX THIS PAGE!
    echo( 'no' );
}
?>index,follow" />

I hope this works as I believe it does. ;)

Arrgg! This code not makes indexed also main blog's homepage: this code only makes indexed singles post...
Sorry!

23 richard_gekko Aug 24, 2005 12:13

<meta name="robots" content="<?php 
$IsSinglePost = ( $disp == 'single' ); 
$IsHomePage = ( $disp=='posts' && ( $paged>1 || $cat!='' || $m!=0 || ( is_array( $catsel ) && count( $catsel )>0 ) || $w>=0 || $s!='' ) ); 
if( !$IsSinglePost || ( $IsHomePage && ( blog == 2 || blog == 3 ) ) ) 
{ 
    // This is NOT a single post page 
    // OR 
    // this is a homepage of blogs ID #2 or #3 
    // SO DON'T INDEX THIS PAGE! 
    echo( 'no' ); 
} 
?>index,follow" />

At line 4 of this code:

if( !$IsSinglePost || ( $IsHomePage && ( $blog == 2 || $blog == 3 ) ) )

I found those bugs but also even I corrected it the code doesn't work!
Arggg!

24 edb Aug 24, 2005 12:35

Here's an alternative way to crack that nut as long as you don't allow skin switching. Copy your skin as many times as you have different blogs, like skin1 and skin2 and so on. In the back office make each blog use the skin with it's blog number. Now you can set each blog's robots meta tag as you see fit by editing _main in the appropriate skin.

25 richard_gekko Aug 24, 2005 13:53

Here's an alternative way to crack that nut as long as you don't allow skin switching

:( In Italy we call it pastrocchio!

26 edb Aug 28, 2005 05:24

You can also do a switch on $blog to select different variations of the (rather long) conditional statement.

<?php switch( $blog ) {
	case '1':
		**conditions for blog 1 indexing**;
		break;
	case '2':
		**conditions for blog 2 indexing**;
		break;
	} ?>

And so on.

27 richard_gekko Aug 28, 2005 11:16

EdB, your are a genius!
Why did not I think to switch before?

It works very well!

28 josie Nov 19, 2005 23:06

kwa wrote:

To sum up, if you want to index your blog's homepage and each post's parmalink page, add the following code to your skin, between the <head> and </head> (X)HTML tags (if a previous <meta name="robots" ...> already appears in your skin, replace it by the following one):
<meta name="robots" content="<?php if( ( $disp!='posts' && $disp!='single' ) || ( $disp=='posts' && ( $paged>1 || $cat!='' || $m!=0 || ( is_array( $catsel ) && count( $catsel )>0 ) || $w>=0 || $s!='' ) ) ) echo( 'no' ); ?>index,follow"/>

When you say add this between the head tags, do you mean in the mainpage.php of the theme I am using?

Thanks )

29 jibberjab Nov 20, 2005 02:28

It would go in the head area of your _main.php for the skin(s) you're using. You probably already have a few meta tags in the head area. Just drop that code among the other ones.

jj.

30 josie Nov 20, 2005 19:20

Thanks... it has taken me nearly a year on and off, but I'm finally getting the hang of this.

----------------------

Quick question.

When I view my page source I don't see this code... is that natural? And yes, the code is there when I view it in my text editor.

:?:

31 jibberjab Nov 20, 2005 20:14

The code you see a couple posts above is a small piece of php code (as is most of the code you'll see in posts around this forum. When a browser requests the larger php code (your _main.php file, for example) it takes all the php code, parses it, and for the most part, generates html.

If you look at the piece a couple posts above, you'll see it begins with

<meta name="robots" content=

In your page source, after the page loads, look for a meta tag that begins with that section, as that is html code. If that appears in your html source after the page has loaded, then that little piece of php is doing its job ans has parsed correctly, rendering html code as its output.

jj.

32 kwa Nov 20, 2005 22:36

Those interested in SEO may read the following hack:

http://forums.b2evolution.net/viewtopic.php?t=6129][HACK

33 josie Nov 21, 2005 01:27

kwa wrote:

Those interested in SEO may read the following hack:

[url=http://forums.b2evolution.net/viewtopic.php?t=6129][HACK] SEO Keywords & Description Hack[/url][/list:u] I've posted recently.[/quote] I have installed this hack and if I am correct, I just type this code somewhere in every post I make?

34 kwa Nov 21, 2005 08:41

Josie wrote:

kwa wrote:
Those interested in SEO may read the following hack:
[url=http://forums.b2evolution.net/viewtopic.php?t=6129][HACK] SEO Keywords & Description Hack[/url][/list:u] I've posted recently.[/quote] I have installed this hack and if I am correct, I just type this code somewhere in every post I make? [/quote] Yes, you do. Using the [url=http://forums.b2evolution.net/viewtopic.php?t=6129]SEO Keywords & Description Hack[/url], you can add both description and keywords to your post (when modified (X)HTML source code, the default edition) like this:


So they will be using as description and keywords supplied in your single post (X)HTML meta tags. That helps search engines to identify page-specific keywords. To check that works fine, after publishing your post, display a single post (using a permalink), then edit the source code. You should be able to find the following (X)HTML code near the beginning of the displayed source code:
<meta name="description" content="My nice description." />
<meta name="keywords" content="keyword1,keyword2 keyword3,keyword4,etc." />
When no description or keywords are given for a post, the default blog information is used, as before installing the [url=http://forums.b2evolution.net/viewtopic.php?t=6129]SEO Keywords & Description Hack[/url]. In addition to this [url=http://forums.b2evolution.net/viewtopic.php?t=6129]SEO Keywords & Description Hack[/url], the [url=http://forums.b2evolution.net/viewtopic.php?t=6038]Rich Tags Plug-in[/url] helps using several tagging systems with your blog.

35 rtomes May 08, 2007 12:05

This is wonderful, thanks Kwa and others. I have been searching all over for the answer to this question. I just added your:

<meta name="robots" content="<?php if( $disp != 'single' ) echo( 'no' ); ?>index,follow"/>

to my _main.php and it works a dream. I look forward to the search engines sending things to the right place real soon.

A question now. Why isn't b2evolution shipped with this code in it? No-one in their right mind would want anything else as a default set-up when starting a blog.

36 edb May 08, 2007 16:09

I do. I mean "I do want a different setup when starting a blog". I have many pages where disp != single but are in fact single pages, so I don't want to tell the search engines to not index them. I run a blog where the main page never changes but disp != single, so again I would not want to tell the search engines to not index them. That blog doesn't get much changes over time, so each blog's main page doesn't change frequently. Again it is in my best interests to have the search engines cache the main pages as they are.

Building a plugin to handle metatags is really what we need to do so that each user can customize their metatags based on what they want for their installation and each specific blog.

37 kwa May 08, 2007 20:59

RTomes wrote:

A question now. Why isn't b2evolution shipped with this code in it? No-one in their right mind would want anything else as a default set-up when starting a blog.

SEO changes over time. Years ago, duplicating content would improve your blog's position in SERPs. Today, it tends to penalize your blog. Who knows what's going to be next?

It would be interesting you to check your blog's analytics to check that improves your audience from search engines, or the average time people stays on your blog, or even the number of pages seen by visitors. In my case, after I applied those SEO patches on my blog, the blog greatly improved its audience from search engines, especially Google. Since then, 75-80% of the whole audience comes from search engines (it tends to be about 15-25% for an average web site).

However those SEO patches are not the only way to improve your blog's position in SERPs and there are many other ways to do so.

EdB wrote:

Building a plugin to handle metatags is really what we need to do so that each user can customize their metatags based on what they want for their installation and each specific blog.

You're right, EdB.

I've been quite lazy those past few months. I'm going to try to publish some nice plugins for both [url=http://b2evolution.net]b2evolution[/url] and [url=http://wordpress.org]WordPress[/url] this summer.

38 rtomes May 09, 2007 05:56

kwa wrote:

It would be interesting you to check your blog's analytics to check that improves your audience from search engines, or the average time people stays on your blog, or even the number of pages seen by visitors. In my case, after I applied those SEO patches on my blog, the blog greatly improved its audience from search engines, especially Google. Since then, 75-80% of the whole audience comes from search engines (it tends to be about 15-25% for an average web site).

If you assume that the other traffic remained constant, then by getting the search engines right you increased your traffic by about 270%. That is a pretty decent effect and worth making.

It also means that people can find what they want. I can see from the searches coming to my blog that a half of them are going to be sadly disappointed. Actually putting a blog index in will help that, but getting the search engines to go to the right page will make everyone happier which is to the good. I will be able to give some statistics in due course.

At present my web sites get worse than the 15-25% search engines that you state, averaging only 10%. Two main causes of that are probably:

* My sites are mostly about cycles, repeating things, and it is really hard to search because it gets swamped by the wheeled ones.

* I put my pages in stumbleupon and this is generating most of my blog traffic at present. (I cannot get technorati to work right for me.)

The causes of what happens with my blog is going to be hard to work out though, because at present it is increasing by 100% compound per month. :-)

39 filthio Jul 08, 2007 17:46

Many thanks kwa for an elegant solution to a perennial problem. For rapidly-changing blogs this is a must- I've been scratching my head about this for some time now. Nicely done.

I'd love to see this and similar features implemented as a plug-in. B2evo SEO - or just plain old good manners to robots - is a bit of an arcane art right now, and when I next grit my teeth and update my installations this hack is one I certainly intend to take with me.

Form is loading...

Search Engines Optimization (SEO)

1 ralphy Jun 13, 2005 18:09

2 isaac Jun 14, 2005 00:37

3 kwa Jul 13, 2005 19:55

4 kwa Jul 14, 2005 21:38

5 malakajoe Jul 25, 2005 16:52

6 tejaaa Jul 26, 2005 08:43

7 kwa Jul 26, 2005 15:37

8 kwa Jul 26, 2005 15:40

9 tejaaa Jul 27, 2005 01:30

10 horschdi Jul 27, 2005 10:54

11 kwa Jul 27, 2005 14:18

12 nomad Jul 27, 2005 17:50

13 kwa Jul 28, 2005 00:12

14 nomad Jul 29, 2005 00:45

15 stephenr Aug 10, 2005 09:25

16 richard_gekko Aug 22, 2005 13:23

17 kwa Aug 22, 2005 19:34

18 edb Aug 22, 2005 20:25

19 richard_gekko Aug 22, 2005 22:35

20 kwa Aug 23, 2005 00:35

21 kwa Aug 23, 2005 00:36

22 richard_gekko Aug 23, 2005 09:40

23 richard_gekko Aug 24, 2005 12:13

24 edb Aug 24, 2005 12:35

25 richard_gekko Aug 24, 2005 13:53

26 edb Aug 28, 2005 05:24

27 richard_gekko Aug 28, 2005 11:16

28 josie Nov 19, 2005 23:06

29 jibberjab Nov 20, 2005 02:28

30 josie Nov 20, 2005 19:20

31 jibberjab Nov 20, 2005 20:14

32 kwa Nov 20, 2005 22:36

33 josie Nov 21, 2005 01:27

34 kwa Nov 21, 2005 08:41

35 rtomes May 08, 2007 12:05

36 edb May 08, 2007 16:09

37 kwa May 08, 2007 20:59

38 rtomes May 09, 2007 05:56

39 filthio Jul 08, 2007 17:46

About b2evolution

Downloads

About us

Webhosting Guide

Docs & Support

Other

Stay in touch