2. Performance bottlenecks

My machine configuration:
CPU: Intel (R) Core (TM) i7-4790 CPU @ 3.60GHz [8]
Memory: 12G

Test process
For testing, I used b2evolution-6.6.7-stable-2015-12-31.zip which made no change, and except the user 'admin', created no data at start.
Then, I used the testing tool to create 5000 users. The user group is set to "New users automatically get a new blog", so also created 5000 blogs. At last I created 10 000 articles in the first blog.
Of course, I removed the front and back office b2evo bar links for they are almost filled all the screen (I used an empay array to replaced the $blog_array, the code is $blog_array=array();), and then the appearance of the display is very simple without any additional content.
Finally, I found the following problems in the course of using:

A.Submit a comment would take a long time (about 12 seconds)
I opened the debug mode and note the code //header_redirect() on the bottom of the content of the file htsrv\comment_post.php, then found the debug infomation as follows:

SELECT DISTINCT user_email, user_ID, uset_value as notify_moderation
  FROM evo_users
  LEFT JOIN evo_blogusers ON bloguser_user_ID = user_ID
  LEFT JOIN evo_bloggroups ON bloggroup_group_ID = user_grp_ID
  LEFT JOIN evo_users__usersettings ON uset_user_ID = user_ID
   AND uset_name = "notify_comment_moderation"
  LEFT JOIN evo_groups ON grp_ID = user_grp_ID
 WHERE ( ( bloguser_blog_ID = 1
   AND bloguser_perm_edit_cmt IN ( "anon", "lt", "le", "all" ) )
    OR ( bloggroup_blog_ID = 1
   AND bloggroup_perm_edit_cmt IN ( "anon", "lt", "le", "all" ) )
    OR ( grp_perm_blogs = "editall" ) )
   AND LENGTH(TRIM(user_email)) > 0 [Show less]
Rows: 2 – Time: 12.1239s (99.87%)

I pasted this code to phpMyAdmin to execute, still performed 12 seconds.

B. Main / Home type site displayed too slow (about 10 seconds)
First, I changed a blog to Main / Home type , and in the widgets page , I removed all of the options except Featured / Intro Post, Recent Posts (Simple Post list), Recent Comments (Comment list).Finally redirect to the page , it took 10 seconds to display. Every time I refresh the page, spent time always the same, just 10 seconds.
However the debug information was shown 3 seconds in total.Then I found a long sql statement, although the execution time of that sql is not long, the transmission may take much time.

C.Admin/Moderators back-office management
Here only for administrators/moderators who get advanced privileges.In the back-office, Review articles and comments took a long time to display. See more, please check the following pictures.

Screenshot as follows

Admin/Moderators back-office management would take much time

@haharen Thank you for your feedback and ideas. I will reply to each of your points below.

1 . The B2evo bar display

In the short term you can customize the global front office header by these instructions: http://b2evolution.net/man/custom-site-skin

I have a way to do so:
(1) For administrator, add an option (just like the 'fav' click at the back-office site-dashboard ) to choose which collections to be shown to the public.

You can already do this by choosing for each collection: http://b2evolution.net/man/collection-list-settings

(2) For ordinary users, also add an option (also just like the 'fav' click) to choose which collections to be shown to themselves.

Ok, we will add such an option. Do you have an idea of where you think ordinary users should marl a collection as a favorite? An annotated screenshot would help.

2. Performance bottlenecks

Thank you for your detailed reports. In the future, please make a separate forum thread for each separate performance problem. This way it's easier to keep track of which ones we can reproduce and fix and which take longer. (I understand they all share the same test methodology, but it would be easier if you just copy/paste the methodology in each form topic).

Problems A & B are clear. We're working on them.

C.Admin/Moderators back-office management
Here only for administrators/moderators who get advanced privileges.In the back-office, Review articles and comments took a long time to display. See more, please check the following pictures.

Can you please show a complete backoffice page? Your screenshot is cropped and I don't know which part of the backoffice it is.

Thank you.

@fplanque wrote earlier:

(2) For ordinary users, also add an option (also just like the 'fav' click) to choose which collections to be shown to themselves.

Ok, we will add such an option. Do you have an idea of where you think ordinary users should marl a collection as a favorite? An annotated screenshot would help.

First of all, I am very appreciate that you pay attention to my question.

My idea is that just using the Fav button to control the collections to display for usrs themselves.Is that better if the front and back office showing the same collections bar? I think it's not necessary to separate the front and back office fav collections bar showing.(If there is another situation and have to separate, just add a button beside the Fav button is OK.)

And then the administrator control the fixed collections which would be shown to the public.(Of course, front and back office bar would be both displayed) Now, the administrator can go into any collection and choose 'Show in front-office list:Always (Public)' to show the collections in front office,but it also need be shown at the back-office fav collections bar, then it can't be choosen by ordinary users.

@fplanque wrote earlier:

> 2. Performance bottlenecks

Thank you for your detailed reports. In the future, please make a separate forum thread for each separate performance problem. This way it's easier to keep track of which ones we can reproduce and fix and which take longer. (I understand they all share the same test methodology, but it would be easier if you just copy/paste the methodology in each form topic).

> C.Admin/Moderators back-office management
Here only for administrators/moderators who get advanced privileges.In the back-office, Review articles and comments took a long time to display. See more, please check the following pictures.

Can you please show a complete backoffice page? Your screenshot is cropped and I don't know which part of the backoffice it is.

Thank you.

OK, next time, I will separate each performance problem.

The categories, when administrator or moderators access the collection which type is "Main/Home", and the file manager, it will be shown all categories of all collections. My idea is that showing the all recently posts but under a judgement to show the categories.if the collections number more than the toplimit which could be set in a conf file or back-office, then :

1. Only the categories of chosen collections which may be moderated or seen frequently would be shown .This method would add a choose button. Or maybe we can use the Fav star button which we talked above.
2. The sql of showing collections categories gets a limit value.

The screenshot as follows:

@haharen Thank you for the additional details. Here are some preliminary answers.

• the slow comment posting is due to finding moderators to notify about the new post. It looks like we will be able to optimize this sigificantly.
• other perf issues: still working on it.
• back-office list of collections/categories : thank you for the screenshot, it's clear now, we'll implement a limit.
• favorite collections. I will think about this some more but I agree it should probably be a per-user setting.
• I removed an additional screenshot you posted because it is yet another issue. If you want to repost it, please use a separate forum thread. But you don't have to -- in reality, I already added it to our internal bug tracker.

@fplanque OK
I find these problems at present, maybe if there are some more then I will continue.
Thank you very much for your help.

@haharen We have internal trackers on all the performance issues that you reported and we will work on all of them.

However I noticed that many of the performance issues you found arise from trying to use the"Home" collection as an aggregation of all other collections (which is the default setting of the Home collection) and actually having 10.000+ collections.

Basically aggregation was designed with aggregated maybe 100 collections max, not 10.000.

We can certainly improve it to some extent but we will always have an issue. Maybe it will fail at 100.000 collections...

So my question is: do you really want to aggregate posts from 10.000+ collections or are you only doing it because the default demo setup is like this?

And if you really need this, the big question is: would you want your home page show all public posts only or would you want it filtered depending on each user's access permissions? (It is the permission checking on 10000 collections that creates the bottleneck)

@fplanque
it's so exciting that b2 would support so many blogs, thank you very much! :)

My idea of using main collection is that:

1. Showing the news of all site.
2. Showing the users who want to show themselves by posting public articles.

So, I think the main site is necessary, for we need a page to show the site news. However, only showing public posts would be OK, no need other conditional statement. Because in principle everyone who have logined the website (for other universities which can open their sites to the public may don't need to login) can read the public posts, If some authors don't want to show their articles, just down the reading permession to community or members would be OK. Users can read various of public articles then choose the author whom they want to pay attention to.

However, thousands of collections and thousands of categories destine articles would be too complicated(especially for an university). How to organize and classify the posts, this is the major problem which is placed in front of me.Some days ago, I want to create a public blog as main blog ,and fixed the article categories. But this way, the user should post twice maybe (One post in the public blog, the same post in his own blog). Another way is enable the function of cross posting, post once then appear the same article both in public blog and own blog by choosing cross categories. But if there are thousands of collections, maybe we would make more bottlenecks or create some unexpected confusions.So the ultimate question is that how to organize and show the public articles(and users), after lots of thinking, maybe the "Main/Home" collection is the best showing way at the moment.

Ok, here are my thoughts:

• Don't use cross posting with 10.000+ blogs. It's very NOT optimized for that.
• It should be possible to make a simplified home page that collects only public posts and it will be much faster.
  • But will the student blogs be public? If they are "members only", they will not be able to have public posts (in recent b2evo versions at least)
    • In that case (if you need b2evo to restrict access to your blogs instead of a firewall), then you may need to have "community" blogs. i-e: blogs that require to be logged in.
• If you want to show unified categories on the home page, you could select a few tags and encourage users to use these tags for posts they want to appear on the homepage.

@fplanque
I am really agree with your opinion.
Think carefully about the types of creating collections in one university, may have the following situations:

1. Main / Home
Administrator-created, only one, which the type of "Main / Home". In this situation, check "Allow access to: Logged-in users only";

2. Public Collection
Administrator-created, one or more collection(s) to the public, in addition to the type of "Main / Home" . In this situation, check "Allow access to: Logged-in users only" and "Advanced permissions", then select the users and groups who have the persession to access, read, post;

3. Personal private collection, not open to the public
User-created. In this situation, check "Allow access to: Only the owner (Private collection)";

4. Personal collection, the content is available to public
User-created. In this situation, check "Allow access to: Logged-in users only";

5. Team collection, private content and not open to the public
Member-created. It is suitable for laboratories, offices, departments which eager to have an internal discussions or collaboration. In this case, check the "Allow access to: Members only" and "Advanced permissions", then select the users and groups who have the persession to access, read, post;

6. Team collection, the content is available to public
Member-created. It is suitable for laboratory, offices, departments or a special enthusiasts team. In this case, check "Allow access to: Logged-in users only" and "Advanced permissions", then select the users and groups who have the persession to access, read, post.

If I'm all right about the creating collection method which I list above, I think there is no problem about the permissions. What I will do is just listing the detailed documentations for users to read. On the other hand, maybe only for me, it requires additional settings:

1. Remove or hidden the option " Allow access to: Everyone (Public Blog) Options" .
2. Limit the types of creating collection based on user groups(avoid user to choose the Main/Home type), or hidden the Main/Home collection type for ordinary users.

I think these two are not a big problem and also very easy setting by noting some codes only. However, it is better if we can set them in the back-office or a config file.

On the other hand, the idea : "Select a few tags and encourage users to use these tags for posts they want to appear on the homepage."
It's really a good idea which I never thought before. :)

Interesting use cases indeed:

2. Limit the types of creating collection based on user groups(avoid user to choose the Main/Home type), or hidden the Main/Home collection type for ordinary users.

I'm adding this to the todo list.

1. Remove or hidden the option " Allow access to: Everyone (Public Blog) Options" .

I'm trying to think about where such a setting should be and how to make it not sound like "[√] we don't like public speech" :p

Maybe this setting would be in the group settings too.

So each group would have:

• users can create a blog?
• what kind of collections can they create?
• what level of publicity can they create?

@fplanque wrote earlier:

So each group would have:

• users can create a blog?
• what kind of collections can they create?
• what level of publicity can they create?

Yes, it's great! This would be perfect, especially for me, for the situation of our university. Thank you.
Actually, I think these functions still apply to other universities or some organizations which would let members creat their own blogs.