Did a little more digging.
The site-wide header is found in skins_site>site_body_header.inc.php. I removed the following items:

// --------------------------------- START OF CONTACT LINK --------------------------------
// Call widget directly (without container):
skin_widget( array(
// CODE for the widget:
'widget' => 'basic_menu_link',
// Optional display params
'block_start' => '',
'block_end' => '',
'block_display_title' => false,
'list_start' => '',
'list_end' => '',
'item_start' => '',
'item_end' => '',
'item_selected_start' => '',
'item_selected_end' => '',
'link_selected_class' => 'swhead_item swhead_item_nav_md swhead_item_selected',
'link_default_class' => 'swhead_item swhead_item_nav_md ',
'link_type' => 'ownercontact',
) );
// --------------------------------- END OF CONTACT LINK --------------------------------

and

// --------------------------------- START OF CONTACT LINK --------------------------------
// Call widget directly (without container):
skin_widget( array(
// CODE for the widget:
'widget' => 'basic_menu_link',
// Optional display params
'block_start' => '',
'block_end' => '',
'block_display_title' => false,
'list_start' => '',
'list_end' => '',
'item_start' => '<li class="swhead_item swhead_item_menu_md">',
'item_end' => '</li>',
'item_selected_start' => '<li class="swhead_item swhead_item_menu_md">',
'item_selected_end' => '</li>',
'link_selected_class' => 'swhead_item swhead_item_selected',
'link_default_class' => 'swhead_item',
'link_type' => 'ownercontact',
) );
// --------------------------------- END OF CONTACT LINK --------------------------------

And it did remove the Contact link and pop-up (if viewed on narrow screen). Quick and dirty, but this did work until the next release of b2evolution where you might need to repeat the process.

Seems strange to have "This Site" appear on the far left of the site-wide header by default. One would think you would want the name of the blog displayed instead.

@blex that top bar is part of the "Site skin" which are sections shared between all the collections of your b2evolution site. You can just disable the whole feature by unchecking the option at: Site Settings > Global Site Settings > Enable site skins.

You can also modify the code, but proceeding as yo did will make those changes to be lost on the next upgrade of you site. So, in order to avoid that, you need to override the files first, as explained here: http://b2evolution.net/man/custom-site-skin

Regards!

Mgsolipa,

Thank you for providing several solutions! Unchecking enable site skins was the easiest, but creating custom-site-skin is a great way to fine-tune the page without having to re-edit skins after an update!

Thank you!