When layer elements are selected in the Map View, using the Invert selection tool you can select the unselected layer elements, and unselect the selected ones.

Before:

NMT_Invert_Selection1

After:

NMT_Invert_Selection2