This is specific to our bugs.python.org instance and it's not in the
default Roundup (I would be willing to contribute this and other UI
enhancements if people are interested).
The reason why there's no [-] button comes from the wider goal of
keeping the UI as clear as possible.  Adding oneself to the Nosy is a
very common operation and it deserves a button, whereas the opposite is
fairly uncommon as far as I can tell.
See also http://psf.upfronthosting.co.za/roundup/meta/issue266
(especially msg1677).

Suggest closing this here (unless you want to turn the issue in a rfe to
add the [+] button).