You can remove an explicit permission from the Edit Permissions dialog by clicking the Delete button to the right of the relevant user or group:
Or, if you want to revert a number of users or groups to the original inherited permission level, select Inherit permissions from parent container, then click the Revert button to the right of the relevant user or group: