Users cannot delete their default "My Collection" ("Default Collection" in v2.3), however it is very simple to delete any other collection.
To delete a collection do the following:
Go to the "Manage My Collections Page"
Click Delete against the collection you wish to delete
Confirm deletion of this collection
Your collection will now be permanently deleted.
Note that deleting a collection will not delete the resources themselves.
To delete multiple collections (Version 2.3):
Select the items required by checking the boxes on the left side of the grid. To select a range, select the first collection and then hold down shift when selecting the next. All boxes between the two will be checked.
A bin icon will be displayed above the grid which when clicked will delete all the selected collections. You will receive a prompt to confirm before the collections are permanently deleted.