You can easily delete a collection by navigating to the main collections page and clicking the trash icon located at the far right of each collection pane. None of the actually icons will be permanently deleted, just the collection data.

Please note: If you delete a collection it cannot be undone.