To delete a folder on disk, usermdir as per Stephen's answer. Matlab path is irrelevant for this.
To remove a folder from matlab's search path, usermpath. This does not delete the folder from disk if it exists. If you've also added subfolders to the path, you need to remove each one of these individually. Alternatively, use