Delete empty directories

find . -type d -empty -delete

This recursively finds and deletes empty directory.