The tax rates that are not applied to any order can be deleted. To do this go to Settings → Company Settings → Tax Details and delete the specific tax.
The tax rates applied to orders can only be made inactive. For this, Go to Settings → Company Settings → Tax Details. You can choose a tax rate and click on the edit button in the right corner. The following overlay will appear and you can un-check the Active option.
