Does anyone know of an easy way (read without going into the XML) of deleting a dataset?

Please see above

In report designer, go to the data tab and in its toolbar click the icon with the red X (deletes the currently selected dataset).

-- Robert