It's just separation of duty. The Published form is the form that a user will be filling out, so in theory it exists in some shared location, network location, SharePoint, etc.
The saved form is going to be useful for the form designer, to make changes and republish.
You could delete the saved form, but it's never a bad idea to have the XSN backed up. I tend to keep various versions of my forms, on a local folder, in case I ever notice a problem and need to roll back.