Hello, and welcome to the forums.
I believe that this can be easily accomplished with views. Within the same InfoPath form template, you can have two views. One view can allow the user to atatch/delete/replace, and the second view can allow to display (read only).
If you are not using SharePoint, how is the XML being submitted/stored? Via email? Is the user just saving a local copy of the XML to his drive?