I've never seen this error, so I would imagine that it's a problem with your form, your workflow, your form library, etc.
I would try a brand new form, a simple browser form, that submits to the form library, and that's it. Does it open in the browser correctly when you reopen the submitted form?