There is no way to redirect the user when they click the Close button. But where the user is directed is based on the Source parameter in the URL - so, open a form or a new form from your library, and copy the URL out - you'll see a parameter in the URL called "Source" and it will probably point back to the library view the form was opened from. If you were to put a link to open a new document some place else, like on the home page of the site, for example, you could change that Source parameter, and redirect the user to the home page (or some other page on the site). More info: https://docs.microsoft.com/en-us/previous-versions/office/developer/sharepoint-2010/ms772417(v=office.14)
Hilary Stoupa
