Wow, that's awful - so, if this were my form, the first things I'd check would be:
1. What happens when you open the form in InfoPath Filler? Still slow?
2. What happens when you open the view in SQL Server Management Studio - is there something up with the view SQL that makes it slow to return data?
That, I'd verify that the data connection is ONLY slow in IPFS.