InfoPath Dev

I have a form that uses the CRM 2011 rest service with a central dataconnection to retrieve a few fields and to populate a dropdown. This works fine when we have less than 400 rows. More rows, then it is not fast enough to retrieve and navigate while the data retrieved is stored in the session. My question is, is there some way to get better performance/workaround? Moreover, will I face the same issue if I have all the data in a SP list? 

400 rows sounds like a fairly big number.  I think the most effective solution would be to find a way to query fewer rows, either by filtering the query, or using cascading dropdowns.