You will need to use the eval function - if you look back at that sample form, you'll see it as a default in the Total field. Something like: sum(eval(group2, "my:Amount * my:Price"))
You use the field picker to select the repeating group, then hand type the XPath for the other two fields - attach a copy of your form if you need a hand. :)