Patrick,
Thanks for the response. I ended up solving the problem using the second option. So, the users enter data into repeating tables on forms for a sharepoint library. This data is concatenated (using a double-eval) into a single string and promoted to a column in the sharepont library like this:
Form 1 dataField: "item 1: feature1: 1, feature2: 2; item 3: feature1: 3, feature2: 4,"
Form 2 dataField: "item 1: feature1: 5, feature2: 0; item 2: feature1: 6, feature2: 1; item 3: feature1: 9, feature2: 13,"
(Note: the first entry does not have any data for "item 2" but the second does.)
Then I read that as a secondary data source in another form, and I want to sum all the related fields in a repeating table that has a new row for each item. Users pick items (itemID), and the form takes all the data from the secondary data source, goes through each feature, and computes the sum. We can this in three steps:
1) We need to store the current row itemID as a temporary variable. I used the secondary data source as a convenient holding spot (convenient as long as I don't want to requery, which I don't). On changing the itemID in the row ofthe destination table, I use a rule (runs when this field changes) to set the field
[secondaryTable path]/q:SharePointListItem_RW/q:ID = concat("item ", .)
2) I need to read all the rows of the secondary data source, lookup my current itemID (now stored in [secondaryTable path]/q:SharePointListItem_RW/q:ID) and sum the values associated with each feature. This requires a separate rule for each feature. Here's a piece of code for feature1:
sum(eval( [secondaryTable path] /d:SharePointListItem_RW/d:dataField, 'concat(substring-before(substring-after(substring-after(., concat("item ",[secondaryTable path]/q:SharePointListItem_RW/q:ID)), "feature1: "), ",")'))
For item 1, feature 2, the eval reads through both rows of of data and finds "2" + "0" = 2. Note: we are using a special feature of InfoPath (and possibly XPath, I'm not sure) that it allows doing math on strings, provided that the strings are composed strictly of numerals. The problem is, that in the example above, the above function will return a null string ("") when looking for "item 2" in the first row of data above, and InfoPath evaluates "" as NaN rather than 0 for math. So the for item 2, feature1 in the sample data the function returns "" + "6" = NaN.
The solution is a conditional substring technique that allows us to use if/then logic to return "0" for empty strings (http://blogs.msdn.com/b/infopath/archive/2006/11/27/conditional-default-values.aspx). The new code reads:
sum(eval( my:fullList, 'concat(substring-before(substring-after(substring-after(., concat("item ",[secondaryTable path]/q:SharePointListItem_RW/q:ID)), "feature1: "), ","), substring("0", 1, not(contains(., concat("item ",[secondaryTable path]/q:SharePointListItem_RW/q:ID))))'))
The second part that concatenates a "0" to the returned string if the item is not found. ie. for the example data above, Form 1 dataField (inside the revised eval function) would evaluate to concat("", "0"). So in the example for item 2, feature 1, the function now returns "0" + "6" = 12.
3) Finally we clean up our mess by resetting our temporary holder with one more rule:
[secondaryTable path]/q:SharePointListItem_RW/q:ID = ""