I looked at this again, and I think I simply coded it wrong (I am much more of a matlab guy so ideas like curry and :: are pretty new to me). My goal was to create a matrix that stored sollutions so that more complex code could be appplied to them (like the solution space dimension finder). Here is the code:
Sols3 := proc (H::algebraic, F::(list(algebraic)), i::posint, j::posint) options operator, arrow; GTS2(H, F, i, j) end proc; n, m := 7, 7; M := Matrix(n, m, curry(Sols3, H, F)); `~`[nops](M)*plots:-matrixplot(`~`[nops](M), heights = histogram)
and here is a worksheet showing it in context - and its output for (4,4) (scroll down to the graph)- i think it doesn't do what i thought it would and that probably caused the memory leak.