Question: How to minimize the usage of memory using Grid:-Seq?

Hello

I had to save previous results of a calculation to files as the number of elements is too big for my computer to handle in one go.   Here it is an idea of what I am doing to read the files and perform the calculations.

mainproc:=proc(arg1,arg2,arg3,arg4) ... end proc:

Grid:-Set(mainproc):
Grid:-Set(arg2,arg3,arg4):   #  They don't change ever.

for i from 1 to number_of_files do 
   read(...):  # it reads arg1 from a file
   Grid:-Set(arg1):
   ans:=Grid:-Seq(mainproc(arg1[i],arg2,arg3,arg4),i=1..numelems(arg1))):
   Grid:-Wait():
   save ans, ....:
   unassign('arg1'):
   unassign('ans'):
   gc():  # An attempt
end do:

The actual code works but, for every step in the loop, the memory used by Maple increases by a certain amount that seems to be mostly related to arg1 (as if arg1 is piling up from iteration to iteration). 

I read some of the earlier posts on a similar subject dated 5 to 10 years old.  I wonder if there is something new that can be done to minimize the usage of memory.   

Many thanks

Ed

 

PS. I am aware of tasksize, numcpus and Threads.   

 

 

 

Please Wait...