Items tagged with memory memory Tagged Items Feed

Hi guys,

I have written a program in Maple 2016 (Windows 7) made of two parts :

  1. An initialization step where a formal system of algebraic equations (some non linear) is built
    Let S(U, P) this system, where U denotes a set of unknowns and P a set of parameters (see step 2 below)
  2. An iterative step where this system is solved with respect to U for different values of parameters P
    This loop has to be executed N times for values P1, ..., Pn, ... of P

I observe that the size of the memory (bottom right of the Maple window) inflates as the value of n increases.

I was able to isolate in the loop corresponding to step 2, the procedure MyProc I wrote which is responsible of this memory inflation.
Now I would like to manage this inflation (typically the memory size grows up to 3 GiB for n about one thousand) because of its very negative effects on the computational time (probably Maple does spend a lot of time in swapping operations).


Suspecting the remember process to be the source of this problem, I tried simple tricks such as

  • systematically write   > quantity := 'quantity';   for fome intermediate quantities
  • use forget  : for example MyProc contains a call to fsolve and,  after "local" declarations, I inserted  the command  forget(fsolve)   
  • in the the loop over n,  I even inserted the command forget(Myproc).

None of these tricks was to some extent efficient to contain the memory inflation.


I suppose it is a very common situation that people who use to develop code are familiar with. So maybe some of you could provide my some advices or move me towards "strategies" or "methodologies" to prevent this situation ?
My purpose here is not to ask you to solve my problem, but rather to ask youy to give me hints to be able to manage such kind of situations by myself.


Maybe this question is unorthodox and doesn't have its place here ?
It that case please let me know.

Thanks In Advance


PS : it would be very difficult for me to provide you the code : if it is a necessary condition for you to help me, just forget it, I will understand

This is somehow an old problem which came back and I decided to post it. So, I have some code which runs in Grid framework locally. Each node saves data directly to hard disk and there is no need to return anything at the end to the node of ID 0. Nevertheless, I noticed that memory usage on one instance of the mserver is very huge.  Moreover, this instance is all the time in S state, namely sleeping. Below you can see dump from top. I deny this is caused by some particular procedures called in my code since in this case I should rather observe more uniform memory usage. I am not sure if for this moment I could share some code with you which demonstrate this behavior. I will try to write some toy example because my productive code cannot be posted here.

%MEM     TIME+ COMMAND

49.1 17:00.19 mserver
0.1 12:43.70 mserver
0.1 12:32.15 mserver
0.1 13:54.02 mserver
0.1 14:45.45 mserver
0.1 12:35.22 mserver
0.1 13:50.77 mserver
0.1 14:14.85 mserver
0.1 8:58.82 mserver
0.1 12:58.48 mserver
0.1 8:51.00 mserver
0.1 10:35.92 mserver
0.1 12:58.13 mserver
0.1 9:34.39 mserver
0.1 10:10.13 mserver
0.1 12:07.77 mserver
0.0 8:21.17 mserver
0.0 11:41.42 mserver
0.0 5:07.27 mserver
0.0 8:31.88 mserver
0.0 6:30.12 mserver

Hej Mapleprimes,

I am making Maple sweat over a simple problem

11.00=11.244522435+log(x) 

right click and solve for variable x. 

So far Maple has been working for 10 minutes at allocatet 2 GB 

memory.  And no answer yet. 

Is there a short way to solve this faster?

 

Kind regards 

Per Kirkegaard

I am faced by the problem where I used Grid:-Seq()---locally---to do some long time computation and I am faced by the exhaustion of memory problem. I think the problem itself is related to the memory management in maple.

So the problematic function is

# Procedure: ComputeEventsCTypeGrid
#   Compute events such that three quadrics intersects in a point.
#
# Parameters:
#   Q          - a set of quadrics
#
# Output:
#   Indexes of quadrics which intersect in a point.
# TODO:
#  - Memory
#  - Cleanups
ComputeEventsCTypeGrid := proc( Q )
  local s, cType, i, j, k, rootNb, univ, sys;
  s := proc (i, j, k)
    sys := [ Q[i], Q[j], Q[k] ];
    univ := PolynomialIdeals[UnivariatePolynomial]( indets( sys )[1], sys );
    if not type( univ, constant ) then
      rootNb := nops(select(proc(x)op(x)[2]>0;end proc,RootFinding:-Isolate( univ, [ op( indets(univ ) ) ] )) ):
      if rootNb > 0 then
        return [ univ, rootNb, sys ];
      end if:
    end if:
  end proc;
  cType := [Grid:-Seq(seq(seq(s(i,j,k),k=j+1..nops(Q)),j=i+1..nops(Q)),i=1..nops(Q))];
  return cType;
end proc:

A dump from top after calling ComputeEventsCTypeGrid for some data(total amount of memory on this machine is 251G + 48G swap)

VIRT    RES    SHR S  %CPU %MEM     TIME+

33.007g 0.031t   3676 S   0.0 12.5   2606:20
31.137g 0.029t   3436 S   0.0 11.7   2274:34
8645756 4.984g   2840 S   0.0  2.0  32:06.40
28.798g 0.026t   1968 S   0.0 10.7   1821:16
26.650g 0.024t   1968 S   0.0  9.8   1520:54
25.346g 0.023t   1968 S   0.0  9.2   1236:44
23.296g 0.021t   1968 S   0.0  8.4   1004:52
20.498g 0.018t   1968 S   0.0  7.4 790:30.81
16.799g 0.014t   1968 S   0.0  5.9 625:28.12
15.011g 0.013t   1968 S   0.0  5.2 469:36.27
7678444 4.149g   1968 S   0.0  1.6 104:34.22
6734804 623904   1968 S   0.0  0.2  73:13.13
4327420 132516   1968 S   0.0  0.1  24:11.51
3420712 111336   1968 S   0.0  0.0  11:26.16
13.671g 0.011t   1964 S   0.0  4.6 380:15.13
11.771g 9.831g   1964 S   0.0  3.9 274:25.59
9936056 7.769g   1964 S   0.0  3.1 201:13.94
8805512 3.394g   1964 S   0.0  1.3 148:09.53
5077056 1.500g   1964 S   0.0  0.6  42:19.29
2384784  82040   1832 S   0.0  0.0   3:56.57

Size of output data calculated with

> length(sprintf("%m", eval(cType)));

> 326260610

Also, I called gc() at the end of the computations but memory allocated by instances of mserver were not released. Does it mean gc() should be called from each instance separately? My final point is that I have to run some other calculations for different and bigger dataset (It will probably take 2 days to finish -- Q has around 700 polynomials of degree 2) and for this moment I cannot do this because I've gotten an error about not enough amount of memory.

Here is a simple program that displays the issue.

yint:=proc(i,a,b) local y,yi; y:=x^i; yi:=int(y,x=a..b); return yi; end:

n:=10^4;

for i from 1 to n do inty:=yint(i,a,b); od:

Running the program gives the following memory usage.

n            memory
10^4      70M
10^5      138M
10^6      562M

Why is the memory increasing so much? Are internal variables to yint stored between calls?

How can I avoid the memory increase with such a loop?

Thanks.

Dear all,

I am using Maple to perform numerical integrations. When the final index in the loop is set to 5, the computation is fast and the results are quickly delivered. When I set a number higher than that, even 6, the program gets really slow and often crashes.

I herewith attach the script I use to generate the results. I guess that there should be a problem of memory management and I tried to use gc() as suggested in some forms but without success. I would appreciate it if someone here could explain the reason behind the problem.

Thank you,
question.mw

restart; Ts := 1.; sigma := 1.; C := 1.; B := 2./(1+C); with(inttrans); beta := B*Ts*omega; assume(Tb > 0); assume(u >= 0); FzzS := -(3/2)*u^3*((2*u+I*beta)*(exp(2*sigma*u)+sigma^2*exp(2*u))-4*sigma*u*((1+sigma)*u-1))/((2*u+I*beta)^2*exp((2*(1+sigma))*u)-4*u^2*((1+sigma)*u-1)^2); InvFzzS := simplify(invfourier(FzzS, omega, t)); logTimeMin := -2; logTimeMax := -1; NumSteps := 6; logTimeStep := evalf(1.0*(logTimeMax-logTimeMin)/NumSteps); curdirectory(); A := matrix(NumSteps, 2); T1 := Array(1 .. NumSteps); AF := Array(1 .. NumSteps); for i to NumSteps do logTime := evalf(i*logTimeStep+logTimeMin); curTime := evalf(10^logTime); A[i, 1] := curTime; A[i, 2] := evalf(Int(eval(InvFzzS, t = curTime), u = 0 .. infinity, epsilon = 10^(-5))); T1[i] := A[i, 1]; AF[i] := A[i, 2] end do

 

 

``

 

Download question.mw

Can anyone confirm what units of memory are used for this option?

Will Maple 2015 automatically use virtual memory if the size of this option exceeds the RAM memory and how can this be confirmed?

Thank you.

 

Why the command

 

coeff(sum(x^i,i=0..999)^500,x,2) prints

 

Error, unable to compute coeff??

 

I believe, that memory/cpu issues are not a relevant answer here.

 

Already

coeff(sum(x^i,i=0..998)^500,x,2) gives the correct result.

Even

coeff(sum(x^i,i=0..998)^500,x,200) is ok.

How do I compute coeffs of longer sums?Or why this limitation is imposed by Maple Server?

Hi all,

I have been trying to figure out why when I run my 'code' maple keeps building up memory. I am currently writing X number of inequalities and plotting the results. I begin the script with the  'restart' command to clear the memory of any previous runs. But this does nothing to the memory allocated. I check with my 'task manager' window.

I've been scouring the internet for similar issues and basically is this a software problem? how can this be avoided?

'restart' does clear the variables but doesn't seem to make the freed up memory accessible.

- pJ

Hi,

When I run a maple program which take a long time (in command line), I have this kind of message every second:

"memory used=43867.3MB, alloc=147.3MB, time=856.28"

Does there exist any way to avoid this display ?

Thanks in advance.

I am running a Maple procedure in CLI Maple (mainly because it just provides colored random numbers for other work). When the procedure runs I am getting this kind of output scrolling by:

memory used=10732.9MB, alloc=344.0MB, time=895.85
memory used=10809.2MB, alloc=345.6MB, time=904.26
memory used=10885.5MB, alloc=347.3MB, time=912.67
memory used=10961.8MB, alloc=348.9MB, time=921.27
.
.
.

So I am wondering what "memory used" and "memory allocated" mean here. In "Activity Monitor" (this is Mac OS X) I see that the "memory allocated" values correspond rather closely to the "real memory" usage. No idea what the "memory used" column is. Note that this is a 32 bit process running on a PPS Mac, and it merrily "uses" unlimited amounts of memory that don't exist on the machine and that a 32 bit process should not be able to use (I think Mac OS X limits 32 bit processes to 4 GB). It is clearly not paging as the machine stays perfectly useable.

So, what am I to make of these numbers?

Mac Dude.

 

Is there any command in maple that can show us an assigned varibale how much bytes occupied on memory ?

a:=["just a string", 3.14 , 123 , x/y]

d:=array( 1 .. 2, 1 .. 2, [( 1, 1 ) = ("just a string"), ( 1, 2 ) = (3.14), ( 2, 1 ) = (123), ( 2, 2 ) = (x/y)  ] )

c:=a x^2 + b x + c

b := Matrix(2, 2, {(1, 1) = "just a string", (1, 2) = 3.14, (2, 1) = 123, (2, 2) = x/y})

b := Matrix(2, 2, {(1, 1) = "just a string", (1, 2) = 3.14, (2, 1) = 123, (2, 2) = x/y})

(1)

``


It should be great for benchmarking !

I am running a huge symbolic problem in Maple 18. At the moment, I dont even know if there is enough computational power to suceed the computation.

 

However, I'd like to save a progress. And maybe to continue computation later.

Is there any way to save progress?

Dear all,

 

Let me first explain a problem I've encountered when working with Maple 18 64 bit in Linux (Ubuntu 12.04). I've tried some things to solve it, but unfortunately without succes

When working with the DifferentialAlgebra package I'm encountering a problem concerning memory. Without going into details, I'm using the RosenfeldGroebner function (DifferentialAlgebra/RosenfeldGroebner) . After about 10 minutes of calculation or so, the computation stops with the error message

>> Error, (in DifferentialAlgebra:-RosenfeldGroebner) out of memory error

This particular error does not have a *help*-file, so no help there.

Now, I'm pretty bumped out because of this error, because I'm working on 16 GB RAM computer which is imo quite a lot. Moreover, I have the feeling, that Maple is not using the full capacity of memory. This, is because I was monitoring the memory during the computation (via the System monitor). Before I start the calculation: the memory usage is about 3.6 GiB (=22.8%). During the calculation it slowly rises until about 8GiB (50%) at which the computation stops and the above-mentioned error message pops up. So it looks there's a big part (about 7.5-8 GiB) of memory still available, but isn't used.

So my question is: What are the possibilities to solve this problem. What can I try to do to have more memory available to be able to do the calculation.


Of course I was looking into the possibilities myself. In my experience I found the information found on this forum and on the Maple help files confusing or not very helpful. Probably, because I have little knowledge of programming or computer architecture. Nevertheless I would like to know some things about it. So, iI'll  present what I tried to do:

- The command kernelopts('datalimit') returns the total amout of heap memory in kibibtyes that Maple may consume   
  In my aplication this is set to infinity. So I presume that this could not explain why the program stops at only 50% Memory usage

- The command kernelopts('bytesalloc') returns the total amount of bytes allocated by Maple kernel. This value cannot be set.  Does this attribute pose a limit on calculations or is this  the amount of memory used by the Maple 'overhead'?   
In my application the value of this attribute is 8781824 bytes, so that's about 8 MB. I suppose it is not related to the problem I'm having, or is it? 

The command kernelopts('stacklimit') returns The total amount of stack space avilable. In my application this was initially set to 8160 kibityes. I've changed this to 8160000000, wel beyond the availble memory. I would think that this now would not anymore present a limit on the memory usage.

The command kernelopts('jvmheaplimit') returns the maximum amout of heap memory for the Java virtual machine. I presume this is the maximum amout of memory for the visiualisation part of Maple (is this correct) In my application this was initially set to 65536 kiB. I've changed this to 65536000000 kBib which is well above the available memroy. I would think that this now would not anymore present a limit on the memory usage

The command   kernelopts('cacheclearlimit'); returns the memory for maintaining temporary elements in cache. In my application this was initially 524288 kiB, but i changed this to 524288000000. I would think that, again, the limit on the memory usage would be gone.

I tried the calculation again with the mentioned changes in the memory attributes. Unfortunately, the error message *out of memory error* still popped up at about 50% memory usage.

Did I overlook something?? What can I do?? Is there actually that I can do? Where should I look?

Many thanks !!

i got kernel connection lost error when memory used reach 2.87GB

how to make it fully used 3.8GB ram, total 4GB ram

indexlistresult := [];
gaga := [seq(i, i=1..512];
nops(indexlistresult);
indexlistresult := [op(indexlistresult), op(gaga)];
nops(indexlistresult);
indexlistresult := [op(indexlistresult), op(gaga)];
nops(indexlistresult);
indexlistresult := [op(indexlistresult), op(gaga)];
nops(indexlistresult);
with(combinat):
gg := permute(indexlistresult, 3);

1 2 3 4 5 Page 1 of 5