Hello,

your answer is very interesting for me. How should I trace the procedure to see what you say?

Is the following code the most usefull?


with(CodeTools[Profiling]):
Profile(test3):
PrintProfiles();

Well, How should I use Maple to solve statistics problem?

Thank you

Vladimir

Keeping the total memory allocation down is often an important part of efficiency. But, also, keeping the memory usage (garbage production and cleanup) down can also leads to time savings.

I suspect that ArrayTools:-Reshape produce a full copy of the rtable argument, while ArrayTools:-Alias actually provides an alternate view of the same data in memory without making a copy. So, using Alias can be more efficient than using Reshape.

Also, in the test3 as given, it was not necessary to keep all of the instances of Vector a around. And indeed in Joe's test3a the Vector a was overwritten due to a being reassigned. But each assignment to a in test3a extracts a new row of A, producing a new Vector each time. Memeory usage can be reduced by instead forming the container for Vector a just once, and then doing a fast copy of the appropriate row of A into Vector a.

So, assuming that I coded it correctly,

test3b := proc(N,n)
local i,a,X,A;
use Statistics in
X := RandomVariable(Normal(0,1)):
A := ArrayTools:-Alias(Sample(X,n*N),[N,n],Fortran_order);
a := Vector[row](n,datatype=float,order=Fortran_order);
for i from 1 to N do
ArrayTools:-Copy(n,A,n*(i-1),1,a,0,1);
end do;
end use;
end proc:

Now, to measure the efficiency, let's also look at memory used and memory allocated, as well as time.

> (st,ba,bu):=time(),kernelopts(bytesalloc),kernelopts(bytesused):
> test3(30000,100):
> time()-st,kernelopts(bytesalloc)-ba,kernelopts(bytesused)-bu;
18.419, 8518120, 899923608

> (st,ba,bu):=time(),kernelopts(bytesalloc),kernelopts(bytesused):
> test3a(30000,100):
> time()-st,kernelopts(bytesalloc)-ba,kernelopts(bytesused)-bu;
0.711, 56088544, 79456288

> (st,ba,bu):=time(),kernelopts(bytesalloc),kernelopts(bytesused):
> test3b(30000,100):
> time()-st,kernelopts(bytesalloc)-ba,kernelopts(bytesused)-bu;
0.571, 31844664, 35960072

Of course, what would be really nice would be to get the speed of test3b or test3a and the lower memory allocation (by at least a factor of four) of test3. This may not be currently possible. One way to get it might be with an enhancement to the Statistics:-Sample routine, so that it accepts as an optional argument the container Vector for the result. It could re-use the same Vector container, and fill it with new sample data, operating in-place. That could allow the total memory allocation to stay low, to generate only n pieces of data at any one time, but avoid production of a lot of Vectors as garbage.

It's a bit of a shame, that it isn't implemented in this way.

Since at default Digits=10 the Vectors have hardware double precision float[8] datatype, there wouldn't be garbage produced from all the entries as software floats.

ps. It may be that A should be C_order, according to how I set up the strides for ArrayTools:-Alias, so that it walks rows of A and not columns. It shouldn't affect the timings much.

acer