Interesting question

epostma — Tue, 02 Feb 2010 08:35:42 Z

This is not an easy question to answer, first of all because people who want to know such things sometimes want different types of answers. I'll show you two methods that can be used, and only the first one will give you a true order calculation - but it's often a lot harder to do than the second one. I'll first give you a somewhat more theoretical overview, then examples.

Basically, you count the number of basic operations that Maple needs to do in order to finish your algorithm, as a function of the size of your input, say n. Often, the number of steps will depend on the actual input, not just its size, and in this case you need to find an upper bound on the number of basic operations. A basic operation should in this case be read as something that can always execute in a bounded amount of time, independent of the parameters of the operation. This can be very difficult to determine, because you have to know what algorithms Maple uses internally for certain operations; this is generally not too difficult for library-level Maple routines, but for Maple kernel routines it is less easy. As an example, assigning an entry in a table or Vector or Matrix is a basic operation, but adding an element to a list is not (in Maple).

If you succeed in determining the number of basic operations, this will give you some algebraic expression involving n. You are then allowed (technically not required) to omit all but the eventually largest term in an addition of a fixed number of terms, and omit constant factors, to simplify this algebraic expression.

Let's take a simple bubble sort-like sorting algorithm as an example - as you said, we hope to get out O(n^2). We'll take a Vector of real numberic constants as input.

bsort := proc(v :: Vector(numeric), $) :: Vector(numeric);
local 
  i :: posint, 
  j :: posint, 
  n :: nonnegint := LinearAlgebra:-Dimension(v);

  for i from n to 2 by -1 do
    for j to i - 1 do
      if v[j] > v[j+1] then
        v[j], v[j+1] := v[j+1], v[j];
      end if;
    end do;
  end do;

  return v;
end proc;

The if-block inside the loop will always execute in bounded time (*), so we'll count that as one operation, adding in the bookkeeping overhead of increasing j and such in the inner for loop. Then the inner for loop takes i - 1 basic steps. The outer loop also has some bookkeeping overhead in each iteration, which we can account for by saying that the full inner loop plus bookkeeping for one iteration of the outer loop can be done in i basic steps. For the full outer loop, we get Sum(i, i=2..n) = (n+1)^2/2 - n/2 - 3/2 steps in total. This is the algebraic expression that's our upper bound for the number of basic operations of the algorithm as a whole. If we expand that (using Maple's expand command) we get n^2/2 + n/2 - 1. Eventually, the quadratic term will be bigger than the other two terms, so we can omit the linear and the constant term. Then we can also omit the constant factor of a half, and we get the answer we were looking for: the algorithm is O(n²).

OK - that's it for the first method, which as I said is the only one that can actually get you a true "order" of an algorithm. However, people who ask for a big-O calculation are sometimes more interested in how well the algorithm will scale to large inputs in practice. The second method gives you such information: it is to just time your procedure, then fit a function to it. This is a piece of software engineering that you can very easily and get rough results, or do very well and spend a lot of time accounting for cache warmup etc. I'll show a very rough and easy solution here.

timeit := proc(n :: posint, k :: posint, $) :: numeric;
local 
  vs :: list(Vector) := [seq(LinearAlgebra:-RandomVector(n, generator = 1. .. 2.), i=1..k)],
  i :: posint,
  t, t0;

  t0 := time();
  map(bsort, vs);
  t := time() - t0;

  return t / k;
end proc;
nvalues := [seq(ceil(5 * 1.2^k), k=0..29)];
times := [seq(timeit(n, min(5000, ceil(10^6/n^2))), n in nvalues)];
plots:-pointplot(nvalues, times, axis=[mode=log]);
Statistics[PowerFit](nvalues[4..], times[4..], n);

This yields, after some computation, first the following plot:

We see that the points are almost on a straight line, except for the first three points (which I therefore discard). Then we find the fitted expression 0.00000276 * n^1.92 (with more digits, but let's forget about those) (and this number is from my new core i7 720QM laptop). This suggests approximately quadratic behaviour.

Hope this helps,

Erik Postma
Maplesoft.

(*) Here's an example of how difficult it is to do this in a general-purpose computer algebra system. In an earlier version of this comment, the procedure took a Vector(realcons) as an argument instead of a Vector(numeric). When I wrote the sentence with the footnote mark, I realized that the procedure wouldn't work in general. Writing if v[j] > v[j+1] is a somewhat dangerous thing to do, specifically because Maple has been written so that you can write that and know that it will execute in bounded time. The price to pay, however, is that the comparison is not very powerful. If v[j] and v[j+1] are both of type numeric, then you're safe. However, symbolic constants like Pi cannot be compared like this, because it might require a long computation to ascertain which is the bigger one (or indeed, if the two constants are different). For example, if you would write if x > y then ... while having x = Pi and y = 2, then Maple will complain. If you want to risk the somewhat more lengthy computation, you can write if is(x > y) or if you like living dangerously, if evalf(x - y > 0). However, then the count for the big-O computation cannot be bounded easily without further knowledge of the structure of x and y.

Thank you?

Jabz — Wed, 03 Feb 2010 01:00:15 Z

Thank you very much for your kind help. Unfortunately i am still an amature on maple but i did read everything you wrote and tried my best to understand it but because of the lack of knowledge in programming i did not grasp the whole concept but i am still grateful that you replied beause the information maybe useful to someone else. thank you

simplistic

pagan — Wed, 03 Feb 2010 01:22:34 Z

Pick a large value of the parameter of interest. Let's say it is value n some numeric value. Evaluate your algorithm at that value, and time it. T1:=time(f(n))

Now do the same at 2*n. T2:=time(f(2*n))

Now look at the log (base 2) of the ratio of T2 to T1. log[2](T2/T1)

If you are fortunate enough for the asymptotic behaviour of f to have manifested itself by size n, then that last value will tell you something.

Let's put it another way. Suppose your algorithm has O(n^3) behavior. If you double the input parameter then you expect to see an 8 times increase in the computation time (if you are lucky enough, if n is already large enough). And log[2](8)=3.

MaplePrimes - answers and comments on Question, Big O of a procedure

Interesting question

Thank you?

simplistic