This appears to be my problem. Unfortunately, I couldn't get it to work out from these suggestions. I am running Mac OSX Version 10.5.8, I have 4GB of RAM, and I am using Maple 13. When I run a calculation, the `Memory' indicator at the bottom cuts out just below 200M (whatever that means; I don't think it's mebibytes), whereas a handy program I have just installed, called `MenuMeters', confirms that the calculation is aborting once it reaches about 400 mebibytes. Note that kernelopts(datalimit) outputs `409600' (which seems to be Maple's default setting?).
Anyway, I have tried altering the files as this, and the linked page indicate, but nothing has changed when I run Maple. There is a difference that may be new with Maple 13: The line that seems to have begun with,
ulimit -d 524288 || ulimit -d 409600 || ulimit -d 307200
to be changed to something like,
ulimit -d 1000000 || ulimit -d 409600 || ulimit -d 307200
ulimit -d unlimited > /dev/null 2>&1 || ulimit -d 524288 > /dev/null 2>&1 || ulimit -d 409600 > /dev/null 2>&1 || ulimit -d 307200 > /dev/null 2>&1
ulimit -s 65536 > /dev/null 2>&1 || ulimit -s 32768 > /dev/null 2>&1 || ulimit -s 16384 > /dev/null 2>&1
I changed the number that was indicated (524288 to 2524288) and saved the file. Then I ran it with Terminal (which wasn't suggested), but I still can't go beyond kernalopts(datalimit); 409600 when I open Maple.
However, I just had an idea which has shed some major light on this issue. I opened both mserver_ulimit and mserver_ulimit32 with my text editor and changed 409600 to 409500, saved, and ran both of them in Terminal. This changed the output of kernalopts(datalimit) to 409500. This changes the hard limit, so that kernelopts(datalimit=409600) gives the error message
Error, cannot raise the datalimit above the hard limit
So I've made some headway. Changing this value to 409700 produces the same result. After some time, I have made the important discovery: My OS is imposing a limit of 512000 mebibytes, which cannot be got around. When I change 409600 to 512000 in the default line I've printed above, it works, but when I change it to 512001, the OS will not allow that, so Maple takes the next value down the line; 307200. This is why changing the value of 524288 to anything larger does nothing for me---it is already too large for system constraints, so the next best value (409600) was already being taken.
So now I have narrowed this down to an operating system issue.
The previous help files should not be useful for anyone whose OS is limited as mine is, because increasing the value of an already-too-large number does nothing; this is already covered in Maple 13 by including an initial value of `unlimited', which some operating systems might accept. Some other OSs, like mine, only allow for 0.5GiB per application, although I can't find a statement of this anywhere on the web. I will have to contact Apple and get back to this thread if I make any headway there.
I should note again, for clarity, one important step which is missing in the previous instructions on how to increase this `hard limit' to the value dictated by OS constraints:
1. Right click on mserver_ulimit or mserver_ulimit32 and open with a text editor. (change to Read & Write if you have to).
2. Change the value which is limiting your allowed memory (you should check this first in Maple with kernelopts(datalimit)) to as high as your operating system will allow (you may have to determine this value through trial and error, but if one exists it should be some power of 2). Save the file and close it.
3. Double-click on mserver_ulimit (or mserver_ulimit32). It will run in Terminal. Close that.
4. Make sure Maple is not running. Open Maple and type kernelopts(datalimit). The output should be your specified limit.
Step 3 is important, because without doing this, Maple won't know that you've changed the value. I believe this is related to what is mentioned in the link that pagan posted.
If I find out anything from Apple, I will post that solution.