User login
Login with your MaplePrimes or Maplesoft.com account
Poll
Collaborative Books
Syndicate
Maple benchmark
1410I have been thinking about creating a Maple benchmark, to show a performance ranking versus operating system and hardware.
Is there any interest in such a thing?
If the results were put in a mapleprimes blog post, then they might be editable, and so could be updated as more items were submitted (messaged or emailed, to me, say).
What about suggestions for design? While a single numeric value might be useful, would a breakdown by area be valuable? Ie, evalhf, hardware-numeric, symbolic, large-integer, large float, involved computation, 2D Math rendering, plotting, etc. Depending on the design, it might be comprised of both 1D and 2D components, possibly split by plaintext/worksheet and Document.
acer
Announcements
Active Main Forum Topics
Active Student Forum Topics
Recent comments
- A procedure
2 hours 24 min ago - ellipse
3 hours 47 min ago - Also try the options scaling, gridrefine and crossingrefine
5 hours 4 min ago - Try implicitplot
5 hours 47 min ago - The patch
7 hours 23 min ago - thx!
7 hours 49 min ago - The source of the bug
8 hours 7 min ago - my concern is ...
9 hours 3 min ago - plotting with the "Standard interface"
9 hours 22 min ago - variant
9 hours 40 min ago
Recent blog posts
Recent book pages
I have been thinking about
I have been thinking about creating a Maple benchmark, to show a performance ranking versus operating system and hardware.
Is there any interest in such a thing?
Yes, however Maple's performance tends to be fairly unaffected by a lot of things. I think it would be better to have benchmarks covering specific functionality: solving a linear system, factoring polynomials, etc. Then you could subtract algorithm time to measure things like the overhead of solve or simplify. I would also like to know the cost to evaluate functions at the default setting of Digits, and the time for the GUI to print X megabytes of 2d math. Designing benchmarks like this is quite a lot of work.
Agreed
Designing benchmarks like this is quite a lot of work.
Sure is! But worthwhile.
However, I disagree with your statement about Maple's performance being fairly unaffected by a lot of things. In theory, that's true. In practice, it's not. You can measure a huge difference (in some cases) on the same hardware but running different OS (because memory-intensive Maple computations are very sensitive to the underlying OS's memory management routines). Maple can also be very sensitive to hardware configurations [like cache size for numerics intensive computations]. And it can vary on whether you are running the computation using TTY, Classic or Standard [why that is so mystifies me, but I have definitely measured such an effect].
A massive but very thorough timing suite would be very useful. We could measure both progress and regressions that way too.
Maple benchmarks
Maple can also be very sensitive to hardware configurations [like cache size for numerics intensive computations]
Optimized, compiled routines can tell you quite a bit about the efficiency of the processor and memory architecture, however hardware review sites already report on this. Synthetic benchmarks are a pretty good indicator of theoretical peak performance. I'm just not sure what the performance of the Maple kernel (never mind the library) would be measuring. Maplesoft could make a small tweak and radically change the timings. It would be useful to have a measure of Maple performance over time, because some things have definitely slowed down. Polynomial multiplication, for example, was much faster in Maple VR3-VR5, than Maple 7: http://www.cs.berkeley.edu/~fateman/papers/fastmult.pdf
And it can vary on whether you are running the computation using TTY, Classic or Standard
I believe this is because parts of the standard interface (like 2d math typesetting) is written in the Maple language. This gets interpreted by the kernel, and the extra memory adds to the garbage collection time.
It would be interesting
But I wonder how all this information can be organized in clear way within the "limited" framework of a MaplePrimes blog. Eg, tables are needed, and apparently html tables are not available. Would be ascii tables enough, within the space available in the central column?
sourceforge project
I believe that it would be better organized as a sourceforge project, with a mapleprimes blog page to indicate progress/updates, with links to full html pages for the details.
this is all welcome
The comments so far are very welcome.
I'll add one or two myself.
I was originally thinking just of a quick data sheet that could assist people who are aquiring systems for running Maple (some individual machines, some for Maple computer labs, etc). Comments by Roman and Jacques bring home the point that a good performance suite would allow tracking of Maple's performance across releases. That could be useful information.
A few more subtleties of a benchmark suite: Some parts could have tasks split by restart (or, maybe better, by wholly fresh session), to minimize interference amongst tasks as memory allocation grew and memeory management costs took effect. OK.
But some other parts might deliberately involve lots of tasks together, because that might get closer to typical usage.
There is also the question of Maple performance over very long continuous durations, possibly with large memory allocation. There's an active thread in comp.soft-sys.math.maple related to this.
Speaking of memory, there is also the question of memory fragmentation. Maple seems to not do best when contiguous memory hardware rtables are allocated and then unreferenced (ie. when they become collectible as garbage). The collected memory is not always available in larger free blocks, in current Maple, due to fragmentation of the freed space. I have heard proposals that garbage collection in Maple might be altered so as to also move memory chunks that were still in use. Such memory packing might release larger free contiguous blocks. Some similarity might be had with respect to Matlab's `pack` command, as far as the final effect (if not the means) goes of such memory consolidation.
The more I think about, the more I see that the two purposes would be better having very different and completely distinct sources: one simple set of codes to show the relative performance of Maple across OS/hardware, and another more sophisticated suite for long-term measurement of Maple as it develops.
acer