A request for support of multi-part MIME within Maple

Doug Meade's picture

Increasingly, I see the need to distribute a Maple worksheet with some auxiliary files. These extra files typically contain additional Maple code. These extra files could be a command file, a Maple libray archive (mla) or an image (for use in a maplet, say).

I can create my own ZIP archive to e-mail, but I would like to be able to post some of these resources on my website in a format that they can be automatically executed when downloaded. I understand that there are some potential security issues here, but I still like to see if there cannot be some support for this functionality.

What I have in mind is something very similar to the way attachments are handled in e-mail with multi-part MIME.

Here's what I think should work: use multi-part MIME to create a document whose main part is a Maple file with type (.mw or .maplet) and each additional file (command file, image, ...) is a separate part (a la attachments). When downloaded, if the browser has been appropriately configured, the parts are separated and the first part is sent to Maple (just as the e-mail reader displays the first part of the message as the body and the other parts as attachments).

Admittedly, I do not have a thorough understanding of this technology. I hope my explanation is sufficiently clear for someone with the technical knowledge to correct my statements or to express them in a more appropriate way - or to tell me why this is impossible.

Doug

---------------------------------------------------------------------
Douglas B. Meade
Math, USC, Columbia, SC 29208  E-mail: mailto:meade@math.sc.edu       
Phone:  (803) 777-6183         URL:    http://www.math.sc.edu/~meade/

Comments

acer's picture

.mla supports some or most of that

The .mla maple archive format itself supports some of what you mention.

It can contain more than just .m files. It can contain image files, other .mla archives. Maybe any other file, I think.

A .mla archive can also be set to execute and unpack files, with a popup progress maplet, when opened in the GUI.

See ?InstallerBuilder . Or run InstallerBuilder:-Interactive() .

acer

Doug Meade's picture

interesting, but problems

Interesting. I am still doubtful this will do what I need, but I'm willing to give it a try.

Consulting the online help, I come across the following example in the help for InstallerBuilder,General,Panels:

with(InstallerBuilder):
Build( "SampleToolbox",
    'uninstaller' = false,
    'author'      = "Some Maple User",
    'welcome'     = ['text' = "Welcome to the SampleToolbox installer"],
    'validation'  = [
         'text'     = "Enter the code below:",
         'validate' = proc()
              local code;
              code := ToolboxInstaller:-Data:-Get("validation_attempt");
              if code = "My Secret Code" then
                  true
              else
                  false
              end if;
         end proc
    ]
);

When I execute this, all I see is the following warning message:

Warning, InstallerBuilder is not a correctly formed package - option `package' is missing

I am running Maple 11.01 with the standard interface, worksheet and 1-D Maple input. Does any of this matter?
Am I missing something? Does this work for anyone?

It's still not clear to me that this will do what I want, but it could be a start. I will try to put together a sammple set of files and see if someone can create a single file to do what I have tried to describe.

More to follow soon,

Doug

---------------------------------------------------------------------
Douglas B. Meade
Math, USC, Columbia, SC 29208  E-mail: mailto:meade@math.sc.edu       
Phone:  (803) 777-6183         URL:    http://www.math.sc.edu/~meade/
acer's picture

InstallerBuilder:-Interactive

I never found the help for InstallerBuilder to be very useful. It's not very full, especially concerning what appears to be ability for scripting.

What I have had success with is the InstallerBuilder:-Interactive() routine. However, what I've used it for in the past is simply to "Add" files (there's a button for that). I add files, specifying where it is to grab them from, and where it is to install them when run.

It outputs a single .mla file, which contains all the bundled files. When this .mla is opened in the GUI, a maplet gets run, with a few panels, just like an installer. The bundled files may contain other .mla files, .hdb files, example worksheets, dynamic/shared libraries, etc.

Once or twice, I've had it create and bundle its own uninstaller, too.

I suspect that it was designed, originally, for 3rd party MapleConnect products, so that authors could create their own installers. I don't know if it's still used.

One thing that I find interesting, is that Maple seems to find directories like <yourmaple>/toolbox/XXXX/lib and <yourmaple>/toolbox/XXXX/bin.<platform> and append libname and PATH with them automatically. Does this not make such locations ideal for placing user-authored packages, especially when bundled/redistributed for others?

I also find interesting the idea that a .mla archive may be autoexecuting in some way. Something clearly gets created, which does the work. So, if the InstallerBuilder is not to taste, I wonder whether one could find and utilize the tech behind it regardless.

acer

Doug Meade's picture

this looks promising

The interactive builder does appear to present some chance that I can do what I want. It will take more time to understand how all of the features work and any limitations.

I will try to put together a small example to demonstrate what I want to do, and what I am able to figure out with the interactive InstallerBuilder.

Thanks for the suggestion.

Doug

---------------------------------------------------------------------
Douglas B. Meade
Math, USC, Columbia, SC 29208  E-mail: mailto:meade@math.sc.edu       
Phone:  (803) 777-6183         URL:    http://www.math.sc.edu/~meade/
Doug Meade's picture

an example of a maplet with an auxiliary file

I'm back, with a (simple) example of what I want to do. I will demonstrate with a very simple (and small) example.

Download the ZIP archive to your computer, then extract the contents to create a folder named MapletWithImage and containing the following four files:

  • Simple.mw -- Maple 11 worksheet defining a maplet that displays an image [17,788 bytes]
  • Simple.maplet -- Simple.mw exported as a maplet (actually, just a text file of commands) [368 bytes]
  • PrimesBack.jpg -- an image file (stolen from MaplePrimes) [8,094 bytes]
  • MapletWithImage.mla -- a Maple Library Archive created by InstallerBuilder[Interactive] [135,075 bytes]

Browse to this folder and double-click on Simple.maplet and the maplet that appears should display an image from MaplePrimes and give you a choice of two buttons to click. To see the actual commands used to create the maplet, either open the .maplet file with your favorite text editor or open the .mw file as a Maple worksheet.

My goal is to be able to serve this maplet via the internet in a way that it automatically executes when downloaded. The Simple.maplet file is available on MaplePrimes. Click on this link and the maplet should be downloaded to your computer and executed using a local copy of Maple. Note that the image is not displayed because the image file is not found in the directory currently saved in currentdir().

The .mla file is my initial attempt to use InstallerBuilder[Interactive] to create a Maple Library Archive containing the maplet and image files. The .mla contains these two files and. because I included the uninstaller, Maple opens and the files are extracted. What I have not yet been able to do is find a way to have the maplet automatically execute after extraction.

The InstallerBuilder does provide for a "script" and "validation" for each step. The online help give me some encouragement that there might be a way to use these to achieve my goal. But, to date my attempts have been unsuccessful. It seems that I might need to use a system (or ssystem) call to send a maplet to the maplet viewer, but this seems like it might be a little overkill.

I hope my explanation is clear and that there is someone out there who is able to put together a single file that can be served by HTTP, extracted and executed automatically. Please let me know if you have any questions about this.

I notice also how inefficient the .mla appears to be. I'll guess that it includes a lot of information needed to ensure the files are executed in an appropriate environemnt, but the .mla file is more than 15 times larger than the combined sizes of the .maplet and .jpg files.

Thanks for reading,

Doug

---------------------------------------------------------------------
Douglas B. Meade
Math, USC, Columbia, SC 29208  E-mail: mailto:meade@math.sc.edu       
Phone:  (803) 777-6183         URL:    http://www.math.sc.edu/~meade/
acer's picture

trick bag

(Pardon my use of Linux filesystem paths.)

It took a while, but I figured out that the InstallerBuilder "scripts" are just supposed to be single procedures.

So, I suppose that you might be able to put the whole Maplet into a single proc, as the Finish panel script. I didn't try that. I chose to do it another way, by embedding the Maplet in an autoexecuting help-page, which gets called by the Finish panel script.

For the Finish panel in Installerbuilder, I added this as the script:

proc() try help(StartMe); catch: end try; end proc;

Next, I edited your Simple.mw worksheet a little. I removed the restart. I joined the rest into a single execution group. I select the whole group and toggled it as autoexecute. Then I reselected it and collasped that selected Document Block. Then I saved it. To test, just reopen it in a new session, and check that it autoexecutes.

Next, I saved that worksheet to a .hdb help-database as an "active" help-page. The .hdb needn't be present, it will get created by the INTERFACE_HELP() call. I did that like so,

libname:="/home/acer/dmeade","/usr/local/maple/maple11/lib";
startmefile:="/home/acer/dmeade/files/Simple.mw";
tmp := readbytes(startmefile, TEXT, infinity):
fclose(startmefile);
INTERFACE_HELP(insert, active = true, topic = "Simple,StartMe",
aliases = ["startme","StartMe"], text = TEXT(tmp),
library = "/home/acer/dmeade/Simple.hdb");

Then I ran the InstallerBuilder:-Interactive() command. I added only these files:

Simple.hdb as lib/Simple.hdb
PrimesBack.jpg as misc/PrimesBack.jpg

It was quite nice to see, that the InstallerBuilder knew to suggest lib as the default location for the .hdb file.

And, of course, when I open the resulting Simple.mla file in Maple it starts the installer and unpacks the materials. On Linux, it unpacks it under /home/acer/maple/toolbox/Simple/ .

And it is sweet, that /home/acer/maple/toolbox/Simple/lib/ is automatically recognized by Maple and added to libname. In fact, by the time the installer gets to its last (Finish) panel, and triggers the script I described above, the .hdb can already be used.

And so the help(StartMe) query actually loads the Simple.mw page, inline in a worksheet, and autoexecutes the collasped Document block, and the Maplet runs. The only action I had to do was to have Maple start/open the Simple.mla file.

acer

Doug Meade's picture

getting closer

Nice explanation. This works exactly as described.

I would like to be able to avoid some of the popups in the installation process, but that is not the issue here.

I created SimpleMaplet.mla exactly (I believe) as acer described. When the SimpleMaplet.mla is double-clicked from the folder in which it was created, all works great. It still works if I move the .mla file to a different folder, but the image is not displayed. The same happens if I access the .mla file via the WWW.

I believe the image file from the MLA is not being seen. I confirmed this (to me) by changing the name of the image file in my working directory. Now, running the maplet from the archive does not show the image. Restore the name of the file and it's ok again.

With a little thought, this makes sense. The hdb is saved in lib and the image is saved in misc and these two folders exist in a folder named toolbox somewhere in the user's file system (/home/acer/maple/toolbox/ in acer's Linux system and C:\Documents and Settings\mead\maple\toolbox on my Windows system).

When I changed the code in the definition of the maplet in the .mw file to explicitly look for the image in this path the maplet and image appear when this MLA is downloaded.

To try this for yourself, use http://www.math.sc.edu/~meade/TEST/SimpleMaplet.mla (if necessary, create a file association for .mla files to Maple). Save the file or execute it directly, my results are the same. (You will have to click through the installation process. Just accept everything as is.)

This works - but only for me. What is missing is a way to write the maplet without using a absolute filename for the image. Would there happen to be an environment variable that contains the location where the MLA will be extracted to? (How does the uninstaller know where to put the extracted files?)

That's enough for this post.

Comments and suggestions are always welcome!

Doug

---------------------------------------------------------------------
Douglas B. Meade
Math, USC, Columbia, SC 29208  E-mail: mailto:meade@math.sc.edu       
Phone:  (803) 777-6183         URL:    http://www.math.sc.edu/~meade/
acer's picture

right

Right. I'd just assumed that you would hard-code the relative path to the image file in the Maplet. And I'd likewise assumed that that path would be relative to the toolbox/Simple/ location.

As far as where the installer puts things, I believe that it may use cat(kernelopts(homedir),"/maple/toolbox/") concatenated further with the title "name" for the toolbox as given when constructing the installer (as seen in the Interactive routine, for example). There is also some option for a "network" installation. This might use kernelopts(mapledir), but I didn't check.

nb. You mentioned uninstaller, but really it is the SimpleMaplet.mla which is technically the *installer*. The uninstaller is a separate .mla which when run should autoexecute and remove the toolbox/Simple files.

As for getting rid of all those panels, well, maybe the techniques used by InstallerBuilder can be inspected and mimicked. For that, I'd suggest studying the workings of the routines InstallerBuilder:-Build and LibraryTools:-ActivationModule . You might be able to study the help-page for LibraryTools:-ActivationModule and construct a more lean and direct was to get an auto-opening/executing .mla archive to do just your particular tasks. InstallerBuilder is just one very specialied instance of using that, I think.

acer

Doug Meade's picture

... and closer

I have now updated the MLA that can be access at the URL http://www.math.sc.edu/~meade/TEST/SimpleMaplet.mla. This version of the file uses kernelopts(homedir) in an attempt to automatically generate the corect absolute path to the image.

It works for me. Does it work for anyone else? ("works" means the maplet and image are extracted, the maplet runs, and you can see an image in the maplet)

Doug

---------------------------------------------------------------------
Douglas B. Meade
Math, USC, Columbia, SC 29208  E-mail: mailto:meade@math.sc.edu       
Phone:  (803) 777-6183         URL:    http://www.math.sc.edu/~meade/
Doug Meade's picture

InstallBuilder:-Builder and InstallBuilder:Interactive

The interactive InstallBuilder is pretty easy to use. But, it can become tedious to have to enter all information about the MLA ou are trying to create.

InstallBuilder:-Interactive does accept optional arguments that can be used to specify any of the values that can be set within the GUI. For example:

InstallerBuilder:-Interactive(
  name = "SimpleMaplet",
  author = "Douglas B. Meade",
  version = "1.0",
  uninstaller = false,
  NULL
);

These values are defaults, and can be changed by the user.

You can also do everything directly from the command line. Here is the command I used to create the MLA discussed in another post in this thread:

InstallerBuilder:-Build(
  name = "SimpleMaplet",
  author = "Douglas B. Meade",
  version = "1.0",
  uninstaller = false,
  manifest = [
   "C:\\Documents and Settings/mead/Desktop/MapletWithImage/Simple.hdb"
    = "lib/Simple.hdb",
   "C:\\Documents and Settings/mead/Desktop/MapletWithImage/PrimesBack.jpg"
    = "misc/PrimesBack.jpg",
   NULL ],
  welcome = [
    text = "Welcome! Just accept all prompts and the maplet should run, eventually.",
    image = none,
    script = none,
    validate = true],
  installation = [
    text = "Two more steps.",
    image = none,
    script = none,
    validate = true],
  finish = [
    text = "Your patience has paid off. This is the final step!",
    image = none,
    script = proc() try help(StartMe); catch: end try; end proc,
    validate = true],
  NULL
);

It is pretty self-expanatory, but I would never have come up with all of this from the online help. (To be honest, the majority of this comes from the Build Command in the Interactive builder.)

I hope this is useful to someone else.

Doug

---------------------------------------------------------------------
Douglas B. Meade
Math, USC, Columbia, SC 29208  E-mail: mailto:meade@math.sc.edu       
Phone:  (803) 777-6183         URL:    http://www.math.sc.edu/~meade/
acer's picture

better help

Doug, you are quite right. The help for InstallerBuilder could do with a lot more detail.

It may not even be your ideal long-term answer. Multi-part mime handling sounded exciting. But the InstallerBuilder is a very nifty little tool for distributing about one's Maple packages. There's all kinds of power behind it. I find that this is true of Maple all round -- so many darker corners hiding a true wealth of functionality.

acer

JacquesC's picture

Hidden wealth

Frustrating isn't it? You are quite right: Maple is in fact substantially more powerful and has a fair bit more functionality than what is actually documented. At least Maple's source and functioning is mostly visible, so that those willing to dig are amply rewarded. Too bad most of Maplesoft's customers can't more easily benefit from some of that hard work done by the developers.

Doug Meade's picture

Maple Reader?

Let's take this discussion in a slightly different direction.

The distribution of Maple 11.01 included a new application, the Maple Reader. Very little documentation exists for this. I was successful in getting a sample file to use with the Maple Reader. I find it interesting that the file I received from Maplesoft was a .mla file. I do not know how they created it, and I have not tried to extract any information from it.

I just thought it was interesting that it does appear Maplesoft has some plans to use MLAs in a way similar to what we are discussing here.

I will add that the Maple Reader interface was pretty nice. I table of contents in the left column, and a worksheet displayed in the main window on the right. In my case, there wasn't much to display, but my maplet was launched once it was selected from the list on the left panel.

Does anyone have more information about the Maple Reader?

Doug

---------------------------------------------------------------------
Douglas B. Meade
Math, USC, Columbia, SC 29208  E-mail: mailto:meade@math.sc.edu       
Phone:  (803) 777-6183         URL:    http://www.math.sc.edu/~meade/
acer's picture

LibraryTools:-ActivationModule

This can be done using an .mla and LibraryTools:-ActivationModule() directly.

try FilesTools:-Remove(".//SimpleTest.mla"); catch: end try:
LibraryTools:-Create(".//SimpleTest.mla");
march('addfile', ".//", "files//PrimesBack.jpg", "SimpleTest.mla");
                                                                                                                                           
initproc:=proc()
local maplet, TEMPDIR, found, x, IMAGELOC;
    TEMPDIR := getenv(TEMP);
    if not (TEMPDIR<>"" and FileTools:-Exists(TEMPDIR)=true
            and FileTools:-IsDirectory(TEMPDIR)=true) then
        TEMPDIR:=cat(kernelopts(homedir),"//Temp//");
        if not (TEMPDIR<>"" and FileTools:-Exists(TEMPDIR)=true
                and FileTools:-IsDirectory(TEMPDIR)=true) then
           error "unable to find directory for temp files";
        end if;
    end if;
    for x in libname do
        try
            march('extractfile', x, "SimpleTest.mla", cat(TEMPDIR,"PrimesBack.jpg"));
            found := true;
        catch:
        end try;
        if found = true then break; end if;
    end do;
    if found<>true then
        error "a problem occurred while attempting to extract the temp files."
    end if;
    IMAGELOC := cat(TEMPDIR,"PrimesBack.jpg");
    use Maplets[Elements] in
    maplet := Maplet(
     [
       Label(Image(IMAGELOC)),
       "Did this maplet show an image stolen from the MaplePrimes website?",
       [
         HorizontalGlue(),
         Button("Yes", Shutdown("Image was shown")),
         Button("No ", Shutdown("Image was not shown")),
         HorizontalGlue()
       ]
     ] ):
    Maplets[Display](maplet);
    end use;
    try
        FileTools:-Remove(IMAGELOC);
    catch:
        error "a problem occurred while attempting to delete the temp files."
    end try;
end proc:
                                                                                                                                           
LibraryTools:-ActivationModule("SimpleTest.mla",initproc);

acer

Doug Meade's picture

initial attempts unsuccessful

I assume you wrote this for Linux. My initial attempts to use this in Windows were not successful. The initproc procedure raised an exception because found<>true ("a problem occurred while attempting to extract the temp files"). I have not had time to debug initproc to see if something needs to be different in the construction of TEMPDIR.

Is it correct to put "files//PrimeBack.jpg" in the archive and to extract cat(TEMPDIR,"PrimeBack.jpg")?

Once this is made more robust, I can see that it would be very easy to pass the maplet filename as an argument so that this could be used to automate the creation of the MLA files for an entire collection of maplets.

Thanks for all of your efforts. I'll get back in touch when I have more time to look at this again.

Doug

---------------------------------------------------------------------
Douglas B. Meade
Math, USC, Columbia, SC 29208  E-mail: mailto:meade@math.sc.edu       
Phone:  (803) 777-6183         URL:    http://www.math.sc.edu/~meade/
Doug Meade's picture

LibraryTools[ActivationModule] works great!

Now that I have taken a little time to better understand libraries and archives I finally understand acer's post. There were a couple of typos and I believe I have made some (minor) improvements.

Here is my current code:

restart;
if StringTools[Search]( "Linux", interface(version) )>0 then
  DOT := ".//";
else
  DOT := cat(currentdir(),"/");
end if:

try
  FileTools[Remove](cat(DOT,"/SimpleTest.mla"));
catch:
#  WARNING( "There was no previous MLA to remove. Proceeding." );
end try:

LibraryTools:-Create("SimpleTest.mla");
march('addfile', DOT, cat(DOT,"files/PrimesBack.jpg"), "PrimesBack.jpg");

initproc:=proc()
local found, maplet, x, IMAGELOC, TEMPDIR, IsValidDirectory;
    TEMPDIR := getenv(TEMP);
    IsValidDirectory := d -> evalb( d<>"" and FileTools[Exists](d) and FileTools[IsDirectory](d) );
    if not IsValidDirectory(TEMPDIR) then
        TEMPDIR:=cat(kernelopts(homedir),"/Temp/");
        if not IsValidDirectory(TEMPDIR) then
           error "unable to find directory for temp files";
        end if;
    end if;
    IMAGELOC := cat(TEMPDIR,"PrimesBack.jpg");
    found:=false;
    for x in libname while not found do
       try
          march('extractfile', x, "PrimesBack.jpg", IMAGELOC);
          found := true;
        catch:
       end try;
    end do;
    if not found then
        error "a problem occurred while attempting to extract the temp files."
    end if;
    use Maplets[Elements] in
    maplet := Maplet(
     [
       Label(Image(IMAGELOC)),
       "Did this maplet show an image stolen from the MaplePrimes website?",
       [
         HorizontalGlue(),
         Button("Yes", Shutdown("Image was shown")),
         Button("No ", Shutdown("Image was not shown")),
         HorizontalGlue()
       ]
     ] ):
    Maplets[Display](maplet);
    end use;
    try
        FileTools:-Remove(IMAGELOC);
    catch:
        error "a problem occurred while attempting to delete the temp files."
    end try;
end proc:
LibraryTools:-ActivationModule(cat(DOT,"SimpleTest.mla"),initproc);

The main changes I made were:

  1. began to provide cross-platform support (not thoroughly tested)
  2. fixed spelling of FileTools in Remove
  3. add the image with an indexname that is related to the image
  4. created local proc to clean up checking for a valid directory

This library archive is now posted on my personal website. It can be accessed at the URL http://www.math.sc.edu/~meade/TEST/SimpleTest.mla.
Compare this to the original MLA created with InstallerBuilder.

Not only is the installation cleaner, but the new MLA file is less than half the size of the original one. (It is still almost twice the size of the two files needed to run the maplet locally.)

I do have one more desire. I would like to have the maplet open without any visual interference from the Maple interface.

FAST FORWARD ONE HOUR

A way to achieve this occurred to me while I was teaching. This is interesting enough that I will post it in a separate blog entry. It will appear as soon as it is composed.

Doug

---------------------------------------------------------------------
Douglas B. Meade
Math, USC, Columbia, SC 29208  E-mail: mailto:meade@math.sc.edu       
Phone:  (803) 777-6183         URL:    http://www.math.sc.edu/~meade/
acer's picture

sure

Yes, I used Linux. I thought that perhaps the // dirsep would work on Windows too.

It matters whether TEMP is set, as an OS environment variable, to a writable directory. If not set then Temp must be a directory in your homedir, as the code is written.

The cat(TEMPDIR,"PrimeBack.jpg") is the location that the image file will be written out to, when the initproc procedure gets run. The location "files//PrimeBack.jpg" was simply the source location, relative to currentdir when I actually ran all that code to create the .mla file. Those two locations, then, aren't related.

The initproc is just three steps. The first is to find a temp directory and write out the image file (storing that location). The second is to use that location, and invoke the Maplet which uses that image file. The third is to delete that unpacked image file. Naturally, I realize that you'll have seen all that.

I didn't really know much about LibraryTools:-ActivationModule before now. It seems a useful thing to learn.

acer

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
}