Carl Love

Carl Love

21223 Reputation

24 Badges

8 years, 249 days
Natick, Massachusetts, United States
My name was formerly Carl Devore.

MaplePrimes Activity


These are replies submitted by Carl Love

@brian bovril Regardless of whether the integration can be done symbolically, there's no benefit to using unapply like that.

You can use a step with add, unlike sum. The only reason I used sum was because the integration could be done symbolically. If it can't be, you should use add anyway.

Isn't the expression u above also "a product of two Gaussians"? Yet it can be integrated symbolically nearly instantaneously. Can you show an example of just one term from your actual expression? 

This is likely of little significance towards fixing this; I just mention it as a curiosity: The assigned value is the final valid value, whereas with a for loop, it'd be the first invalid value.

@nmacsai You asked:

  • The data set I am working with is a csv file.  Is there a way to convert it into a binary format?

Once the numeric data is in a Maple Array (or Matrix or rtable) with a hardware datatype (such as float[8]), it can be made into a binary file with the command 

FileTools:-Binary:-WriteFile("file name", array name);

Reconstruction of the array from the file requires (at the very least) knowledge of the number of rows and columns (and other dimensions, if applicable). Making a complete and automatic conversion followed by reconstruction, including all of the array's options and attributes, is the subject of the first Reply to my Answer below.

The command convert(n, binary) that you mentioned represents a number n using the literal digits 1 and 0 (and possibly a decimal point). It is mainly for display rather than computational purposes.

@mmcdara If you're saying that the CSV file should be read into Maple once and then saved as a binary file for re-use within Maple, then I agree with you, and I've already said so, and my Answers show how to do it while retaining all the structural information (number of rows, number of columns, etc.).

But both the OP and I have interpretted "it is not recommended to work with csv files this size. It is much more efficient to use binary files instead" to mean that you're advocating the complete elimination of the CSV. That's what I disagree with. The OP's data is in a CSV, so they have no choice but to "work with a csv file of this size" at least once. And the binary file is worthless without that structural information (including the byte order), so I'd keep the CSV for long-term (aka archival) storage, or use one of those other formats (JSON, etc.) that are human readable. I'll note that CSV is one of the "acceptable" formats discussed on the "big data" web page that you linked.

Also, I have no idea what "essentially raises the main question" means. Perhaps it's a phrase that doesn't directly translate well from French to English.

@acer Does having a large DataTable slow down the GUI, like having a large plot or large prettyprinted formulas slows it down even when they're off screen? What about the auto-saves?

@Muhammad Usman There's no computational reason to put the ands in the piecewise conditions; indeed, they slow down the computation a tiny bit. So, Tom's form is functionally equivalent to yours. The only reason to use the ands would be to achieve a certain display.

The traditional textbook-style display of piecewise functions supposes that one views the conditions independently of each other. The lines could be rearranged in any order and they'd still make sense (although it would reduce the readability). But the conditions in a Maple piecewise are not processed independently of each other. They are processed strictly in the order presented, and the processing stops as soon as one condition is true. This is why the ands aren't needed when the conditions are given in real-number order.

The benefit of using a ".m" file is that it automatically stores all the information needed to reconstruct all the objects stored in it, whether they be arrays or anything else; and the entire reconstruction is handled quickly by a simple read command. But storing the array in a binary file is even much faster than a ".m" file (by a factor of 10 or so).

In the code below, my procedure SaveArray stores three files: the first a pure binary file containing the array's data, the second a small ".m" file containing the array's attributes (you don't need to know about attributes to understand the rest of this), and the third a small file of plaintext Maple code that can be used to exactly reconstruct the array from the first two files in a new session so that it's structured exactly the way that it was in the originating session.

restart
:
(*---------------------------------------------------------------------
Procedure to save a hardware-datatype rtable in a binary file along
with all the info needed for its reconstruction in a new session.

The 1st argument is the rtable, which must've been assigned to a 
name. The 2nd argument is a file name "stem" to use for three files.
It's assumed that currentdir() has been set to the desired directory.

The output is 3 files and a snippet of text on screen. For example,
if the stem is "mydata", then the output files are
    mydata.bin    | the raw form of the rtable
    mydataAttr.m  | the rtable's 'attributes'
    mydata.mpl    | plaintext Maple code file to be 'read' to 
                  | reconstruct the rtable from the 1st 2 files
The snippet of text is a line of code that can be copy & pasted to a
new session to further automate the reconstruction.
---------------------------------------------------------------------*)
SaveArray:= proc(A::evaln(rtable), F::string)
description 
    "Save a hardware-datatype rtable in binary file along with all "
    "info needed for its reconstruction in a new session           "
;
option `Author: Carl Love <carl.j.love@gmail.com> 2021-May-27`;
local 
    O:= [rtable_options](eval(A)), 
    OE:= select(type, O, `=`)
;
    (:-_Attr, O):= selectremove(
        type, O, identical(attributes)=anything
    );
    :-_Attr:= :-_Attr[];
    save :-_Attr, cat(F, "Attr.m");
    FileTools:-Binary:-WriteFile(cat(F, ".bin"), eval(A));
    FileTools:-Text:-WriteFile(
        cat(F, ".mpl"),
        sprintf(
            "read \"%sAttr.m\":                                  \n"
            "%a:= ArrayTools:-Alias(                             \n"
            "    FileTools:-Binary:-ReadFile(                    \n"
            "        \"%s.bin\", ':-datatype'= ':-%a'            \n"
            "    ),                                              \n"
            "    %a, ':-%a', %a[]                                \n"
            "):                                                  \n"
            "rtable_options(                                     \n"
            "    %a, %a[],                                       \n"
            "    attributes= [                                   \n"
            "        eval(                                       \n"
            "            attributes,                             \n"
            "            select(type, [rtable_options](%a), `=`) \n"
            "        )[],                                        \n"
            "        rhs(:-_Attr)[]                              \n"
            "    ]                                               \n"
            "):                                                  \n"
            ":-_Attr:= ':-_Attr':                                \n",
            F,                     
            A, 
            F, eval(':-datatype', OE),
            [rtable_dims](eval(A)), eval(':-order', OE),
            `if`(':-readonly' in O, [':-readonly'= true], []),
            A, remove(type, O, identical(datatype, storage)=anything),
            A
        )
    );
    :-_Attr:= ':-_Attr';
    printf("#Copy & paste next line to new session and execute:\n");
    (printf@cat)(
        "currentdir(\"",
        StringTools:-SubstituteAll(currentdir(), "\\", "/"), 
        "\"): ",
        "read \"", F, ".mpl\":\n"
    );
    return
end proc
: 
#random array simulating your conditions:
B:= rtable(
    (1..434000, 1..28), frandom(0..1), 
    datatype= hfloat, order= C_order, readonly, subtype= Matrix,
    attributes= ["This is the good data"]
):
B[42, 23]; #some random entry for verification
                       0.528682555121851

currentdir("/Users/carlj/desktop"):
st:= time[real]():
   SaveArray(B, "mydata");
time = (time[real]()-st)*Unit(second);
#Copy & paste next line to new session and execute:
currentdir("C:/Users/carlj/desktop"): read "mydata.mpl":

                      time = 0.126 Unit(s)

restart:
st:= time[real]():
    currentdir("C:/Users/carlj/desktop"): read "mydata.mpl":
time = (time[real]() - st)*Unit(second);
                      time = 0.164 Unit(s)

B[42, 23];
                       0.528682555121851

rtable_options(B);
datatype = float[8], subtype = Matrix, storage = rectangular, 
  order = C_order, readonly, attributes = [source_rtable = 
  [...numeric data elided...],
  "This is the good data"]


 

@2cUniverse Both the original procedure and VV's version require zero padding of the lists if either are shorter than d. I've made that correction and also added some other error checking:


`&*`:= proc(A::list, B::list, b::And(posint, Not(1)):= 10, d::posint:= 4) 
local i, k, c:= 0, A0:= table(sparse, A), B0:= table(sparse, B);
    if [A,B]::list(list(integer[0..b-1])) then 
        [seq](irem(c + add(A0[i]*B0[k+1-i], i= 1..k), b, 'c'), k= 1..d)
    else
        error "expected lists of base-%1 digits, received %2", b, [A,B]
    fi
end proc:

@Carl Love Actually, the same workaround can be done without overload because the eval pre- and post-wrappers do nothing in the usual case that the input doesn't contain a free _Z. I'd rather not use overload if it can be avoided because the structure that it returns is a bit awkward to deconstruct. Thus, 

restart:

interface(warnlevel=4):
kernelopts('assertlevel'=2):

CI__orig:= eval(SolveTools:-CancelInverses):
unprotect(SolveTools:-CancelInverses):
SolveTools:-CancelInverses:= e->
local _A;
    eval(CI__orig(eval(e, _Z= _A), _rest), _A= _Z)
:
protect(SolveTools:-CancelInverses, CI__orig):

expr:= exp(x)*sin(y)-3*x^2+(exp(x)*cos(y)+1/3/y^(2/3))*Z = 0;
solve(expr, y);

 

@tomleslie Ah, I was led astray by an incomplete 3rd-party help file https://exceljet.net/excel-functions/excel-mround-function which doesn't specify the type of m, and shows numerous examples using integer m

@MapleEnthusiast Your array has numIters rows. Row 1 stores the variable labels. Row 2 stores the initial guess. That leaves numIters-2 rows available for storing iterations. You need to create the array with 1 more row.

@mmcdara You said:

  •  it is not recommended to work with csv files this size.

That depends on who's doing the recommending. If computational efficiency is the concern, then I'd agree that you don't want to be constantly rereading a huge CSV. On the other hand, if data archiving is a concern, then I'd strongly recommend keeping it as CSV, which is essentially a universal format which'll remain easily readable for many years.

To the OP: If you use either acer's or my solution to this problem, then this is not an issue. So I recommend that you don't change that file!

@kiraMou You're still confusing singular "solution" with plural "solutions". Consider the solution list4 from your posted worksheet. It happens that this solution has imaginary parts -Pi/2 for all four variables. But you think that half of them should be +Pi/2. That's not what's meant by "the solutions (plural!) are symmetric with respect to the imaginary axis." That statement doesn't mean that each individual solution (each of which contains 4 variables) is symmetric. Instead, it means that if you take the conjugates of all 4 values, then you'll get another solution. Let's do that:

list4c:= evalindets(list4, complex(numeric), conjugate);
eval({t[1],t[2],t[3],t[4]}, list4c);

Now you can see that the 4 imaginary parts have been changed to +Pi/2. And you can see that the residual imaginary parts from this new solution are effectively 0, the deviations being due simply to roundoff error, and the residual real parts are effectively -1. Thus list4c is also a solution. It is the symmetric counterpart of list4.

@Carl Love Furthermore, if one did want to represent a curve as a surface (and there are good reasons that one might want to do that, such as elaborate colorings), then the better command for that is tubeplot, as shown by mmcdara.

@kiraMou A correct assignment of numeric values to x[1], ..., x[4] is a single solution. You've been referring to such a thing as "solutions" (plural). It's possible that this misclassification has led to some of your confusion.

2 3 4 5 6 7 8 Last Page 4 of 592