@alex_01 I understand. You yourself would like to see a pattern visually (perhaps sometimes from a smaller example). People who work regularly with linear algebra and math can often see such patterns from the notation itself, regardless of the size or number of individual equations forming the patterns.

I converted your example from the top of this post, to the Matrix form that I showed, in my head and directly typed into the worksheet. I did not use that internal routine to do it, or to detect the pattern. It was very easy, and took only about ten minutes.

The Matrix form target ([A,b,Aeq,beq],[bl,bu]) is extremely straightforward. It was quite clear how to obtain it from the Matrix-Vector multiplications which you did using W.

The proof is in the pudding. The fact that your Matrix-Vector multiplications were near trivial to convert is reflected in the simple fact that the data Matrices R and ER appear directly as the major portion of the Matrix form Matrix A.

Note that I had also shifted the ur and dr to the left-hand-side since that makes it clea that they are variables just like the w[i]. The rhs's are just numbers (mostly zero). The bottom left parts of A, Aeq are for the W, and handling only two more variable ur and dr required slightly larger matrices and a only few more lines of code.

If you don't think that the two-fold speedup at size n=500 is good enough then you have a simple choice, moving forward: if it's too hard, and the benefit too slight, then simply don't bother with it.

Careful optimization of code often requires several kinds of modifications, picking up a factor of 2 here, and some more there, sometimes leading to more impressive overall improvement. You are the one who originally wrote that you want the code here "as fast and efficient as possible", and that usually means that any modest improvements (taken piecemeal or not) are wanted. If your examples already run fast enough and you don't need them to scale up in size then you aren't forced down that road. Nobody else will care.

[I recall now that we have discussed this kind of thing before...]

@alex_01 I understand. You yourself would like to see a pattern visually (perhaps sometimes from a smaller example). People who work regularly with linear algebra and math can often see such patterns from the notation itself, regardless of the size or number of individual equations forming the patterns.

I converted your example from the top of this post, to the Matrix form that I showed, in my head and directly typed into the worksheet. I did not use that internal routine to do it, or to detect the pattern. It was very easy, and took only about ten minutes.

The Matrix form target ([A,b,Aeq,beq],[bl,bu]) is extremely straightforward. It was quite clear how to obtain it from the Matrix-Vector multiplications which you did using W.

The proof is in the pudding. The fact that your Matrix-Vector multiplications were near trivial to convert is reflected in the simple fact that the data Matrices R and ER appear directly as the major portion of the Matrix form Matrix A.

Note that I had also shifted the ur and dr to the left-hand-side since that makes it clea that they are variables just like the w[i]. The rhs's are just numbers (mostly zero). The bottom left parts of A, Aeq are for the W, and handling only two more variable ur and dr required slightly larger matrices and a only few more lines of code.

If you don't think that the two-fold speedup at size n=500 is good enough then you have a simple choice, moving forward: if it's too hard, and the benefit too slight, then simply don't bother with it.

Careful optimization of code often requires several kinds of modifications, picking up a factor of 2 here, and some more there, sometimes leading to more impressive overall improvement. You are the one who originally wrote that you want the code here "as fast and efficient as possible", and that usually means that any modest improvements (taken piecemeal or not) are wanted. If your examples already run fast enough and you don't need them to scale up in size then you aren't forced down that road. Nobody else will care.

[I recall now that we have discussed this kind of thing before...]

gem 18 is a nice keystroke saver.

acer

@alex_01 You appear to have a fundamental misunderstanding about the purpose of passing an LP problem to LPSolve in Matrix form.

The evidence for this is in your Comment above, where you first form the algebraic form, then convert it, and then pass it in. You even write that you are amazed that anyone would ever try and form the Matrix-form directly (without just using a utility to convert it).

That works. But there is no benefit to doing that conversion. The whole point of using Matrix form is to never form the algebraic equalities/inequalities/bounds-as-ranges.

The way you have done it (algebraic form, then conversion by command, then pass in the Matrix form) is exactly what LPSolve would do for you if you just passed it the algebraic form.

If you have formed the algebraic form, ever, at all, with symbolic variable names like w[i] etc, then you have already wrecked all the advantaages of using the Matrix form for calling LPSolve.

If you cannot understand this then I apologize but I'm afraid that I cannot help you further.

@alex_01 You appear to have a fundamental misunderstanding about the purpose of passing an LP problem to LPSolve in Matrix form.

The evidence for this is in your Comment above, where you first form the algebraic form, then convert it, and then pass it in. You even write that you are amazed that anyone would ever try and form the Matrix-form directly (without just using a utility to convert it).

That works. But there is no benefit to doing that conversion. The whole point of using Matrix form is to never form the algebraic equalities/inequalities/bounds-as-ranges.

The way you have done it (algebraic form, then conversion by command, then pass in the Matrix form) is exactly what LPSolve would do for you if you just passed it the algebraic form.

If you have formed the algebraic form, ever, at all, with symbolic variable names like w[i] etc, then you have already wrecked all the advantaages of using the Matrix form for calling LPSolve.

If you cannot understand this then I apologize but I'm afraid that I cannot help you further.

Are you sure that the current (serial, non-parallel) version of your code is well optimized?

Given (only) that overview of the problem I might guess that Grid would be easier, provided that you have enough memory to hold the multiple full instances of the whole problem instantiated at various parameter value sets. That would also alleviate your having to ensure that your code is Thread-safe.

But you might consider uploading your code here (green arrow in the editor). We have a little experience with optimizing code, though of course there are no promises. Feel free to contact me, if you'd prefer to not post your code publicly.

acer

@alex_01 I think that if you are having trouble understanding how to formulate an NLPSolve problem in Matrix-form then it would be better if you asked in a separate question (giving a full example that explains exactly what you are trying to do).

That way, the two threads remain clear to all late-comers.

@alex_01 I think that if you are having trouble understanding how to formulate an NLPSolve problem in Matrix-form then it would be better if you asked in a separate question (giving a full example that explains exactly what you are trying to do).

That way, the two threads remain clear to all late-comers.

@Joe Riel For this particular problematic file, the two characters are in stranges places. Doesn't one appear between otherwise conjoined numerals 2 and a 4, in a text section?

I think that the GUI should be able 1) to remove such when pasting, 2) not save with such, and 3) remove such when re-opening. Right now it seems to just halt all further reading/parsing and give up, when re-opening, which is weak.

@Joe Riel For this particular problematic file, the two characters are in stranges places. Doesn't one appear between otherwise conjoined numerals 2 and a 4, in a text section?

I think that the GUI should be able 1) to remove such when pasting, 2) not save with such, and 3) remove such when re-opening. Right now it seems to just halt all further reading/parsing and give up, when re-opening, which is weak.

It's disturbing how often instances of this problem get reported here. I count 4 or 5 times in the past couple of months. Which raises the question of how often it occurs and is not reported here.

I wonder how it occurs, whether the cause is preventable and, if not, why the GUI itself cannot accomplish a repair more reliably.

acer

It's disturbing how often instances of this problem get reported here. I count 4 or 5 times in the past couple of months. Which raises the question of how often it occurs and is not reported here.

I wonder how it occurs, whether the cause is preventable and, if not, why the GUI itself cannot accomplish a repair more reliably.

acer

@alex_01 In order for the debugger to be invoked when LPSolve gets called all one needs to do is,

stopat(Optimization:-LPSolve);

and then call Optimization:-LPSolve on some example.

@alex_01 In order for the debugger to be invoked when LPSolve gets called all one needs to do is,

stopat(Optimization:-LPSolve);

and then call Optimization:-LPSolve on some example.

@alex_01 You are very welcome, though you are giving me too much credit. All I did was run the first example of ?LPSolve under the debugger, and walked through until it came to the routine that did the conversion.

What you might find most interesting is that your R and ER appear "as is" inside `A` the Matrix of inequalities. You can also see this visually, if you set N=3 and print `A`. So it may illustrate the principal -- in Matrix form, there are just numeric coefficients, and no variables appearing explicitly by (symbolic) name.