What is missing from MATLAB #2 - the next decade edition

Meta threads have a tendency to grow large. This has happened several times before (the wishlist threads #1 #2 #3 #4 #5, and 'What frustrates you about MATLAB?' #1 and #2).

No wonder that a thread from early 2011 has also kept growing. After just under a decade there are (at time of writing) 119 answers, making the page slow to load and navigate (especially on mobile). So after a friendly nudge; here is a new thread for the things that are missing from Matlab.

Same question: are there things you think should be possible in Matlab, but aren't? What things are possible with software packages similar to Matlab that Matlab would benefit from? (note that you can also submit an enhancement request through support, although I suspect they will be monitoring activity on this thread as well)

What should you post where?

Wishlist threads (#1 #2 #3 #4 #5): bugs and feature requests for Matlab Answers

Frustation threads (#1 #2): frustations about usage and capabilities of Matlab itself

Missing feature threads (#1 #2): features that you whish Matlab would have had

Next Gen threads (#1): features that would break compatibility with previous versions, but would be nice to have

@anyone posting a new thread when the last one gets too large (about 50 answers seems a reasonable limit per thread), please update this list in all last threads. (if you don't have editing privileges, just post a comment asking someone to do the edit)

[madhan ravi]

Thanks Rik!

[Walter Roberson]

Thanks, Rik!

[Adam Danz]

Thanks for the lists of curated threads, @Rik!

Answer 1

[cui]

Deep Learning toolbox:

How to visualize the changes in dlarray/weight distribution with histograms in deep learning?

Histogram displays how the trend of tensor (weight, bias, gradient, etc.) changes during the training process in the form of histogram. Developers can adjust the model structures accurately by having an in-depth understanding of the effect of each layer.

source

source2

[cui]

Related issues：

visualize the deep neural network architecture ？？？

It seems that Mathworks is not interested in many of the improvement questions (above and below) raised by me, and I regret that there is still no answer!

[dpb]

TMW will not comment in open forum on pending development or toolbox direction; you can be confident they have seen and read the above, but you cannot interpret no response as no interest.

If you wanted more direct input, submit items as enhancement requests, but again, TMW simply does not share its plans in open forums.

[Bruno Luong]

This thread is not an official enhancement request.

To me it's simply a place where people come to emplty the bag of complains.

[Adam Danz]

The first plot is similar to Matlab's waterfall plotting function.

The GIF is too quick for me to make any sense out of it.

Perhaps you could directly suggest these ideas using Contact Us - MATLAB & Simulink

[cui]

@Adam Danz Thank you for your suggestions, I look forward to the official future version will be greatly improved！

[cui]

it's lucky to hear that recently matlab version support deepnetwork visualize weight distribution. here is one example link

https://www.mathworks.com/help/deeplearning/ug/detect-vanishing-gradients-in-deep-neural-networks.html

Answer 2

[cui]

Deep Learning toolbox:

• action recognition: mmaction2
• video understanding, X3D: https://github.com/facebookresearch/SlowFast ，https://github.com/kenshohara/3D-ResNets-PyTorch
• reID: https://github.com/JDAI-CV/fast-reid
• track: https://github.com/ZQPei/deep_sort_pytorch，https://votchallenge.net/challenges.html
• Metric Learning: https://github.com/KevinMusgrave/pytorch-metric-learning
• image Classification: Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition
• object detection: Centernet, https://github.com/xingyizhou/CenterNet , efficientDet, https://github.com/google/automl/tree/master/efficientdet，detectron2，https://github.com/facebookresearch/detectron2 ,PointRCNN,https://github.com/sshaoshuai/PointRCNN
• face recognition: insightface, https://github.com/deepinsight/insightface, sphereface,https://github.com/wy1iu/sphereface, arcface,https://github.com/ronghuaiyang/arcface-pytorch
• Semantic segmentation, instance segmentation，https://github.com/facebookresearch/maskrcnn-benchmark , Semantic Segmentation (Most of them are difficult to implement with matlab)
• deeplearning preprocess data: https://github.com/NVIDIA/DALI
• StyleGAN: https://github.com/NVlabs/stylegan
• DeepNetwork Architecture：Mnasnet，https://github.com/tensorflow/tpu/tree/master/models/official/mnasnet , Bayesian-Neural-Networks , Graph Convolutional Networks
• similarity search: https://github.com/facebookresearch/faiss
• deepfakes: https://github.com/deepfakes/faceswap
• deepnetwork inference performance: https://github.com/microsoft/onnxruntime ,https://github.com/zuoqing1988/ZQCNN
• image Labeler app: https://www.mathworks.com/matlabcentral/answers/789159-how-does-image-labeler-app-realize-this-interactive-operation-by-writing-a-program?s_tid=srchtitle

and so on....

The above are influential applications of deep learning in various aspects, but it is difficult to reproduce in matlab. Although Matlab2019b version supports automatic differentiation mechanism, it is still difficult to implement algorithms in matlab. The efficiency of the differentiation mechanism is not high, and many operators do not support it. I tried to implement the more famous yolov3/v4 algorithm with the latest MATLAB2020a version, but it is still not satisfactory

In summary, my personal suggestions are like my personal answer above, and I hope that future versions can improve a lot!

以上都为深度学习在各个方面有影响力的应用，但是在matlab中复现困难，虽然Matlab2019b版本支持自动微分机制，但仍然不易在matlab实现算法，微分机制效率不高，很多operators也不支持。。。

总之，我的个人建议就像上面的个人回答建议一样，我希望将来的版本可以有所改善！

[cui]

I have been following the latest development progress of "deeplearning toolbox", and although there are many new features listed, most of them are superficial improvements with very limited substantive support for the "operator", which is still very inadequate for experienced and in-depth researchers.

I hope that mathworks will seriously develop basic modules for the toolbox, instead of small features that are not very important to attract beginners.

我一直在关注“deeplearning toolbox”的最新开发进展，虽然有很多新的功能项被列出来，但这些大部分都是表面上改进，实质上的支持operator非常有限，对于有经验的深入研究者来说还显现的非常不足。

Answer 3

[dpb]

Editor won't restrict a substituion to selected area...best it knows of is the function. Uncool in the max! I'm now having to fix up a bunch of stuff shouldn't have to have done... :(

Answer 4

[Walter Roberson]

A "soft interrupt" -- that is, an interrupt that can be handled with try/catch .

For example I am running a long calculation at the moment that iterates several times, and each iteration can take more than an hour. I don't always have the patience to wait through as many iterations as I asked for. For my purposes, the output of the last successful iteration would be "good enough" if I were to ask to interrupt the function.

I can control-C, but that interrupts the function completely, losing all of the outputs, and losing the function variables.

If I could somehow "soft interrupt" and have it return the current values of all variables, then that would be good enough for my current task. But the generalization of that would be the ability to catch a soft interrupt in a try/catch so that the code could make whatever final summariziation it needed to in order to create usable output variables. Furthermore, even though my own routine might be happy to return the "last good" values of the output variables, any soft-interrupt I requested might well get received while some lower-level routine had control that did not know about soft-interrupts, so I would want interrupt (with no return values) to apply to those layers until control reached my handler for a clean run-down.

[Steven Lord]

This isn't exactly what you're asking for but you could pause your running program (which may pause in a lower-level routine.) Then select your function's workspace and save the variables you want to keep. Or if you want MATLAB to continue the current iteration and stop just before it starts the next, add a breakpoint on the line where the next iteration will start and continue the current iteration. This functionality has been part of MATLAB since release R2016a.

[Walter Roberson]

It can be a pain to dig into the program each time, and requires detailed knowledge of the program.

... I've already held off several releasing this program because I am not happy with the command line interface I have for it (too many obscure options that are rarely (but sometimes) useful)... I don't expect people to manually poke through the code to put in breakpoints and know which variables to save... but it would be common for people to get tired of the program executing for hours and to hope not to lose the progress so-far if they interrupted. Yeah, I could save to global variables, but...

ICK

[Rik]

Another solution in current Matlab would be a GUI with a flag controlled by a button. But I believe you have previously talked about a function that would need a manual for a GUI to be feasible.

That last section sounds exactly like try-catch, so essentially you want to be able to trigger an error, which will then cascade up to your softtry-catch block. Either way this sounds like a useful feature.

[Walter Roberson]

a function that would need a manual for a GUI to be feasible

That's probably the same program being referred to ;-)

[Bruno Luong]

Just program an onCleanup to save whatever that needs to be saved. Ctrl-C do the rest.

What you propose is a dream but how to deal with variables/outputs not assigned and function interrupted at unknown state?

Only the programmer can know his workflow. So no possible to implement this lazy feature IMO.

[Walter Roberson]

onCleanup would have to either save() or store into global variables. The point is to be able to have a smooth return of values.

Outputs not assigned would still generate the same error messages as at present.

Function interrupted at unknown state is no worse of a problem than functions that error and are caught by try/catch at present.

It is common for programming systems to have a control-C handler .

[Sindar]

My solution in a similar case was to check whether a certain file ('FAIL_MODE.txt') exists in the directory at the end of each iteration. If it does, break the loop. Then, to stop the program, all I had to do was create the file (which is pretty trivial unless Matlab has frozen the whole computer)

Answer 5

[Michael Foote]

In the code analyzer, there is no easy way to find what #ok directive controls a particular flagged issue.

(And no overall list of the available #ok directives, at least that I can find. Even the enable/disable messages in preferences don't show them.)

[Rik]

Maybe I've only looked in the wrong places, but the changes from release to release about what is marked as a warning by mlint are poorly (or not) documented in the release notes.

So there are more things leaving room for improvement regarding this.

[Steven Lord]

Perhaps the codeIssues function introduced in release R2022b will be of use to you. The CheckID variable in the issues table stored in the codeIssues object is the identifier you'd use in the %#ok pragma to suppress Code Analyzer's reporting of that particular issue.

Answer 6

[Shae Morgan]

facet_wrap or facet_grid (or general ggplot2 functionality) version of subplots, or some altered, simpler customizability for subplotting subsets of data.

gramm is an available toolbox, but it'd be nice to have it built in

Answer 7

[Rafael S.T. Vieira]

I would love to have a command TeX2sym and sym2TeX. With it, we could bring formulas from LaTeX and run them in MATLAB and the other way around. It is a tedious work doing it by hand, and I could bet most MATLAB users do it or will do it eventually.

Another useful feature to add would be arbitrary-precision arithmetic...languages such as python and Java (with Bignum), allows unlimited precision when dealing with integers at least. Granted that It is slow, but I believe that the MATLAB team could do something better. And I honestly feel like MATLAB is missing a feature by not having it...even some competing software has it.

And finally I was considering buying the Text Analytics Toolbox, and it would be nice, if it could have a grammar/spell checker. Even if it is not as advanced or doesn't contain all words. With live scripts, we can write an interactive document, so it would be nice if MATLAB could correct our spelling (even if it required a ToolBox to do it).

[Rik]

Neat idea. With enough patience it shouldn't be too hard to write a basic tex2sym, but I don't have any good ideas for sym2tex that wouldn't require a lot of cleaning up.

I think the main problem with tex2sym would be interpreting the subscript and superscript. The main problem with sym2tex would probably be that a clear and concise mathematical description is not necessarily the same as compact Matlab code, even when working with symbolic variables.

[Walter Roberson]

sym2tex exists as latex()

tex2sym is pretty hard. Pretty mathematical notation is often ambiguous.

[Walter Roberson]

For example

• sin x+y -> %observed in the wild. People are likely to interpret as sin(x)+y . Not recommended latex style for trig functions
• \sin x+y -> %is it sin(x+y) or sin(x)+y? People are likely to interpret as sin(x)+y
• \sin{x+y} -> %sin(x+y) but people are likely to guess sin(x)+y without being sure
• \sin{x}+y -> %sin(x)+y but people are likely to guess sin(x)+y without being sure
• sin(x+y) -> %sin(x+y). Not recommended latex style for trig functions
• \sin(x+y) -> %sin(x+y). Better latex syntax for trig functions
• \sin\left( x+y \right) -> %sin(x+y) . Considered better latex style because it allows latex to match heights of () when internal content is varying sizes, but has too much space
• mx+b -> %m*x+b which people understand from long convention. But it could be a variable named mx that is being added to b . This is, however, the primary recommended latex style
• m.x+b -> %m*x+b but not recommended latex style. However, lower dot for multiplication tends to show up in papers that have matrix multiplication
• m*x+b -> %m*x+b but actively recommended against latex style
• {m}{x}+b -> %m*x+b which people understand from long convention, but visually it could be a variable named mx that is being added to b. Easier for mechanical interpretation but not typical latex style
• m{\cdot}x+b -> %m*x+b understandable both mechanically and human. One of the recommended latex styles
• m{\times}x+b -> -> %m*x+b understandable both mechanically and human, but less common in practice. Considered to be one of the valid latex styles if needed
• f{\textrm{\"{o}}}rl{\textrm{\aa}}t -> %a single word. Using {} inside an expression does not reliably indicate multiplication. And it was harder to get the special characters to work here than it should have been

Implied multiplication without any syntactic separation is very common in mathematics and latex.

The above shows just some of the ways that simple operations can be written in Latex, and tex2sym would have to try to understand them all.

There is also the issue that a lot of latex uses constructs that MATLAB does not support, especially \usepackage and amsmath mode.

[madhan ravi]

Wow , deep sir Walter!

[Rafael S.T. Vieira]

Thank you for your interest, Walter and Rik. I believe that It is best to teach ourselves to code in a particular way, which will allow MATLAB to convert it to sym, than to be doing the task by hand every time.

Operator precedence could take care some of these issues. If \sin x + y was the input, then left-to-right precedence could dictate the output to be sin(x) + y. And to obtain sin(x+y), we would have to code in LaTeX \sin{x+y} or \sin(x+y).

Implied multiplication is indeed very common in mathematics and latex. On the other hand, it is also almost ubiquitous that variables are written just as single letters (especially if we are writing implied multiplication).

Finally, a command like tex2sym does not need to contemplate every math package imho, just some set of commands and macros. MATLAB could even return its best guess, and let us do the remainder of the task. Of course, ideally, we would just copy the contents from environments like $$ and \[ ... \] and paste them into MATLAB for a tex2sym conversion.

Answer 8

[Walter Roberson]

I know I've said it before, but it's still missing and still important:

We need a way to gather all of the outputs of function call into a cell in the middle of an expression .

I know this may be tricky to implement internally. There are internal rules that are hard to work out, that have to do with how many outputs to request from functions. For example,

[A, B] = cellfun(@C, D, 'uniform', 0)

somehow passes knowledge of "two outputs" to C -- for example if you were to use @max then A would be a cell array of the maxima and B would be a cell array of the indices. The situation can get more complicated than this, and figuring out all the cases can make your head hurt. But we do know that any expression C(D(E)) that D(E) will be evaluated asking for exactly one output and that would be passed into C... but the knowledge of multiple outputs would be passed to C and yet not D.

The number of outputs to use is not inherently clear. If for example you call ode45 and ask to gather the outputs, are you asking for the common TOUT, YOUT case, or the full TOUT,YOUT,TE,YE,IE ? There are some cases where extra outputs can be expensive to calculate, so even though an operation that gathered "maximum" outputs might be useful, it is not always desireable, so ability to select the number would be useful.

Then there are issues with, for example, deal(), where you can have any number of outputs with just a single input:

[A,B,C,D] = deal(123)

would initialize A, B, C, and D all to 123. So if you ask to gather "all" of the outputs from deal(123), that number is not well defined.

Working these things out is not trivial -- but it is a really missing bit of the language.

There might be an opportunity for a syntax such as {}name -- e.g.,

arrayfun(@(X0) {}ode45(@f, tspan, X0), x0s)

meaning to gather all of the outputs of the call. At present, {} is not valid before a name.

[Bruno Luong]

Why nargout doesn't meet what you ask?

function varargout = foo(varargin)
varargout = cell(1,nargout);
[varargout{:}] = deal('dummy');
fprintf('request with %d outputs\n', nargout);

Test

>> [A]=cellfun(@foo, {1 2 3}, 'UniformOutput', false);
request with 1 outputs
request with 1 outputs
request with 1 outputs
>> [A,B]=cellfun(@foo, {1 2 3}, 'UniformOutput', false);
request with 2 outputs
request with 2 outputs
request with 2 outputs
>> 

[Walter Roberson]

Bruno,

Imagine that you want to implement

[temp{1:3}] = ndgrid(-1:.1:1);
C = cell2mat(cellfun(@(M) M(:), temp, 'uniform', 0);

That is, you want a single array in which each of the columns is one of the outputs of ndgrid(). And you want to do it as a single expression.

ndgrid() does not have a fixed number of outputs -- it is not like sin() with one fixed outputs, or max with two fixed outputs. nargout(@ndgrid) is -1 -- in other words the declaration is like

function varargout = ndgrid(varargin)

If you wanted to capture all of the outputs of max() then you could query ndgrid(@max) to get 2:

function outputs = gather_outputs(f, varargin)
  n = nargout(f);
  [outputs{1:n}] = f(varargin{:});
end

and you could

 gather_outputs(@max, rand(3,5))

and this would be fine for gathering the two outputs of max into a single cell array.

But if we try to

 gather_outputs(@deal, [])

then nargout(@deal) is -1 and if we said "Okay, take the absolute value of that -1" then we would be doing

 [outputs{1:1}] = deal([])

which would give you just {[]} as the output.

This shows that you cannot just look at nargout() of the function you are invoking, such as @max or @deal .

Can we just look at nargout of the overall expression to determine the number of outputs to use for deal? No,

 C = cell2mat(cellfun(@(M) M(:), gather_outputs(@ndgrid, -1:.1:1), 'uniform', 0));

would at best tell you that nargout is 1 (the C variable). You need something else to tell you the number of outputs you want to get -- something like

function outputs = gather_n_outputs(f, n, varargin)
  [outputs{1:n}] = f(varargin{:});
end

and then you could

 C = cell2mat(cellfun(@(M) M(:), gather_n_outputs(@ndgrid, 3, -1:.1:1), 'uniform', 0));
 

The request is to build this kind of facility in to MATLAB instead of having to write true functions like gather_n_outputs and have to pass function handles into them. Some syntax like

 C = cell2mat(cellfun(@(M) M(:), {3}ndgrid(-1:.1:1), 'uniform',0));
 

where the hypothethical new syntax {3} indicates that you are requesting 3 outputs and that you want them gathered in a cell array. A common alternate syntax would be {} to request all of the outputs, like

 C = cellfun(@(V) V.^2, {}max(rand(3,5)), 'uniform', 0)

which would hypothetically gather both (all) outputs of the max() call into a cell array that would then be available for further processing.

If you just used

  C = cellfun(@(V) V.^2, max(rand(3,5)), 'uniform', 0)

then the knowledge of the single output would be carried through into the cellfun, which would tell max() to only emit a single output, so you would not get the second output processed. And the first output would be numeric not cell, so you would need arrayfun instead of cellfun...)

Answer 9

[cui]

MATLAB

I very much hope that the official version will strengthen the readstruct function in the future! Lack of complete uniformity to support more format requirements.

How to effectively read ".yml" files as structures(and similar '.json','.xml',and so on)?

Answer 10

[Jim Svensson]

Fix the semantics of "clear".

"clear all" does not clear all, but "clear classes" does. Go figure. I want a way to clear classes and only classes.

[Jim Svensson]

Clear also cannot clear specific classes in different packages if they have the same name. I.e. pkg1.my_class, and pkg2.my_class. Matlab's package system is broken, but at least some things could be improved.

Answer 11

[Seth Wagenman]

Ability to convert Python items to MATLAB data types (other than numpy arrays) inside of MATLAB, rather than inside of Python using the admin-rights-required API: https://www.mathworks.com/help/matlab/matlab_external/install-the-matlab-engine-for-python.html?s_tid=srchtitle

[Steven Lord]

I'm not 100% sure if you had specific types in mind when you said "Python items" but as of release R2022a you can convert Python list and tuple types to MATLAB types.

[Steven Lord]

There's an entry in the Release Notes for release R2022b that discusses converting scalar logical and numeric Python types to MATLAB types.

Answer 12

[Sindar]

Ok, this is pretty minor, but:

For common functions that return values as the first output and indices as the second, it would be nice if there was a direct way of getting the indices, so I could do things like this:

x = [1 2 5 4 5];
y = 1:5;
y_xsorted = y(sortInd(x));
% or
y_xunique = y(unique(x,'ia'));

instead of needing to create temporary variables:

x = [1 2 5 4 3];
y = 1:5;
[~,idx] = sort(x);
y_xsorted = y(idx);
[~,idx] = unique(x);
y_xunique = y(idx);

I know I could make wrappers myself, but this seems like a case where a builtin function could potentially be noticeably optimized. (And, is more straightforward than rebuilding the whole output system)

Answer 13

[Tim]

The following changes to the internal volume renderer would make Matlab much more useful for volume visualization:

• True RGB volume visualization (vs. just scalar data + colormap)
• Independent, decoupled specification of alpha values and intensity values
• The ability to combine volume images with standard axes and axes objects like points, lines, patches, etc.

[Walter Roberson]

And add the ability to pass axes scales, similar to the image() XData property, and maybe even support irregularly spaced points

Answer 14

[Mario Malic]

When typing code, if user wants to reference a variable or a function, one could click tab and get a matching list of functions and variables. Would it be useful to split this functionality with modifier if user wants to reference one of the two? As an example Shift +Tab for variables and Tab for functions and variables (not to break the current functionality).

Answer 15

[Bruno Luong]

No big deal but I wish SIGN(X) function could return 1 for X=0 instead of 0. May be implementing an option to not break the compatibility.

I just rarely use SIGN because of this exception choice.

[Adam Danz]

In such cases I've used, sign(x+realmin)

Example,

sign((-2:2)+realmin)

ans = 1×5

-1 -1 1 1 1

●

[Walter Roberson]

Wanting sign to return 1 at 0 would seem to be for use as Heaviside function.

Heaviside is available in the Symbolic Toolbox as heaviside() -- though you would want to sympref('heavisideatorigin', 1).

For numeric work it can often be replaced by (x>=0)... with the usual warnings that if what is being multiplied is nan or inf then you get nan out... but that would happen for sign() as well.

[Adam Danz]

sign() returns the sign of -inf/+inf (-1/1) and returns NaN for NaN inputs.

[Walter Roberson]

It is common for people to write terms in the form

something + value * u[x-offset]              %unit step function

and common for people to implement that either as

something + value * heaviside(x-offset)     %OR
something + value .* (x>= offset)

and occasionally as

something + value .* (sign(x-offset)+1)/2

except wanting sign() to return 1 at 0 to achieve the same boundary condition.

And that is fine provided that value is finite when it is unselected, with non-finite being a problem because 0*inf -> nan, 0*-inf -> nan, 0*nan -> nan.

This is not a problem that can be cured by using the heaviside() or sign() functions or the >= operators... it needs something more like piecewise:

something + piecewise(x >= offset, value, 0)

It is not a problem in theoretical mathematics because Heaviside is more a distribution than a function (just like Dirac Delta is a distribution) and Limit of the distribution acts to remove the problem.

[Bruno Luong]

Why I prefer sign(0) = 1?

To me y = sign(x) should satisfy these two important properties

norm(y) = 1
x = y * abs(x)

This uniquely determines y for any x ~= 0 (real, complex or even quaternion). For x=0, any y=exp(1i*alpha) with alpha real meet both properties, so just select it as y=1 for arbitrary convention.

However the MATLAB SIGN() function does NOT satisfy the first property for x=0.

Adams's trick sign(x+realmin) is interesting, but it just shift the issue to x=-realmin.

[Stephen23]

Rather than getting it to return 1, why not simply (and correctly) return the sign of the zero?:

https://www.mathworks.com/matlabcentral/answers/661023-atan2-not-consisent

[Walter Roberson]

Personally, I do not believe that sign(0) = 1 is reasonable. sign() is signum which is defined as 0 for 0.

https://en.wikipedia.org/wiki/Sign_function

Making sign(0) = 1 would disturb a number of the mathematical properties.

[Stephen23]

"Making sign(0) = 1 would disturb a number of the mathematical properties."

It already disturbs properties of the binary floating point numbers.

Why do some "mathematical properties" have a higher importance than binary floating point properties? I have on occasion required the sign of a binary floating point number (including zeros): are my usecases unimportant?

Perhaps two functions would resolve this:

SIGN (mathematical, consistent with current function)

SIGNF (of floating point number stored in memory)

[Walter Roberson]

@Stephen

You might have needed access to the "sign bit", but that is different than the sign.

Getting to the sign bit of double precision numbers is not difficult:

A = [-100/3, 0, 42.1234]

A = 1×3

-33.3333 0 42.1234

bitget(typecast(A, 'uint64'), 64)

ans = 1×3

1 0 0

For single precision, extract bit 32.

I do not recall ever having encountered a programming language in which the "sign" operator returned the "sign bit" instead of the signum value.

FORTRAN's SIGN functions are "transfer of sign", and so avoid the issue of representation. However, SIGN(1,0) would be 1 not 0.

ALGOL 60's SIGN function was defined in terms of -1, 0, +1.

LISP's signum function was defined in terms of -1, 0, +1.

[Stephen23]

"You might have needed access to the "sign bit", but that is different than the sign."

No, I did not need to "access" the sign bit, I needed to the sign of the value, i.e. the information that those bits represent, but I certainly did not want not the bits themselves.

I did not confuse information with how information is encoded.

[Walter Roberson]

Could you remind me what output you are hoping for, for each of these values:

A = [nan, -inf, -1, -0, 0, 1, inf]

A = 1×7

NaN -Inf -1 0 0 1 Inf

sign(A)

ans = 1×7

NaN -1 -1 0 0 1 1

[Bruno Luong]

I would implement the sign like this

A = [nan, -inf, -1, -0, 0, 1, inf]

A = 1×7

NaN -Inf -1 0 0 1 Inf

normsign(A)

ans = 1×7

NaN -1 -1 -1 1 1 1

function s = normsign(A)
b = typecast(double(A),'uint8');
s = 1-2*(bitand(b(8:8:end),0x80)>0);
s(isnan(A)) = NaN; % arguably need
s = reshape(s,size(A));
end

[Rik]

For the record (because I couldn't immediatly see what the effect would be):

normsign([nan, -inf, -1, -0, 0, 1, inf])

ans = 1×7

-1 -1 -1 -1 1 1 1

[Bruno Luong]

I correct m code, the first value for NaN input should be NaN (arguably need)

[Star Strider]

Perhaps I’m missing something, however if 0 can never be either positive or negative, I don’t understand the rationale of giving it a sign here. Obviously the sign of -eps (or -realmin) is -1 and of eps (or realmin) is +1 however those are not actually 0.

.

[Bruno Luong]

There are a lot of math formula that can be wrtten down very compact and simplified implementation with those convention (normsign), such as recursive formula for derivativex of Zernike functions, etc...

I can't think I ever really need TMW sign() convention. The only nice property is actually it's a limits of a smooth series of functions.

[Bruno Luong]

Another way to look for a logic is that the normsign(z) is the formula

s = exp(1i*angle(z))

or

s = exp(1i*atan2(imag(z),real(z)))

z is real or complex.

[Stephen23]

"Perhaps I’m missing something, however if 0 can never be either positive or negative"

You might be missing the fact that IEEE 754 has signed zeros. Signed zeros explain why this happens:

x = -0;
1/x

ans = -Inf

Bruno Luong and I would like a function that gives the sign of that zero before we do a calculation with it.

Sometimes I work with zeros and infinities as values, so this is actually information that I have needed to use. Currently I have to perform an operation on the value to determine the sign, or do some bit-bashing and scale the results of that (much as Bruno Luong showed). The goal is that sign.

[Star Strider]

I’m not up on IEEE 754 standards obviously, so signed zeros are new.

Answer 16

[Camilo Andres Manrique Escobar]

One thing would be: defining classes in notebooks (Live Scripts in MATLAB case). That's a normal thing for years now in Python, for example.

Answer 17

[Jim Svensson]

Maintain dimensions when getting a field of a class.

If Data is a 3x2x4 array of objects/structs with a field "foo", then Data.foo should be an 3x2x4 array of the type of "foo". Not a 1x24 array, or is it the other way around.

[Image Analyst]

I agree. Makes intuitive sense to me and would be convenient for extracting foo.

[Rik]

It currently is neither: it's a comma separated list.

for i1=1:3,for i2=1:2,for i3=1:4,s(i1,i2,i3).Data=rand;end,end,end
s.Data

ans = 0.9787

ans = 0.5449

ans = 0.2691

ans = 0.9030

ans = 0.7619

ans = 0.5311

ans = 0.8777

ans = 0.5484

ans = 0.2540

ans = 0.7540

ans = 0.6646

ans = 0.9153

ans = 0.6058

ans = 0.2581

ans = 0.6459

ans = 0.4237

ans = 0.1189

ans = 0.1992

ans = 0.3687

ans = 0.4382

ans = 0.7179

ans = 0.2574

ans = 0.5783

ans = 0.1151

To do something with it you'll have to capture it with [] or {} or a function that allows arbitrary input.

[Jim Svensson]

Yes you are right, now I remember. Always have to do something like "foo = [Data.foo]".

Answer 18

[gwoo]

Being able to access the properties/methods/fields of an object/struct even after indexing into one would be a big deal.

Currently, if you have a struct A, with fields size and count, you could do A. then tab to show the options for the fields. But if you do A(1). you cannot now tab to show the options for the fields. Same with an object. So the discoverability is gone.

I want to be able to index into an object/struct and still have the available fields/properties/methods for that parent show up.

[Mario Malic]

+1 on this: Currently, if you have a struct A, with fields size and count, you could do A. then tab to show the options for the fields. But if you do A(1). you cannot now tab to show the options for the fields. Same with an object. So the discoverability is gone.

[Paul]

I just tried this on 2020b and hitting tab after A(1). showed the fields.

Answer 19

[Bruno Luong]

Probably it breaks compatibility, but it would make functions/operators svd, *, ', .' dealing with n-d arrays similar to pagesvd, pagemtimes, ... followed by appropriated reshape.

Extend the pagexxx with matrix left/right division, lu, qr, chol, eig, etc...

[Steven Lord]

You've listed a bunch of functions there. Out of curiosity, if you had to prioritize which additional functions would you like to see get the pagemtimes, pagetranspose, pagectranspose, and pagesvd treatment first? What would be your top say half a dozen?

[Bruno Luong]

Priority in term of usefullness IMO should be

1. pagetranspose, pagectranspose
2. pagemtimes
3. pagemrdivide (pagemldivide can just be a wrapparound), but as mrdivide is based on qr factorization, interm of dev I guess TMW woul need to make dave of pageqr first.
4. pagesvd, pageeig
5. pageqr, pagelu, pagechol
6. pageldl
7. pageexpm
8. pagehess, pageqz, pageschur
9. pageinv, pagetrace, pagedet

[Steven Lord]

The pagemldivide, pagemrdivide, and pageinv functions are new as of release R2022a. We also have several other page* functions (including all but one of the items in your top 4 levels, we don't have a pageeig function) as shown in the functions list.

So does your list still start with pageeig or have one of the items at a lower level jumped ahead of it in your prioritization?

[Bruno Luong]

Hi Steve, the priority remains in this order to me. Thanks

[Steven Lord]

We only added one page function in release R2022b. It's not on your list, but you might find pagenorm useful.

[Bruno Luong]

Thank you Steve, and also to the dedicated developpers.

[Adam]

I really find these page functions very usefull. Some usefull additions in my opinion would be:

Have the option to multiply more than two matrices with pagemtimes

I now always end up writing pagemtimes(A,pagemtimes(B,C)). It would be very usefull to be able to write pagemtimes(A,B,C) in the future for the readability of the code.

Paged version of decomposition

The decomposition is also a usefull tool and would be nice to have a page version of it available

[Bruno Luong]

Good idea

pagemtimes(A,B,C)

User should be able somehow to control the associativity, meaning equivalent to

pagemtimes(A,pagemtimes(B,C))

or

pagemtimes(pagemtimes(A,B),C)

[Adam]

I don't really understand why associativity control is important.

Is the result not the same for both cases?

Maybe I'm missing something.

[Stephen23]

@Adam: binary floating point addition and multiplication are not associative:

https://en.wikipedia.org/wiki/Associative_property#Nonassociativity_of_floating_point_calculation

This means the order matters.

[Adam]

Ah, that's the thing I was missing, thanks for clarifying that

[Bruno Luong]

@Adam Not only the floating point rounding can create different numerically result, but what matters even more is that the floating point counts when multiply matrices can be different.

Consider the product

A*B*x

with

• A: p x m,
• B: m x n,
• x: m x 1

with p = 100, n = 100, m = 1000,

It is much cheaper to perform

A*(B*x) than (A*B)*x.

And in general the less operations is carried out, the less round-off has effect on the final result.

Answer 20

[dpb]

read/writetable from/to Excel workbooks should be able to return/write the comment (now called something else, I forget what) field and the formula associated with the cell as well as the value. While one can write COM to do so, it then takes either using COM entirely (can be a lot of work) or have to use both high-level and COM on the same file; either of which options isn't ideal. Sure, one could forego Excel entirely since have MATLAB, but unfortunately can't always have what we want in that regards.

One can mung on the old xlsread/write routine and add stuff into it, but it's also kinda' klunky and while more convenient in some ways as compared to writing all COM totally from scratch isn't nearly as convenient as would be a higher-level interface with the other current toolset.

There's a hook to a function call in xlsread although I was never able to get it to work to do either of the above; because the Excel COM object isn't exposed to the function to be able to make direct access from that function.

Answer 21

[dpb]

Maybe there's a way I've not found, but I wish arrayfun and cellfun would have automagic argument expansion so one could pass other arguments to the anonymous function without having to replicate them manually to match the others. This would add greatly to the convenience in using either; the present working example happens to be building a set of target range expressions to stuff into a cell array that will be written to Excel although that really has nothing to do with the request/enhancement, just happens to be current time was frustrated that there's no way to pass constants to the anonymous functions to allow them to be generalized.

Example:

Building a variably-sized workbook where it is desirable that the sums over a section of the sheet be formulas rather than the current fixed constant value of the data as the sheet will subsequently be modified by hand; the tool is to build the original working pattern by compending various data sources and arranging for the end user...

It boils down to the point at which one has a set of row range indices and a set of columns over which to build the formula and insert into the cell array which is subsequently written to the workbook. That code looks something like

% anonymous function that builds Excel =SUM(r1:r2) expression for given row range, column
xlsSumRange=@(r1,r2,c)strcat('=SUM(',xlsAddr(r1,c),':',xlsAddr(r2,c),')');
% typical use
col=xlsCol2Col('G');        % another internal translation layer of local array position from Excel column
cOut(isTotal,col)=arrayfun(@(r1,r2,c)xlsSumRange(r1,r2,c),RS1,RS2,repmat('G',numel(RS)-1,1),'UniformOutput',0);
col=xlsCol2Col('K');        % another internal translation layer of local array position from Excel column
cOut(isTotal,col)=arrayfun(@(r1,r2,c)xlsSumRange(r1,r2,c),RS1,RS2,repmat('K',numel(RS)-1,1),'UniformOutput',0);

This works, but the expression "repmat('G',numel(RS)-1,1)" needed is really inconvenient and clutters up the code legibility greatly. I've had any number of similar case in the past where an anonymous function is useful shorthand but then to use more than once requires a workaround like the above.

The alternative is to redefine the anonymous function dynamically and embed the constant inside it for the given invocation.

To visualize, an example output for the above for one invocation looks like for the call with column 'G'

K>> arrayfun(@(r1,r2,c)xlsSumRange(r1,r2,c),RS1,RS2,repmat('K',numel(RS)-1,1),'UniformOutput',0);
ans =
  3×1 cell array
    {'=SUM($G$3:$G$17)'   }
    {'=SUM($G$22:$G$372)' }
    {'=SUM($G$377:$G$414)'}
K>> 

The above could be in a loop over the number of columns instead of using the explicit columns, but was part of still rearranging the output file structure at the time...and, haven't taken the time to clean it all up yet...

[Stephen23]

This just boils down to function parameterization, which is indeed simpler and more efficient:

c = 'G';
xlsSumRange = @(r1,r2)strcat('=SUM(',xlsAddr(r1,c),':',xlsAddr(r2,c),')');
% typical use
col = xlsCol2Col(c);
cOut(isTotal,col) = arrayfun(xlsSumRange,RS1,RS2, 'UniformOutput',0);

https://www.mathworks.com/help/matlab/math/parameterizing-functions.html

Replacing STRCAT with SPRINTF would also be more efficient:

xlsSumRange = @(r1,r2) sprintf('=SUM(%s:%s)',xlsAddr(r1,c),xlsAddr(r2,c));

[dpb]

Somehow I developed an aversion to redefining the function to parameterize; I guess on due consideration it would be the way to simplify with current syntax. Thanks for the prod, @Stephen23...but, I still think the suggestion would be a worthwhile extension to the [array/cell]fun syntax.

[Walter Roberson]

I wonder if string operations would be faster these days?

xlsSumRange = @(r1,r2)"=SUM(" + xlsAddr(r1,c) + ":" + xlsAddr(r2,c) + ")");

in which case you would not need 'UniformOutput', 0 directly because string arrays are a thing. (On the other hand you cannot store a string array into a scalar location.)

[dpb]

Possibly, Walter.

In the end I gave up the arrayfun syntax entirely in this application because there are M rows and N ranges per row over which to build a compound Excel range so the number of elements in the argument list isn't the same and the complexity just got to be too much.

So, I ended up reverting to a doubly-nested loop using the direct sprintf() form for each combination and then join'ed them in the end to pass to Excel COM instruction...sometimes we fall into the trap of trying to remove explicit loops too often.

But, I STILL think the idea has merit...there have been a number of times I'd really, really like to have been able to have done.

[Bruno Luong]

@dpb The whole point of TMW anymous function design is to be able to call it with only changing variables, and constant variable in the workspace is captured in the context without explicitly entered as input argument. This logic is tailored with arrayfun/cellfun, and other functions that requires function handles as input (fmincon and friends).

What you ask seems to be the contrary of this design logic., as @Stephen23 as pointed out.

Or if you want it properly you could cascade the anonymous functions as following

xlsAddr = @(r,c) sprintf('$%c$%d', c, r);
xlsSumRangeCol = @(c) @(r1,r2)strcat('=SUM(',xlsAddr(r1,c),':',xlsAddr(r2,c),')');
c = 'G'; 
xlsSumRange = xlsSumRangeCol(c);

RS1 = 1:3;
RS2 = RS1+10;

arrayfun(@(r1,r2)xlsSumRange(r1,r2),RS1,RS2,'UniformOutput',0)

ans = 1×3 cell array
    {'=SUM($G$1:$G$11)'}    {'=SUM($G$2:$G$12)'}    {'=SUM($G$3:$G$13)'}

Admitly the 'Uniform' argument does look ugly and for-loop is much more readable.

[Steven Lord]

You could avoid the need for 'UniformOutput' by using string arrays. I also changed xlsSumRangeCol to use string concatenation (with +) rather than strcat.

xlsAddr = @(r,c) sprintf("$%c$%d", c, r);
xlsSumRangeCol = @(c) @(r1,r2) "=SUM(" + xlsAddr(r1,c) + ":" + xlsAddr(r2,c) + ")";
c = 'G'; 
xlsSumRange = xlsSumRangeCol(c);

RS1 = 1:3;
RS2 = RS1+10;

arrayfun(xlsSumRange,RS1,RS2)

ans = 1×3 string array
    "=SUM($G$1:$G$11)"    "=SUM($G$2:$G$12)"    "=SUM($G$3:$G$13)"

Or you could vectorize xlsSumRangeCol to eliminate the need for arrayfun.

xlsSumRangeCol = @(c) @(r1,r2) "=SUM($" + c + "$" + r1 + ":$" + c + "$" + r2 + ")";
xlsSumRange = xlsSumRangeCol(c);
xlsSumRange(RS1, RS2)

ans = 1×3 string array
    "=SUM($G$1:$G$11)"    "=SUM($G$2:$G$12)"    "=SUM($G$3:$G$13)"

Or if you want the range to always be a certain number of elements long:

xlsSumRange1 = @(c, r1, n) "=SUM($" + c + "$" + r1 + ":$" + c + "$" + (r1+n) + ")";
xlsSumRange1(c, RS1, 10)

ans = 1×3 string array
    "=SUM($G$1:$G$11)"    "=SUM($G$2:$G$12)"    "=SUM($G$3:$G$13)"

[dpb]

Thanks @Stephen23, @Bruno Luong and @Steven Lord for the feedback and example coding...I'll snag the code snippets and stuff away with the present code in a comment block and come back and revisit when some of the time pressure lessens...

[Andrew Janke]

+1 on this, @dpb. This "implicit scalar expansion for (array|cell)fun args" is a use case I've had occasional cause to want, and haven't been able to figure out a good way to do, except for ditching arrayfun and Functional Programming style and inverting the logic/control-flow to just write a vectorized or "regular Matlab" version of the function I'm trying to do.

In other use cases, as long as I don't care about method-call overhead, I've been able to wrap the value to be expanded in an "infinite copies" array, like this SPAM thing here:

classdef SPAM
    % "spam" a value to every element of an infinite virtual array
    properties
        Value
    end
    methods
        function this = SPAM(x)
            this.Value = x;
        end
        function out = subsref(this, S) %#ok<INUSD>
            out = this.Value;
        end
    end
end

But that doesn't seem to work with cellfun/arrayfun, because it looks like they do an explicit "size(...)" test on their inputs before trying to index in to them.

>> x = magic(3);
>> z = arrayfun(@plus, x, SPAM(420))
Error using arrayfun
All of the input arguments must be of the same size and shape.
Previous inputs had size 3 in dimension 1. Input #3 has size 1 
>> spam = SPAM(420);
>> z = NaN(size(x)); for i = 1:numel(z); z(i) = x(i) + spam(i); end
>> z
z =
   428   421   426
   423   425   427
   424   429   422
>> 

Can't say I blame Matlab for this, though. I like explicit size-conformity checks; catches bugs more often than it prevents features.

Answer 22

[dpb]

I wish clearvars had an optional "do-nothing" flag a la the /L switch in CMD XCOPY to display the variable that would be cleared with the given variables list...would let one confirm a wildcard expression didn't accidentally wipe out something wished to have kept; particular with the -except clause.

[Andrew Janke]

Oooh, good idea. PowerShell does this for a bunch of their cmdlets like xcopy. I call this a "dry-run" or "what-if" run. I think PowerShell and recent Windows have standardized on the -WhatIf option for this.

Answer 23

[Adam]

When defining Abstract methods on a class, I would like to be able to fix the arguments on the abstract class.

Something like this would be usefull:

classdef AbstractSuperClass
    methods (Abstract)
        function AbstractMethod(self,input1,input2)
            arguments 
                self
                input1 (1,1) double
                input2 (1,1) double
            end
            % no function body because it is abstract
        end
    end
end

Right now, I always end writing two versions of the same method to avoid the need to copy the arguments validation block to the subclass:

classdef AbstractSuperClass
    methods
        function Method(self,input1,input2)
            arguments 
                self 
                input1 (1,1) double
                input2 (1,1) double
            end
            abstract_version_of_method(self,input1,input2)
        end
    end
    methods (Abstract,access=protected)
        abstract_version_of_method(self,input1,input2)
    end
end

but this feels wrong to do it and results in programming errors because I forget what the input signature of the method is, the above syntax would be more elegant and would allow for checking whether the input signature of the implementation of the method follows the definition in the abstract class.

With the new arguments (output), I would even be able to fix the output type of my abstract function

[Andrew Janke]

Oooh, tentative +1 on this. This is something I hadn't even thought of, and I'm not sure what all the implications of doing this in Matlab would be. But it's basically how traditional static OOP languages like Java and C++ work, and there it's obviously the right thing and we just take it for granted. There, the types etc of the input arguments are part of the interface or signature of those methods, and subclasses must conform to them, and that just makes things easy to reason about. (IMHO.)

Answer 24

[Mikhail]

Some (well, most) dynamic programming languages allow for an experienced developer to have an insight look into how their code is actually executed by the runtime environment.

They are able to see the parse tree and the compiled byte code.

They are able to see what parts of code are JIT-compiled, and how exactly the are compiled. When something is not compiled, they are able to see why.

The developer doesn't have to guess whether a particular optimization has kicked in or not. They know for sure how each and every object in their code is handled (whether it is CoW, passed by reference, passed by value).

I'd love to see these capabilities in MATLAB, too.

[Rik]

The problem with this idea is the long term portability of code. It is fairly easy to write code that will run on Matlab 6.5 (released in 2002) and R2020a, as you can see in my FEX submissions (although some of those functions are examples of when it is hard to so). If you want to optimize for the JIT engine of every release, you blow up your dev time. If your code runs 10% faster because you take more advantage of the JIT, but you spend weeks on that function, is it worth it?

The engineers at Mathworks are very aware of good coding practices taught to Matlab users, and tend to try to optimize for such code. There are absolutely more obscure parts of Matlab that allow speedups in unexpected places: cellfun(@isempty,data) versus cellfun('isempty',data) is an example of that.

As a last aside: diving deep into the inner workings of Matlab might be blocked by Mathworks for a completely different reason: if you can see how everything is put together it is possibly easier to reverse engineer, allowing competitors to leverage engineering time from Mathworks in their own products. I have no clue whether or not this is a consideration (since I don't have any insider knowledge, nor personally know anyone working at Mathworks), but it might be.

[Mikhail]

I'm not talking about optimizing for JIT. Having insight into what exactly JIT did, a developer can understand the reason why their code e.g. became slower in a newer release. Or why a particular construct is faster than another (without speculating with other MATLAB users as to why this is happening - which I seen here at FEX quite often).

You also seem to have missed my point about parse tree. This information can be usefull and can spawn a wide range of third-party MATLAB utilities - such as code analyzers, linters, profilers, dependency analyzers and so on.

But you are right that this is probably what MathWorks tries to avoid.

But having said that, there should be no doubt that any professional programmer will choose a decent programming language (such as Julia, R, Python or even Lua), which doesn't limit his (programmer's) knowledge and urge to dig as deep as he wants to dig.

If MathWorks wants to attract those professional programmers and not only position MATLAB as a "language for engineers and scientists", they will have to open up.

[Bruno Luong]

"If MathWorks wants to attract those professional programmers and not only position MATLAB as a "language for engineers and scientists", they will have to open up."

Unfortunately I think this won't happen. I have observed them to close down (and not open up) more and more over the years. Examples come to mind is: The mxArray internal structure was hidden, they removed access to undocumented functions, they advises user to code normally and should not care about understanding why some method is faster or slower, telling speed is their business.

Yair's undocument webpage has a huge success not without reason. And I can imagine it take a lot of effort to make it up-to-date.

What you request, and I'm 100% second, seems to hane negative correlation coefficient with TMW past/ current politics. They might have a reason to do it, mainly prevent plagiagism from competitors.

[Walter Roberson]

When trying to understand these kinds of decisions, I find that it helps to keep in mind a couple of factors:

1) The purpose of the Free Software Foundation projects such as Octave is not to provide high quality or long term supported or "affordable even in developing countries" software. The purpose of the FSF is explicitly political: to make it effectively impossible for anyone anywhere in the world to "own" (control) what is commonly known as "intellectual property". The FSF is explicitly intended to destroy the economy of companies such as Microsoft, and Apple, and Mathworks that is based on "owning" the rights to sell software licenses. The FSF believes that once an intellectual idea is produced and instantiated, that it belongs to the entire world and should be available to everyone (**with** source code) for no more than a service fee for the time/resources to copy it. To the FSF, it is not necessary that companies such as Microsoft and Mathworks cease to exist, but rather that such companies should earn their income by providing consulting services to people who want modified versions of the software, or want programs written (with the new software being released in source code form to the public) -- or through sidelines in mechandising physical objects such as Mathworks-branded t-shirts, or Mathworks-branded toys for McDonalds Happy Meals. Therefore Mathworks is not just competing in a marketplace of providing high quality software that users are willing to pay for: Mathworks is competing in a marketplace where one of the significant players is determined to force Mathworks to give away the many millions of dollars invested into intellectual property development, for political reasons. The FSF actively wants proprietary software firms to fail in anything recognizable as their current forms, and the FSF has no plans to mitigate problems that would cause in the world: the FSF just has the nebulous idea that if software is worth doing, then someone will be willing to hire programmers to develop the software (that would then have to be given away to everyone.) Thus, to companies such as Mathworks, the FSF is not just a "respected competitor", it is a malignant actor determined to destroy the right of Mathworks to exist at all (except as a consulting firm.)

2) Google vs Oracle America https://en.wikipedia.org/wiki/Google_v._Oracle_America is an active legal case in the USA. To summarize very briefly: Sun developed Java and encouraged its wide adoption as a portable write-once platform; companies did that, including copying the major java interface components but writing their own version of the internals; Oracle bought Sun; Oracle now claims ownership of the very interface structure for Java and is trying to get companies to pay big $$$ for licensing what Sun once encouraged everyone to use. Oracle sued Google for $US8.8 billion in copyright damages with respect to Android. Google won in lower court, but the decisions were reversed in a higher court, and the matter is now on the docket of the US Supreme Court.

The relevance of this lawsuit to Mathworks is the question of whether Mathworks "owns" the structure and library interfaces it established for MATLAB (and so could potentially force FSF Octave project to modify or shut down). Likewise, if Mathworks were to expose the Execution Engine level, would Mathworks effectively be giving away those decades of intellecutal work for anyone to develop for free, or would they retain enough control to be able to prevent their competitors and organizations such as FSF from using the work against Mathworks? I know that if I were in the position of a company such as Mathworks, that I would not be willing to release significant information about internal infrastructure until the question of ownership / control of APIs was legally settled.

[Rik]

(sorry for the wall of text)

I don't understand why people resort to calling the Matlab language bad (even if only by inference) if they disagree with the direction Mathworks has chosen. I don't think R is better as a language, or Python. They have their pros and cons. If I think a language has a con, why should I start calling it names? Do I have a right to say Python is stupid because you need to specify what exact version of what exact package you want to use, because otherwise everything will break or at least run the risk of not being able to reproduce the result for a paper? Of course not.

I don't think Matlab (as a language) is not mature. You may not like the direction that Mathworks is choosing to move the language, but that is something else.

It makes sense to hide implementation details on some level. Every time I post a function to the FEX I do exactly that. I tell people to use readfile(filename_or_url) and use that result, instead of telling them how to figure out if a file is UTF8 or ANSI and which exact conversion that requires for their specific OS and version of Matlab/Octave. If you want to know, read the m-file. I could have chosen to publish the function as a p-file, but I didn't. That was my call to make. The function is not suddenly horrible because you can't have a peek at the implementation and optimize your workflow accordingly. Why would the language as a whole be any different?

More on topic: I don't see why you would want to understand what the JIT is doing to your code, without wanting to optimize for that. What would be the use? I'm all for creating more understanding, but I don't see how this achieves more than what good coding practices would teach you.

I have no experience working with a parse tree, so I can't comment on that.

About that 'any professional programmer': that reeks of a true Schotsman fallacy. Apart from what I mentioned above, I don't see what more I could say. It sounds more like venting frustration than actually making a point.

@Bruno, it makes sense on some level to let speed be their business. It litterally is. GNU Octave has fewer debug tools, fewer implemented function, but most importantly for most people: it is a lot slower. If you write code following the guidelines everyone is being taught on this forum and in the documentation, Mathworks can focus engineering effort on only a single style of programming. conv used to be single-threaded, now I can saturate 12 threads with a convolution. The ismember function used to create memory issues for large amounts of text in a cell array (my use case: about 3-4 MB), now it works fast. The more you open up your internal mechanisms, the harder it is to make the fundamental changes required for orders of magnitude speedups.

None of this is to say that I am a big fan of every decision Mathworks makes. I just think there is enough valid criticism to level that you don't need to vent frustration under the guise of commentary. Just be honest with yourself and others.

[Walter Roberson]

The parse tree can be accessed through the undocumented mtree() function https://www.mathworks.com/matlabcentral/answers/180048-list-built-in-commands-used-by-m-function#answer_169083

[Mikhail]

> I tell people to use readfile(filename_or_url) and use that result, instead of telling them how to figure out if a file is UTF8 or ANSI and which exact conversion that requires for their specific OS and version of Matlab/Octave.

> ...

> The function is not suddenly horrible because you can't have a peek at the implementation and optimize your workflow accordingly. Why would the language as a whole be any different?

Given that I can't peek at the function's implementation, this function suddenly becomes horrible when I have a file double encoded to UTF-8 (and so your function returns garbage) and you considered your users too dumb to allow them to mend this function for their needs.

Same goes for the language.

[Rik]

That doesn't make the function horrible, it just means the function is not suited to your needs. I don't complain that my hypothetical electric stove can't cool things down, even though with a TEC it is possible to both heat and cool down things with a single electrically powered device.

Although you could re-use a lot of the internals of readfile to fix double UTF-8 encoded files, the problem is not the function, it's the double encoding. Fixing double encoding is a different task from reading a file correctly. It wouldn't be a matter of 'mending my function', it would be re-using internal functions for a different goal. The word 'mending' implies my function is broken, but returning a double encoded file in a corrected form is not living up to the contract that the name of the function is offering.

Also, there are more reasons to close down the source than just thinking your users are dumb (suggesting otherwise sounds a lot like assuming bad faith). As an example: if you want to provide a function for a license fee, it would be plain stupid to have your license check in an m-file, since it would be almost trivial to circumvent.

Answer 25

[dpb]

A ready-built insertion method for tables, arrays, etc., so don't have to do all the under-the-hood indexing manually...given an array of the right size and matching index, if had table (say), then

tMyTable=insert(tMyTable,indices,tNewData);

ideally, an option with keyword 'Empty', T|F would be available as well.

Maybe I'm missing some magic pixie dust, but I've been unable to figure out a way to this without physically moving a section at a time to insert the new row which means working from rear to front to avoid changing the indices to the insertion points or by catenating the pieces starting from the front.

[Walter Roberson]

Hmmm... what would the Empty option do?

[Walter Roberson]

You can do it without moving a section at a time by building a cumulative destination index that increases by the size of each gap at the appropriate places (though I need to think more about how to vectorize this step.) Then you assign the existing values into a new variable, destination indexed by the cumulative index. Then you can assign the new values into the gaps with a single assignment statement using a destination that is just the "holes".

Not sure exactly how to vectorize the index construction... but the data moving itself does not have to be done a section at a time.

[dpb]

"what would the Empty option do?"

Insert blank record(s). With table you have to construct a record of the right size and type for each column and with the matching variable name; would just be "syntactic sugar" to have the routine automagically handle that as well. (Probably, the keyword is enough, likely the only use would be with the value 'T')

Useful if you know where the new record(s) goes(go) but don't yet have the data for it(them) at hand but will be delivered on the next train to arrive at the station. Or, if only have part of the full record with the rest not arriving until the container ship from China gets to port. Insert the blanks then fill in the fields later.

Possibly there's an 'Incomplete' option that would be the composite of the two steps for the partial data in hand case?

Not that big a deal for an array of native type, agreed, but have run into the case enough times with tables for the thought should be a builtin -- at least for the table/timetable/timeseries(?) classes.

[dpb]

Yeah, that's undoubtedly a better way on the construction, Walter -- I hadn't really thought about the implementation too much other than going "dead ahead" to "just get 'er done!" for the immediate task at hand.

Taking a break, I thought I'd throw it out as an idea for an enhancement for discussion...with the sidelight maybe somebody did have a "trick" hadn't thought of.

[dpb]

Related to my previous re: insert() function for tables @ <What is Missing is insert() Function> would be enhancement to addvars to extend the indexing 'Before','After' index to be a vector of indices/existing variable names to all multiple variables to be added in one call at specified locations in the table.

As is, have to either add one-at-a-time where go or rearrange all afterwards. It may be a somewhat easier search to find the location want to go when adding by comparing to existing variable names before the new ones are added than after -- or simpler to put where want to begin with than construct the later indexing sequence.

[Steven Lord]

I missed this item before. For table and timetable arrays this function has existed since release R2018a. See the addvars function.

[dpb]

No, the feature/extension doesn't yet exist; the 'Before/After' position is one value, whether it is provided as index, variable name or logical vector. The logical vector, while a vector, can have only one true position.

I'm suggesting N new variables should be able to be added, each at some specified position -- use case/example is to add the column of means following each variable, for example. As is, they all have to go in one place and then get rearranged or added one-at-a-time.

Answer 26

[Walter Roberson]

A "select" clause for readtable() and kin.

For example one user only wanted to read the rows in which one particular variable had a particular value

https://www.mathworks.com/matlabcentral/answers/610491-how-to-select-specific-content-in-text-file-and-read-it-into-a-matrix

It could at some point be implemented in terms of a rowfun() type function that got passed the variables for a row and could make arbitrary decisions based upon the row contents.

However an earlier stage could potentially have pairs of variable names (or numbers) and a vector or cell array of values, in which the selection code did an ismember(). Such a facility could be further improved if there were a "sorted" option (so the code could figure out when to give up looking -- any one value should no longer be looked for if a larger value were encountered). Or even "grouped", which would not imply sorted as such but would imply that when you find a value that all the instances of that value that exist will be in adjacent rows and so as soon as you detect a change you can know to give up looking for that value.

Answer 27

[Bruno Luong]

Implement family of (numerical) data structure with O(1) inserttion, removal, that includes chain-list, binary tree, Fibonachi tree, red-black tree etc, etc, ... the performance must be focus point. Don't care if they are encapsulated or not in the OP, just don't make that reduce the performace.

Answer 28

[Bruno Luong]

Implement equivalent to C inline-function (or macro) so as calling this function on small data won't be penalized in speed with the over-head.

Answer 29