MATLAB's JIT Engine

Do you think MATLAB's JIT engine is oudated? It does look so.

Look at Julia Language: http://julialang.org/

What do you think?

I hope Mathworks will surprise us in the next version.

Answer 1

[Jan]

I find this code for the comparison between Julia and Matlab:

function n = parseintperf(t)
for i = 1:t
   n = randi([0,2^32-1], 1, 'uint32');
   s = dec2hex(n);
   m = hex2dec(s);
   assert(m == n);
end

This takes in the provided test a time of 179.23 milliseconds.

It would be fair to omit the assert() and use more efficient conversions:

function n = parseintperf(t)
for i = 1:t
   n = randi([0,2^32-1], 1, 'uint32');
   s = sprintf('%8X', n);
   m = sscanf(s, '%X');
end

44.18 milliseconds (Matlab 2009a/64, Win7).

I did not find similar mistakes in the other tests.

Answer 2

[per isakson]

The following comparison might not be interesting from a CS point of view, but it is to a typical Matlab user.

Sure, the performance of Julia is impressive and The Mathworks might have some catch-up to do. However, backward compatibility; they must not break our code.

With Matlab vectorization is a powerful measure to achieve speed. Julia doesn't support vectorization.

I did the following test on R2012a, 64bit, Windows7.

    >> Julia_test_parseintperf
    matlab, parse_int, 2189.9
    matlab, parse_int_vec, 18.1

where the result is in milliseconds and

    function Julia_test_parseintperf
        julia_timeit( 'parse_int'    , @parseintperf    , 1e4 )
        julia_timeit( 'parse_int_vec', @parseintperf_vec, 1e4 )
    end
    function n = parseintperf(t)
        for ii = 1:t
            n = randi([0,2^32-1],1,'uint32');
            s = dec2hex(n);
            m = hex2dec(s);
            assert(m == n);
        end
    end
    function n = parseintperf_vec( t )
        n = randi( [0,2^32-1], t, 1, 'uint32' );
        s = dec2hex( n );
        m = hex2dec( s );
        assert( all( m == n ) );
    end

@Jan, I run this test with and without assert and the profiler and found that assert does not add much to the time. That function is improved - it seems. [See below.]

I had to rename the the function, timeit, used in their benchmark.

.

--- In response to comment 1 ---

I have added two functions proposed by Jan and run the test with the assert included

    >> Julia_test_parseintperf
    matlab, parse_int, 1698.9
    matlab, parse_int_vec, 18.0
    matlab, parse_int_jan, 394.6
    matlab, parse_int_jan_vec, 12.0

and the result from a second run with the assert commented out

    >> Julia_test_parseintperf
    matlab, parse_int, 1666.8
    matlab, parse_int_vec, 17.9
    matlab, parse_int_jan, 359.6
    matlab, parse_int_jan_vec, 12.0

where I have added these two functions proposed by Jan

function n = parseintperf_jan(t)
    for ii = 1:t
        n = randi([0,2^32-1], 1, 'uint32');
        s = sprintf('%8X', n);
        m = sscanf(s, '%8X');
        % assert(m == n);
    end
end
function n = parseintperf_jan_vec(t)
    n = randi([0,2^32-1], t, 1, 'uint32');
    s = sprintf('%8X ', n);
    m = sscanf(s, '%8X');
    % assert( all(m == n));
end

@Jan, my comment above regarding assert might have been wishful thinking. I was looking for order of magnitudes and was happy to have the asserts.

Some months ago I listen to presentation of Julia on Youtube (Stanford?). One thing I remember is that their understanding of "JIT-theory" influenced he design of the Julia language. With Matlab is more of an afterthought. However, lets hope Julia will influence the "Matlab accelerator" to improve. (You just indicated a way to improve the speed of dec2hex/hex2dec by a third.)

Yes, the JavaScript benchmark results are remarkable. JavaScript goes back to Netscape? I just saw this:

    Numeric Javascript is a library for numerical computations in
    Javascript. You can write Javascript programs that manipulate 
    matrices, solve linear problems, find eigenvalues and solve 
    optimization problems in the browser using Numeric Workshop or 
    in your own web pages by downloading numeric.js. You can also 
    use Numeric Javascript outside the browser in any Javascript
    environment.

[Jan]

Under 2011b the assert() increases the runtime by 10% in my loop code with SPRINTF/SSCANF.

The vectorization increases the efficiency of HEX2DEC. Still SPRINTF and SSCANF are faster: 13.7 ms instead of 21.4 ms (R2011b/64, Win7):

s = sprintf('%8X ', n);  % Format needs the trailing space
m = sscanf(s, '%8X');

Anyway, I agree with you, Per: For speed comparisons, which reflect the real world usage of Matlab, vectorization is important. Omitting it means a strong bias. It seems like Julia's JIT is faster than Matlab's. When I see what V8 squeezes out of JavaScript code, I guess, that even Matlab's JIT has still a great potential without breaking the backward compatibility.

[per isakson]

Jan, see my response in the answer above.

[Royi Avital]

I think we need to ask for more from Mathworks. MATLAB should and must be faster.

[Royi Avital]

What do you mean Julia doen't support vectorization?

[amin ya]

Julia natively supports vectorization and SIMD. SIMD is much faster than parfor which slows down the code instead of speeding it up

Answer 3

[Daniel Shub]

I am not sure it is possible to separate the MATLAB JIT accelerator from the rest of MATLAB. MATLAB is fast enough for most of what I do. In the rare cases I need something faster I tend to write C code. I do think the performance of MATLAB could be improved with a major overhaul similar to the move from Python 2 to 3 in which concerns for backwards compatibility were ignored. But just as I still use Python 2, I would stick with the older MATLAB.

Answer 4

[Jan]

When I ask my favorite search engine for "Matlab compile auto parallelize" I get a bunch of links about tools, which compile Matlab code with an automatic parallelization. Some of them are quite comfortable and produce efficient programs. But the drawback is that there can be two different sources of problems in case of bugs: The M-code and the compiler, and interferences of both.

[Royi Avital]

Could you elaborate on that?

Answer 5

[Royi Avital]

Another look on the subject:

http://justindomke.wordpress.com/2012/09/17/julia-matlab-and-c/

And they got much faster.

I really hope Mathworks will do something.

IF MATLAB one day will be as fast as C, or at least close to it as close as Julia gets, it might become an application development language.