Increase efficiency of gif generation and/or plotting

I have a gif that I want to make but when I run my script, it takes upwards of 15 minutes to generate the gif file and uses a ton of memory on my computer. The data set I am using is farily large. I supressed the figure window to speed things up as well.

For context, I have tried removing everything from the loop and only replacing the 'Zdata' but that doesn't speed things up. Is there a way to preallocate frames or do something to speed up the code and make it use less memory? Is there a particular line of code that is slowing things down that I could change?

I also get this as an occasional error

% Warning: MATLAB previously crashed due to a low-level graphics error while
% using software OpenGL. For more information, see Resolving Low-Level Graphics
% Issues. 
% > In matlab.graphics.internal.initialize (line 15) 

I've included an example of how my code is set up below. Note that the loop only has four iteration but it still takes quite a long time. my matlab program also crashes when I do more than four loops. I want to do upwards of 100 interations.

Thanks!

%%%% example code

%clearing
clear
clc
close all

%generate a random data set
time = 1:1:100;

for i = 1:length(time)
z{i} = rand(3600,2020); %this is the same size as my dataset
end
x = 1:1:size(z{1},2);
y = 1:1:size(z{1},1);

f1 = figure('visible','off');
axis tight manual
filename = 'mygiftest.gif'

%% begin loop
for i = 1:4

    %%% plotting
    data = z{i};
    subplot(2,1,1)
        h1 = surf(x(1,1700:end),y',data(:,1700:end));
        hold on
        b1 = surf(x(1,1:1600),y',data(:,1:1600));
        xlabel('x')
        ylabel('y')
        zlabel('z')
        colorbar('visible','off') 
        set(h1,'LineStyle','none')
        set(b1,'LineStyle','none')
        view(3)
        p = get(gca,'position');
        set(gca,'FontName','Arial','FontSize',16,'position',[p(1) p(2) p(3) p(4)])

        subplot(2,1,2)
        h2= surf(x(1,1700:end),y',data(:,1700:end));
        hold on
        b2 = surf(x(1,1:1600),y',data(:,1:1600));
        xlabel('x')
        ylabel('y')
        zlabel('z')
        cc = colorbar('location','southoutside');
        set(cc,'position',[0.13 0.06 0.775 0.03])
        cc.Label
        cc.Label.String = ['z'];
        cc.Label.FontSize = 16;
        set(h2,'LineStyle','none')
        set(b2,'LineStyle','none')
        view(2)
        p = get(gca,'position');
        set(gca,'FontName','Arial','FontSize',16,'position',[p(1) p(2)+0.06 p(3) p(4)])
        set(gca,'FontName','Arial','FontSize',16)

        set(gcf,'position',[2 42 600 924],'visible','off')
        sgtitle(datestr(time(i)),'FontName','Arial','Fontsize',18)
        hold off
        
        drawnow
        
   
    %%% write gif
    frame = getframe(f1);
      im = frame2im(frame); 
      [imind,cm] = rgb2ind(im,256); 
      % Write to the GIF File 
      if i == 1 
          imwrite(imind,cm,filename,'gif', 'Loopcount',inf); 
      else 
          imwrite(imind,cm,filename,'gif','DelayTime',0.05,'WriteMode','append'); 
      end 
    clf(f1,'reset')
end

[DGM]

You're wrangling nearly 6GB of data in memory, and writing 600x900px frames to a GIF. I'm going to go out on a limb and suggest that writing the GIF has next to nothing to do with how slow it is.

You can try using the profiler or just tic/toc to try to see which parts are slowest. As it is, I don't even have enough system resources to run this. Running a reduced version, the most expensive lines are getframe() and drawnow(). To put it in perspective, getframe() takes 250x as long as the entire conversion and saving process that follows.

I'm not sure of a good way to streamline the rendering/capture process. Maybe someone else can chime in.

[Eileen Lukens]

Thanks for your advice on how to look through the code! It's good to know that getframe is an expensive line of code.

[Dave B]

To make the getframe/drawnow part faster you could consider whether you could reduce the resolution quite a bit, i.e. do you have more points rendered than you can actually see in the gif? if you took every other point of x,y,z...would the result look basically the same? Every 5th point?

For the rest, which sounds like it's not the slow part based on DGM's comment, I noticed that you're doing a lot of work in the loop that's really not changing with each iteration. Here's a refactored version where I pulled out the parts that aren't changing and just set the ZData in the loop, this will speed it up but it may be insiginificant wrt. the time spent drawing. It makes the code a little cleaner too (in my opinion at least), so I figured I'd include it even though it might not solve the core issue.

I also cleaned up a few odds and ends that seemed unnecessary but I might have missed something (so please double check!)

I did this for a greatly subsampled version, 10 frames at 10% of the pixels, just to make it quick to make sure it ran!

%generate a random data set
time = 1:10;
for i = 1:length(time)
    z{i} = rand(360,202); %this is the same size as my dataset
end
x = 1:size(z{1},2);
y = 1:size(z{1},1);

f1 = figure('visible','off');
filename = 'mygiftest.gif';
%% Prepare figure
clf
data=z{1};
%% First axes
subplot(2,1,1)
h1 = surf(x(1,170:end),y',data(:,170:end),'LineStyle','none');
hold on
b1 = surf(x(1,1:160),y',data(:,1:160),'LineStyle','none');
xlabel('x')
ylabel('y')
zlabel('z')
view(3)
p = get(gca,'position');
set(gca,'FontName','Arial','FontSize',16,'position',p)
%% Second Axes
ax2 =subplot(2,1,2);
h2 = surf(x(1,170:end),y',data(:,170:end),'LineStyle','none');
hold on
b2 = surf(x(1,1:160),y',data(:,1:160),'LineStyle','none');
xlabel('x')
ylabel('y')
zlabel('z')
view(2)
p = get(gca,'position');
set(gca,'FontName','Arial','FontSize',16,'position',[p(1) p(2)+0.06 p(3) p(4)])
%% colorbar
cc = colorbar('location','southoutside','position',[0.13 0.06 0.775 0.03]);
cc.Label.String = ['z'];
cc.Label.FontSize = 16;

%%
set(gcf,'position',[2 42 600 924],'visible','off')
sgtitle(datestr(time(i)),'FontName','Arial','Fontsize',18)

%% Save first frame
frame = getframe(f1);
[imind,cm] = rgb2ind(im,256);
imwrite(imind,cm,filename,'gif', 'Loopcount',inf);

%% iterate over remaining frames and change data
for i = 2:length(time)    
    data = z{i};
    set(h1,'ZData',data(:,170:end))
    set(b1,'ZData',data(:,1:160))
    
    set(h2,'ZData',data(:,170:end))
    set(b2,'ZData',data(:,1:160))
    % drawnow % I think you can probably drop this because getframe will call it
    
    frame = getframe(f1);
    [imind,cm] = rgb2ind(im,256);
    imwrite(imind,cm,filename,'gif','DelayTime',0.05,'WriteMode','append');
end

[DGM]

It may be worth noting that for me, I can render/capture/write one or two frames from the original script in about 40s per frame -- but only if I only generate the amount of data I intend to render. All of those timings go out the window once I try to preallocate 6GB of data in memory. That pushes me deep into swap and I doubt I'd be able to write a single frame without something crashing. It might be worth considering how much memory load your data poses and whether it's making operations slower than they should be. It might be possible to load the data as needed for processing if you're using literal dumpster hardware like I am.

[Eileen Lukens]

Thanks for all the suggestions! I will give them a try. I may be able to display fewer points and still get the same general effect. I appreciate the effort both of you put in to help me out!

[Jan]

I tried it with a similar approach as Dave B. The time for the first frame takes 18 sec on my i5 mobile, Matlab R2018b, and 12 sec for the following frames.

I've tried to set some of the drawing modes to 'manual', to avoid letting Matlab check the dimensions in each iteration:

ax1 = subplot(2,1,1, 'NextPlot', 'add');  % Includes the "hold on"
...
ax2 = subplot(2,1,2, 'NextPlot', 'add');
...
set(ax1, 'XLimMode', 'manual', 'YLimMode', 'manual', 'ZLimMode', 'manual', ...
         'XTickMode', 'manual', 'YTickMode', 'manual', 'ZTickMode', 'manual');
set(ax2, 'XLimMode', 'manual', 'YLimMode', 'manual', 'ZLimMode', 'manual', ...
         'XTickMode', 'manual', 'YTickMode', 'manual', 'ZTickMode', 'manual');

This has no large effects.