How do I plot a spectrogram of a real time plugin?

I managed to program a plugin that does a bit of reverb and a low pass filter. The plugin itself just shows me the spectrum of a signal during the time it is played but I need the entire spectrogram of that signal. Is there an option to plot the entire spectrogram with the audioTestBench function?

classdef FDNreverb2 < audioPlugin
    properties
        preDelayT = 0;              % pre delay [ms]
        reverbTime = 1.0;           % reverb time [s]
        roomSize = 5;
        mix = 70;                  % mix of wet and dry signal [Percent], 100 % -> only wet
        order = enumFDNorder.order8;                      % order of FDN, should be power of 2
        in_coeff = 1/2;         % coeff for in and output lines --- just for now one value for all
        out_coeff = 0.7;
    end

    properties (Access = private)
        N = 8;                  % order of FDN 
        a = 1.1;                    % multiplying factor for delays
        cSound = 343.2;             % speed of sound
        
        primeDelays = [2 3 5 7 11 13 17 19 23 29 31 37 41 43 47 53 59 61 ...
            67 71 73 79 83 89 97 101 103 107 109 113 127 131];

        % following values have to be set manually for Interface
        maxPreDelay = 0.2;          % max pre delay [s] (200ms)
        maxRoomSize = 20.0;         % max room size [m] 

        % following variables initialized in reset method
        
        fs;         % sample rate
        A;                  % feedback matrix
        b;                  % input gain coefficients
        c;                  % output gain coefficients
        f;                  % feedback lines after matrix
        v;                  % signal before delay - v_i(n) = b_i * x(n) + f_i(n)
        s;                  % output lines
        d;                  % signal to be sent to matrix A       
        buffDelayLines;     % buffer delay lines, initialized in reset method
        m;                  % delay in samples of each delay line
        g;                  % gain coefficients of delay lines
        maxDelay;           % maximum delay in delay lines
        buffInput;          % input buffer for pre delay
        pBuffDelayLines;    % pointer for delay lines buffer
        pBuffInput;         % pointer for input buffer 
        preDelayS;          % preDelay in samples
        alpha;
        v_prev2;
        v_prev1;
        v_filt_prev1;
        v_filt_prev2;
        
    end

    properties(Constant)
        PluginInterface = audioPluginInterface( ...
            audioPluginParameter('preDelayT', ...
            'DisplayName', 'Pre Delay [ms]', ...
            'Mapping', {'int', 0, 200}, ...
            'Style', 'rotary', ...
            'Layout', [1,1]), ...
            audioPluginParameter('roomSize', ...
            'DisplayName', 'Room Size [m]', ...
            'Mapping', {'lin', 1.0, 20.0}, ...
            'Style', 'rotary', ...
            'Layout', [1,2]), ...
            audioPluginParameter('reverbTime', ...
            'DisplayName', 'Reverb Time [s]', ...
            'Mapping', {'lin', 0.1, 5.0}, ...
            'Style', 'rotary', ...
            'Layout', [1,3]), ...
            audioPluginParameter('order', ...
            'DisplayName', 'Order of FDN', ...
            'Mapping', {'enum', '8', '16', '32'}, ...
            'Layout', [3,2]), ...   
            audioPluginParameter('in_coeff', ...
            'DisplayName', 'Input Gain', ...
            'Mapping', {'lin', 0.0, 1.0}, ...
            'Style', 'rotary', ...
            'Layout', [3,3]), ...
            audioPluginParameter('out_coeff', ...
            'DisplayName', 'Output Gain', ...
            'Mapping', {'lin', 0.0, 1.0}, ...
            'Style', 'rotary', ...
            'Layout', [3,4]), ...
            audioPluginParameter('mix', ...
            'DisplayName', 'Mix', ...
            'Mapping', {'int', 0, 100}, ...
            'Style', 'rotary', ...
            'Layout', [3,1]), ...
            audioPluginGridLayout( ...
            'RowHeight', [100 100 100 100 100 100], ...
            'ColumnWidth',[100, 100 100 100 100 100], ...
            'RowSpacing', 10, ...
            'ColumnSpacing', 10, ...
            'Padding', [10 10 10 10]) ...
            );
    end

    methods

        function plugin = FDNreverb
            init(plugin)
        end

        function out = process(plugin, in) 
            out = zeros(size(in));
            %Butterworth coefficients
            [bx, ax] = butter(2, 3000/(plugin.fs/2)); % 2. Ordnung, Grenzfrequenz 1000 Hz
            b0 = bx(1);
            b1 = bx(2);
            b2 = bx(3);
            a1 = ax(2);
            a2 = ax(3);
            % rawVn(1) = b0*plugin.v(1);
            % rawVn(2) = b0*plugin.v(2) + b1*plugin.v(1) - a1* rawVn(1);

            % Define the cutoff frequency and calculate alpha
            cutoffFreq = 1000; % Example cutoff frequency in Hz
            plugin.alpha = (2 * pi * cutoffFreq) / (plugin.fs + 2 * pi * cutoffFreq);
            
            % Initialize the previous output for the filter
            prevY = zeros(plugin.N, 1);
            
            for i = 1:size(in,1)
                % Summieren der L/R - Kan�le
                inL = in(i,1);
                inR = in(i,2);
                inSum = (inL + inR)/2;
                plugin.buffInput(plugin.pBuffInput + 1) = inSum;
                
                % loop over delay lines
                for n=1:plugin.N
                    % d_n = gain * delayed v_n
                    for k=1:plugin.N
                        plugin.d(k) = plugin.g(k) * plugin.buffDelayLines(k, mod(plugin.pBuffDelayLines + plugin.m(k), plugin.maxDelay +1) + 1);
                    end
                    % f_n = A(n,:) * d'
                    plugin.f(n) = plugin.A(n,:) * plugin.d(:);
                    % v_n with pre delay
                    rawVn = plugin.b(n) * plugin.buffInput(mod(plugin.pBuffInput + plugin.preDelayS, (plugin.maxPreDelay * plugin.fs + 1)) + 1) + plugin.f(n);
    
                    % Filter anwenden
                    plugin.v(n) = b0 * rawVn + b1 * plugin.v_prev1(n) + b2 * plugin.v_prev2(n) ...
                                - a1 * plugin.v_filt_prev1(n) - a2 * plugin.v_filt_prev2(n);
                    
                    % Update der vorherigen Filterwerte
                    plugin.v_prev2(n) = plugin.v_prev1(n);
                    plugin.v_prev1(n) = rawVn;
                    plugin.v_filt_prev2(n) = plugin.v_filt_prev1(n);
                    plugin.v_filt_prev1(n) = plugin.v(n);
                    
                    plugin.buffDelayLines(n, plugin.pBuffDelayLines + 1) = plugin.v(n);

                    % output lines
                    plugin.s(n) = plugin.c(n) * plugin.d(n);
                
                    out(i,:) = out(i,:) + real(plugin.s(n));
                    
                end

                % Assign to output
                out(i,1) = plugin.mix/100 * out(i,1) + (1.0 - plugin.mix/100) * in(i,1);
                out(i,2) = plugin.mix/100 * out(i,2) + (1.0 - plugin.mix/100) * in(i,2);

                calculatePointer(plugin);
            end 
        end

        function calculatePointer(plugin)
            if (plugin.pBuffDelayLines==0)
                plugin.pBuffDelayLines = plugin.maxDelay;
            else
                plugin.pBuffDelayLines = plugin.pBuffDelayLines - 1;
            end

            if (plugin.pBuffInput==0)
                plugin.pBuffInput = plugin.maxPreDelay * plugin.fs;
            else
                plugin.pBuffInput = plugin.pBuffInput - 1;
            end
        end

        function set.in_coeff(plugin, val)
            plugin.in_coeff = val;
            plugin.b = ones(plugin.N,1) * val;
        end

        function set.out_coeff(plugin, val)
            plugin.out_coeff = val;
            plugin.c = ones(plugin.N,1) * val;
        end

        function set.order(plugin, val)
            plugin.order = val;
            switch (plugin.order)
                case enumFDNorder.order8
                    plugin.N = 8;
                case enumFDNorder.order16
                    plugin.N = 16;
                case enumFDNorder.order32
                    plugin.N = 32;    
            end
            reset(plugin)
        end

        function set.reverbTime(plugin, val)
            plugin.reverbTime = val;
            calculateGainCoeffs(plugin, plugin.reverbTime)
            calculateDelays(plugin)
            %sprintf('Set Reverb Time: %f', plugin.reverbTime)
            %disp(['Set Reverb Time: ', num2str(plugin.reverbTime), ' s.']);
        end

        function set.preDelayT(plugin, val)
            plugin.preDelayT = val;
            calculatePreDelayS(plugin, plugin.preDelayT)
            %disp(['Set Predelay: ', int2str(plugin.preDelayT), ' ms.']);
        end

        function set.mix(plugin, val)
            plugin.mix = val;
            %disp(['Set Mix value: ', int2str(plugin.mix), ' %.']);
        end

        function calculateMatrix(plugin)
            plugin.A = generateFDNmatrix(plugin.N);
        end

        function calculateDelays(plugin)
            % calculate sample delays dependent on room size (val) and
            % sample rate (fs)
            % m_1 = trace of sound / cSound * fs
            
            M = ceil(0.15 * plugin.reverbTime * plugin.fs);
            %disp(['M = ', int2str(M)]);
            plugin.m = zeros(plugin.N,1);
            interval = M/(plugin.N*4);
            %test = 0;
            for i=1:plugin.N
                tmp = interval/2 + (i-1) * interval;
                e = floor(0.5 + log(tmp)/log(plugin.primeDelays(i)));
                plugin.m(i) = plugin.primeDelays(i)^(e);
            end
        end

        function calculateGainCoeffs(plugin, val)
            for i=1:plugin.N
                plugin.g(i) = 10^((-3) * (plugin.m(i)/plugin.fs) / val);
            end
        end

        function calculatePreDelayS(plugin, val)
            plugin.preDelayS = val * plugin.fs / 1000;
        end

        function init(plugin)
            plugin.fs = getSampleRate(plugin);

            % initialize buffDelayLines & pointer
            %plugin.maxDelay = floor(plugin.maxRoomSize/plugin.cSound * plugin.fs * plugin.a^(plugin.N)); %probably higher than actual max delay as actual delays get rounded down
            %plugin.maxDelay = ceil(0.15 * 5 * plugin.fs);   % maximum possible delay for max reverb time = 5s
            plugin.maxDelay = 131^2;
            plugin.buffDelayLines = zeros(plugin.N, plugin.maxDelay + 1);
            plugin.pBuffDelayLines = plugin.maxDelay;
            
            %initialize filter elements
            plugin.v_prev1 = zeros(plugin.N,1);
            plugin.v_prev2 = zeros(plugin.N,1);
            plugin.v_filt_prev2 = zeros(plugin.N,1);
            plugin.v_filt_prev1 = zeros(plugin.N,1);
            
            % initialize 
            plugin.A = zeros(plugin.N);
            plugin.b = ones(plugin.N,1) * plugin.in_coeff; % input gain coefficients
            plugin.c = ones(plugin.N,1) * plugin.out_coeff; % output gain coefficients
            plugin.f = zeros(plugin.N,1);             % feedback lines after matrix
            plugin.v = zeros(plugin.N,1);             % signal before delay - v_i(n) = b_i * x(n) + f_i(n)
            plugin.s = zeros(plugin.N,1);             % output lines
            plugin.d = zeros(plugin.N,1);             % signal to be sent fsto matrix A
            plugin.m = zeros(plugin.N,1);             % delay in samples of each delay line
            plugin.g = zeros(plugin.N,1);             % gain coefficients of delay lines
            plugin.buffInput = zeros((plugin.maxPreDelay * plugin.fs) + 1, 1);   % input buffer for pre delay, 0.2 = max pre delay of 200ms
            plugin.preDelayS = 0;                     % pre delay in samples
            plugin.pBuffInput = plugin.maxPreDelay * plugin.fs;    % pointer for input buffer

            % calculate sample delays of delay lines
            calculateDelays(plugin)

            % calculate gain coeffs of delay lines
            calculateGainCoeffs(plugin, plugin.reverbTime)
            
            % calculate FDN matrix based on set order
            calculateMatrix(plugin) 
        end

        function reset(plugin)
            init(plugin)
        end

        function generateFDNmatrix(order)
            % generates FDN matrix with evenly distributed eigenvalues at
            % the unit circle
            eig_nr = order/2;
            FDNmatrix = zeros(order, order);
            M = orth((rand(order,order)));
            BC_n = zeros(order, order, eig_nr);
            DtD_n = zeros(order, order, eig_nr);
            k1 = 1;
            for k=2:2:eig_nr*2
                x = M(:,k-1);
                y = M(:,k);
                x = x / sqrt(2);
                y = y / sqrt(2);
                B = x * x.';
                C = y * y.';
                D = x * y.';
                BC_n(1:order,1:order,k1) = B + C;
                DtD_n(1:order,1:order,k1) = D.' - D;
                w = pi * k / (eig_nr * 2 + 2);
                temp = 2 * BC_n(:,:,k1) * cos(w) + 2 * DtD_n(:,:,k1) * sin(w);
                FDNmatrix = FDNmatrix + temp;
                k1 = k1 + 1;
            end
        end
    end
end

Answer 1

[Ayush]

Hi @Muhsin Zerey,

I understand that you need to generate and plot the entire spectrogram of signal processed by the audio plugin. To achieve this, you need to capture the output of the plugin over time and then use functions like “spectrogram” in MATLAB to visualize it.

The “audioTestBench” function in MATLAB is primarily for testing and tuning audio plugins interactively, but it doesn’t provide a way to capture the entire output for spectrogram plotting.

Here’s a code snippet to illustrate the above approach:

% Load the audio signal 
[inputSignal, fs] = audioread('your_audio_file.wav'); % your_audio_file : sample audio file

% Creating the plugin object 
plugin = FDNreverb2; %FDNreverb2 is plugin class as in your code 

% Process the input signal 
outputSignal = plugin.process(inputSignal);

% Compute and plot the spectrogram 
windowLength = 1024; 
overlap = 512; 
nfft = 2048; 
spectrogram(outputSignal(:,1), windowLength, overlap, nfft, fs, 'yaxis'); 
title('Spectrogram of Processed Signal'); 
xlabel('Time (s)'); 
ylabel('Frequency (Hz)'); 

I've taken a sample audio file and following is the spectrogram I obtained:

You can read more about “audioread” function here:

https://www.mathworks.com/help/matlab/ref/audioread.html

You can read more about “spectrogram” here:

https://www.mathworks.com/help/signal/ref/spectrogram.html

Hope this helps!

Answer 2

[jibrahim]

audioTestBench has a spectrum analyzer which includes a spectrogram option.

Launch the spectrum analyzer from audioTstBench by clicking the Spectrum analyzer button in the toopstrip.

In the spectrum analyzer, turn on spectrogram mode by going under the scope menu and clicking spectrogram.

Now, when you execute audioTestBench, you will able to see the spectrogram of the output signal as the algorithm is running.