R2024b cannot reproduce documented example result

When I am ready to use imufilter, "Tune imufilter to Optimize Orientation Estimate",the current latest version R2024b cannot reproduce the documented example results? Please fix it in time!

TEST in MATLAB online and desktop MATLAB

ld = load('imufilterTuneData.mat');
qTrue = ld.groundTruth.Orientation; % true orientation

fuse = imufilter;
qEstUntuned = fuse(ld.sensorData.Accelerometer, ...
    ld.sensorData.Gyroscope);

% reset(fuse);
cfg = tunerconfig('imufilter');
tune(fuse, ld.sensorData, ld.groundTruth, cfg)

qEstTuned = fuse(ld.sensorData.Accelerometer, ...
    ld.sensorData.Gyroscope);

dUntuned = rad2deg(dist(qEstUntuned, qTrue));
dTuned = rad2deg(dist(qEstTuned, qTrue));
rmsUntuned = sqrt(mean(dUntuned.^2))
rmsTuned = sqrt(mean(dTuned.^2))

N = numel(dUntuned);
t = (0:N-1)./ fuse.SampleRate;
plot(t, dUntuned, 'r', t, dTuned, 'b');
legend('Untuned', 'Tuned');
title('imufilter - Tuned vs Untuned Error')
xlabel('Time (s)');
ylabel('Orientation Error (degrees)');

[Aquatris]

The problem seems to be the default objective limit is too high for this example. Set it to a low value and it works fine. Basically this means the initial imufilter parameters in the newer Matlab versions are a lot better then the previous implementations :P

ld = load('imufilterTuneData.mat');
qTrue = ld.groundTruth.Orientation; % true orientation

fuse = imufilter;
qEstUntuned = fuse(ld.sensorData.Accelerometer, ...
    ld.sensorData.Gyroscope);

reset(fuse);
cfg = tunerconfig('imufilter');
cfg.ObjectiveLimit = 0.01; % HERE set objective limit so that it can do more iterations
tune(fuse, ld.sensorData, ld.groundTruth, cfg)

    Iteration    Parameter                        Metric
    _________    _________                        ______
    1            AccelerometerNoise               0.0857
    1            GyroscopeNoise                   0.0855
    1            GyroscopeDriftNoise              0.0855
    1            LinearAccelerationNoise          0.0851
    1            LinearAccelerationDecayFactor    0.0844
    2            AccelerometerNoise               0.0844
    2            GyroscopeNoise                   0.0842
    2            GyroscopeDriftNoise              0.0842
    2            LinearAccelerationNoise          0.0840
    2            LinearAccelerationDecayFactor    0.0836
    3            AccelerometerNoise               0.0836
    3            GyroscopeNoise                   0.0834
    3            GyroscopeDriftNoise              0.0834
    3            LinearAccelerationNoise          0.0834
    3            LinearAccelerationDecayFactor    0.0831
    4            AccelerometerNoise               0.0831
    4            GyroscopeNoise                   0.0829
    4            GyroscopeDriftNoise              0.0829
    4            LinearAccelerationNoise          0.0829
    4            LinearAccelerationDecayFactor    0.0827
    5            AccelerometerNoise               0.0827
    5            GyroscopeNoise                   0.0824
    5            GyroscopeDriftNoise              0.0824
    5            LinearAccelerationNoise          0.0824
    5            LinearAccelerationDecayFactor    0.0822
    6            AccelerometerNoise               0.0822
    6            GyroscopeNoise                   0.0819
    6            GyroscopeDriftNoise              0.0819
    6            LinearAccelerationNoise          0.0819
    6            LinearAccelerationDecayFactor    0.0818
    7            AccelerometerNoise               0.0818
    7            GyroscopeNoise                   0.0814
    7            GyroscopeDriftNoise              0.0814
    7            LinearAccelerationNoise          0.0814
    7            LinearAccelerationDecayFactor    0.0813
    8            AccelerometerNoise               0.0813
    8            GyroscopeNoise                   0.0808
    8            GyroscopeDriftNoise              0.0808
    8            LinearAccelerationNoise          0.0808
    8            LinearAccelerationDecayFactor    0.0807
    9            AccelerometerNoise               0.0807
    9            GyroscopeNoise                   0.0802
    9            GyroscopeDriftNoise              0.0802
    9            LinearAccelerationNoise          0.0802
    9            LinearAccelerationDecayFactor    0.0801
    10           AccelerometerNoise               0.0801
    10           GyroscopeNoise                   0.0794
    10           GyroscopeDriftNoise              0.0794
    10           LinearAccelerationNoise          0.0794
    10           LinearAccelerationDecayFactor    0.0794
    11           AccelerometerNoise               0.0794
    11           GyroscopeNoise                   0.0785
    11           GyroscopeDriftNoise              0.0785
    11           LinearAccelerationNoise          0.0785
    11           LinearAccelerationDecayFactor    0.0785
    12           AccelerometerNoise               0.0785
    12           GyroscopeNoise                   0.0775
    12           GyroscopeDriftNoise              0.0775
    12           LinearAccelerationNoise          0.0775
    12           LinearAccelerationDecayFactor    0.0774
    13           AccelerometerNoise               0.0774
    13           GyroscopeNoise                   0.0762
    13           GyroscopeDriftNoise              0.0762
    13           LinearAccelerationNoise          0.0762
    13           LinearAccelerationDecayFactor    0.0761
    14           AccelerometerNoise               0.0761
    14           GyroscopeNoise                   0.0746
    14           GyroscopeDriftNoise              0.0746
    14           LinearAccelerationNoise          0.0746
    14           LinearAccelerationDecayFactor    0.0745
    15           AccelerometerNoise               0.0745
    15           GyroscopeNoise                   0.0727
    15           GyroscopeDriftNoise              0.0727
    15           LinearAccelerationNoise          0.0727
    15           LinearAccelerationDecayFactor    0.0726
    16           AccelerometerNoise               0.0726
    16           GyroscopeNoise                   0.0706
    16           GyroscopeDriftNoise              0.0706
    16           LinearAccelerationNoise          0.0705
    16           LinearAccelerationDecayFactor    0.0705
    17           AccelerometerNoise               0.0705
    17           GyroscopeNoise                   0.0684
    17           GyroscopeDriftNoise              0.0684
    17           LinearAccelerationNoise          0.0683
    17           LinearAccelerationDecayFactor    0.0683
    18           AccelerometerNoise               0.0683
    18           GyroscopeNoise                   0.0662
    18           GyroscopeDriftNoise              0.0662
    18           LinearAccelerationNoise          0.0662
    18           LinearAccelerationDecayFactor    0.0662
    19           AccelerometerNoise               0.0662
    19           GyroscopeNoise                   0.0644
    19           GyroscopeDriftNoise              0.0644
    19           LinearAccelerationNoise          0.0644
    19           LinearAccelerationDecayFactor    0.0644
    20           AccelerometerNoise               0.0644
    20           GyroscopeNoise                   0.0630
    20           GyroscopeDriftNoise              0.0630
    20           LinearAccelerationNoise          0.0630
    20           LinearAccelerationDecayFactor    0.0630

qEstTuned = fuse(ld.sensorData.Accelerometer, ...
    ld.sensorData.Gyroscope);

dUntuned = rad2deg(dist(qEstUntuned, qTrue));
dTuned = rad2deg(dist(qEstTuned, qTrue));
rmsUntuned = sqrt(mean(dUntuned.^2))

rmsUntuned = 4.9108

rmsTuned = sqrt(mean(dTuned.^2))

rmsTuned = 3.6116

N = numel(dUntuned);
t = (0:N-1)./ fuse.SampleRate;
plot(t, dUntuned, 'r', t, dTuned, 'b');
legend('Untuned', 'Tuned');
title('imufilter - Tuned vs Untuned Error')
xlabel('Time (s)');
ylabel('Orientation Error (degrees)');

[cui,xingxing]

@AquatrisThank you for your expermient! The default value for ObjectiveLimit in R2024b is 0.1. Have you tried the default value in R2024a and earlier versions?

I couldn't find any indications of changes in the version history.

[Aquatris]

I only ran the code here, which uses the latest version of matlab, 2024b. From my experience, it is hard to get the default parameters used in a function in version history.

If you have access to previous versions, check what the default parameters imufilter is using via running the below command in previous versions. That might give an idea on whether the initial parameters are changed or the implementation is changed.

a = imufilter

a = 
  imufilter with properties:

                       SampleRate: 100             Hz      
                 DecimationFactor: 1                       
               AccelerometerNoise: 0.00019247      (m/s²)² 
                   GyroscopeNoise: 9.1385e-05      (rad/s)²
              GyroscopeDriftNoise: 3.0462e-13      (rad/s)²
          LinearAccelerationNoise: 0.0096236       (m/s²)² 
    LinearAccelerationDecayFactor: 0.5                     
              InitialProcessNoise: [9⨯9 double]            
                OrientationFormat: 'quaternion'            

a.InitialProcessNoise

ans = 9×9

0.0000 0 0 0 0 0 0 0 0 0 0.0000 0 0 0 0 0 0 0 0 0 0.0000 0 0 0 0 0 0 0 0 0 0.0001 0 0 0 0 0 0 0 0 0 0.0001 0 0 0 0 0 0 0 0 0 0.0001 0 0 0 0 0 0 0 0 0 0.0096 0 0 0 0 0 0 0 0 0 0.0096 0 0 0 0 0 0 0 0 0 0.0096

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

[Cris LaPierre]

I was able to run the example in R2021b. The default ObjectiveLimit there is 0.1

[cui,xingxing]

Hi,@all, I tested the same scenario in R2023b and R2024b, and the output parameters of imufilter are consistent. Therefore, I suspect that the changes started from R2022a (according to @Cris LaPierre, R2021b worked correctly, with the ObjectiveLimit set to the default value of 0.1). Did some other internal implementation change?

Since I only have R2023b and R2024a versions on hand, it would be great if someone could help me investigate further. Thank you in advance!

[Paul]

I'm curious about the code in the doc page example (repeated above in the question).

Why does the code not reset the imufilter immediately prior to and immediatey after the call to tune?

Does tune() reset the filter on entry and on exit?

[cui,xingxing]

“Does tune() reset the filter on entry and on exit?”

NO，it doesn't reset filter, i add reset code above,but it can't make sense,you can see source code here:<MATLABROOT>\toolbox\shared\positioning\positioning\+fusion\+internal\+tuner\FilterTuner.m, LINE190,function varargout = tune(obj, varargin)

[cui,xingxing]

Another issue I'd like to complain about is that whenever I ask this question, I can never seem to display the output results below the code. I just can't find the right adjustment button.

[Paul]

If tune doesn't reset the filter on input and again on output, then I'd question the code snippet in the example. Seems like that would cause the tuning to be corrupted by the state of the filter on the call to tune, and subsquent use of the filter will be corrupted by its state after tune completes. That doesn't sound like a good system, and it should be documented in bold that that's how the function works.

As for displaying the output on Answers, what happens after you enter code (using cod formatting as you've done) and the click the green triangle in the Run section of the ribbon?

Answer 1

[Brian Fanous]

A bug in the imufilter and ahrsfilter that had a minor affect on the output was fixed in R2021b. For the setup in the code above the filter reached the ObjectiveLimit faster once the bug was addressed. This doc page was not updated after the fix. I have let our documentation group know.

Set the ObjectiveLimit to 0.03 and you'll see the filter tune itself for several iterations.

I would suggest always resetting the filter prior to tuning.

[cui,xingxing]

@Brian Fanous Thanks for the clarification, clear and accurate documentation is important!