Azzera filtri
Azzera filtri

Rounding using 'fix'... possibly a bug?

3 visualizzazioni (ultimi 30 giorni)
Brad
Brad il 14 Ott 2013
Commentato: Brad il 15 Ott 2013
Could someone tell me if this is a bug, or is there some logical reason 'fix' would round:
.3/.1 = 2
whereas
.4/.1 = 4
K>> fix(.3/.1)
ans =
2
K>> fix(.4/.1)
ans =
4
K>> fix(.7/.1)
ans =
6
K>> fix(.5/.1)
ans =
5
  1 Commento
Jan
Jan il 14 Ott 2013
It is such surprising that the well known floating point effects appear for beginners usually for the value 0.3 . Usually it is:
find((0:0.1:1) == 0.3) % replies [] !!!

Accedi per commentare.

Risposta accettata

James Tursa
James Tursa il 14 Ott 2013
To see what is going on you can examine the exact floating point values. Note that none of your original numbers except 0.5 can be represented in IEEE double format exactly:
>> num2strexact(.7)
ans =
0.6999999999999999555910790149937383830547332763671875
>> num2strexact(.5)
ans =
0.5
>> num2strexact(.4)
ans =
0.40000000000000002220446049250313080847263336181640625
>> num2strexact(.3)
ans =
0.299999999999999988897769753748434595763683319091796875
>> num2strexact(.1)
ans =
0.1000000000000000055511151231257827021181583404541015625
>> num2strexact(.7/.1)
ans =
6.99999999999999911182158029987476766109466552734375
>> num2strexact(.5/.1)
ans =
5
>> num2strexact(.4/.1)
ans =
4
>> num2strexact(.3/.1)
ans =
2.999999999999999555910790149937383830547332763671875
You can find num2strexact here:

Più risposte (1)

Image Analyst
Image Analyst il 14 Ott 2013
Looks like another case explained by the FAQ.

Categorie

Scopri di più su Coordinate Transformations and Trajectories in Help Center e File Exchange

Prodotti

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by