POSIX and float rounding

[Denis Ovsienko <denis () ovsienko info>]

Hello all.

QNX SDP 8.0.3 QSTI build 245 (2026-03-09) includes Perl (again!), so I
ran tcpdump tests on it and one test failed:

Failed tests:
    olsr-oobr-1                             : diff exited with 1
-         vtime 0.062s, msg-seq 0x0008, length 127 [|olsr]
+         vtime 0.063s, msg-seq 0x0008, length 127 [|olsr]
-         vtime 0.062s, msg-seq 0x0008, length 100 [|olsr]
+         vtime 0.063s, msg-seq 0x0008, length 100 [|olsr]
-         vtime 0.062s, msg-seq 0x5c50, length 185 [|olsr]
+         vtime 0.063s, msg-seq 0x5c50, length 185 [|olsr]

The "0.063" is the result of a "%.3f" that takes ME_TO_DOUBLE(), which
in turn takes VTIME_SCALE_FACTOR:

#define VTIME_SCALE_FACTOR    0.0625
#define ME_TO_DOUBLE(me) \
  (double)(VTIME_SCALE_FACTOR*(1+(double)(me>>4)/16)*(double)(1<<(me&0x0F)))

In a bash session on the same QNX host the command-line "printf"
produces the same, so I presume it uses the printf(3) in the libc:

$ printf '%.3f\n' 0.0625
0.063

Every other OS where the tests have been run (automatically or
manually) produces 0.062 in tcpdump tests.  A quick test on FreeBSD,
Haiku, illumos, Linux, NetBSD and OpenBSD confirms that the command-line
"printf" on these OSes produces 0.062 too.  So seemingly QNX libc is
off, but if you look at Linux printf(3), it says:

f, F   The double argument is rounded and converted to decimal notation

Then, seemingly QNX printf() does it right (0.0625 to 0.063), but then
does every other implementation do it incorrectly?  [1] explains that
two factors are usually at play: the representation of decimal float in
binary and the rounding method.  Because 0.0625 equals exactly 1/16,
which is perfectly presentable in binary, it seems the former is not a
factor for this specific value, but the latter maps to "half away from
zero" in QNX and "half to even" in all other implementations, as the
following test indicates.

--------------------------------------------------
#!/bin/sh -e

round_and_print()
{
        format=${1:?}
        value=${2:?}
        printf "Format "%s" rounds %s to $format.\n" "$format" "$value" "$value"
}

for i in 0 1 2 3 4; do
        for f in 49 50 51; do
                for prec in 0 1 2; do
                        round_and_print "%.${prec}f" "$i.$f"
                done
        done
done
--------------------------------------------------

On Linux it produces:

Format %.0f rounds 0.49 to 0.
Format %.1f rounds 0.49 to 0.5.
Format %.2f rounds 0.49 to 0.49.
Format %.0f rounds 0.50 to 0.
Format %.1f rounds 0.50 to 0.5.
Format %.2f rounds 0.50 to 0.50.
Format %.0f rounds 0.51 to 1.
Format %.1f rounds 0.51 to 0.5.
Format %.2f rounds 0.51 to 0.51.
Format %.0f rounds 1.49 to 1.
Format %.1f rounds 1.49 to 1.5.
Format %.2f rounds 1.49 to 1.49.
Format %.0f rounds 1.50 to 2.
Format %.1f rounds 1.50 to 1.5.
Format %.2f rounds 1.50 to 1.50.
Format %.0f rounds 1.51 to 2.
Format %.1f rounds 1.51 to 1.5.
Format %.2f rounds 1.51 to 1.51.
Format %.0f rounds 2.49 to 2.
Format %.1f rounds 2.49 to 2.5.
Format %.2f rounds 2.49 to 2.49.
Format %.0f rounds 2.50 to 2.
Format %.1f rounds 2.50 to 2.5.
Format %.2f rounds 2.50 to 2.50.
Format %.0f rounds 2.51 to 3.
Format %.1f rounds 2.51 to 2.5.
Format %.2f rounds 2.51 to 2.51.
Format %.0f rounds 3.49 to 3.
Format %.1f rounds 3.49 to 3.5.
Format %.2f rounds 3.49 to 3.49.
Format %.0f rounds 3.50 to 4.
Format %.1f rounds 3.50 to 3.5.
Format %.2f rounds 3.50 to 3.50.
Format %.0f rounds 3.51 to 4.
Format %.1f rounds 3.51 to 3.5.
Format %.2f rounds 3.51 to 3.51.
Format %.0f rounds 4.49 to 4.
Format %.1f rounds 4.49 to 4.5.
Format %.2f rounds 4.49 to 4.49.
Format %.0f rounds 4.50 to 4.
Format %.1f rounds 4.50 to 4.5.
Format %.2f rounds 4.50 to 4.50.
Format %.0f rounds 4.51 to 5.
Format %.1f rounds 4.51 to 4.5.
Format %.2f rounds 4.51 to 4.51.

On QNX it produces almost the same, except:

Format %.0f rounds 0.50 to 1.
Format %.0f rounds 2.50 to 3.
Format %.0f rounds 4.50 to 5.

Does POSIX define which method printf(3) is supposed to use for
rounding a float?

1: https://floating-point-gui.de/errors/rounding/

-- 
    Denis Ovsienko
_______________________________________________
tcpdump-workers mailing list -- tcpdump-workers () lists tcpdump org
To unsubscribe send an email to tcpdump-workers-leave () lists tcpdump org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s