From Newsgroup: comp.arch

On Thu, 1 Oct 2026 10:17:57 +0200
David Brown <[email protected]> wrote:

On 01/10/2026 03:33, bart wrote:

On 01/10/2026 01:47, Lawrence D’Oliveiro wrote:

A couple of things stand out immediately: one is that the “long
double” type doesn’t seem to make use of all the 128 bits it
occupies. I was expecting a mantissa length closer to 100 bits,
but it’s nowhere near that.

Probably it uses Intel's 80-bit x87 FPU format. That uses a 64-bit mantissa (with explicit top bit).

Yes, that's the default for "long double" in the standard x86-64 ABI.

On 32-bit x86, these 10-byte doubles were often stored in 12-byte
(96-bit) containers for better alignment, but I don't know the ABI
standards here.

On 64-bit x86, for better alignment they are stored in 16-byte
containers. The rest of the space will be padding.

gcc supports "-mlong-double-64", "-mlong-double-80" and
"-mlong-double-128" flags.

The latter appears to be a default on ARM64 Linux.

With 64-bit long doubles, they are
effectively the same as doubles, and thus faster. 80-bit long
doubles are the standard, while 128-bit long doubles give
quad-precision range and precision, but must be handled in software
through the gcc-provided "libquad" library functions.

It is more complicated than that.
libquadmath supplies 128-bit equivalents of of standard C math library functions and functions for conveersion betwen fp128 and strings.
But basic arithmetic operations are coming from other source. One would
suspect libgcc, but reality is more complicated than just that. It
seems that Jakub Jelinek knows details, but I am not sure that there is somebody else that knows all of them for all platforms.

You can also use the _Float128 or _Float128x types, which are
optional types in the C standards (along with 16, 32 and 64-bit
versions). The "x" types are for calculations and might be bigger
than their name applies, while the non-x types are "interchange"
types that have the same format on all systems.

Another name supported on many platforms is __float128.

10 bytes would be an odd size though, so it's rounded up to 16
bytes in the implementation.

I don't know if current hardware would directly support a full 128
bits, or it needs to be emulated.

IBM's POWER processors are, I think, the only cpus with hardware quad-precision floating point that are realistic. RISC-V has the Q extensions specified, but I do not believe anyone actually makes
RISC-V cores with quad floating point.

Don't forget IBM Z.
BTW, the next generation of IBM Z chips will run ARM64 natively
alongside Z. I wonder if it would mativate IBM to ask ARM Inc to add
128-bit BFP (and may be DFP) as optional extension(s) to Arm ISA.

However, float limits (in float.h) should give a clue as to the
actual range and precision.

Yes.

You can also just write some code in godbolt.org, and see if it calls library functions like "__multf3" instead of generating hardware instructions, as you play around with the compiler flags.

--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.arch

On 01/10/2026 13:12, Michael S wrote:

On Thu, 1 Oct 2026 10:17:57 +0200
David Brown <[email protected]> wrote:

On 01/10/2026 03:33, bart wrote:

On 01/10/2026 01:47, Lawrence D’Oliveiro wrote:

A couple of things stand out immediately: one is that the “long
double” type doesn’t seem to make use of all the 128 bits it
occupies. I was expecting a mantissa length closer to 100 bits,
but it’s nowhere near that.

Probably it uses Intel's 80-bit x87 FPU format. That uses a 64-bit
mantissa (with explicit top bit).

Yes, that's the default for "long double" in the standard x86-64 ABI.

On 32-bit x86, these 10-byte doubles were often stored in 12-byte
(96-bit) containers for better alignment, but I don't know the ABI
standards here.

On 64-bit x86, for better alignment they are stored in 16-byte
containers. The rest of the space will be padding.

gcc supports "-mlong-double-64", "-mlong-double-80" and
"-mlong-double-128" flags.

The latter appears to be a default on ARM64 Linux.

That makes sense. Although software 128-bit floating point is going to
be very slow compared to hardware 64-bit, the reason you would use "long double" is to get more than "double".

With 64-bit long doubles, they are
effectively the same as doubles, and thus faster. 80-bit long
doubles are the standard, while 128-bit long doubles give
quad-precision range and precision, but must be handled in software
through the gcc-provided "libquad" library functions.

It is more complicated than that.
libquadmath supplies 128-bit equivalents of of standard C math library functions and functions for conveersion betwen fp128 and strings.
But basic arithmetic operations are coming from other source. One would suspect libgcc, but reality is more complicated than just that. It
seems that Jakub Jelinek knows details, but I am not sure that there is somebody else that knows all of them for all platforms.

For targets without hardware floating point for float and/or double, the normal floating arithmetic operations are provided in a "language
support library" - this roughly covers any library code needed for C
that can be written without any standard headers. On small
microcontrollers, it will also include some integer operations, and
maybe other bits of code. This library is used and linked implicitly by
gcc. I suppose the basic 128-bit floating point arithmetic operations
will be there too (on targets that support it) - and not in a specific
bit of "libquadmath" as I first guessed (and which I also named
inaccurately).

You can also use the _Float128 or _Float128x types, which are
optional types in the C standards (along with 16, 32 and 64-bit
versions). The "x" types are for calculations and might be bigger
than their name applies, while the non-x types are "interchange"
types that have the same format on all systems.

Another name supported on many platforms is __float128.

Yes. But names with double underscores like that are primarily for implementation use, such as for typedefs in standard headers. It is
good to use C standard names where practical.

10 bytes would be an odd size though, so it's rounded up to 16
bytes in the implementation.

I don't know if current hardware would directly support a full 128
bits, or it needs to be emulated.

IBM's POWER processors are, I think, the only cpus with hardware
quad-precision floating point that are realistic. RISC-V has the Q
extensions specified, but I do not believe anyone actually makes
RISC-V cores with quad floating point.

Don't forget IBM Z.

Yes, I forgot them - I have absolutely no experience with them. (I have
no experience with POWER either, but I have used microcontrollers with
its smaller cousin PowerPC.)

BTW, the next generation of IBM Z chips will run ARM64 natively
alongside Z. I wonder if it would mativate IBM to ask ARM Inc to add
128-bit BFP (and may be DFP) as optional extension(s) to Arm ISA.

I wonder how much call there is for fast 128-bit precision floating
point in practice.

--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.arch

David Brown wrote:

On 01/10/2026 13:12, Michael S wrote:

On Thu, 1 Oct 2026 10:17:57 +0200
David Brown <[email protected]> wrote:

On 01/10/2026 03:33, bart wrote:

On 01/10/2026 01:47, Lawrence D’Oliveiro wrote:

A couple of things stand out immediately: one is that the “long >>>>> double” type doesn’t seem to make use of all the 128 bits it >>>>> occupies. I was expecting a mantissa length closer to 100 bits,
but it’s nowhere near that.

Probably it uses Intel's 80-bit x87 FPU format. That uses a 64-bit
mantissa (with explicit top bit).

Yes, that's the default for "long double" in the standard x86-64 ABI.>>> >>> On 32-bit x86, these 10-byte doubles were often stored in 12-byte
(96-bit) containers for better alignment, but I don't know the ABI
standards here.

On 64-bit x86, for better alignment they are stored in 16-byte
containers.  The rest of the space will be padding.

gcc supports "-mlong-double-64", "-mlong-double-80" and
"-mlong-double-128" flags.

The latter appears to be a default on ARM64 Linux.

That makes sense.  Although software 128-bit floating point is going to
be very slow compared to hardware 64-bit, the reason you would use "long double" is to get more than "double".

Significantly slower, yes.
For Mill we figured out that if the HW provided a small number of helper functions (easy to do in HW, much harder in SW), then you can do most
ops in maybe 5x the HW cycle count.
Without that help, you need to manually extract
sign/exp/mantissa(inserting leading bit unless subnormal), then for
add/sub you must normalize the smaller number (including sticky bit), do the add, normalize again, then merge with exponent and round.
For FMUL you don't pre-normalize (so no extra cycles for subnormal), but you have to handle the potential for up to 111 bits of post-normalization.
Michael S is my current benchmark source here, I'd guess FADD128 in less than 40 cycles, about the same for FMUL128, while FMAC128 has to be a
little bit harder with a _very_ wide intermediate result.
Terje
--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.arch

On 10/2/2026 1:35 PM, Terje Mathisen wrote:

David Brown wrote:

On 01/10/2026 13:12, Michael S wrote:

On Thu, 1 Oct 2026 10:17:57 +0200
David Brown <[email protected]> wrote:

On 01/10/2026 03:33, bart wrote:

On 01/10/2026 01:47, Lawrence D’Oliveiro wrote:

A couple of things stand out immediately: one is that the “long >>>>>> double” type doesn’t seem to make use of all the 128 bits it
occupies. I was expecting a mantissa length closer to 100 bits,
but it’s nowhere near that.

Probably it uses Intel's 80-bit x87 FPU format. That uses a 64-bit
mantissa (with explicit top bit).

Yes, that's the default for "long double" in the standard x86-64 ABI.

On 32-bit x86, these 10-byte doubles were often stored in 12-byte
(96-bit) containers for better alignment, but I don't know the ABI
standards here.

On 64-bit x86, for better alignment they are stored in 16-byte
containers.  The rest of the space will be padding.

gcc supports "-mlong-double-64", "-mlong-double-80" and
"-mlong-double-128" flags.

The latter appears to be a default on ARM64 Linux.

That makes sense.  Although software 128-bit floating point is going
to be very slow compared to hardware 64-bit, the reason you would use
"long double" is to get more than "double".

Significantly slower, yes.

For cases where one needs more than "double" the slowness is justified.

And, if the person expects it to be as fast as double, they are at fault
for casually using "long double".

For Mill we figured out that if the HW provided a small number of helper functions (easy to do in HW, much harder in SW), then you can do most
ops in maybe 5x the HW cycle count.

Without that help, you need to manually extract sign/exp/
mantissa(inserting leading bit unless subnormal), then for add/sub you
must normalize the smaller number (including sticky bit), do the add, normalize again, then merge with exponent and round.

For FMUL you don't pre-normalize (so no extra cycles for subnormal), but
you have to handle the potential for up to 111 bits of post-normalization.

Michael S is my current benchmark source here, I'd guess FADD128 in less than 40 cycles, about the same for FMUL128, while FMAC128 has to be a
little bit harder with a _very_ wide intermediate result.

In my case, was just doing it in software with integer math.
Nothing particularly notable here though.

Did eventually end up deciding to support ISA level ops for 128-bit
floating point even with them being always traps, because in some cases
the use of traps can be "less bad" than the use of function calls
(though, mostly on RV64).

Though, strict adherence to RV64G creates a bit of a problem here for
"long double", as expecting any particular behavior from the 'Q'
extension ops is inherently undefined:
Does it just trap and terminate the program;
Does it trap and emulate Binary128 using register pairs (my approach);
Does it actually implement the 'Q' extension (other option).


Though, maybe controversial in that personally I feel that implementing
a reduced version of 'Q' on top of even-numbered register pairs is a
better option than implementing the Q extension proper (or, IOW, that in
this case, that natively 128-bit registers would actually make things
worse).

Though, as can be noted, I also take the design approach/assumption that
even pairs can be combined in hardware to perform 128-bit operations as needed. So, the use of 64-bit as a base size does not preclude the
possibility of natively 128-bit operations in hardware (had also taken
this approach for 128-bit ALU and SIMD).

Though, mostly this is because I expect that either way, Binary64 is
likely to remain by far dominant over Binary128 even in the presence of
native support for Binary128 in hardware (because, even if not slow, it
is still likely to be overkill; much like how cheap support for "double" doesn't mean that no one uses "float"...).

Terje

--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.arch

On Fri, 2 Oct 2026 17:13:35 -0500, BGB wrote:

... much like how cheap support for "double" doesn't mean that no
one uses "float"...).

Double is pretty much the default precision, though. High-level
languages like Python and JavaScript only support one real-number
precision, and that’s double.

Even the naming of the C run-time routines -- an “f” suffix denotes
single precision, “l” denotes long double, while no suffix defaults to meaning double.
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.arch

On Fri, 2 Oct 2026 20:35:56 +0200
Terje Mathisen <[email protected]> wrote:

David Brown wrote:

On 01/10/2026 13:12, Michael S wrote:

On Thu, 1 Oct 2026 10:17:57 +0200
David Brown <[email protected]> wrote:

On 01/10/2026 03:33, bart wrote:

On 01/10/2026 01:47, Lawrence D’Oliveiro wrote:

A couple of things stand out immediately: one is that the
“long double” type doesn’t seem to make use of all the
128 bits it occupies. I was expecting a mantissa length closer
to 100 bits, but it’s nowhere near that.

Probably it uses Intel's 80-bit x87 FPU format. That uses a
64-bit mantissa (with explicit top bit).

Yes, that's the default for "long double" in the standard x86-64
ABI.

On 32-bit x86, these 10-byte doubles were often stored in 12-byte
(96-bit) containers for better alignment, but I don't know the ABI
standards here.

On 64-bit x86, for better alignment they are stored in 16-byte
containers.  The rest of the space will be padding.

gcc supports "-mlong-double-64", "-mlong-double-80" and
"-mlong-double-128" flags.

The latter appears to be a default on ARM64 Linux.

That makes sense.  Although software 128-bit floating point is
going to be very slow compared to hardware 64-bit, the reason you
would use "long double" is to get more than "double".

Significantly slower, yes.

For Mill we figured out that if the HW provided a small number of
helper functions (easy to do in HW, much harder in SW), then you can
do most ops in maybe 5x the HW cycle count.

Without that help, you need to manually extract
sign/exp/mantissa(inserting leading bit unless subnormal), then for
add/sub you must normalize the smaller number (including sticky bit),
do the add, normalize again, then merge with exponent and round.

For FMUL you don't pre-normalize (so no extra cycles for subnormal),
but you have to handle the potential for up to 111 bits of post-normalization.

Michael S is my current benchmark source here, I'd guess FADD128 in
less than 40 cycles, about the same for FMUL128, while FMAC128 has to
be a little bit harder with a _very_ wide intermediate result.

Terje

Of course, an actual cycle count depends on what you measure, latency
or throughput, on your CPU, on how many corners you are willing to cut
w.r.t. support for rounding modes and for FP exceptions and on the ABI.
Current x86-64 SYSV ABI is quite problematic and rather far from
well-thought.
Windows currently has no official FP128 ABI at all, the closest to
official is the ABI implemented by gcc under msys2. It is even more
problematic than SysV.
As far as I am concerned, the most problematic part of both ABIs is
program status word that is shared with FP32/64. But registers choice
is also bad.
According to what I hear from Thomas Koenig, in Fortran quite a few
corners can be cut without violating language assumptions. For C it
would be harder.
With most corners cut, i.e. without non-default rounding modes and
without support for Inexact exception, and for throughput rather than
latency Zen3/Linux runs at ~50 clocks per FMUL+FADD. I never tried to
separate between the two. Would think that they are about the same.
Pay attention, that nearly all cost of implementing non-default
rounding mode is because of need to read rounding mode from HW register.
The same applies to implementing Inexact exception - the main cost is
update of HW register.
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.arch

Michael S wrote:

On Fri, 2 Oct 2026 20:35:56 +0200
Terje Mathisen <[email protected]> wrote:

David Brown wrote:

On 01/10/2026 13:12, Michael S wrote:

On Thu, 1 Oct 2026 10:17:57 +0200
David Brown <[email protected]> wrote:

On 01/10/2026 03:33, bart wrote:

On 01/10/2026 01:47, Lawrence D’Oliveiro wrote:

A couple of things stand out immediately: one is that the
“long double” type doesn’t seem to make use of all the
128 bits it occupies. I was expecting a mantissa length closer
to 100 bits, but it’s nowhere near that.

Probably it uses Intel's 80-bit x87 FPU format. That uses a
64-bit mantissa (with explicit top bit).

Yes, that's the default for "long double" in the standard x86-64
ABI.

On 32-bit x86, these 10-byte doubles were often stored in 12-byte
(96-bit) containers for better alignment, but I don't know the ABI
standards here.

On 64-bit x86, for better alignment they are stored in 16-byte
containers.  The rest of the space will be padding.

gcc supports "-mlong-double-64", "-mlong-double-80" and
"-mlong-double-128" flags.

The latter appears to be a default on ARM64 Linux.

That makes sense.  Although software 128-bit floating point is
going to be very slow compared to hardware 64-bit, the reason you
would use "long double" is to get more than "double".

Significantly slower, yes.

For Mill we figured out that if the HW provided a small number of
helper functions (easy to do in HW, much harder in SW), then you can
do most ops in maybe 5x the HW cycle count.

Without that help, you need to manually extract
sign/exp/mantissa(inserting leading bit unless subnormal), then for
add/sub you must normalize the smaller number (including sticky bit),
do the add, normalize again, then merge with exponent and round.

For FMUL you don't pre-normalize (so no extra cycles for subnormal),
but you have to handle the potential for up to 111 bits of
post-normalization.

Michael S is my current benchmark source here, I'd guess FADD128 in
less than 40 cycles, about the same for FMUL128, while FMAC128 has to
be a little bit harder with a _very_ wide intermediate result.

Terje

Of course, an actual cycle count depends on what you measure, latency
or throughput, on your CPU, on how many corners you are willing to cut
w.r.t. support for rounding modes and for FP exceptions and on the ABI.> Current x86-64 SYSV ABI is quite problematic and rather far from
well-thought.
Windows currently has no official FP128 ABI at all, the closest to
official is the ABI implemented by gcc under msys2. It is even more problematic than SysV.
As far as I am concerned, the most problematic part of both ABIs is
program status word that is shared with FP32/64. But registers choice
is also bad.
According to what I hear from Thomas Koenig, in Fortran quite a few
corners can be cut without violating language assumptions. For C it
would be harder.
With most corners cut, i.e. without non-default rounding modes and
without support for Inexact exception, and for throughput rather than
latency Zen3/Linux runs at ~50 clocks per FMUL+FADD. I never tried to separate between the two. Would think that they are about the same.

Pay attention, that nearly all cost of implementing non-default
rounding mode is because of need to read rounding mode from HW register.
The same applies to implementing Inexact exception - the main cost is
update of HW register.

So what you are saying is the my guesstimate was in the right ballpark,
but that it would be far better for fp128 to be a totally separate environment, with all flags and modes maintained in the library instead
of sharing anything with the HW?
Regarding rounding modes, in my own Mill work a single 64-bit const
contain all the rounding rules for the four non-truncate alternatives,
so this didn't really cost much.
Terje
--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.arch

On Sat, 3 Oct 2026 20:22:51 +0200
Terje Mathisen <[email protected]> wrote:

Michael S wrote:

On Fri, 2 Oct 2026 20:35:56 +0200
Terje Mathisen <[email protected]> wrote:

David Brown wrote:

On 01/10/2026 13:12, Michael S wrote:

On Thu, 1 Oct 2026 10:17:57 +0200
David Brown <[email protected]> wrote:

On 01/10/2026 03:33, bart wrote:

On 01/10/2026 01:47, Lawrence D’Oliveiro wrote: >>>>>>> A couple of things stand out immediately: one is that the

“long double” type doesn’t seem to make
use of all the 128 bits it occupies. I was expecting a
mantissa length closer to 100 bits, but it’s nowhere >>>>>>> near that.

Probably it uses Intel's 80-bit x87 FPU format. That uses a
64-bit mantissa (with explicit top bit).

Yes, that's the default for "long double" in the standard x86-64
ABI.

On 32-bit x86, these 10-byte doubles were often stored in
12-byte (96-bit) containers for better alignment, but I don't
know the ABI standards here.

On 64-bit x86, for better alignment they are stored in 16-byte
containers.  The rest of the space will be padding.

gcc supports "-mlong-double-64", "-mlong-double-80" and
"-mlong-double-128" flags.

The latter appears to be a default on ARM64 Linux.

That makes sense.  Although software 128-bit floating point is
going to be very slow compared to hardware 64-bit, the reason you
would use "long double" is to get more than "double".

Significantly slower, yes.

For Mill we figured out that if the HW provided a small number of
helper functions (easy to do in HW, much harder in SW), then you
can do most ops in maybe 5x the HW cycle count.

Without that help, you need to manually extract
sign/exp/mantissa(inserting leading bit unless subnormal), then for
add/sub you must normalize the smaller number (including sticky
bit), do the add, normalize again, then merge with exponent and
round.

For FMUL you don't pre-normalize (so no extra cycles for
subnormal), but you have to handle the potential for up to 111
bits of post-normalization.

Michael S is my current benchmark source here, I'd guess FADD128 in
less than 40 cycles, about the same for FMUL128, while FMAC128 has
to be a little bit harder with a _very_ wide intermediate result.

Terje

Of course, an actual cycle count depends on what you measure,
latency or throughput, on your CPU, on how many corners you are
willing to cut w.r.t. support for rounding modes and for FP
exceptions and on the ABI. Current x86-64 SYSV ABI is quite
problematic and rather far from well-thought.
Windows currently has no official FP128 ABI at all, the closest to
official is the ABI implemented by gcc under msys2. It is even more problematic than SysV.
As far as I am concerned, the most problematic part of both ABIs is
program status word that is shared with FP32/64. But registers
choice is also bad.
According to what I hear from Thomas Koenig, in Fortran quite a few
corners can be cut without violating language assumptions. For C it
would be harder.
With most corners cut, i.e. without non-default rounding modes and
without support for Inexact exception, and for throughput rather
than latency Zen3/Linux runs at ~50 clocks per FMUL+FADD. I never
tried to separate between the two. Would think that they are about
the same.

Pay attention, that nearly all cost of implementing non-default
rounding mode is because of need to read rounding mode from HW
register. The same applies to implementing Inexact exception - the
main cost is update of HW register.

So what you are saying is the my guesstimate was in the right
ballpark, but that it would be far better for fp128 to be a totally
separate environment, with all flags and modes maintained in the
library instead of sharing anything with the HW?

Yes.
Also it would be better if parameteres/return values to/from
support routines (i.e. fadd128, fmul128 etc) passed either in GPRs or in
memory instead of SIMD registers.

Regarding rounding modes, in my own Mill work a single 64-bit const
contain all the rounding rules for the four non-truncate
alternatives, so this didn't really cost much.

And what happens when FP32 implemented in HW, but FP64 emoulated in
software as is the most common case in ARM-based MCUs?

Terje

--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.arch

Terje Mathisen <[email protected]> schrieb:

So what you are saying is the my guesstimate was in the right ballpark,
but that it would be far better for fp128 to be a totally separate environment, with all flags and modes maintained in the library instead
of sharing anything with the HW?

Apart from implementaiton details, it might actually be a good
idea - it makes sense to do FP128 in RNO and FP64 in RNE.
--
This USENET posting was made without artificial intelligence,
artificial impertinence, artificial arrogance, artificial stupidity,
artificial flavorings or artificial colorants.
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.arch

On 10/2/2026 9:17 PM, Lawrence D’Oliveiro wrote:

On Fri, 2 Oct 2026 17:13:35 -0500, BGB wrote:

... much like how cheap support for "double" doesn't mean that no
one uses "float"...).

Double is pretty much the default precision, though. High-level
languages like Python and JavaScript only support one real-number
precision, and that’s double.

Even the naming of the C run-time routines -- an “f” suffix denotes single precision, “l” denotes long double, while no suffix defaults to meaning double.

Can note that Binary64/double does make more sense as a general purpose format.

It is also kinda interesting that people 40 years ago seemingly knew
which format was best for general purpose use, and the needle hasn't
really moved since then.


Say, 40 years ago:
int : 2 bytes
char* : 2 bytes
float : 4 bytes
double : 8 bytes
long double : 10 bytes
Now:
int : 4 bytes
char* : 8 bytes
float : 4 bytes
double : 8 bytes
long double : 16 bytes

We have since gained "half" / "short float", sometimes...

So, say:
long double : 16 bytes, rarely used, overkill
double : 8 bytes, general purpose
float : 4 bytes, 3D and lots of stuff
short float : 2 bytes, graphics, sound, and NNs

Or, smaller still:
FP8 : 1 byte (S.E4.M3), NNs mostly
FP8A : 1 byte (S.E3.M4), audio, rotation quats, *1
FP8U : 1 byte (E4.M4), HDR graphics, *2.


*1: 4x FP8A is a usable format for rotation quaternions when precision
isn't a high priority. Otherwise 4xBinary16 or 4xBinary32 are more
typical in my uses. Though, 4xFp8A isn't used as an in-register format,
more for in-memory, with 4xFP16 as the corresponding in-register format.


*2: Image fidelity is comparable to RGB555 in this case, but with a
larger dynamic range.

Had observed (in TKRA-GL) that one can do HDR 3D rendering by shoving
this through an LDR RGBA32 path (reusing most of the LDR math on the HDR
color values), which isn't super accurate, but interestingly gave
limitations and artifacts very close to those of HDR on the Radeon HD
4850 card I was using in the early 2010s.

The main obvious feature being that things like wonk with texture interpolation, blending, and the restrictions on the use of an alpha
channel; and that (in the final output) the pixels only really had
limited accuracy (if fetching the framebuffer image as HALF_FLOAT, low 6
bits of mantissa were zeroed, etc).

If asking AI: its response is that this card did not have native HDR
paths in HW and was in-fact shoving HDR through the LDR path with a
similar strategy (hence the visual similarity in the results).



Can note though that despite "float" being widely used in 3D, had noted
that it isn't quite good enough for things like CSG clipping for 3D mesh building; as it can leave small gaps between vertices.

To some extent this issue can be reduced after the fact by finding
vertices that are close together, averaging them, and then snapping them
to some level of grid points (typically something smallish).


--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.arch

Michael S wrote:

On Sat, 3 Oct 2026 20:22:51 +0200
Terje Mathisen <[email protected]> wrote:

Michael S wrote:

On Fri, 2 Oct 2026 20:35:56 +0200
Terje Mathisen <[email protected]> wrote:

David Brown wrote:

On 01/10/2026 13:12, Michael S wrote:

On Thu, 1 Oct 2026 10:17:57 +0200
David Brown <[email protected]> wrote:

On 01/10/2026 03:33, bart wrote:

On 01/10/2026 01:47, Lawrence D’Oliveiro wrote:

A couple of things stand out immediately: one is that the
“long double” type doesn’t seem to make
use of all the 128 bits it occupies. I was expecting a
mantissa length closer to 100 bits, but it’s nowhere
near that.

Probably it uses Intel's 80-bit x87 FPU format. That uses a
64-bit mantissa (with explicit top bit).

Yes, that's the default for "long double" in the standard x86-64 >>>>>>> ABI.

On 32-bit x86, these 10-byte doubles were often stored in
12-byte (96-bit) containers for better alignment, but I don't
know the ABI standards here.

On 64-bit x86, for better alignment they are stored in 16-byte
containers.  The rest of the space will be padding.

gcc supports "-mlong-double-64", "-mlong-double-80" and
"-mlong-double-128" flags.

The latter appears to be a default on ARM64 Linux.

That makes sense.  Although software 128-bit floating point is >>>>> going to be very slow compared to hardware 64-bit, the reason you
would use "long double" is to get more than "double".

Significantly slower, yes.

For Mill we figured out that if the HW provided a small number of
helper functions (easy to do in HW, much harder in SW), then you
can do most ops in maybe 5x the HW cycle count.

Without that help, you need to manually extract
sign/exp/mantissa(inserting leading bit unless subnormal), then for
add/sub you must normalize the smaller number (including sticky
bit), do the add, normalize again, then merge with exponent and
round.

For FMUL you don't pre-normalize (so no extra cycles for
subnormal), but you have to handle the potential for up to 111
bits of post-normalization.

Michael S is my current benchmark source here, I'd guess FADD128 in
less than 40 cycles, about the same for FMUL128, while FMAC128 has
to be a little bit harder with a _very_ wide intermediate result.

Terje

Of course, an actual cycle count depends on what you measure,
latency or throughput, on your CPU, on how many corners you are
willing to cut w.r.t. support for rounding modes and for FP
exceptions and on the ABI. Current x86-64 SYSV ABI is quite
problematic and rather far from well-thought.
Windows currently has no official FP128 ABI at all, the closest to
official is the ABI implemented by gcc under msys2. It is even more
problematic than SysV.
As far as I am concerned, the most problematic part of both ABIs is
program status word that is shared with FP32/64. But registers
choice is also bad.
According to what I hear from Thomas Koenig, in Fortran quite a few
corners can be cut without violating language assumptions. For C it
would be harder.
With most corners cut, i.e. without non-default rounding modes and
without support for Inexact exception, and for throughput rather
than latency Zen3/Linux runs at ~50 clocks per FMUL+FADD. I never
tried to separate between the two. Would think that they are about
the same.

Pay attention, that nearly all cost of implementing non-default
rounding mode is because of need to read rounding mode from HW
register. The same applies to implementing Inexact exception - the
main cost is update of HW register.

So what you are saying is the my guesstimate was in the right
ballpark, but that it would be far better for fp128 to be a totally
separate environment, with all flags and modes maintained in the
library instead of sharing anything with the HW?

Yes.
Also it would be better if parameteres/return values to/from
support routines (i.e. fadd128, fmul128 etc) passed either in GPRs or in memory instead of SIMD registers.

I used memory/stack for all parameters in the limited functionality
fp128 library I wrote over a weekend in January 1995. I only needed it
to verify our FDIV workaround code.

Regarding rounding modes, in my own Mill work a single 64-bit const
contain all the rounding rules for the four non-truncate
alternatives, so this didn't really cost much.

And what happens when FP32 implemented in HW, but FP64 emoulated in
software as is the most common case in ARM-based MCUs?

Can you trap on updates to those registers? That would make it much
easier to maintain a proper SW shadow of the HW state.
Terje
--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.arch

On Sat, 3 Oct 2026 15:20:39 -0500, BGB wrote:

It is also kinda interesting that people 40 years ago seemingly knew
which format was best for general purpose use, and the needle hasn't
really moved since then.

You mean the Bell Labs C/Unix folks?

I don’t think their design choice was very popular at the time. It
simplified the design and implementation of the C language, which I
think was their primary consideration. I imagine the FORTRAN folks
threw their hands up in horror.
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.arch

On 10/3/2026 4:46 PM, Lawrence D’Oliveiro wrote:

On Sat, 3 Oct 2026 15:20:39 -0500, BGB wrote:

It is also kinda interesting that people 40 years ago seemingly knew
which format was best for general purpose use, and the needle hasn't
really moved since then.

You mean the Bell Labs C/Unix folks?

I was more thinking of things around the time of the IEEE-754 standard,
which defined single/double, which mapped 1:1 with C float/double.

And, double was the default, and also the most useful...
And, they chose double rather than single as the default, despite double
being gigantic compared with the integer or pointer types in use at the
time.


Say, CPU types:
PDP-11: 16 bit address space, no native FPU
6502: 16-bit address space, no multiply or variable shift, ...
Z80: Mostly like 6502 in this sense;
8088: 16-bit, still no FPU, 640K of RAM (very big at the time);
...

Late 1980s had M68K for a while (eg: Macintosh).
And, the Apple II/gs using the 65C816
Extended 6502 with 24-bit address bus.

...

Knowing of the hardware that was in use at the time, and much of the
code from the 1990s, it seems almost prescient for them to have known
that 'double' was likely to win out in the long run as the dominant
floating point format for general purpose use (rather than, say, single/float).

...

I don’t think their design choice was very popular at the time. It simplified the design and implementation of the C language, which I
think was their primary consideration. I imagine the FORTRAN folks
threw their hands up in horror.

OK.


--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.arch


Michael S <[email protected]> posted:

On Sat, 3 Oct 2026 20:22:51 +0200
Terje Mathisen <[email protected]> wrote:

Michael S wrote:

On Fri, 2 Oct 2026 20:35:56 +0200
Terje Mathisen <[email protected]> wrote:

David Brown wrote:

On 01/10/2026 13:12, Michael S wrote:

On Thu, 1 Oct 2026 10:17:57 +0200
David Brown <[email protected]> wrote:

On 01/10/2026 03:33, bart wrote:

On 01/10/2026 01:47, Lawrence D’Oliveiro wrote: >>>>>>> A couple of things stand out immediately: one is that the

“long double” type doesn’t seem to make
use of all the 128 bits it occupies. I was expecting a
mantissa length closer to 100 bits, but it’s nowhere
near that.

Probably it uses Intel's 80-bit x87 FPU format. That uses a
64-bit mantissa (with explicit top bit).

Yes, that's the default for "long double" in the standard x86-64 >>>>> ABI.

On 32-bit x86, these 10-byte doubles were often stored in
12-byte (96-bit) containers for better alignment, but I don't
know the ABI standards here.

On 64-bit x86, for better alignment they are stored in 16-byte
containers.  The rest of the space will be padding.

gcc supports "-mlong-double-64", "-mlong-double-80" and
"-mlong-double-128" flags.

The latter appears to be a default on ARM64 Linux.

That makes sense.  Although software 128-bit floating point is
going to be very slow compared to hardware 64-bit, the reason you
would use "long double" is to get more than "double".

Significantly slower, yes.

For Mill we figured out that if the HW provided a small number of
helper functions (easy to do in HW, much harder in SW), then you
can do most ops in maybe 5x the HW cycle count.

Without that help, you need to manually extract
sign/exp/mantissa(inserting leading bit unless subnormal), then for
add/sub you must normalize the smaller number (including sticky
bit), do the add, normalize again, then merge with exponent and
round.

For FMUL you don't pre-normalize (so no extra cycles for
subnormal), but you have to handle the potential for up to 111
bits of post-normalization.

Michael S is my current benchmark source here, I'd guess FADD128 in
less than 40 cycles, about the same for FMUL128, while FMAC128 has
to be a little bit harder with a _very_ wide intermediate result.

Terje

Of course, an actual cycle count depends on what you measure,
latency or throughput, on your CPU, on how many corners you are
willing to cut w.r.t. support for rounding modes and for FP
exceptions and on the ABI. Current x86-64 SYSV ABI is quite
problematic and rather far from well-thought.
Windows currently has no official FP128 ABI at all, the closest to official is the ABI implemented by gcc under msys2. It is even more problematic than SysV.
As far as I am concerned, the most problematic part of both ABIs is program status word that is shared with FP32/64. But registers
choice is also bad.
According to what I hear from Thomas Koenig, in Fortran quite a few corners can be cut without violating language assumptions. For C it
would be harder.
With most corners cut, i.e. without non-default rounding modes and without support for Inexact exception, and for throughput rather
than latency Zen3/Linux runs at ~50 clocks per FMUL+FADD. I never
tried to separate between the two. Would think that they are about
the same.

Pay attention, that nearly all cost of implementing non-default
rounding mode is because of need to read rounding mode from HW
register. The same applies to implementing Inexact exception - the
main cost is update of HW register.

So what you are saying is the my guesstimate was in the right
ballpark, but that it would be far better for fp128 to be a totally separate environment, with all flags and modes maintained in the
library instead of sharing anything with the HW?

Yes.

This, also, argues for FP8, FP16, FP32 have entirely separate
environments, too.

Also it would be better if parameteres/return values to/from
support routines (i.e. fadd128, fmul128 etc) passed either in GPRs or in memory instead of SIMD registers.

Regarding rounding modes, in my own Mill work a single 64-bit const contain all the rounding rules for the four non-truncate
alternatives, so this didn't really cost much.

And what happens when FP32 implemented in HW, but FP64 emoulated in
software as is the most common case in ARM-based MCUs?

Terje

--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.arch

On Sat, 3 Oct 2026 20:35:35 -0500, BGB wrote:

On 10/3/2026 4:46 PM, Lawrence D’Oliveiro wrote:

On Sat, 3 Oct 2026 15:20:39 -0500, BGB wrote:

It is also kinda interesting that people 40 years ago seemingly
knew which format was best for general purpose use, and the needle
hasn't really moved since then.

You mean the Bell Labs C/Unix folks?

I was more thinking of things around the time of the IEEE-754
standard, which defined single/double, which mapped 1:1 with C
float/double.

And, double was the default, and also the most useful...
And, they chose double rather than single as the default, despite double being gigantic compared with the integer or pointer types in use at the
time.

You’re thinking of C. Looking at the original IEEE-754 spec from 1985,
it doesn’t define anything resembling a “default” precision. The
closest thing that might sound like that would be section 4.3,
“Rounding Precision”:

Normally, a result is rounded to the precision of its destination.
However, some systems deliver results only to double or extended
destinations. On such a system the user, which may be a high-level
language compiler, shall be able to specify that a result be
rounded instead to single precision, though it may be stored in
the double or extended format with its wider exponent range.
[FOOTNOTE 4: Control of rounding precision is intended to allow
systems whose destinations are always double or extended to mimic,
in the absence of over/underflow, the precisions of systems with
single and double destinations. An implementation should not
provide operations that combine double or extended operands to
produce a single result, nor operations that combine double
extended operands to produce a double result, with only one
rounding.] Similarly, a system that delivers results only to
double extended destinations shall permit the user to specify
rounding to single or double precision. Note that to meet the
specifications in 4.1, the result cannot suffer more than one
rounding error.
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.arch


Michael S <[email protected]> posted:

On Thu, 1 Oct 2026 10:17:57 +0200
David Brown <[email protected]> wrote:

On 01/10/2026 03:33, bart wrote:

On 01/10/2026 01:47, Lawrence D’Oliveiro wrote:

A couple of things stand out immediately: one is that the “long
double” type doesn’t seem to make use of all the 128 bits it
occupies. I was expecting a mantissa length closer to 100 bits,
but it’s nowhere near that.

Probably it uses Intel's 80-bit x87 FPU format. That uses a 64-bit mantissa (with explicit top bit).

Yes, that's the default for "long double" in the standard x86-64 ABI.

On 32-bit x86, these 10-byte doubles were often stored in 12-byte
(96-bit) containers for better alignment, but I don't know the ABI standards here.

On 64-bit x86, for better alignment they are stored in 16-byte
containers. The rest of the space will be padding.

gcc supports "-mlong-double-64", "-mlong-double-80" and "-mlong-double-128" flags.

The latter appears to be a default on ARM64 Linux.

I personally have not seen an extended FP format supported for a long time
{20 years)--yes, x86 still gives lip service to then, but "support is iffy"
at best.

For example, I have eXcel from 2003, and on AMD machines I get 80-bit
precision (when real careful in eXcel math)--but those same programs
on Intel and Arm display 64-bit precision.

I think looking forward, one should anticipate long double being 128-bits rather than 80-bits.
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.arch

On Tue, 13 Oct 2026 21:33:23 GMT, MitchAlsup wrote:

I think looking forward, one should anticipate long double being
128-bits rather than 80-bits.

Allocation or actual usage?

My test script indicated that, even though sizeof(long double) = 16,
the mantissa was only using about 64 bits, and the exponent around 15.
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.arch

MitchAlsup <[email protected]d> writes:

I personally have not seen an extended FP format supported for a long time >{20 years)--yes, x86 still gives lip service to then, but "support is iffy" >at best.

Not in the least. 387 FP works fine, and, e.g., VFX Forth a first
switched to SSE2 in their 64-bit port, but due to customer complaints
about the reduced precision, they switched back to using 387 FP.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <[email protected]>
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.arch

On Tue, 13 Oct 2026 21:33:23 GMT
MitchAlsup <[email protected]d> wrote:

For example, I have eXcel from 2003, and on AMD machines I get 80-bit precision (when real careful in eXcel math)--but those same programs
on Intel and Arm display 64-bit precision.

Are you sure that you use exactly the same version of Excel on AMD
and Intel? Also, what OS do you have on each machine?

Also, how exactly do you detect that precision is higher than IEEE
binary64? By forcing mantisa to hold more than 53 bits (incl.hidden) or
by forcing exponent to hold more than 11 bits?



--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.arch


Michael S <[email protected]> posted:

On Tue, 13 Oct 2026 21:33:23 GMT
MitchAlsup <[email protected]d> wrote:

For example, I have eXcel from 2003, and on AMD machines I get 80-bit precision (when real careful in eXcel math)--but those same programs
on Intel and Arm display 64-bit precision.

Are you sure that you use exactly the same version of Excel on AMD
and Intel? Also, what OS do you have on each machine?

Also, how exactly do you detect that precision is higher than IEEE
binary64? By forcing mantisa to hold more than 53 bits (incl.hidden) or
by forcing exponent to hold more than 11 bits?

Very carefully crafted arithmetic. I calculated the Chebyshev points on
a 128-bit FP machine and then transported the ASCII data back and gained
1.2ULP instead of calculating them using COS( point/k ) which is the
standard formula.

Also, instead of calculating a bunch of terms in multiple cells, I use SUMPRODUCT() which I found to be ~0.3ULP better than writing the
equations out by hand.

I have no idea why, just that I can measure {LOG(ABS(value),2)} as the
number of bits of precision--where value is the {AMD,Intel} builtin
minus my Chebyshev polynomial evaluated at the same point. Given 1,000 evaluations points between [a, b] with random dither at each point
the shape of the graph shows the accuracy while the noise on the graph
shows the arithmetic noise.



--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.arch

On Wed, 14 Oct 2026 22:13:04 GMT
MitchAlsup <[email protected]d> wrote:

Michael S <[email protected]> posted:

On Tue, 13 Oct 2026 21:33:23 GMT
MitchAlsup <[email protected]d> wrote:

For example, I have eXcel from 2003, and on AMD machines I get
80-bit precision (when real careful in eXcel math)--but those
same programs on Intel and Arm display 64-bit precision.

Are you sure that you use exactly the same version of Excel on AMD
and Intel? Also, what OS do you have on each machine?

Also, how exactly do you detect that precision is higher than IEEE binary64? By forcing mantisa to hold more than 53 bits
(incl.hidden) or by forcing exponent to hold more than 11 bits?

Very carefully crafted arithmetic. I calculated the Chebyshev points
on a 128-bit FP machine and then transported the ASCII data back and
gained 1.2ULP instead of calculating them using COS( point/k ) which
is the standard formula.

Also, instead of calculating a bunch of terms in multiple cells, I use SUMPRODUCT() which I found to be ~0.3ULP better than writing the
equations out by hand.

I have no idea why, just that I can measure {LOG(ABS(value),2)} as the
number of bits of precision--where value is the {AMD,Intel} builtin
minus my Chebyshev polynomial evaluated at the same point. Given 1,000 evaluations points between [a, b] with random dither at each point
the shape of the graph shows the accuracy while the noise on the graph
shows the arithmetic noise.

All that sounds non-trivial. The difference can be easily related to
something else, e.g. to results of evaluation of trigs. Which do differ
a little between AMD and Intel.

What about trivial tests?
A = 1; B = A + 1e-18; C = B - 1;










--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.arch


Michael S <[email protected]> posted:

On Wed, 14 Oct 2026 22:13:04 GMT
MitchAlsup <[email protected]d> wrote:

Michael S <[email protected]> posted:

On Tue, 13 Oct 2026 21:33:23 GMT
MitchAlsup <[email protected]d> wrote:

For example, I have eXcel from 2003, and on AMD machines I get
80-bit precision (when real careful in eXcel math)--but those
same programs on Intel and Arm display 64-bit precision.

Are you sure that you use exactly the same version of Excel on AMD
and Intel? Also, what OS do you have on each machine?

Also, how exactly do you detect that precision is higher than IEEE binary64? By forcing mantisa to hold more than 53 bits
(incl.hidden) or by forcing exponent to hold more than 11 bits?

Very carefully crafted arithmetic. I calculated the Chebyshev points
on a 128-bit FP machine and then transported the ASCII data back and
gained 1.2ULP instead of calculating them using COS( point/k ) which
is the standard formula.

Also, instead of calculating a bunch of terms in multiple cells, I use SUMPRODUCT() which I found to be ~0.3ULP better than writing the
equations out by hand.

I have no idea why, just that I can measure {LOG(ABS(value),2)} as the number of bits of precision--where value is the {AMD,Intel} builtin
minus my Chebyshev polynomial evaluated at the same point. Given 1,000 evaluations points between [a, b] with random dither at each point
the shape of the graph shows the accuracy while the noise on the graph shows the arithmetic noise.

All that sounds non-trivial. The difference can be easily related to something else, e.g. to results of evaluation of trigs. Which do differ
a little between AMD and Intel.

What about trivial tests?
A = 1; B = A + 1e-18; C = B - 1;

It is really something which is a lot easier to see in graphical form--
like the difference between a multimeter and an oscilloscope.

And yes, there is a lot more numerical noise when I compare EXPM1 with
EXP-1 or LNP1 with Ln+1. But mostly the noise stays down near 52.3-bits
of precision {after all the careful attention to "things"; whereas it
started near 51-bits of precision before care was applied.
--- Synchronet 3.22a-Linux NewsLink 1.2