24 bit vs floating point

Some advantages of working with floating point files are clear, but I’ve never been quite sure about quantization error. With integer math it is straightforward: the quantization error size of transforms on 24 bit files is only 1/256 that when doing the same operations on 16 bit files.

In general, floating point, having digits to the right of the decimal point, allows greater calculation precision. Is that precision 256 times greater than 24 bit integer? Is it a consistent amount greater or does it vary based on some aspect of the data? Does it always, sometimes, or never result in smaller quantization errors than those generated by manipulations of 24 bit files? Yes, I probably should have this clear from long ago math classes, but unfortunately I have not followed a path that has used this kind of information in the past forty years.

For a full scale floating point signal the quantisation errors are the same as 24 bit. One could expect that to be 1/16777216 of the full scale amplitude. With processed floating point signals the error has the potential of being reduced in proportion to the amplitude, i.e. 1/16777216 of the amplitude of the signal, whatever it may be. (For the purposes of this discussion I don't think it's particularly relevant that fp signals may be greater than "full scale").

The real difference between integer and floating point is the difference between accuracy and precision. The first is about how well you can define something, and the other is about how much information you can store about it. But despite this, 32-bit FP is still inherently more accurate than 16-bit int. The quantising noise introduced during processing is theoretically a lot lower, and rather than just repeat the arguments about it, you will find the relevant numbers here.

[snip] you will find the relevant numbers here.

forced to add my 2 cent (that's 0.012636 Euro) plug for the text from which the above link is drawn: "The Scientist and Engineer's Guide to Digital Signal Processing"   ©1997-1998 by Steven W. Smith

I think I originally recommended it, what? maybe seven years ago (or maybe only six) after Andy's original post on the Syntrillium forum.  It remains a valuable bedside read for anyone involved in digital audio (& just might save you money on green marking pens and cold fusion speaker cable, oh & $149 plug ins that promise to fix MP3 data loss)

Paperback new it's $46, used it's $45 (which for me is a pretty good metaphorical indication of the books signal2noise ratio)