SoX Volume

Less data loss aiming for integers below 24-bit

Using only -v [FACTOR] values without 32 bit floting point rounding

Preface

This idea took form after reading about Mr. Milot's Leedh Processing Volume. A lossless volume control that work on the simple prinsipple of

Minimise the number of bits used to quantize volume control coefficients so that information loss is minimized at truncation stage. In other words, it trades volume control coefficients precision against information loss minimization.

Would it be possible to implement a similar approach using Sound eXchange (SoX)?
Rather then input volume values on a whim have volume adjustment done from a lookup table selecting only [FACTOR] values known to use minimum bit resolution?

Not being a math wizard how do we find usefull floating point values with minimum rounding loss and with minimum fuzz? Being simple we decided to use brute force and made a script to loop while doing adjustments to -v [FACTOR] and read out the resulting bit-depth

SoX stats

sox -v [FACTOR] [file] -n stats
sox -v.562341 '03 -*' -n stats

The stats command will output somehing like this for a 16-bit audio file when -v paramter are used in front of the filename:

             Overall     Left      Right
DC offset  -0.000009 -0.000009 -0.000009
Min level  -0.561483 -0.561483 -0.561466
Max level   0.561483  0.561466  0.561483
Pk lev dB      -5.01     -5.01     -5.01
RMS lev dB    -18.47    -18.54    -18.40
RMS Pk dB     -13.88    -13.90    -13.88
RMS Tr dB       -inf      -inf      -inf
Crest factor       -      4.75      4.67
Flat factor     0.00      0.00      0.00
Pk count        3.50         5         2
Bit-depth      32/32     32/32     32/32
Num samples    10.9M
Length s     246.267
Scale max   1.000000
Window s       0.050
For this discussion the line containing Bit-depth are the interesting part. Here vi see -v.562341 will need 32 bit-depth for the resample. While -v 1 no change will return 16 bit near all nine hundred fifty million possibilities will need more than 24 bit to hold the resample with adjustet volume. Most of them 31 to 32 bits.

The challenge

Leedh Processor Volume have suggested a list of effective attenuation values giving least amount of bit used to quantize volume control coefficients. Calculating voltage factor values from these attenuation values give the following chart.

   attenat  |  factor
   -1.16 dB   0.874984
   -1.80 dB   0.812831
   -3.25 dB   0.687860
   -4.08 dB   0.625173
   -5.00 dB   0.562341
   -6.02 dB   0.5

SoX use 32 bit floating point as input for its -v [FACTOR] parameter. As it turns out presission of 2 decimals are not enought to do this voltage calculation. With the exception of .5 value all of the calculated [FACTOR] values need 32 bit. Later tests would also reveal that the [FACTOR] value accept presission of 9.5 decimals.

-v [FACTOR] of 0.5 will shift data one bit to the right giving -6.0206 dB with no rounding. Half of .5 = .25 will shift two bits with -12.012 attenuation. 3 bits -18 dB and so on. i.e. division by a power of 2 (X2), or in computer programming this is arithmetic right shift.
A -v factor value of 0.9715 could also be thought of as 97.15% of 100% (-v 1 no attenuation).

SoX equivalent attenuation table for Lossless Lessloss Digital Volume control

Attenuation dB [FACTOR] Bit-depth
0 1.0000 16
-1.1596 0.8750 19
-1.8032 0.8125 20
-2.4981 0.7500 18
-3.2538 0.6875 20
-4.0825 0.6250 19
-4.9971 0.5625 19
-6.0206 0.5000 16
The full attenuation table is then built by dividing each [FACTOR] value by the power of two over and over again. See Bash script elsewhere on this page.

While we learned SoX's -v [FACTOR] have a resolution of 9.5 decimals the first four digits seem to be the sweetspot where we find the minimum use of bits.
With SoX adjusting regain levels to 0 dBFS will in many, possible most cases introduce clipping. Experimenting with levels show targeting factor value at 0.999 rather than 1.0 will eliminate clipped samples.
Using SoX to test for intersample overs.
When upsampling we learned by input a 44.1/16 file of white noise normalized to 0 dBFS with no clipping that all sample rates 2x, 4x, 8x and 16x (705600 Hz) ned to have volume reduced by a factor of 0.5552 (-5.14 dB) to avoid clipping.

sox -v .5552 'filename' -r 705600 -n

While the number of unique peak count, and number of clipped samples increase with higher resampling rates the attenuation level -5.14 dB stay identical for all sample rates.

All tests in this presentation are done with Debian on Linux subsystem for Windows 10

Build 0 to -96 dB attenuation list from 0 to -6 dB factor list

This bash script will build -96 dB attenuation list from array of [FACTOR] values and verify the result using sox stats

#!/bin/bash
# dependencies sox (Sound eXchange), bc (Basic Calculator), a short audio file
# generate attenuation list down to -96 dB from array of factors valid for 0 to -6 dB
# v.0.0.3 - 12 april 2021
#
audio='noise.wav'
function mySoX(){
	# function take [FACTOR] as parameter $1
	local tmp=$(sox -v $1 "$audio" -n --multi-threaded --buffer 32768 stats 2>&1 | grep -i "Bit-depth" | sed 's/[^0-9.]*//g')
	local bitd=$(head <<< "$tmp")
	local bitdepth="${bitd:0:2}"
	local b="${bitd:0:2}/${bitd:2:2} ${bitd:4:2}/${bitd:6:2} ${bitd:8:2}/${bitd:10:2}"
	# calculate decibel: dB = 20 × log10(FACTOR)  
	local mydB=$(echo "scale=4;20*l($1)/l(10)" | bc -l)
	echo "$1   $mydB dB   Bit-depth: $bitdepth  [$b]"
}

function attenuationlist(){
	factors=$1 # this only work if single value are passed, else the 'factors' array should be used
	loops=$((${#factors[*]} * 15))
	orgsize=${#factors[@]}
	echo "Preparing $(("$loops" + orgsize)) attenuation values"

	# ensure we have 10 decimals for visual pleasure
	for ((i = 0 ; i < ${#factors[@]} ; i++ ))
	do
		# divide by one tricked scale to work for us here
		factors[$i]=$(bc <<< "scale=10; ${factors[$i]}/1")
	done

	for ((i = 0 ; i < $loops ; i++ ))
	do
		newstart=${factors[$i]}
		# -6dB step
		#tmp=$(echo "scale=10;$newstart/2" | bc -l)
		tmp=$(bc <<< "scale=10;$newstart/2")
		factors[${#factors[*]}]="$tmp"
		printf "$tmp $i/$(($loops-$i)) \r"            # progressbar
	done
	
	echo "Verifying ${#factors[*]} attenuation values using SoX"
	for factor in "${factors[@]}"
	do
		mySoX $factor
	done
	echo "
${#factors[*]} attenuation factors:
${factors[@]}
"
}
# uncomment to test one of these arrays
# 7 decimals, 64 factors give total of 1040 factors for 0 to -96dB
#factors=( 1 .9921875 .9843750 .9765625 .9687500 .9609375 .9531250 .9453125 .9375000 .9296875 .9218750 .9140625 .9062500 .8984375 .8906250 .8828125 .8750000 .8671875 .8593750 .8515625 .8437500 .8359375 .8281250 .8203125 .8125000 .8046875 .7968750 .7890625 .7812500 .7734375 .7656250 .7578125 .7500000 .7421875 .7343750 .7265625 .7187500 .7109375 .7031250 .6953125 .6875000 .6796875 .6718750 .6640625 .6562500 .6484375 .6406250 .6328125 .6250000 .6171875 .6093750 .6015625 .5937500 .5859375 .5781250 .5703125 .5625000 .5546875 .5468750 .5390625 .5312500 .5234375 .5156250 .5078125 .5000000)
# 6 decimals, 34 factors give total of 528 factors for 0 to -96dB 
factors=(1.000000 .984375 .968750 .953125 .937500 .921875 .906250 .890625 .875000 .859375 .843750 .828125 .812500 .796875 .781250 .765625 .750000 .734375 .718750 .703125 .687500 .671875 .656250 .640625 .625000 .609375 .593750 .578125 .562500 .546875 .531250 .515625 .500000)
# 4 decimals, 8 factors give total of 128 factors for 0 to -96dB 
#factors=( 1.000 .8750 .8125 .7500 .6875 .6250 .5625 .5 )

#factors=(.8750) 

attenuationlist ${factors[@]} # -v [FACTOR] single value can also be used

#attenuationlist .8750

exit 0

This is the complete 128 step attenuation list generated from these eight factors:

1.0000 .8750 .8125 .7500 .6875 .6250 .5625 .5000

1.0000000000        0 dB   Bit-depth: 16  [16/16 16/16 16/16]
.8750000000   -1.1596 dB   Bit-depth: 19  [19/19 19/19 19/19]
.8125000000   -1.8032 dB   Bit-depth: 20  [20/20 20/20 20/20]
.7500000000   -2.4981 dB   Bit-depth: 18  [18/18 18/18 18/18]
.6875000000   -3.2538 dB   Bit-depth: 20  [20/20 20/20 20/20]
.6250000000   -4.0825 dB   Bit-depth: 19  [19/19 19/19 19/19]
.5625000000   -4.9971 dB   Bit-depth: 19  [19/20 19/20 19/20]
.5000000000   -6.0204 dB   Bit-depth: 16  [16/17 16/17 16/17]
.5000000000   -6.0204 dB   Bit-depth: 16  [16/17 16/17 16/17]
.4375000000   -7.1800 dB   Bit-depth: 19  [19/20 19/20 19/20]
.4062500000   -7.8236 dB   Bit-depth: 20  [20/21 20/21 20/21]
.3750000000   -8.5194 dB   Bit-depth: 18  [18/19 18/19 18/19]
.3437500000   -9.2751 dB   Bit-depth: 20  [20/21 20/21 20/21]
.3125000000   -10.1029 dB   Bit-depth: 19  [19/20 19/20 19/20]
.2812500000   -11.0184 dB   Bit-depth: 19  [19/21 19/21 19/21]
.2500000000   -12.0408 dB   Bit-depth: 16  [16/18 16/18 16/18]
.2500000000   -12.0408 dB   Bit-depth: 16  [16/18 16/18 16/18]
.2187500000   -13.2013 dB   Bit-depth: 19  [19/21 19/21 19/21]
.2031250000   -13.8449 dB   Bit-depth: 20  [20/22 20/22 20/22]
.1875000000   -14.5398 dB   Bit-depth: 18  [18/20 18/20 18/20]
.1718750000   -15.2955 dB   Bit-depth: 20  [20/22 20/22 20/22]
.1562500000   -16.1233 dB   Bit-depth: 19  [19/21 19/21 19/21]
.1406250000   -17.0388 dB   Bit-depth: 19  [19/22 19/22 19/22]
.1250000000   -18.0621 dB   Bit-depth: 16  [16/19 16/19 16/19]
.1250000000   -18.0621 dB   Bit-depth: 16  [16/19 16/19 16/19]
.1093750000   -19.2217 dB   Bit-depth: 19  [19/22 19/22 19/22]
.1015625000   -19.8653 dB   Bit-depth: 20  [20/23 20/23 20/23]
.0937500000   -20.5611 dB   Bit-depth: 18  [18/21 18/21 18/21]
.0859375000   -21.3168 dB   Bit-depth: 20  [20/23 20/23 20/23]
.0781250000   -22.1446 dB   Bit-depth: 19  [19/22 19/22 19/22]
.0703125000   -23.0601 dB   Bit-depth: 19  [19/23 19/23 19/23]
.0625000000   -24.0825 dB   Bit-depth: 16  [16/20 16/20 16/20]
.0625000000   -24.0825 dB   Bit-depth: 16  [16/20 16/20 16/20]
.0546875000   -25.2429 dB   Bit-depth: 19  [19/23 19/23 19/23]
.0507812500   -25.8866 dB   Bit-depth: 20  [20/24 20/24 20/24]
.0468750000   -26.5815 dB   Bit-depth: 18  [18/22 18/22 18/22]
.0429687500   -27.3372 dB   Bit-depth: 20  [20/24 20/24 20/24]
.0390625000   -28.1650 dB   Bit-depth: 19  [19/23 19/23 19/23]
.0351562500   -29.0805 dB   Bit-depth: 19  [19/24 19/24 19/24]
.0312500000   -30.1038 dB   Bit-depth: 16  [16/21 16/21 16/21]
.0312500000   -30.1038 dB   Bit-depth: 16  [16/21 16/21 16/21]
.0273437500   -31.2634 dB   Bit-depth: 19  [19/24 19/24 19/24]
.0253906250   -31.9070 dB   Bit-depth: 20  [20/25 20/25 20/25]
.0234375000   -32.6028 dB   Bit-depth: 18  [18/23 18/23 18/23]
.0214843750   -33.3585 dB   Bit-depth: 20  [20/25 20/25 20/25]
.0195312500   -34.1863 dB   Bit-depth: 19  [19/24 19/24 19/24]
.0175781250   -35.1018 dB   Bit-depth: 19  [19/25 19/25 19/25]
.0156250000   -36.1242 dB   Bit-depth: 16  [16/22 16/22 16/22]
.0156250000   -36.1242 dB   Bit-depth: 16  [16/22 16/22 16/22]
.0136718750   -37.2846 dB   Bit-depth: 19  [19/25 19/25 19/25]
.0126953125   -37.9283 dB   Bit-depth: 20  [20/26 20/26 20/26]
.0117187500   -38.6232 dB   Bit-depth: 18  [18/24 18/24 18/24]
.0107421875   -39.3789 dB   Bit-depth: 20  [20/26 20/26 20/26]
.0097656250   -40.2067 dB   Bit-depth: 19  [19/25 19/25 19/25]
.0087890625   -41.1222 dB   Bit-depth: 19  [19/26 19/26 19/26]
.0078125000   -42.1454 dB   Bit-depth: 16  [16/23 16/23 16/23]
.0078125000   -42.1454 dB   Bit-depth: 16  [16/23 16/23 16/23]
.0068359375   -43.3051 dB   Bit-depth: 19  [19/26 19/26 19/26]
.0063476562   -43.9487 dB   Bit-depth: 20  [20/27 20/27 20/27]
.0058593750   -44.6445 dB   Bit-depth: 18  [18/25 18/25 18/25]
.0053710937   -45.4002 dB   Bit-depth: 20  [20/27 20/27 20/27]
.0048828125   -46.2280 dB   Bit-depth: 19  [19/26 19/26 19/26]
.0043945312   -47.1426 dB   Bit-depth: 19  [19/27 19/27 19/27]
.0039062500   -48.1659 dB   Bit-depth: 16  [16/24 16/24 16/24]
.0039062500   -48.1659 dB   Bit-depth: 16  [16/24 16/24 16/24]
.0034179687   -49.3263 dB   Bit-depth: 19  [19/27 19/27 19/27]
.0031738281   -49.9700 dB   Bit-depth: 20  [20/28 20/28 20/28]
.0029296875   -50.6649 dB   Bit-depth: 18  [18/26 18/26 18/26]
.0026855468   -51.4206 dB   Bit-depth: 20  [20/28 20/28 20/28]
.0024414062   -52.2484 dB   Bit-depth: 19  [19/27 19/27 19/27]
.0021972656   -53.1639 dB   Bit-depth: 19  [19/28 19/28 19/28]
.0019531250   -54.1871 dB   Bit-depth: 16  [16/25 16/25 16/25]
.0019531250   -54.1871 dB   Bit-depth: 16  [16/25 16/25 16/25]
.0017089843   -55.3467 dB   Bit-depth: 19  [19/28 19/28 19/28]
.0015869140   -55.9904 dB   Bit-depth: 20  [20/29 20/29 20/29]
.0014648437   -56.6862 dB   Bit-depth: 18  [18/27 18/27 18/27]
.0013427734   -57.4419 dB   Bit-depth: 20  [20/29 20/29 20/29]
.0012207031   -58.2697 dB   Bit-depth: 19  [19/28 19/28 19/28]
.0010986328   -59.1843 dB   Bit-depth: 19  [19/29 19/29 19/29]
.0009765625   -60.2076 dB   Bit-depth: 16  [16/26 16/26 16/26]
.0009765625   -60.2076 dB   Bit-depth: 16  [16/26 16/26 16/26]
.0008544921   -61.3680 dB   Bit-depth: 19  [19/29 19/29 19/29]
.0007934570   -62.0117 dB   Bit-depth: 20  [20/30 20/30 20/30]
.0007324218   -62.7066 dB   Bit-depth: 18  [18/28 18/28 18/28]
.0006713867   -63.4623 dB   Bit-depth: 20  [20/30 20/30 20/30]
.0006103515   -64.2901 dB   Bit-depth: 19  [19/29 19/29 19/29]
.0005493164   -65.2056 dB   Bit-depth: 19  [19/30 19/30 19/30]
.0004882812   -66.2288 dB   Bit-depth: 16  [16/27 16/27 16/27]
.0004882812   -66.2288 dB   Bit-depth: 16  [16/27 16/27 16/27]
.0004272460   -67.3884 dB   Bit-depth: 19  [19/30 19/30 19/30]
.0003967285   -68.0321 dB   Bit-depth: 20  [20/31 20/31 20/31]
.0003662109   -68.7279 dB   Bit-depth: 18  [18/29 18/29 18/29]
.0003356933   -69.4836 dB   Bit-depth: 20  [20/31 20/31 20/31]
.0003051757   -70.3114 dB   Bit-depth: 19  [19/30 19/30 19/30]
.0002746582   -71.2260 dB   Bit-depth: 19  [19/31 19/31 19/31]
.0002441406   -72.2492 dB   Bit-depth: 16  [16/28 16/28 16/28]
.0002441406   -72.2492 dB   Bit-depth: 16  [16/28 16/28 16/28]
.0002136230   -73.4089 dB   Bit-depth: 19  [19/31 19/31 19/31]
.0001983642   -74.0534 dB   Bit-depth: 20  [20/32 20/32 20/32]
.0001831054   -74.7483 dB   Bit-depth: 18  [18/30 18/30 18/30]
.0001678466   -75.5040 dB   Bit-depth: 20  [20/32 20/32 20/32]
.0001525878   -76.3318 dB   Bit-depth: 19  [19/31 19/31 19/31]
.0001373291   -77.2473 dB   Bit-depth: 19  [19/32 19/32 19/32]
.0001220703   -78.2705 dB   Bit-depth: 16  [16/29 16/29 16/29]
.0001220703   -78.2705 dB   Bit-depth: 16  [16/29 16/29 16/29]
.0001068115   -79.4301 dB   Bit-depth: 19  [19/32 19/32 19/32]
.0000991821   -80.0738 dB   Bit-depth: 19  [19/32 19/32 19/32]
.0000915527   -80.7687 dB   Bit-depth: 18  [18/31 18/31 18/31]
.0000839233   -81.5252 dB   Bit-depth: 19  [19/32 19/32 19/32]
.0000762939   -82.3530 dB   Bit-depth: 19  [19/32 19/32 19/32]
.0000686645   -83.2677 dB   Bit-depth: 18  [18/32 18/32 18/32]
.0000610351   -84.2909 dB   Bit-depth: 16  [16/30 16/30 16/30]
.0000610351   -84.2909 dB   Bit-depth: 16  [16/30 16/30 16/30]
.0000534057   -85.4505 dB   Bit-depth: 18  [18/32 18/32 18/32]
.0000495910   -86.0951 dB   Bit-depth: 18  [18/32 18/32 18/32]
.0000457763   -86.7900 dB   Bit-depth: 18  [18/32 18/32 18/32]
.0000419616   -87.5457 dB   Bit-depth: 18  [18/32 18/32 18/32]
.0000381469   -88.3735 dB   Bit-depth: 18  [18/32 18/32 18/32]
.0000343322   -89.2890 dB   Bit-depth: 17  [17/32 17/32 17/32]
.0000305175   -90.3122 dB   Bit-depth: 16  [16/31 16/31 16/31]
.0000305175   -90.3122 dB   Bit-depth: 16  [16/31 16/31 16/31]
.0000267028   -91.4718 dB   Bit-depth: 17  [17/32 17/32 17/32]
.0000247955   -92.1155 dB   Bit-depth: 17  [17/32 17/32 17/32]
.0000228881   -92.8104 dB   Bit-depth: 17  [17/32 17/32 17/32]
.0000209808   -93.5669 dB   Bit-depth: 17  [17/32 17/32 17/32]
.0000190734   -94.3947 dB   Bit-depth: 17  [17/32 17/32 17/32]
.0000171661   -95.3094 dB   Bit-depth: 16  [16/32 16/32 16/32]
.0000152587   -96.3326 dB   Bit-depth: 16  [16/32 16/32 16/32]

When better resolution are needed then these 64 factors using seven decimals precision might be helpful. Most of them fall in the 22-23 bit range. Possible better to implement the full -96 dB table in program code as the full list will generate some 1040 factors.

List below contain only the 0 to -6 dB factors to visualize the resolution. Use the script if you need to see the complete output.

1.0000000 .9921875 .9843750 .9765625 .9687500 .9609375 .9531250 .9453125 .9375000 .9296875 .9218750 .9140625 .9062500 .8984375 .8906250 .8828125 .8750000 .8671875 .8593750 .8515625 .8437500 .8359375 .8281250 .8203125 .8125000 .8046875 .7968750 .7890625 .7812500 .7734375 .7656250 .7578125 .7500000 .7421875 .7343750 .7265625 .7187500 .7109375 .7031250 .6953125 .6875000 .6796875 .6718750 .6640625 .6562500 .6484375 .6406250 .6328125 .6250000 .6171875 .6093750 .6015625 .5937500 .5859375 .5781250 .5703125 .5625000 .5546875 .5468750 .5390625 .5312500 .5234375 .5156250 .5078125 .5000000

1.0000000000   0 dB   Bit-depth: 16  [16/16 16/16 16/16]
.9921875000   -.0677 dB   Bit-depth: 23  [23/23 23/23 23/23]
.9843750000   -.1363 dB   Bit-depth: 22  [22/22 22/22 22/22]
.9765625000   -.2058 dB   Bit-depth: 23  [23/23 23/23 23/23]
.9687500000   -.2753 dB   Bit-depth: 21  [21/21 21/21 21/21]
.9609375000   -.3457 dB   Bit-depth: 23  [23/23 23/23 23/23]
.9531250000   -.4169 dB   Bit-depth: 22  [22/22 22/22 22/22]
.9453125000   -.4881 dB   Bit-depth: 23  [23/23 23/23 23/23]
.9375000000   -.5602 dB   Bit-depth: 20  [20/20 20/20 20/20]
.9296875000   -.6332 dB   Bit-depth: 23  [23/23 23/23 23/23]
.9218750000   -.7061 dB   Bit-depth: 22  [22/22 22/22 22/22]
.9140625000   -.7800 dB   Bit-depth: 23  [23/23 23/23 23/23]
.9062500000   -.8547 dB   Bit-depth: 21  [21/21 21/21 21/21]
.8984375000   -.9294 dB   Bit-depth: 23  [23/23 23/23 23/23]
.8906250000   -1.0058 dB   Bit-depth: 22  [22/22 22/22 22/22]
.8828125000   -1.0823 dB   Bit-depth: 23  [23/23 23/23 23/23]
.8750000000   -1.1596 dB   Bit-depth: 19  [19/19 19/19 19/19]
.8671875000   -1.2377 dB   Bit-depth: 23  [23/23 23/23 23/23]
.8593750000   -1.3159 dB   Bit-depth: 22  [22/22 22/22 22/22]
.8515625000   -1.3950 dB   Bit-depth: 23  [23/23 23/23 23/23]
.8437500000   -1.4749 dB   Bit-depth: 21  [21/21 21/21 21/21]
.8359375000   -1.5565 dB   Bit-depth: 23  [23/23 23/23 23/23]
.8281250000   -1.6373 dB   Bit-depth: 22  [22/22 22/22 22/22]
.8203125000   -1.7198 dB   Bit-depth: 23  [23/23 23/23 23/23]
.8125000000   -1.8032 dB   Bit-depth: 20  [20/20 20/20 20/20]
.8046875000   -1.8875 dB   Bit-depth: 23  [23/23 23/23 23/23]
.7968750000   -1.9717 dB   Bit-depth: 22  [22/22 22/22 22/22]
.7890625000   -2.0577 dB   Bit-depth: 23  [23/23 23/23 23/23]
.7812500000   -2.1437 dB   Bit-depth: 21  [21/21 21/21 21/21]
.7734375000   -2.2314 dB   Bit-depth: 23  [23/23 23/23 23/23]
.7656250000   -2.3192 dB   Bit-depth: 22  [22/22 22/22 22/22]
.7578125000   -2.4086 dB   Bit-depth: 23  [23/23 23/23 23/23]
.7500000000   -2.4981 dB   Bit-depth: 18  [18/18 18/18 18/18]
.7421875000   -2.5893 dB   Bit-depth: 23  [23/23 23/23 23/23]
.7343750000   -2.6814 dB   Bit-depth: 22  [22/22 22/22 22/22]
.7265625000   -2.7743 dB   Bit-depth: 23  [23/23 23/23 23/23]
.7187500000   -2.8681 dB   Bit-depth: 21  [21/21 21/21 21/21]
.7109375000   -2.9628 dB   Bit-depth: 23  [23/23 23/23 23/23]
.7031250000   -3.0592 dB   Bit-depth: 22  [22/22 22/22 22/22]
.6953125000   -3.1557 dB   Bit-depth: 23  [23/23 23/23 23/23]
.6875000000   -3.2538 dB   Bit-depth: 20  [20/20 20/20 20/20]
.6796875000   -3.3537 dB   Bit-depth: 23  [23/23 23/23 23/23]
.6718750000   -3.4536 dB   Bit-depth: 22  [22/22 22/22 22/22]
.6640625000   -3.5552 dB   Bit-depth: 23  [23/23 23/23 23/23]
.6562500000   -3.6586 dB   Bit-depth: 21  [21/21 21/21 21/21]
.6484375000   -3.7619 dB   Bit-depth: 23  [23/23 23/23 23/23]
.6406250000   -3.8679 dB   Bit-depth: 22  [22/22 22/22 22/22]
.6328125000   -3.9739 dB   Bit-depth: 23  [23/23 23/23 23/23]
.6250000000   -4.0825 dB   Bit-depth: 19  [19/19 19/19 19/19]
.6171875000   -4.1910 dB   Bit-depth: 22  [22/23 22/23 22/23]
.6093750000   -4.3022 dB   Bit-depth: 21  [21/22 21/22 21/22]
.6015625000   -4.4143 dB   Bit-depth: 22  [22/23 22/23 22/23]
.5937500000   -4.5272 dB   Bit-depth: 20  [20/21 20/21 20/21]
.5859375000   -4.6427 dB   Bit-depth: 22  [22/23 22/23 22/23]
.5781250000   -4.7591 dB   Bit-depth: 21  [21/22 21/22 21/22]
.5703125000   -4.8773 dB   Bit-depth: 22  [22/23 22/23 22/23]
.5625000000   -4.9971 dB   Bit-depth: 19  [19/20 19/20 19/20]
.5546875000   -5.1187 dB   Bit-depth: 22  [22/23 22/23 22/23]
.5468750000   -5.2421 dB   Bit-depth: 21  [21/22 21/22 21/22]
.5390625000   -5.3672 dB   Bit-depth: 22  [22/23 22/23 22/23]
.5312500000   -5.4940 dB   Bit-depth: 20  [20/21 20/21 20/21]
.5234375000   -5.6225 dB   Bit-depth: 22  [22/23 22/23 22/23]
.5156250000   -5.7528 dB   Bit-depth: 21  [21/22 21/22 21/22]
.5078125000   -5.8857 dB   Bit-depth: 22  [22/23 22/23 22/23]
.5000000000   -6.0204 dB   Bit-depth: 16  [16/17 16/17 16/17]

6 decimals give these 34 factors with total of 528 factors for 0 to -96 dB.

List below contain only the 0 to -6 dB factors to visualize the resolution. Use the script if you need to see the complete output.

1.000000 .984375 .968750 .953125 .937500 .921875 .906250 .890625 .875000 .859375 .843750 .828125 .812500 .796875 .781250 .765625 .750000 .734375 .718750 .703125 .687500 .671875 .656250 .640625 .625000 .609375 .593750 .578125 .562500 .546875 .531250 .515625 .500000

1.0000000000   0 dB   Bit-depth: 16  [16/16 16/16 16/16]
.9843750000   -.1363 dB   Bit-depth: 22  [22/22 22/22 22/22]
.9687500000   -.2753 dB   Bit-depth: 21  [21/21 21/21 21/21]
.9531250000   -.4169 dB   Bit-depth: 22  [22/22 22/22 22/22]
.9375000000   -.5602 dB   Bit-depth: 20  [20/20 20/20 20/20]
.9218750000   -.7061 dB   Bit-depth: 22  [22/22 22/22 22/22]
.9062500000   -.8547 dB   Bit-depth: 21  [21/21 21/21 21/21]
.8906250000   -1.0058 dB   Bit-depth: 22  [22/22 22/22 22/22]
.8750000000   -1.1596 dB   Bit-depth: 19  [19/19 19/19 19/19]
.8593750000   -1.3159 dB   Bit-depth: 22  [22/22 22/22 22/22]
.8437500000   -1.4749 dB   Bit-depth: 21  [21/21 21/21 21/21]
.8281250000   -1.6373 dB   Bit-depth: 22  [22/22 22/22 22/22]
.8125000000   -1.8032 dB   Bit-depth: 20  [20/20 20/20 20/20]
.7968750000   -1.9717 dB   Bit-depth: 22  [22/22 22/22 22/22]
.7812500000   -2.1437 dB   Bit-depth: 21  [21/21 21/21 21/21]
.7656250000   -2.3192 dB   Bit-depth: 22  [22/22 22/22 22/22]
.7500000000   -2.4981 dB   Bit-depth: 18  [18/18 18/18 18/18]
.7343750000   -2.6814 dB   Bit-depth: 22  [22/22 22/22 22/22]
.7187500000   -2.8681 dB   Bit-depth: 21  [21/21 21/21 21/21]
.7031250000   -3.0592 dB   Bit-depth: 22  [22/22 22/22 22/22]
.6875000000   -3.2538 dB   Bit-depth: 20  [20/20 20/20 20/20]
.6718750000   -3.4536 dB   Bit-depth: 22  [22/22 22/22 22/22]
.6562500000   -3.6586 dB   Bit-depth: 21  [21/21 21/21 21/21]
.6406250000   -3.8679 dB   Bit-depth: 22  [22/22 22/22 22/22]
.6250000000   -4.0825 dB   Bit-depth: 19  [19/19 19/19 19/19]
.6093750000   -4.3022 dB   Bit-depth: 21  [21/22 21/22 21/22]
.5937500000   -4.5272 dB   Bit-depth: 20  [20/21 20/21 20/21]
.5781250000   -4.7591 dB   Bit-depth: 21  [21/22 21/22 21/22]
.5625000000   -4.9971 dB   Bit-depth: 19  [19/20 19/20 19/20]
.5468750000   -5.2421 dB   Bit-depth: 21  [21/22 21/22 21/22]
.5312500000   -5.4940 dB   Bit-depth: 20  [20/21 20/21 20/21]
.5156250000   -5.7528 dB   Bit-depth: 21  [21/22 21/22 21/22]
.5000000000   -6.0204 dB   Bit-depth: 16  [16/17 16/17 16/17]
.5000000000   -6.0204 dB   Bit-depth: 16  [16/17 16/17 16/17]

Comments

Your comment are welcome

Name:


Email: optional, will be hidden


Comment: