This idea took form after reading about Mr. Milot's Leedh Processing Volume. A lossless volume control that work on the simple prinsipple of
Minimise the number of bits used to quantize volume control coefficients so that information loss is minimized at truncation stage. In other words, it trades volume control coefficients precision against information loss minimization.
Would it be possible to implement a similar approach using Sound eXchange (SoX)?
Rather then input volume values on a whim have volume adjustment done from a lookup table selecting only [FACTOR] values known to use minimum bit resolution?
Now, there is one big elephant in the room we probably should discuss. The caveat with this lossless volume control, — including the Leedh Processesing are with practical implementation. Where the volume control often are implementet between one or two re-samplers. The audio signal are typically assigned a reconstruction filter before upsampling and another one when bringing the signal back to it original format if needed. Meaning the loss of mathematical accuracy for the volume control would be negligibly in the scheme of things. The filter calculation probably will have more loss and therefore make it very hard for the listener to tell if there's a difference if only the amplitude adjustment are lossless.
Also this will only function with 16-bit audio. 24-bit hi-res audio will need at least 28-bit for any useful implementation.
pow(10, -6.0205999132796239042747778944898 / 20) = 0.5
log(0.25) * 20 = -12.04119982655924780854955578898 dB
sox -v [FACTOR] [file] -n stats
sox -v.562341 '03 -*' -n stats
The stats command will output somehing like this for a 16-bit audio file when -v paramter are used in front of the filename:
Overall Left Right
DC offset -0.000009 -0.000009 -0.000009
Min level -0.561483 -0.561483 -0.561466
Max level 0.561483 0.561466 0.561483
Pk lev dB -5.01 -5.01 -5.01
RMS lev dB -18.47 -18.54 -18.40
RMS Pk dB -13.88 -13.90 -13.88
RMS Tr dB -inf -inf -inf
Crest factor - 4.75 4.67
Flat factor 0.00 0.00 0.00
Pk count 3.50 5 2
Bit-depth 32/32 32/32 32/32
Num samples 10.9M
Length s 246.267
Scale max 1.000000
Window s 0.050
For this discussion the line containing Bit-depth are the interesting part. Here vi see -v.562341 will need 32 bit-depth for the resample. While -v 1 no change will return 16 bit near all the other nine hundred fifty million possibilities will need more than 24 bit to hold the resample with adjustet volume. Most of them 31 to 32 bits.
attenat | factor -1.16 dB 0.874984 -1.80 dB 0.812831 -3.25 dB 0.687860 -4.08 dB 0.625173 -5.00 dB 0.562341 -6.02 dB 0.5
SoX use 32 bit floating point as input for its -v [FACTOR] parameter. As it turns out presission of 2 decimals are not enought to do this voltage calculation. With the exception of .5 value all of the calculated [FACTOR] values need 32 bit. Later tests would also reveal that the [FACTOR] value accept presission of 9.5 decimals.
| Attenuation dB | [FACTOR] | Bit-depth |
|---|---|---|
| 0 | 1.0000 | 16 |
| -1.1596 | 0.8750 | 19 |
| -1.8032 | 0.8125 | 20 |
| -2.4981 | 0.7500 | 18 |
| -3.2538 | 0.6875 | 20 |
| -4.0825 | 0.6250 | 19 |
| -4.9971 | 0.5625 | 19 |
| -6.0206 | 0.5000 | 16 |
sox -v .5552 'filename' -r 705600 -n
While the number of unique peak count, and number of clipped samples increase with higher resampling rates the attenuation level -5.14 dB stay identical for all sample rates.
This bash script will build -96 dB attenuation list from array of [FACTOR] values and verify the result using sox stats
#!/bin/bash
# dependencies sox (Sound eXchange), bc (Basic Calculator), a short audio file
# generate attenuation list down to -96 dB from array of factors valid for 0 to -6 dB
# v.0.0.3 - 12 april 2021
#
audio='noise.wav'
if [ ! -f "$audio" ]; then
echo "Audio file '$audio' does not exist"
exit 1
fi
function mySoX(){
# function take [FACTOR] as parameter $1
local tmp=$(sox -v $1 "$audio" -n --multi-threaded --buffer 32768 stats 2>&1 | grep -i "Bit-depth" | sed 's/[^0-9.]*//g')
local bitd=$(head <<< "$tmp")
local bitdepth="${bitd:0:2}"
local b="${bitd:0:2}/${bitd:2:2} ${bitd:4:2}/${bitd:6:2} ${bitd:8:2}/${bitd:10:2}"
# calculate decibel: dB = 20 × log10(FACTOR)
local mydB=$(echo "scale=4;20*l($1)/l(10)" | bc -l)
echo "$1 $mydB dB Bit-depth: $bitdepth [$b]"
}
function attenuationlist(){
factors=$1 # this only work if single value are passed, else the 'factors' array should be used
loops=$((${#factors[*]} * 15))
orgsize=${#factors[@]}
echo "Preparing $(("$loops" + orgsize)) attenuation values"
# ensure we have 10 decimals for visual pleasure
for ((i = 0 ; i < ${#factors[@]} ; i++ ))
do
# divide by one tricked scale to work for us here
factors[$i]=$(bc <<< "scale=10; ${factors[$i]}/1")
done
for ((i = 0 ; i < $loops ; i++ ))
do
newstart=${factors[$i]}
# -6dB step
#tmp=$(echo "scale=10;$newstart/2" | bc -l)
tmp=$(bc <<< "scale=10;$newstart/2")
factors[${#factors[*]}]="$tmp"
printf "$tmp $i/$(($loops-$i)) \r" # progressbar
done
echo "Verifying ${#factors[*]} attenuation values using SoX"
for factor in "${factors[@]}"
do
mySoX $factor
done
echo "
${#factors[*]} attenuation factors:
${factors[@]}
"
}
# uncomment to test one of these arrays
# 7 decimals, 64 factors give total of 1040 factors for 0 to -96dB
#factors=( 1 .9921875 .9843750 .9765625 .9687500 .9609375 .9531250 .9453125 .9375000 .9296875 .9218750 .9140625 .9062500 .8984375 .8906250 .8828125 .8750000 .8671875 .8593750 .8515625 .8437500 .8359375 .8281250 .8203125 .8125000 .8046875 .7968750 .7890625 .7812500 .7734375 .7656250 .7578125 .7500000 .7421875 .7343750 .7265625 .7187500 .7109375 .7031250 .6953125 .6875000 .6796875 .6718750 .6640625 .6562500 .6484375 .6406250 .6328125 .6250000 .6171875 .6093750 .6015625 .5937500 .5859375 .5781250 .5703125 .5625000 .5546875 .5468750 .5390625 .5312500 .5234375 .5156250 .5078125 .5000000)
# 6 decimals, 34 factors give total of 528 factors for 0 to -96dB
factors=(1.000000 .984375 .968750 .953125 .937500 .921875 .906250 .890625 .875000 .859375 .843750 .828125 .812500 .796875 .781250 .765625 .750000 .734375 .718750 .703125 .687500 .671875 .656250 .640625 .625000 .609375 .593750 .578125 .562500 .546875 .531250 .515625 .500000)
# 4 decimals, 8 factors give total of 128 factors for 0 to -96dB
#factors=( 1.000 .8750 .8125 .7500 .6875 .6250 .5625 .5 )
#factors=(.8750)
attenuationlist ${factors[@]} # -v [FACTOR] single value can also be used
#attenuationlist .8750
exit 0
This is the complete 114 step attenuation list generated from these eight factors:
1.0000 .8750 .8125 .7500 .6875 .6250 .5625 .5000
1.0000000000 0 dB Bit-depth: 16 [16/16 16/16 16/16] .8750000000 -1.1596 dB Bit-depth: 19 [19/19 19/19 19/19] .8125000000 -1.8032 dB Bit-depth: 20 [20/20 20/20 20/20] .7500000000 -2.4981 dB Bit-depth: 18 [18/18 18/18 18/18] .6875000000 -3.2538 dB Bit-depth: 20 [20/20 20/20 20/20] .6250000000 -4.0825 dB Bit-depth: 19 [19/19 19/19 19/19] .5625000000 -4.9971 dB Bit-depth: 19 [19/20 19/20 19/20] .5000000000 -6.0204 dB Bit-depth: 16 [16/17 16/17 16/17] .4375000000 -7.1800 dB Bit-depth: 19 [19/20 19/20 19/20] .4062500000 -7.8236 dB Bit-depth: 20 [20/21 20/21 20/21] .3750000000 -8.5194 dB Bit-depth: 18 [18/19 18/19 18/19] .3437500000 -9.2751 dB Bit-depth: 20 [20/21 20/21 20/21] .3125000000 -10.1029 dB Bit-depth: 19 [19/20 19/20 19/20] .2812500000 -11.0184 dB Bit-depth: 19 [19/21 19/21 19/21] .2500000000 -12.0408 dB Bit-depth: 16 [16/18 16/18 16/18] .2187500000 -13.2013 dB Bit-depth: 19 [19/21 19/21 19/21] .2031250000 -13.8449 dB Bit-depth: 20 [20/22 20/22 20/22] .1875000000 -14.5398 dB Bit-depth: 18 [18/20 18/20 18/20] .1718750000 -15.2955 dB Bit-depth: 20 [20/22 20/22 20/22] .1562500000 -16.1233 dB Bit-depth: 19 [19/21 19/21 19/21] .1406250000 -17.0388 dB Bit-depth: 19 [19/22 19/22 19/22] .1250000000 -18.0621 dB Bit-depth: 16 [16/19 16/19 16/19] .1093750000 -19.2217 dB Bit-depth: 19 [19/22 19/22 19/22] .1015625000 -19.8653 dB Bit-depth: 20 [20/23 20/23 20/23] .0937500000 -20.5611 dB Bit-depth: 18 [18/21 18/21 18/21] .0859375000 -21.3168 dB Bit-depth: 20 [20/23 20/23 20/23] .0781250000 -22.1446 dB Bit-depth: 19 [19/22 19/22 19/22] .0703125000 -23.0601 dB Bit-depth: 19 [19/23 19/23 19/23] .0625000000 -24.0825 dB Bit-depth: 16 [16/20 16/20 16/20] .0546875000 -25.2429 dB Bit-depth: 19 [19/23 19/23 19/23] .0507812500 -25.8866 dB Bit-depth: 20 [20/24 20/24 20/24] .0468750000 -26.5815 dB Bit-depth: 18 [18/22 18/22 18/22] .0429687500 -27.3372 dB Bit-depth: 20 [20/24 20/24 20/24] .0390625000 -28.1650 dB Bit-depth: 19 [19/23 19/23 19/23] .0351562500 -29.0805 dB Bit-depth: 19 [19/24 19/24 19/24] .0312500000 -30.1038 dB Bit-depth: 16 [16/21 16/21 16/21] .0273437500 -31.2634 dB Bit-depth: 19 [19/24 19/24 19/24] .0253906250 -31.9070 dB Bit-depth: 20 [20/25 20/25 20/25] .0234375000 -32.6028 dB Bit-depth: 18 [18/23 18/23 18/23] .0214843750 -33.3585 dB Bit-depth: 20 [20/25 20/25 20/25] .0195312500 -34.1863 dB Bit-depth: 19 [19/24 19/24 19/24] .0175781250 -35.1018 dB Bit-depth: 19 [19/25 19/25 19/25] .0156250000 -36.1242 dB Bit-depth: 16 [16/22 16/22 16/22] .0136718750 -37.2846 dB Bit-depth: 19 [19/25 19/25 19/25] .0126953125 -37.9283 dB Bit-depth: 20 [20/26 20/26 20/26] .0117187500 -38.6232 dB Bit-depth: 18 [18/24 18/24 18/24] .0107421875 -39.3789 dB Bit-depth: 20 [20/26 20/26 20/26] .0097656250 -40.2067 dB Bit-depth: 19 [19/25 19/25 19/25] .0087890625 -41.1222 dB Bit-depth: 19 [19/26 19/26 19/26] .0078125000 -42.1454 dB Bit-depth: 16 [16/23 16/23 16/23] .0068359375 -43.3051 dB Bit-depth: 19 [19/26 19/26 19/26] .0063476562 -43.9487 dB Bit-depth: 20 [20/27 20/27 20/27] .0058593750 -44.6445 dB Bit-depth: 18 [18/25 18/25 18/25] .0053710937 -45.4002 dB Bit-depth: 20 [20/27 20/27 20/27] .0048828125 -46.2280 dB Bit-depth: 19 [19/26 19/26 19/26] .0043945312 -47.1426 dB Bit-depth: 19 [19/27 19/27 19/27] .0039062500 -48.1659 dB Bit-depth: 16 [16/24 16/24 16/24] .0034179687 -49.3263 dB Bit-depth: 19 [19/27 19/27 19/27] .0031738281 -49.9700 dB Bit-depth: 20 [20/28 20/28 20/28] .0029296875 -50.6649 dB Bit-depth: 18 [18/26 18/26 18/26] .0026855468 -51.4206 dB Bit-depth: 20 [20/28 20/28 20/28] .0024414062 -52.2484 dB Bit-depth: 19 [19/27 19/27 19/27] .0021972656 -53.1639 dB Bit-depth: 19 [19/28 19/28 19/28] .0019531250 -54.1871 dB Bit-depth: 16 [16/25 16/25 16/25] .0017089843 -55.3467 dB Bit-depth: 19 [19/28 19/28 19/28] .0015869140 -55.9904 dB Bit-depth: 20 [20/29 20/29 20/29] .0014648437 -56.6862 dB Bit-depth: 18 [18/27 18/27 18/27] .0013427734 -57.4419 dB Bit-depth: 20 [20/29 20/29 20/29] .0012207031 -58.2697 dB Bit-depth: 19 [19/28 19/28 19/28] .0010986328 -59.1843 dB Bit-depth: 19 [19/29 19/29 19/29] .0009765625 -60.2076 dB Bit-depth: 16 [16/26 16/26 16/26] .0009765625 -60.2076 dB Bit-depth: 16 [16/26 16/26 16/26] .0008544921 -61.3680 dB Bit-depth: 19 [19/29 19/29 19/29] .0007934570 -62.0117 dB Bit-depth: 20 [20/30 20/30 20/30] .0007324218 -62.7066 dB Bit-depth: 18 [18/28 18/28 18/28] .0006713867 -63.4623 dB Bit-depth: 20 [20/30 20/30 20/30] .0006103515 -64.2901 dB Bit-depth: 19 [19/29 19/29 19/29] .0005493164 -65.2056 dB Bit-depth: 19 [19/30 19/30 19/30] .0004882812 -66.2288 dB Bit-depth: 16 [16/27 16/27 16/27] .0004272460 -67.3884 dB Bit-depth: 19 [19/30 19/30 19/30] .0003967285 -68.0321 dB Bit-depth: 20 [20/31 20/31 20/31] .0003662109 -68.7279 dB Bit-depth: 18 [18/29 18/29 18/29] .0003356933 -69.4836 dB Bit-depth: 20 [20/31 20/31 20/31] .0003051757 -70.3114 dB Bit-depth: 19 [19/30 19/30 19/30] .0002746582 -71.2260 dB Bit-depth: 19 [19/31 19/31 19/31] .0002441406 -72.2492 dB Bit-depth: 16 [16/28 16/28 16/28] .0002136230 -73.4089 dB Bit-depth: 19 [19/31 19/31 19/31] .0001983642 -74.0534 dB Bit-depth: 20 [20/32 20/32 20/32] .0001831054 -74.7483 dB Bit-depth: 18 [18/30 18/30 18/30] .0001678466 -75.5040 dB Bit-depth: 20 [20/32 20/32 20/32] .0001525878 -76.3318 dB Bit-depth: 19 [19/31 19/31 19/31] .0001373291 -77.2473 dB Bit-depth: 19 [19/32 19/32 19/32] .0001220703 -78.2705 dB Bit-depth: 16 [16/29 16/29 16/29] .0001068115 -79.4301 dB Bit-depth: 19 [19/32 19/32 19/32] .0000991821 -80.0738 dB Bit-depth: 19 [19/32 19/32 19/32] .0000915527 -80.7687 dB Bit-depth: 18 [18/31 18/31 18/31] .0000839233 -81.5252 dB Bit-depth: 19 [19/32 19/32 19/32] .0000762939 -82.3530 dB Bit-depth: 19 [19/32 19/32 19/32] .0000686645 -83.2677 dB Bit-depth: 18 [18/32 18/32 18/32] .0000610351 -84.2909 dB Bit-depth: 16 [16/30 16/30 16/30] .0000534057 -85.4505 dB Bit-depth: 18 [18/32 18/32 18/32] .0000495910 -86.0951 dB Bit-depth: 18 [18/32 18/32 18/32] .0000457763 -86.7900 dB Bit-depth: 18 [18/32 18/32 18/32] .0000419616 -87.5457 dB Bit-depth: 18 [18/32 18/32 18/32] .0000381469 -88.3735 dB Bit-depth: 18 [18/32 18/32 18/32] .0000343322 -89.2890 dB Bit-depth: 17 [17/32 17/32 17/32] .0000305175 -90.3122 dB Bit-depth: 16 [16/31 16/31 16/31] .0000267028 -91.4718 dB Bit-depth: 17 [17/32 17/32 17/32] .0000247955 -92.1155 dB Bit-depth: 17 [17/32 17/32 17/32] .0000228881 -92.8104 dB Bit-depth: 17 [17/32 17/32 17/32] .0000209808 -93.5669 dB Bit-depth: 17 [17/32 17/32 17/32] .0000190734 -94.3947 dB Bit-depth: 17 [17/32 17/32 17/32] .0000171661 -95.3094 dB Bit-depth: 16 [16/32 16/32 16/32] .0000152587 -96.3326 dB Bit-depth: 16 [16/32 16/32 16/32]
When better resolution are needed then these 64 factors using seven decimals precision might be helpful. Most of them fall in the 22-23 bit range. Possible better to implement the full -96 dB table in program code as the full list will generate some 1040 factors.
List below contain only the 0 to -6 dB factors to visualize the resolution. Use the script if you need to see the complete output.
1.0000000 .9921875 .9843750 .9765625 .9687500 .9609375 .9531250 .9453125 .9375000 .9296875 .9218750 .9140625 .9062500 .8984375 .8906250 .8828125 .8750000 .8671875 .8593750 .8515625 .8437500 .8359375 .8281250 .8203125 .8125000 .8046875 .7968750 .7890625 .7812500 .7734375 .7656250 .7578125 .7500000 .7421875 .7343750 .7265625 .7187500 .7109375 .7031250 .6953125 .6875000 .6796875 .6718750 .6640625 .6562500 .6484375 .6406250 .6328125 .6250000 .6171875 .6093750 .6015625 .5937500 .5859375 .5781250 .5703125 .5625000 .5546875 .5468750 .5390625 .5312500 .5234375 .5156250 .5078125 .5000000
1.0000000000 0 dB Bit-depth: 16 [16/16 16/16 16/16] .9921875000 -.0677 dB Bit-depth: 23 [23/23 23/23 23/23] .9843750000 -.1363 dB Bit-depth: 22 [22/22 22/22 22/22] .9765625000 -.2058 dB Bit-depth: 23 [23/23 23/23 23/23] .9687500000 -.2753 dB Bit-depth: 21 [21/21 21/21 21/21] .9609375000 -.3457 dB Bit-depth: 23 [23/23 23/23 23/23] .9531250000 -.4169 dB Bit-depth: 22 [22/22 22/22 22/22] .9453125000 -.4881 dB Bit-depth: 23 [23/23 23/23 23/23] .9375000000 -.5602 dB Bit-depth: 20 [20/20 20/20 20/20] .9296875000 -.6332 dB Bit-depth: 23 [23/23 23/23 23/23] .9218750000 -.7061 dB Bit-depth: 22 [22/22 22/22 22/22] .9140625000 -.7800 dB Bit-depth: 23 [23/23 23/23 23/23] .9062500000 -.8547 dB Bit-depth: 21 [21/21 21/21 21/21] .8984375000 -.9294 dB Bit-depth: 23 [23/23 23/23 23/23] .8906250000 -1.0058 dB Bit-depth: 22 [22/22 22/22 22/22] .8828125000 -1.0823 dB Bit-depth: 23 [23/23 23/23 23/23] .8750000000 -1.1596 dB Bit-depth: 19 [19/19 19/19 19/19] .8671875000 -1.2377 dB Bit-depth: 23 [23/23 23/23 23/23] .8593750000 -1.3159 dB Bit-depth: 22 [22/22 22/22 22/22] .8515625000 -1.3950 dB Bit-depth: 23 [23/23 23/23 23/23] .8437500000 -1.4749 dB Bit-depth: 21 [21/21 21/21 21/21] .8359375000 -1.5565 dB Bit-depth: 23 [23/23 23/23 23/23] .8281250000 -1.6373 dB Bit-depth: 22 [22/22 22/22 22/22] .8203125000 -1.7198 dB Bit-depth: 23 [23/23 23/23 23/23] .8125000000 -1.8032 dB Bit-depth: 20 [20/20 20/20 20/20] .8046875000 -1.8875 dB Bit-depth: 23 [23/23 23/23 23/23] .7968750000 -1.9717 dB Bit-depth: 22 [22/22 22/22 22/22] .7890625000 -2.0577 dB Bit-depth: 23 [23/23 23/23 23/23] .7812500000 -2.1437 dB Bit-depth: 21 [21/21 21/21 21/21] .7734375000 -2.2314 dB Bit-depth: 23 [23/23 23/23 23/23] .7656250000 -2.3192 dB Bit-depth: 22 [22/22 22/22 22/22] .7578125000 -2.4086 dB Bit-depth: 23 [23/23 23/23 23/23] .7500000000 -2.4981 dB Bit-depth: 18 [18/18 18/18 18/18] .7421875000 -2.5893 dB Bit-depth: 23 [23/23 23/23 23/23] .7343750000 -2.6814 dB Bit-depth: 22 [22/22 22/22 22/22] .7265625000 -2.7743 dB Bit-depth: 23 [23/23 23/23 23/23] .7187500000 -2.8681 dB Bit-depth: 21 [21/21 21/21 21/21] .7109375000 -2.9628 dB Bit-depth: 23 [23/23 23/23 23/23] .7031250000 -3.0592 dB Bit-depth: 22 [22/22 22/22 22/22] .6953125000 -3.1557 dB Bit-depth: 23 [23/23 23/23 23/23] .6875000000 -3.2538 dB Bit-depth: 20 [20/20 20/20 20/20] .6796875000 -3.3537 dB Bit-depth: 23 [23/23 23/23 23/23] .6718750000 -3.4536 dB Bit-depth: 22 [22/22 22/22 22/22] .6640625000 -3.5552 dB Bit-depth: 23 [23/23 23/23 23/23] .6562500000 -3.6586 dB Bit-depth: 21 [21/21 21/21 21/21] .6484375000 -3.7619 dB Bit-depth: 23 [23/23 23/23 23/23] .6406250000 -3.8679 dB Bit-depth: 22 [22/22 22/22 22/22] .6328125000 -3.9739 dB Bit-depth: 23 [23/23 23/23 23/23] .6250000000 -4.0825 dB Bit-depth: 19 [19/19 19/19 19/19] .6171875000 -4.1910 dB Bit-depth: 22 [22/23 22/23 22/23] .6093750000 -4.3022 dB Bit-depth: 21 [21/22 21/22 21/22] .6015625000 -4.4143 dB Bit-depth: 22 [22/23 22/23 22/23] .5937500000 -4.5272 dB Bit-depth: 20 [20/21 20/21 20/21] .5859375000 -4.6427 dB Bit-depth: 22 [22/23 22/23 22/23] .5781250000 -4.7591 dB Bit-depth: 21 [21/22 21/22 21/22] .5703125000 -4.8773 dB Bit-depth: 22 [22/23 22/23 22/23] .5625000000 -4.9971 dB Bit-depth: 19 [19/20 19/20 19/20] .5546875000 -5.1187 dB Bit-depth: 22 [22/23 22/23 22/23] .5468750000 -5.2421 dB Bit-depth: 21 [21/22 21/22 21/22] .5390625000 -5.3672 dB Bit-depth: 22 [22/23 22/23 22/23] .5312500000 -5.4940 dB Bit-depth: 20 [20/21 20/21 20/21] .5234375000 -5.6225 dB Bit-depth: 22 [22/23 22/23 22/23] .5156250000 -5.7528 dB Bit-depth: 21 [21/22 21/22 21/22] .5078125000 -5.8857 dB Bit-depth: 22 [22/23 22/23 22/23] .5000000000 -6.0204 dB Bit-depth: 16 [16/17 16/17 16/17]
6 decimals give these 34 factors with total of 528 factors for 0 to -96 dB.
List below contain only the 0 to -6 dB factors to visualize the resolution. Use the script if you need to see the complete output.
1.000000 .984375 .968750 .953125 .937500 .921875 .906250 .890625 .875000 .859375 .843750 .828125 .812500 .796875 .781250 .765625 .750000 .734375 .718750 .703125 .687500 .671875 .656250 .640625 .625000 .609375 .593750 .578125 .562500 .546875 .531250 .515625 .500000
1.0000000000 0 dB Bit-depth: 16 [16/16 16/16 16/16] .9843750000 -.1363 dB Bit-depth: 22 [22/22 22/22 22/22] .9687500000 -.2753 dB Bit-depth: 21 [21/21 21/21 21/21] .9531250000 -.4169 dB Bit-depth: 22 [22/22 22/22 22/22] .9375000000 -.5602 dB Bit-depth: 20 [20/20 20/20 20/20] .9218750000 -.7061 dB Bit-depth: 22 [22/22 22/22 22/22] .9062500000 -.8547 dB Bit-depth: 21 [21/21 21/21 21/21] .8906250000 -1.0058 dB Bit-depth: 22 [22/22 22/22 22/22] .8750000000 -1.1596 dB Bit-depth: 19 [19/19 19/19 19/19] .8593750000 -1.3159 dB Bit-depth: 22 [22/22 22/22 22/22] .8437500000 -1.4749 dB Bit-depth: 21 [21/21 21/21 21/21] .8281250000 -1.6373 dB Bit-depth: 22 [22/22 22/22 22/22] .8125000000 -1.8032 dB Bit-depth: 20 [20/20 20/20 20/20] .7968750000 -1.9717 dB Bit-depth: 22 [22/22 22/22 22/22] .7812500000 -2.1437 dB Bit-depth: 21 [21/21 21/21 21/21] .7656250000 -2.3192 dB Bit-depth: 22 [22/22 22/22 22/22] .7500000000 -2.4981 dB Bit-depth: 18 [18/18 18/18 18/18] .7343750000 -2.6814 dB Bit-depth: 22 [22/22 22/22 22/22] .7187500000 -2.8681 dB Bit-depth: 21 [21/21 21/21 21/21] .7031250000 -3.0592 dB Bit-depth: 22 [22/22 22/22 22/22] .6875000000 -3.2538 dB Bit-depth: 20 [20/20 20/20 20/20] .6718750000 -3.4536 dB Bit-depth: 22 [22/22 22/22 22/22] .6562500000 -3.6586 dB Bit-depth: 21 [21/21 21/21 21/21] .6406250000 -3.8679 dB Bit-depth: 22 [22/22 22/22 22/22] .6250000000 -4.0825 dB Bit-depth: 19 [19/19 19/19 19/19] .6093750000 -4.3022 dB Bit-depth: 21 [21/22 21/22 21/22] .5937500000 -4.5272 dB Bit-depth: 20 [20/21 20/21 20/21] .5781250000 -4.7591 dB Bit-depth: 21 [21/22 21/22 21/22] .5625000000 -4.9971 dB Bit-depth: 19 [19/20 19/20 19/20] .5468750000 -5.2421 dB Bit-depth: 21 [21/22 21/22 21/22] .5312500000 -5.4940 dB Bit-depth: 20 [20/21 20/21 20/21] .5156250000 -5.7528 dB Bit-depth: 21 [21/22 21/22 21/22] .5000000000 -6.0204 dB Bit-depth: 16 [16/17 16/17 16/17] .5000000000 -6.0204 dB Bit-depth: 16 [16/17 16/17 16/17]
Using this method of testing bit length needed by SoX we learned we can use integer calculation with constants to create the full coefficient list. We first select the constant value with desired max bit length then we can substract this value from previous coefficient to get a integer value for every next step. Then convert these values to float from string to create a valid coefficient. Compared to float calculation this proved to give more accurate results.
When we start out with the constant of 625 representing 4 decimals we can multiply this by 5 to add another decimal precision and double step resolution. This seem work up to 9 decimals from where rounding inside SoX seem to give less precise results.
While 4 decimals return results with 20-bit and lower bit length now adding a decimal will double step resolution but also return coefficients with higher bit length.
Python script utilizing this method can be read here
24-bit audio can make use of the same constants but will of course result with bit length larger than 24-bit. Typical 28-bit for 4 decimals, 29-bit for 5 decimals and so on.
Your comment are welcome