SpectralDenoise - not working with higher sampling rate -96K

[Aclarky1969]

Hi All

Tested the example ino which worked fine at a SR of 44100 KHZ but doesn't work with a higher SR of 96KHZ.
The audio sounds distorted. Anybody else tested with 96K ?

Thanks
Alan

[grahamwhaley]

Hi @Aclarky1969 . I think I only tested at 44.1KHz when I ported the code.

Refreshing my memory on the code ;-), if you look at the comment above the setup() function then it gives us some clues:

  //Our default FFT size is 256. That is time and space efficient, but
  // if you are running at a 'high' sample rate, the NR 'buckets' might
  // be quite small. You may want to use a 1024 FFT if running at 44.1KHz
  // for instance, if you can afford the time and space overheads.
  int setup(const AudioSettings_F32 & settings, const int _N_FFT = 256);

Basically, the Spectral NR works by analysing a number of 'buckets' across your spectrum (sample rate), and deciding if each bucket needs muting/processing or not. If you have a higher sample rate then each bucket covers a larger chunk of the audio spectrum, making the process more granular. So, two things you can try:

1. increase the number of buckets to bring the granularity back down. This will cost processing time and space. The example passes the number of bins in as a setup() parameter. Modify the setup code to double the bins as a start (as your sample rate is double the 44.1k we tested at). Bump 1024 to 2048, for example:

  //Use a 2048 FFT in this example for 96k sample rate
  if (Spectral.setup(audio_settings, 2048) < 0 ) {

2. The Spectral NR focusses its effort on the vocal range of audio - if you have a look in the Spectral header file at the VAD_low_freq and VAD_high_freq you'll see the current default range is 100-3600Hz. So, most of your audio spectrum from the 96KHz sample rate is probably not being processed anyway. Even the example of 44.1KHz is rather wasteful, but that is I think the default sample rate of the library? In one of my projects I decimated the sample stream down to 11KHz before processing through the Spectral NR, as that adequately covered the voice frequency range and saved time and space. Just a thought :-)

[Aclarky1969]

Hi Graham

Thanks for your reply much appreciated! I'm quite new to F32 lib.
I tried increasing to 2048 but still no joy at 96K. Works at 44100.0 K
This is the orginal ino in the lib.

Regards

Alan

//Use a 2048 FFT in this example for 96k sample rate
if (Spectral.setup(audio_settings, 2048) < 0 ) {

/* Spectral Noise reduction test program.
*

• The example takes sound in from both the I2S and USB of the Teensy/audio-daughtercard,
• processes it, and sends it back out the I2S/headphone ports.
• Every 10 seconds it switches from Spectral processing to data-passthrough and back,
• to aid comparison.
• Some information is printed on the serial monitor.
• This example requires the Teensy Board 'USB Type' in the Arduino Tools menu to be set
• to a type that includes 'Audio', and ideally 'Serial' as well. Tested with
• 'Serial+MIDI+Audio'.
• If you do not set 'Audio', you will get a compliation errors similar to:

•  "... OpenAudio_ArduinoLibrary/USB_Audio_F32.h: In member function 'virtual void AudioOutputUSB_F32::update()':"
  

•  "... OpenAudio_ArduinoLibrary/USB_Audio_F32.h:139:3: error: 'usb_out' was not declared in this scope"
  

• MIT License. use at your own risk.
  */

#include "OpenAudio_ArduinoLibrary.h"
#include "AudioStream_F32.h"
#include "USB_Audio_F32.h"
#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <SerialFlash.h>

// GUItool: begin automatically generated code
AudioInputI2S_F32 audioInI2S1; //xy=117,343
AudioInputUSB_F32 audioInUSB1; //xy=146,397
AudioMixer4_F32 input_mixer; //xy=370,321
AudioSpectralDenoise_F32 Spectral; //xy=852,250
AudioMixer4_F32 output_mixer; //xy=993,296
AudioSwitch4_OA_F32 processing_switch;
AudioOutputI2S_F32 audioOutI2S1; //xy=1257,367
AudioOutputUSB_F32 audioOutUSB1; //xy=1261,418

//Inputs - mixed into one stream
AudioConnection_F32 patchCord1(audioInI2S1, 0, input_mixer, 0);
AudioConnection_F32 patchCord2(audioInUSB1, 0, input_mixer, 1);

//route through a switch, so we can switch Spectral in/out
AudioConnection_F32 patchCord3(input_mixer, 0, processing_switch, 0);

//First route is direct - direct to the output mixer
AudioConnection_F32 patchCord4(processing_switch, 0, output_mixer, 0);

//Second route is through Spectral to the output mixer
AudioConnection_F32 patchCord5(processing_switch, 1, Spectral, 0);
AudioConnection_F32 patchCord6(Spectral, 0, output_mixer, 1);

//And finally output the mixer to the output channels
AudioConnection_F32 patchCord7(output_mixer, 0, audioOutI2S1, 0);
AudioConnection_F32 patchCord8(output_mixer, 0, audioOutI2S1, 1);
AudioConnection_F32 patchCord9(output_mixer, 0, audioOutUSB1, 0);
AudioConnection_F32 patchCord10(output_mixer, 0, audioOutUSB1, 1);

AudioControlSGTL5000 sgtl5000_1; //xy=519,146
// GUItool: end automatically generated code

AudioSettings_F32 audio_settings(AUDIO_SAMPLE_RATE_EXACT, AUDIO_BLOCK_SAMPLES);

int current_cycle = 0; //Choose how we route the audio - to process or not

static void spectralSetup(void){
//Use a 1024 FFT in this example
if (Spectral.setup(audio_settings, 1024) < 0 ) {
Serial.println("Failed to setup Spectral");
} else {
Serial.println("Spectral setup OK");
}
Serial.flush();
}

//The setup function is called once when the system starts up
void setup(void) {
//Start the USB serial link (to aid debugging)
Serial.begin(115200); delay(500);
Serial.println("Setup starting...");

//Allocate dynamically shuffled memory for the audio subsystem
AudioMemory(30); AudioMemory_F32(30);

Serial.println("Calling Spectral setup");
spectralSetup();
Serial.println("Spectral Setup done");
sgtl5000_1.enable();
sgtl5000_1.unmuteHeadphone();
sgtl5000_1.volume(0.5);

//End of setup
Serial.println("Setup complete.");
};

//After setup(), the loop function loops forever.
//Note that the audio modules are called in the background.
//They do not need to be serviced by the loop() function.
void loop(void) {
// every 'n' seconds move to the next cycle of processing.
if ( ((millis()/1000) % 10) == 0 ) {
current_cycle++;
if (current_cycle >= 2) current_cycle = 0;

switch( current_cycle ) {
  case 0:
    Serial.println("Passthrough");
    processing_switch.setChannel(0);
    break;

  case 1:
    Serial.println("Run Spectral NR");
    processing_switch.setChannel(1);
    break;

  default:
    current_cycle = 0;  //oops - reset to start
    break;
}

}

//Nap - we don't need to hard-spin...
delay(1000);
};

[grahamwhaley]

Hi @Aclarky1969 . Thx for the feedback. Hmm, going to have to ask some more questions then ;-)

1. What codec are you using? Are you using the Teensy with an SGTL5000 audio card for instance? (that's the only combo I have here ;-) ).
2. How have you set your system to run at 96k ? Best if you show the code you are using rather than the original code (which we can see in the repo already). btw, if you wrap your code paste in some github markdown to note it is code when pasting, then it will render better on the ticket. Have a look at this for some info. The code 'type' will be c or cpp for instance. Or, if the code is more than a snippet, maybe attach the files to the Issue.
3. Looking at the Spectral code, it derives the sample frequency from the audio setting structure that gets passed in by the audio framework - have a look at the top of the setup() function where it looks at settings.sample_rate_Hz for instance. It is probably worth you checking with some debug if that is set to 96KHz or not
4. Related to (3) - did changing the FFT size from 1024 to 2048 make any difference at all?

I might not be able to re-create your setup here - you might have to get your hands dirty and do some debugging... :-) It is definitely possible I missed some code in the Spectral that needs to adapt to the sample rate...