Tutorial 1 - Audio programming with gtkIOStream

[flatmax]

gtkIOStream is a versatile software for signal processing and audio processing, amongst other features (such as GUI programming).

In this tutorial we will focus on loading audio from file (almost any type of audio file format) and saving it back to file (almost any audio file format).

Before we start, you have to have compiled and installed gtkIOStream. For certain components of gtkIOStream its compilation and installation isn't necessary, but for certain parts of audio processing it is necessary. Tutorial 0 guides you through how to install and test the installation of gtkIOStream. Before you start this tutorial, you must have it installed first.

Our first step is to open the standard lightweight Raspberry Pi IDE with a blank file for us to code up in ... like so :

Code: Select all

geany SoxInOut.C

Now if you have used ssh to get onto your Pi, then you may have to logout and log in with the "-X" flag like so :

Code: Select all

ssh -X pi@raspberrypiIP

In the geany editor, enter the following code :

Code: Select all

#include <Sox.H>
#include <iostream>

int main(int argc, char *argv[]) {
	if (argc<3){ // input check
		cout<<"Usage:\n"<<argv[0]<<" audioFileNameIn audioFileNameOut"<<endl;
		return -1;
	}

	Sox<int> sox; // declare our sox object reading/writing int (32 bits)
	
	int ret; // return variable on error
	if ((ret=sox.openRead(argv[1]))<0  && ret!=SOX_READ_MAXSCALE_ERROR)
		return SoxDebug().evaluateError(ret, argv[argc-2]);

	Eigen::Array<float, Eigen::Dynamic, Eigen::Dynamic>  x; // the buffer
	int N=100*1024; // number of frames to load in each time
	ret=sox.read(x, N);
	if ((ret=sox.read(x, N))!=0) // read in again
		return SoxDebug().evaluateError(ret);

	// Now that we know how many channels the file is, open the output file
	if ((ret=sox.openWrite(argv[2], sox.getFSIn(), x.cols(), pow(2.,31)-1)))
		return SoxDebug().evaluateError(ret);

	// main loop, write and read
	int i=0; // remember how many times we have looped
	while (x.rows()!=0){ // wait till no more read in
		// process your audio here
		if ((ret=sox.write(x))!=x.rows()*x.cols())
			return SoxDebug().evaluateError(ret);

		cout<<'\r'<<((float)(++i*N)/sox.getFSIn())<< " seconds written";

		if ((ret=sox.read(x, N))!=0) // read in again
			return SoxDebug().evaluateError(ret);
	}

	sox.closeWrite();
	sox.closeRead();

	return 0;
}

At this point, we will step through the code in logical flow to get an understanding of audio file processing.

The first step is to include our Sox and iostream headers. We then enter our "main" function. Like so :

Code: Select all

#include <Sox.H>
#include <iostream>

int main(int argc, char *argv[]) {

The first thing we want to do is make sure the user has specified an input and output file :

Code: Select all

	if (argc<3){ // input check
		cout<<"Usage:\n"<<argv[0]<<" audioFileNameIn audioFileNameOut"<<endl;
		return -1;
	}

if not then we tell them to use it like so : SoxInOut inFile outFile

We instantiate our Sox object. Now at this point we have to tell sox what type we want it to use for reading and writing audio.

Code: Select all

Sox<int> sox;

Here we tell it to use type int, because that is 32 bits and Sox's internal type is 32 bits as well. This typename is used mainly for working out the word size (a sox requirement) when opening write files.

Next we want to open the input file, the read file :

Code: Select all

	int ret; // return variable on error
	if ((ret=sox.openRead(argv[1]))<0  && ret!=SOX_READ_MAXSCALE_ERROR)
		return SoxDebug().evaluateError(ret, argv[argc-2]);

Now here we open the first input argument "argv[1]" and test for standard errors. You will also notice that we ignore the SOX_READ_MAXSCALE_ERROR error. This MAXSCALE_ERROR is thrown when Sox can't fund the inputFile.max text file. This text file is used to rescale the sox input data according to the scale factor in that file. If that file exists and say the factor is 0.0001, then sox will load in audio data from the audio file and rescale it from maxscale down to a maxscale value of 0.0001. This is rather significant, because 0.0001 represents around 13 bits of data! That means, say you processed a file and its highest value was 0.0001, then by saving it to an audio file, you may loose 13 bits of data ... wow ! Say for example you were using 16 bit audio file formats - that leaves 3 bits for audio dynamic range ... now I think you start to understand the problem I am getting at here - we need to preserve low noise levels and still be able to capture audio waveforms at their original sound level - a very improtant thing for signal processing - particularly when the system is calibrated.

We now make an buffer to hold audio data (perhaps you want to do computations on that buffer before writing back out to file) :

Code: Select all

	Eigen::Array<float, Eigen::Dynamic, Eigen::Dynamic>  x; // the buffer

You can do all sorts of sophisticated things with "x" using Eigen.

We have to specify how many frames of audio we want to process each time (audio frames are one sample per channel - i.e. one frame can be one sample of two channels), we do that like so :

Code: Select all

	int N=100*1024; // number of frames to load in each time
	ret=sox.read(x, N);
	if ((ret=sox.read(x, N))!=0) // read in again
		return SoxDebug().evaluateError(ret);

We set N to 100k samples per channel and we ask sox to read that audio data in (sox.read). We check if there was an error, and if so then we report the error and return it - exiting. The SoxDebug class prints out error information and allows you to print error info and return the error in one command. It is currently out of the scope of this tutorial and will be left there.

Now we know the sample rate of the input audio file (the sox.getFSIn method), the number of channels (x.cols) and we can open the output file for writing :

Code: Select all

	if ((ret=sox.openWrite(argv[2], sox.getFSIn(), x.cols(), pow(2.,31)-1)))
		return SoxDebug().evaluateError(ret);

The output file name is arvv[2] and pow(2.,31)-1 is the maximum signed value representable by signed 32 bits.

We now enter the read write loop like so :

Code: Select all

	int i=0; // remember how many times we have looped
	while (x.rows()!=0){ // wait till no more read in

Here we check to see that more then 0 frames have been loaded into x, if not, then we are at the end of the file and the while loop stops.

At this point if you want to process the audio data, you should ... we choose not to !

We write the audio to the output file :

Code: Select all

		if ((ret=sox.write(x))!=x.rows()*x.cols())
			return SoxDebug().evaluateError(ret);

Nothing fancy here, just a write (sox.write) and check that we wrote all the sample out (rows frames times cols channels).

Next we print out how many seconds we have processed and read in more data for processing :

Code: Select all

		cout<<'\r'<<((float)(++i*N)/sox.getFSIn())<< " seconds written";

		if ((ret=sox.read(x, N))!=0) // read in again
			return SoxDebug().evaluateError(ret);

The seconds is the number of frames per read (N) times the number of times we have done this (i) divided by the number of frames per seconds - which is the sample rate in Hz.

At this point we loop and loop - processing - until we run out of audio samples to process. We then close the audio files which we opened and return :

Code: Select all

	sox.closeWrite();
	sox.closeRead();

	return 0;

Actially Sox will close them for you when the class destructs, but what if we wanted to open a new file again ? Actually if you call openWeite or openRead again it will first close an open file ... just for robustness - it also allows you to simplify your code base minimising calls.

Thats it !

We need an audio file for testing, so lets download a free one :
wget "https://ogg.jamendo.com/download/track/206411/ogg1/" -O flatmax.CentralTransmission.ogg

Compile your code :

Code: Select all

g++ -o SoxInOut SoxInOut.C `pkg-config --cflags --libs gtkIOStream`

The pkg-config statement tells g++ about preprocessor flags and what libraries to link against - you can type this into the command line and see it for yourself : pkg-config --cflags --libs gtkIOStream
We compile and create the output file SoxInOut. It will print a couple of warnings like so :

Code: Select all

$ g++ `pkg-config --cflags --libs gtkIOStream` -o SoxInOut SoxInOut.C
In file included from SoxInOut.C:1:0:
/usr/include/gtkIOStream/Sox.H:27:32: warning: unknown option after ‘#pragma GCC diagnostic’ kind [-Wpragmas]
 #pragma GCC diagnostic ignored "-Wignored-attributes"
                                ^

Now lets run SoxInOut on our downloaded ogg file ... say ogg to wav so we can use aplay to listen to it ...

Code: Select all

$ ./SoxInOut flatmax.CentralTransmission.ogg flatmax.CentralTransmission.wav

It takes a little while because we have chosen a small number of frames to process - but it is still many many times faster then realtime.

Thats it ! You can now process audio files of many many different formats with your own algorithms. Here is a list of some of the audio formats handled
by the Sox object :

Code: Select all

AUDIO FILE FORMATS:The known output file extensions (output file formats) are the following :
8svx aif aifc aiff aiffc al amb amr-nb amr-wb anb au avr awb caf cdda cdr cvs cvsd cvu dat dvms f32 f4 f64 f8 fap flac fssd gsm gsrt hcom htk ima ircam la lpc lpc10 lu mat mat4 mat5 maud mp2 mp3 nist ogg paf prc pvf raw s1 s16 s2 s24 s3 s32 s4 s8 sb sd2 sds sf sl sln smp snd sndfile sndr sndt sou sox sph sw txw u1 u16 u2 u24 u3 u32 u4 u8 ub ul uw vms voc vorbis vox w64 wav wavpcm wv wve xa xi 

[buddy]

Hi, Thanks for this tutorial it will be very helpful!

I feel like I'm missing somthing very small but i've tried everything i can think of. when I try to compile the program I get this error "g++: error: SoxInOut.C: No such file or directory". (similar issue when i try tutorial 2)

my ./configure output is the same. I've located /usr/include/gtkIOStream/Sox.H and it exists. If i hard code its location i get the same error but with debug.h

this is the output of pkg-config --cflags --libs gtkIOStream. I don't see Sox.h but im not sure thats the issue.

Code: Select all

pi@raspberrypi:~ $ pkg-config --cflags --libs gtkIOStream
-pthread -I/usr/include/gtkIOStream -I/usr/include/alsa -I/usr/include/gtk-2.0 -I/usr/lib/arm-linux-gnueabihf/gtk-2.0/include -I/usr/include/gio-unix-2.0/ -I/usr/include/cairo -I/usr/include/pango-1.0 -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/pixman-1 -I/usr/include/libpng16 -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/libpng16 -I/usr/include/pango-1.0 -I/usr/include/harfbuzz -I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib/arm-linux-gnueabihf/glib-2.0/include -I/usr/include/freetype2 -I/usr/include/eigen3 -ldsp -lgtkIOStream -lasound -lgtk-x11-2.0 -lgdk-x11-2.0 -lpangocairo-1.0 -latk-1.0 -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lpangoft2-1.0 -lpango-1.0 -lgobject-2.0 -lglib-2.0 -lfontconfig -lfreetype -lsox -ljack -lfftw3

thanks again for the tutorials and any help

Buddy

[flatmax]

OK, lets see...
SoxInOut.C is the file you have created in this tutorial.

Can you confirm you can see SoxInOut.C ?

Code: Select all

ls SoxInOut.C

If you can't, then you have to find where it is !

Matt

[buddy]

the file exists at ~/Test/SoxInOut.c

i think it is an include issue with Sox.h because when I go to compile it through geany instead, i get an error that says Sox.h: No such file or directory

i did some poking around online and I found some sources confirming this error comes from include issues
https://stackoverflow.com/questions/129 ... -directory

I just need to figure out how to fix it

thanks again,
buddy

[flatmax]

ok,

Did you use the path ~/Test/SoxInOut.c when compiling ?

Code: Select all

g++ `pkg-config --cflags --libs gtkIOStream` -o ~/Test/SoxInOut ~/Test/SoxInOut.c

You will need to execute like so :

Code: Select all

~/Test/SoxInOut flatmax.CentralTransmission.ogg flatmax.CentralTransmission.wav

Matt

[buddy]

wow i feel very dumb. I had a lowercase c in SoxInOut.C. Thanks for the help!

[flatmax]

These types of problems get even the most senior programmers !

Matt

[Franshej]

Hi
I get the following error when trying to compile:

In file included from SoxInOut.C:1:0:
/usr/include/gtkIOStream/Sox.H:50:20: fatal error: config.h: No such file or directory
#include "config.h"
^
compilation terminated.

Frans

[FelixNiev]

Hi Frans,

i have the same error message.

In file included from SoxInOut.C:1:0:
/usr/include/gtkIOStream/Sox.H:50:20: fatal error: config.h

Did you find a solution?

Felix

[mikkkkky]

Me too, with the same problem.

This is the solution:
g++ `pkg-config --cflags --libs gtkIOStream` -I/home/pi/gtkiostream -o SoxInOut SoxInOut.C