Tutorial 1 - Audio programming with gtkIOStream

Moderator: flatmax

Post Reply
flatmax
Posts: 603
Joined: Sat Jul 23, 2016 11:39 pm

Tutorial 1 - Audio programming with gtkIOStream

Post by flatmax » Mon Jul 24, 2017 11:11 pm

gtkIOStream is a versatile software for signal processing and audio processing, amongst other features (such as GUI programming).

In this tutorial we will focus on loading audio from file (almost any type of audio file format) and saving it back to file (almost any audio file format).

Before we start, you have to have compiled and installed gtkIOStream. For certain components of gtkIOStream its compilation and installation isn't necessary, but for certain parts of audio processing it is necessary. Tutorial 0 guides you through how to install and test the installation of gtkIOStream. Before you start this tutorial, you must have it installed first.

Our first step is to open the standard lightweight Raspberry Pi IDE with a blank file for us to code up in ... like so :

Code: Select all

geany SoxInOut.C
Now if you have used ssh to get onto your Pi, then you may have to logout and log in with the "-X" flag like so :

Code: Select all

ssh -X pi@raspberrypiIP
In the geany editor, enter the following code :

Code: Select all

#include <Sox.H>
#include <iostream>

int main(int argc, char *argv[]) {
	if (argc<3){ // input check
		cout<<"Usage:\n"<<argv[0]<<" audioFileNameIn audioFileNameOut"<<endl;
		return -1;
	}

	Sox<int> sox; // declare our sox object reading/writing int (32 bits)
	
	int ret; // return variable on error
	if ((ret=sox.openRead(argv[1]))<0  && ret!=SOX_READ_MAXSCALE_ERROR)
		return SoxDebug().evaluateError(ret, argv[argc-2]);

	Eigen::Array<float, Eigen::Dynamic, Eigen::Dynamic>  x; // the buffer
	int N=100*1024; // number of frames to load in each time
	ret=sox.read(x, N);
	if ((ret=sox.read(x, N))!=0) // read in again
		return SoxDebug().evaluateError(ret);

	// Now that we know how many channels the file is, open the output file
	if ((ret=sox.openWrite(argv[2], sox.getFSIn(), x.cols(), pow(2.,31)-1)))
		return SoxDebug().evaluateError(ret);

	// main loop, write and read
	int i=0; // remember how many times we have looped
	while (x.rows()!=0){ // wait till no more read in
		// process your audio here
		if ((ret=sox.write(x))!=x.rows()*x.cols())
			return SoxDebug().evaluateError(ret);

		cout<<'\r'<<((float)(++i*N)/sox.getFSIn())<< " seconds written";

		if ((ret=sox.read(x, N))!=0) // read in again
			return SoxDebug().evaluateError(ret);
	}

	sox.closeWrite();
	sox.closeRead();

	return 0;
}
At this point, we will step through the code in logical flow to get an understanding of audio file processing.

The first step is to include our Sox and iostream headers. We then enter our "main" function. Like so :

Code: Select all

#include <Sox.H>
#include <iostream>

int main(int argc, char *argv[]) {
The first thing we want to do is make sure the user has specified an input and output file :

Code: Select all

	if (argc<3){ // input check
		cout<<"Usage:\n"<<argv[0]<<" audioFileNameIn audioFileNameOut"<<endl;
		return -1;
	}
if not then we tell them to use it like so : SoxInOut inFile outFile

We instantiate our Sox object. Now at this point we have to tell sox what type we want it to use for reading and writing audio.

Code: Select all

Sox<int> sox;
Here we tell it to use type int, because that is 32 bits and Sox's internal type is 32 bits as well. This typename is used mainly for working out the word size (a sox requirement) when opening write files.

Next we want to open the input file, the read file :

Code: Select all

	int ret; // return variable on error
	if ((ret=sox.openRead(argv[1]))<0  && ret!=SOX_READ_MAXSCALE_ERROR)
		return SoxDebug().evaluateError(ret, argv[argc-2]);
Now here we open the first input argument "argv[1]" and test for standard errors. You will also notice that we ignore the SOX_READ_MAXSCALE_ERROR error. This MAXSCALE_ERROR is thrown when Sox can't fund the inputFile.max text file. This text file is used to rescale the sox input data according to the scale factor in that file. If that file exists and say the factor is 0.0001, then sox will load in audio data from the audio file and rescale it from maxscale down to a maxscale value of 0.0001. This is rather significant, because 0.0001 represents around 13 bits of data! That means, say you processed a file and its highest value was 0.0001, then by saving it to an audio file, you may loose 13 bits of data ... wow ! Say for example you were using 16 bit audio file formats - that leaves 3 bits for audio dynamic range ... now I think you start to understand the problem I am getting at here - we need to preserve low noise levels and still be able to capture audio waveforms at their original sound level - a very improtant thing for signal processing - particularly when the system is calibrated.

We now make an buffer to hold audio data (perhaps you want to do computations on that buffer before writing back out to file) :

Code: Select all

	Eigen::Array<float, Eigen::Dynamic, Eigen::Dynamic>  x; // the buffer
You can do all sorts of sophisticated things with "x" using Eigen.

We have to specify how many frames of audio we want to process each time (audio frames are one sample per channel - i.e. one frame can be one sample of two channels), we do that like so :

Code: Select all

	int N=100*1024; // number of frames to load in each time
	ret=sox.read(x, N);
	if ((ret=sox.read(x, N))!=0) // read in again
		return SoxDebug().evaluateError(ret);
We set N to 100k samples per channel and we ask sox to read that audio data in (sox.read). We check if there was an error, and if so then we report the error and return it - exiting. The SoxDebug class prints out error information and allows you to print error info and return the error in one command. It is currently out of the scope of this tutorial and will be left there.

Now we know the sample rate of the input audio file (the sox.getFSIn method), the number of channels (x.cols) and we can open the output file for writing :

Code: Select all

	if ((ret=sox.openWrite(argv[2], sox.getFSIn(), x.cols(), pow(2.,31)-1)))
		return SoxDebug().evaluateError(ret);
The output file name is arvv[2] and pow(2.,31)-1 is the maximum signed value representable by signed 32 bits.

We now enter the read write loop like so :

Code: Select all

	int i=0; // remember how many times we have looped
	while (x.rows()!=0){ // wait till no more read in
Here we check to see that more then 0 frames have been loaded into x, if not, then we are at the end of the file and the while loop stops.

At this point if you want to process the audio data, you should ... we choose not to !

We write the audio to the output file :

Code: Select all

		if ((ret=sox.write(x))!=x.rows()*x.cols())
			return SoxDebug().evaluateError(ret);
Nothing fancy here, just a write (sox.write) and check that we wrote all the sample out (rows frames times cols channels).

Next we print out how many seconds we have processed and read in more data for processing :

Code: Select all

		cout<<'\r'<<((float)(++i*N)/sox.getFSIn())<< " seconds written";

		if ((ret=sox.read(x, N))!=0) // read in again
			return SoxDebug().evaluateError(ret);
The seconds is the number of frames per read (N) times the number of times we have done this (i) divided by the number of frames per seconds - which is the sample rate in Hz.

At this point we loop and loop - processing - until we run out of audio samples to process. We then close the audio files which we opened and return :

Code: Select all

	sox.closeWrite();
	sox.closeRead();

	return 0;
Actially Sox will close them for you when the class destructs, but what if we wanted to open a new file again ? Actually if you call openWeite or openRead again it will first close an open file ... just for robustness - it also allows you to simplify your code base minimising calls.

Thats it !

We need an audio file for testing, so lets download a free one :
wget "https://ogg.jamendo.com/download/track/206411/ogg1/" -O flatmax.CentralTransmission.ogg

Compile your code :

Code: Select all

g++ `pkg-config --cflags --libs gtkIOStream` -o SoxInOut SoxInOut.C
The pkg-config statement tells g++ about preprocessor flags and what libraries to link against - you can type this into the command line and see it for yourself : pkg-config --cflags --libs gtkIOStream
We compile and create the output file SoxInOut. It will print a couple of warnings like so :

Code: Select all

$ g++ `pkg-config --cflags --libs gtkIOStream` -o SoxInOut SoxInOut.C
In file included from SoxInOut.C:1:0:
/usr/include/gtkIOStream/Sox.H:27:32: warning: unknown option after ‘#pragma GCC diagnostic’ kind [-Wpragmas]
 #pragma GCC diagnostic ignored "-Wignored-attributes"
                                ^
Now lets run SoxInOut on our downloaded ogg file ... say ogg to wav so we can use aplay to listen to it ...

Code: Select all

$ ./SoxInOut flatmax.CentralTransmission.ogg flatmax.CentralTransmission.wav
It takes a little while because we have chosen a small number of frames to process - but it is still many many times faster then realtime.

Thats it ! You can now process audio files of many many different formats with your own algorithms. Here is a list of some of the audio formats handled
by the Sox object :

Code: Select all

AUDIO FILE FORMATS:The known output file extensions (output file formats) are the following :
8svx aif aifc aiff aiffc al amb amr-nb amr-wb anb au avr awb caf cdda cdr cvs cvsd cvu dat dvms f32 f4 f64 f8 fap flac fssd gsm gsrt hcom htk ima ircam la lpc lpc10 lu mat mat4 mat5 maud mp2 mp3 nist ogg paf prc pvf raw s1 s16 s2 s24 s3 s32 s4 s8 sb sd2 sds sf sl sln smp snd sndfile sndr sndt sou sox sph sw txw u1 u16 u2 u24 u3 u32 u4 u8 ub ul uw vms voc vorbis vox w64 wav wavpcm wv wve xa xi 
Checkout the next kickstarter campaign - the Ultra 2 : https://kck.st/2xwM3DL
Please review the Zero sound card on Amazon USA : https://www.amazon.com/dp/B075V1VNDD
---
Check out our new forum on github : https://github.com/Audio-Injector

buddy
Posts: 4
Joined: Fri Sep 08, 2017 9:31 am

Re: Tutorial 1 - Audio programming with gtkIOStream

Post by buddy » Fri Sep 08, 2017 11:51 am

Hi, Thanks for this tutorial it will be very helpful!

I feel like I'm missing somthing very small but i've tried everything i can think of. when I try to compile the program I get this error "g++: error: SoxInOut.C: No such file or directory". (similar issue when i try tutorial 2)

my ./configure output is the same. I've located /usr/include/gtkIOStream/Sox.H and it exists. If i hard code its location i get the same error but with debug.h

this is the output of pkg-config --cflags --libs gtkIOStream. I don't see Sox.h but im not sure thats the issue.

Code: Select all

pi@raspberrypi:~ $ pkg-config --cflags --libs gtkIOStream
-pthread -I/usr/include/gtkIOStream -I/usr/include/alsa -I/usr/include/gtk-2.0 -I/usr/lib/arm-linux-gnueabihf/gtk-2.0/include -I/usr/include/gio-unix-2.0/ -I/usr/include/cairo -I/usr/include/pango-1.0 -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/pixman-1 -I/usr/include/libpng16 -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/libpng16 -I/usr/include/pango-1.0 -I/usr/include/harfbuzz -I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib/arm-linux-gnueabihf/glib-2.0/include -I/usr/include/freetype2 -I/usr/include/eigen3 -ldsp -lgtkIOStream -lasound -lgtk-x11-2.0 -lgdk-x11-2.0 -lpangocairo-1.0 -latk-1.0 -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lpangoft2-1.0 -lpango-1.0 -lgobject-2.0 -lglib-2.0 -lfontconfig -lfreetype -lsox -ljack -lfftw3
thanks again for the tutorials and any help

Buddy

flatmax
Posts: 603
Joined: Sat Jul 23, 2016 11:39 pm

Re: Tutorial 1 - Audio programming with gtkIOStream

Post by flatmax » Tue Sep 12, 2017 5:25 pm

OK, lets see...
SoxInOut.C is the file you have created in this tutorial.

Can you confirm you can see SoxInOut.C ?

Code: Select all

ls SoxInOut.C
If you can't, then you have to find where it is !

Matt
Checkout the next kickstarter campaign - the Ultra 2 : https://kck.st/2xwM3DL
Please review the Zero sound card on Amazon USA : https://www.amazon.com/dp/B075V1VNDD
---
Check out our new forum on github : https://github.com/Audio-Injector

buddy
Posts: 4
Joined: Fri Sep 08, 2017 9:31 am

Re: Tutorial 1 - Audio programming with gtkIOStream

Post by buddy » Wed Sep 13, 2017 9:27 am

the file exists at ~/Test/SoxInOut.c

i think it is an include issue with Sox.h because when I go to compile it through geany instead, i get an error that says Sox.h: No such file or directory

i did some poking around online and I found some sources confirming this error comes from include issues
https://stackoverflow.com/questions/129 ... -directory

I just need to figure out how to fix it

thanks again,
buddy

flatmax
Posts: 603
Joined: Sat Jul 23, 2016 11:39 pm

Re: Tutorial 1 - Audio programming with gtkIOStream

Post by flatmax » Thu Sep 14, 2017 9:43 pm

ok,

Did you use the path ~/Test/SoxInOut.c when compiling ?

Code: Select all

g++ `pkg-config --cflags --libs gtkIOStream` -o ~/Test/SoxInOut ~/Test/SoxInOut.c
You will need to execute like so :

Code: Select all

~/Test/SoxInOut flatmax.CentralTransmission.ogg flatmax.CentralTransmission.wav
Matt
Checkout the next kickstarter campaign - the Ultra 2 : https://kck.st/2xwM3DL
Please review the Zero sound card on Amazon USA : https://www.amazon.com/dp/B075V1VNDD
---
Check out our new forum on github : https://github.com/Audio-Injector

buddy
Posts: 4
Joined: Fri Sep 08, 2017 9:31 am

Re: Tutorial 1 - Audio programming with gtkIOStream

Post by buddy » Wed Sep 20, 2017 3:55 am

wow i feel very dumb. I had a lowercase c in SoxInOut.C. Thanks for the help!

flatmax
Posts: 603
Joined: Sat Jul 23, 2016 11:39 pm

Re: Tutorial 1 - Audio programming with gtkIOStream

Post by flatmax » Wed Sep 20, 2017 5:50 am

These types of problems get even the most senior programmers !

Matt
Checkout the next kickstarter campaign - the Ultra 2 : https://kck.st/2xwM3DL
Please review the Zero sound card on Amazon USA : https://www.amazon.com/dp/B075V1VNDD
---
Check out our new forum on github : https://github.com/Audio-Injector

Franshej
Posts: 9
Joined: Mon Jan 22, 2018 4:13 am

Re: Tutorial 1 - Audio programming with gtkIOStream

Post by Franshej » Thu Apr 12, 2018 5:58 pm

Hi
I get the following error when trying to compile:

In file included from SoxInOut.C:1:0:
/usr/include/gtkIOStream/Sox.H:50:20: fatal error: config.h: No such file or directory
#include "config.h"
^
compilation terminated.

Frans

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest