\documentclass[12pt, a4paper]{article} \usepackage{graphicx, url, color, lscape, listings} \lstset{language=Java, commentstyle=\color{blue}, keywordstyle=\bfseries\color{red}} \lstset{numbers=left, numberstyle=\tiny, stepnumber=2, numbersep=5pt} \begin{document} \setlength{\parindent}{0pt} \title{\textbf{CS-M39 Hardware and Devices\\} {\em{Coursework 1: SHAKEsynth}}} \author{Richard Byrne \& Christopher Elsmore\\366455 \& 344400} \date{} \maketitle \thispagestyle{empty} \vfill \begin{center} \includegraphics[scale=0.2]{unilogo.pdf} \end{center} \begin{center} FIT Lab\\ Department of Computer Science\\ Swansea University \end{center} \thispagestyle{empty} \newpage \newpage \tableofcontents \pagenumbering{arabic} \newpage %Literature Review %Built System %Comparison %Conclusions \setcounter{page}{1} \section{Introduction} For our first coursework submission, we decided to create a system using the SHAKE. Of the several different ideas we had for this project we decided to settle on a system we have called SHAKEsynth. SHAKEsynth is a system which uses the SHAKE to sense kinetic motion and combines this with a sound generator, called a synthesiser, which uses different waveforms to create different sounds. \subsection{Background and Similar Systems}\label{related} %Mention history of synth, theremin, airsynth The inspiration for our system comes from our love of music, and the knowledge of electronic instruments that have been used to create music - specifically, the theremin and the synthesiser. We realised that the SHAKE had the potential to imitate such a device, and we decided to explore it's features and those of the mentioned instruments, in order to see which instrument would best match the SHAKEs features, and allow us to create our own version. The theremin uses two antennas which allow a player to alter the output frequency and amplitude of the device hands-free, by simply moving their hands closer to, or further away from the antennas. The position of the players hands affects the radio waves that surround each antenna, and hence allow for even the most minute changes in the radio frequency to be detected by the device, and to allow the sound output to be altered accordingly.\\ \begin{figure} \centering \includegraphics[angle=0, width=60mm]{synths.jpg} \caption{Similar Systems (Clockwise from top left: Alesis AirSynth, Korg Kaoss, Kaossilator \& Theremin).} \label{synths} \end{figure} A synthesiser works by generating one or more signals of varying frequencies, amplitudes and wave types, and filtering and combining them. They have enjoyed increasing popularity since their conception, and whilst initially quite expensive, are now relatively affordable. Whilst most are normally operated with a standard piano like keyboard, with additional controls for altering the filtering and combining process, alternatives use differing input methods, such as using the hand position above the device to reflect Infrared Light (Alesis AirSynth).\\ Whilst not strictly a synthesiser itself, the Korg Kaoss pad uses a similar movement and position based input method. A touch pad is employed, and the co-ordinates of a users finger are used to apply various effects to an already provided signal. Korg have recently created a device call the Kaossilator that combined a simple synthesiser and digital sampler with the same touch pad interface. The operation of the devices will be explained in more detail in Section \ref{Compare}, where we compare how our system operates, compared to the ones mentioned above. Images of the discussed systems can be seen in figure \ref{synths}. \section{Constructed System} %Made use of Processing, talk about code, and what the shake does %Nice picture, can say step by step, this axis does this, pressing the button does this. We built the system using a development environment called {\em Processing}\footnote{Available from \url{www.processing.org}.} Processing is Java based, and contained several libraries and examples that we were able to look at in order to provide inspiration and guidance when designing and building our system.\\ The system is shown in figure \ref{shakeAxis}, and consists of just one SHAKE. The arrows indicate the movements that are detected by our system, and these movements have the following affects: \begin{itemize} \item{\bf Red Arrow: Downwards\\} Decreases frequency. \item{\bf Red Arrow: Upwards\\ } Increases frequency. \item{\bf Blue Arrow: Left\\} Pans sound to the left. \item{\bf Blue Arrow: Right\\} Pans sound to the right. \item{\bf Device facing upwards (Upper Green Arrow) :\\} Sound Plays. \item{\bf Device facing downwards (Lower Green Arrow):\\} Sound is muted. \end{itemize} \begin{figure} \centering \includegraphics[angle=0, width=60mm]{shakeAxis.png} \caption{SHAKEsynth: Arrows depicting appropriate movements.} \label{shakeAxis} \end{figure} The SHAKE also has a navigation wheel, this is indicated in figure \ref{shakeAxis} by the yellow arrow. We also decided to make use of this component and hence decided to add the functionality, of allowing the user to alter the wave form of the sound that would be emitted. There were 4 potential sound waves that could be selected, these were: Saw Wave, Sine Wave, Triangle Wave and a Square Wave. The user can select these simply by scrolling the navigation wheel up and down. Doing so selects the next wave in a loop type format. That is to say, if the user just keeps pressing the navigation wheel up (or vice-versa), then the waves will cycle accordingly (Saw, then Sine etc.).\\ Below is a sample of code we wrote to control which wave the user had selected. For ease, we simply had all of the waves outputting at the same time, but set their amplitude to 0, or 1, (Muted, or Unmuted respectfully). This meant that we didn't have to recreate the sound signal from scratch every time we switched to a different wave. Therefore, in each instance the case statement meant that only one particular wave would have its amplitude set to unmuted, at any one time. \begin{lstlisting}[frame = single, breaklines=true] void switchWave() { switch(selectedWave) { case 0: mySawWave.setAmp(1); mySineWave.setAmp(0); myTriangleWave.setAmp(0); mySquareWave.setAmp(0); break; case 1: mySawWave.setAmp(0); mySineWave.setAmp(1); myTriangleWave.setAmp(0); mySquareWave.setAmp(0); break; }//end switch } \end{lstlisting} \section{Comparison with Existing Devices}\label{Compare} %Short section! This does it this ways ours does it AWESOMER! Our system differs from those mentioned in section \ref{related} in terms of how you use the system. The Korg Kaoss and Kaossilator require some physical touch interaction in order to control the system, an example of the Kaosilator in use can be seen in figure \ref{kaossilator}. The AirSynth and Theremin are hands-free devices, so the user interacts with them by placing their hands at various proximity's to the sensors on the devices. The hands-free nature of these two devices is more similar to our system, however.\\ As such, our system takes inspiration found in these existing devices, but takes advantage of its portability in a way that the other devices do not. The other devices work by either needed direct, touch based input, or by generating a field, whether with infrared light (AirSynth), or radio waves (Theremin). Our device however, is self contained and highly portable, simulating the effects of these systems, but in a far more manageable way. \begin{figure} \centering \includegraphics[angle=0, width=60mm]{Kaossilator.jpg} \caption{SHAKEsynth: Arrows depicting appropriate movements.} \label{kaossilator} \end{figure} \section{Conclusions and Future Work } %Could be expanded to include 2 SHAKES, combine with existing devices . Whilst developing the system we had several ideas for how to improve and further enhance the functionality. However due to the nature of the project, and time frame these ideas were not possible to facilitate to fruition. We discuss these ideas here, and propose them instead as ideas for future work.\\ Currently the SHAKE is held in the hand, it could be potentially beneficial to actually strap the SHAKE to the wrist, or the back of that hand, to allow for further degrees of rotation. Additionally we only make use of one SHAKE unit, but it would be possible to use two SHAKEs, perhaps one on each hand or wrist, in conjunction with each other.\\ Another additional improvement could be to combine the SHAKE with an existing synthesiser device, in order to enhance the usability experience of the existing device.\\ However, despite our ideas for future improvements we believe that the device we created is a novel system, simulating devices already in use for a similar purpose, but doing so in a new way. The system was also extremely fun to build and greatly improved our knowledge of how to interact with the SHAKE and to develop software and systems to use with it. \newpage \begin{appendix} \footnotesize \begin{landscape} \section{Full Code Listing} \begin{lstlisting}[frame = single, breaklines=true] import processing.serial.*; import ddf.minim.*; import ddf.minim.signals.*; Minim myMinim; // Create Minim object AudioOutput myAudioOut; // Create AudioOut object SawWave mySawWave; // Create Wave objects SineWave mySineWave; TriangleWave myTriangleWave; SquareWave mySquareWave; Serial myPort; // Create myPort object from Serial class String serialString; // Data received from the serial port as a String int cr = 13; // ASCII char Carridge Return = 13 int lf = 10; // ASCII char Line Feed = 10 int selectedWave = 0; // Int to store the current slected wave int portamento; // Amount of portamento to use Boolean mute = false; // Mute toggle void setup() { size(512, 200, P2D); // Draw a 512x200 canvas myMinim = new Minim(this); // Create Minim sound object myAudioOut = myMinim.getLineOut(Minim.STEREO); // Create lineout from minim, default bufferSize is 1024, samplerate 44,100 - 16bit mySawWave = new SawWave(440, 0.5, myAudioOut.sampleRate()); // Create waveforms, at A440Hz mySineWave = new SineWave(440, 0.5, myAudioOut.sampleRate()); myTriangleWave = new TriangleWave(440, 0.5, myAudioOut.sampleRate()); mySquareWave = new SquareWave(440, 0.5, myAudioOut.sampleRate()); portamento = 200; // Set the portamento to use on all waveforms mySawWave.portamento(portamento); mySineWave.portamento(portamento); myTriangleWave.portamento(portamento); mySquareWave.portamento(portamento); mySawWave.setAmp(0); // Set the amplitude of all waves to zero, muting them mySineWave.setAmp(0); myTriangleWave.setAmp(0); mySquareWave.setAmp(0); myAudioOut.addSignal(mySawWave); // Add the signals to the audio lineout myAudioOut.addSignal(mySineWave); myAudioOut.addSignal(myTriangleWave); myAudioOut.addSignal(mySquareWave); String portName = Serial.list()[0]; // Create Port name from the []th serial in the list myPort = new Serial(this, portName, 230400); // Setup coms to SHAKE myPort.write("$WRI,0000,82"); // Exclude all data but ACC + Nav Button data myPort.clear(); // Clear port // Discard first serial string, in case its partly formed println(serialString); serialString = null; } void draw() { background(0); // Use a black background stroke(0,255,0); // Use green stroke colour while (myPort.available() > 0) { // When data is avalible, serialString = myPort.readStringUntil(lf); // Read data until LF (Linefeed) char is found (End of SHAKE data string) if (serialString != null) { // If the data read in does not equal null; String[] serialArray = split(serialString, ','); // Split the string using commas as a delimiter if (serialArray[0].equals("$ACC")) { // If string is data from the accelerometer; float x = float(serialArray[1]); // Split the XYZ data into seperate floats float y = float(serialArray[2]); float z = float(serialArray[3]); if (z > 600){mute = false; switchWave();} // If the SHAKE is facing up, enable waves else if (z < -600){mute = true; muteAll();} // If the SHAKE is facing down, mute all waves if (y > 600){increaseAllPan();} // Control panning from left and right accel. data else if (y < -600){decreaseAllPan();} setAllFrequency(x); // Set the wave frequency to current X accel. value }//End if else { serialArray = split(serialString, char(cr)); // If the string is not ACC data, remove the last CR char if (serialArray[0].equals("$NVD")) { // If string is a Nav button Down packet; if (selectedWave == 0){ // Decrease selectedWave, or return to 3 if it = 0 selectedWave = 3; } else { selectedWave--; } }//End if if (serialArray[0].equals("$NVU")) { // If the string is a Nav button Up packet; if (selectedWave == 3){ // Increase selectedWave, or return to 0 if it = 3 selectedWave = 0; } else { selectedWave++; } }//End if }//End else if(!mute){switchWave();} // As selected wave may have changed, if not muted, enable wave } // Display wave code, based on Processing example code "SineWaveSignal" for(int i = 0; i < myAudioOut.bufferSize() - 1; i++) { // For a whole buffer's worth of audio; float x1 = map(i, 0, myAudioOut.bufferSize(), 0, width); // Calculate x co-ords, re-mapped from ratio of buffersize to ratio of window float x2 = map(i+1, 0, myAudioOut.bufferSize(), 0, width); line(x1, 50 + myAudioOut.left.get(i)*50, x2, 50 + myAudioOut.left.get(i+1)*50); // Draw lines for each sample in audio buffer line(x1, 150 + myAudioOut.right.get(i)*50, x2, 150 + myAudioOut.right.get(i+1)*50); }//End for }//End while }//End draw void setAllFrequency(float frequency ){ // Sets the frequency of all waves, after summing 1000 to the frequency to prevent negative Hz frequency += 1000; mySawWave.setFreq(frequency); mySineWave.setFreq(frequency); myTriangleWave.setFreq(frequency); mySquareWave.setFreq(frequency); } void increaseAllPan() { // Increase pan by adding 0.075 to current value, and applying it float pan = mySawWave.pan(); pan += 0.075; mySawWave.setPan(pan); mySineWave.setPan(pan); myTriangleWave.setPan(pan); mySquareWave.setPan(pan); } void decreaseAllPan() { // Decrease pan by subtracting 0.075 to current value, and applying it float pan = mySawWave.pan(); pan -= 0.075; mySawWave.setPan(pan); mySineWave.setPan(pan); myTriangleWave.setPan(pan); mySquareWave.setPan(pan); } void muteAll() { // Set amplitude of all signals to zero mySawWave.setAmp(0); mySineWave.setAmp(0); myTriangleWave.setAmp(0); mySquareWave.setAmp(0); } void switchWave() { // Switch to the currently selected wave switch(selectedWave) { case 0: // Sawtooth wave mySawWave.setAmp(1); mySineWave.setAmp(0); myTriangleWave.setAmp(0); mySquareWave.setAmp(0); break; case 1: // Sine wave mySawWave.setAmp(0); mySineWave.setAmp(1); myTriangleWave.setAmp(0); mySquareWave.setAmp(0); break; case 2: // Triangle wave mySawWave.setAmp(0); mySineWave.setAmp(0); myTriangleWave.setAmp(1); mySquareWave.setAmp(0); break; case 3: // Square wave mySawWave.setAmp(0); mySineWave.setAmp(0); myTriangleWave.setAmp(0); mySquareWave.setAmp(1); break; }//End switch } void stop() { // Stop all audio and close outputs on exit myAudioOut.close(); myMinim.stop(); super.stop(); } \end{lstlisting} \end{landscape} \end{appendix} \end{document}