SHAFT

This projects purpose is to explore the principles of time and space in simple in simple third person games in a 3D OPENGL Processing Environment. In this project we use sound and imagery to convey a space and time warping effect and bewilder the player. We will be externally sourcing our music.
I first started by creating a blog and assigning it to a domain name for the project (the projects address is www.projectfireball.com). We used this blog to post any updates to code we could. This allowed us to go back and find any fixes that we added. We also used it for adding video links and images for project ideas.
I designed and uploaded the original class layout for the game which has had changes made to it as we developed the product. After Several weeks of discussion on how the wall code should be made I came up with a simple solution in which I concluded to keep the code simple the camera itself should be used to generate the four sides of the wall. This code for generating a floor does not have to be modified instead only the camera angle has to be modified.
Without the code for generating a single wall the basic principle to this can be found in the code below:

void update(){     
     this.rate = 0.5; // the rate at which the tiles move relative to the cam height.
     this.ypos = (frameCount%this.rate);  
     
  // ------------------------- DRAW FLOOR TILES ------------------------------------
  
     camera(0,0,this.cameraHeight,0,this.ypos+400,40,0,-1,0);  
     drawtiles(this.tiles_floor, this.rate, this.ypos);
  
  // -------------------------- DRAW CEILING TILES ----------------------------------
  
     camera(0,0,this.cameraHeight+24,0,this.ypos+400,73,0,1,0);
     drawtiles(this.tiles_floor, this.rate, this.ypos);
      
      
  // -------------------------- DRAW RIGHT WALL TILES ------------------------------
  
     camera(10,0,this.cameraHeight+10,20,this.ypos+400,60,1,0,0);
     drawtiles(this.tiles_floor, this.rate, this.ypos);
     
  //--------------------------- DRAW LEFT WALL TILES -------------------------------
     
     camera(-10,0,this.cameraHeight+10,-20,this.ypos+400,60,-1,0,0);
     drawtiles(this.tiles_floor, this.rate, this.ypos);  
    
    menu.update();    }

as you can see from the code I have changed the way the camera is facing then generated a new wall. Although there was much disagreement in the group on this way of generating the walls as there was misunderstanding as to how it was generating it. A better solution was never found so this was the method we ended up using, although I would have much preferred the rest of the group to have seen the benefits.
How is a single wall generated?
The tiles are draw using a single for loop (with nested for loops) to fill array values which is then used to determine the x and y position of some rectangles. The camera is then rotated so that you are looking at the side of the rectangles as though you are lying down and looking along the floor. Then the floor is rotated and drawn again. Then the values were carefully tweaked and the sides were put together.

Making the walls interactive with sound

The next step I took to making the walls look more interesting was adding sound interaction to them. To do this I have used Matthew Yee-King’s classes to get samples of sound and put them into a buffer. This means that you can read the various frequency values and use it for beat detection and for the sound to interact with the walls.
Here is a short part of the code for a very basic example of how I have used the frequency to alter the walls size positon and color:

if(abs(world.audioData[world.readHead2]) > 0.3) {
if(frameCount%900<600 && frameCount%900>300){
stroke(abs(world.audioData[world.readHead2]*500),
 abs(world.audioData[world.readHead2]*10), 
abs(world.audioData[world.readHead2]*10));
fill(abs(world.audioData[world.readHead2]*500), 
abs(world.audioData[world.readHead2]*10),
 abs(world.audioData[world.readHead2]*10), green);
}
if(frameCount%900<300){
stroke(abs(world.audioData[world.readHead2]*10),
 abs(world.audioData[world.readHead2]*500),
 abs(world.audioData[world.readHead2]*10));
fill(abs(world.audioData[world.readHead2]*10), 
abs(world.audioData[world.readHead2]*500), 
abs(world.audioData[world.readHead2]*10), green);
}
if(frameCount%900>600){
stroke(abs(world.audioData[world.readHead2]*10), 
abs(world.audioData[world.readHead2]*10), 
abs(world.audioData[world.readHead]*500));
fill(abs(world.audioData[world.readHead2]*10),
 abs(world.audioData[world.readHead2]*10), 
abs(world.audioData[world.readHead2]*500), green);
}
  }

Here is an example of a bug-fix I implemented designed to try and stop background noise in the wav recordings used from effecting the look and feel of the game (the visual sound element).

    if(abs(world.audioData[world.readHead2]) > 0.005) {
    rect(this.position.x+inc,this.position.y+inc,this.w-  abs(inc),this.h-abs(inc));
    }
    else{
     rect(this.position.x,this.position.y,this.w-(abs(inc)*0.1),this.h-(abs(inc)*0.1)); 
    }

Another idea I had to implement was to smooth out the way the rectangles change size. With just the code above the rectangles will change color, size and position in a jittery way and for the human eyes it doesn’t represent the music very well. To fix this I used a threshold. and until the threshold is met it runs this code:

if(inc > 20 || inc < -20){
   inc = 0;
  }
  if((world.audioData[world.readHead]) < 0){
  inc++;
  }
  else{
   inc--; 
  }   
}

This means the blocks, walls, player or whatever is effected by inc can’t go passed a certain width and height, won’t be able to go above a certain pigmentation and won’t be able to move more than 20 pixels up, down , left or right. This gives a smoother look and feel meaning that the walls itself (as well as looking more hypnotic) will represent the music more heavily and set the player into more of a trance.
Increasing the effect
To make the effect look and feel more in-sync I have added another ‘bug-fix’. Because the speed at which the tiles are rendered and the human eyes can interpret them then the brain relate that to the sound I have made it so that the Audio Data is read 4096 readings ahead of time by implementing a second Head. This means the video should be more in-sync with the sound in terms of the way it is processed. However I have not fully researched this and gone by the word of a well renowned researcher on its effectiveness. In my own opinion and from my own tests it does look more effective, however we can sometimes trick our minds into thinking something is different when really it isn’t.
Here is the code for this fix:

        buffer[i] = audioData[readHead];
        readHead = (readHead) % audioData.length;
        if(readHead % audioData.length >= audioData.length - 5000){
          readHead = 0;
          readHead2 = 0;
        }
        else{
        readHead2 = (readHead) % audioData.length +4096;
        }
                
        readHead += speed;  // set the speed of the music
        wait++;

So simply put, if the second head gets right near the end then move it back to the beginning. Also move back the first head to the beginning just for that section as the second head is a future representation of the first head.
Putting a menu in a OPENGL environment
Although the title sounds like putting in a menu would be relatively simple.... it is not... when putting in menu to an OPENGL environment the way in which it renders breaks the timeline rule. What I mean by this is that OPENGL has its own depth buffer and this overrides itself and overlays its rendering on anything else if it is closer in depth.
So for example... it renders more like an eye... than how your eyes would perceive the stacking of several books. It may not necessarily be the last book put on the pile that you can see... but its actually the book that is closest to you.

The problem I had was that the tiles were rendering themselves over the menu causing the menu to be unreadable. To fix this I had to clear the OPENGL depth buffer and then draw the menu to get the menu to overlay the rectangles. I actually was closing and opening me eyes then realised that... when looking at an optical illusion... sometimes certain layers can look like they are in-front of others or visa-versa after closing and opening our eyes again... however they are not... it is just the way our eyes render the image, hence the solution to this problem.

Here is the code:

pushStyle();
fill(0);
gl.glClear(GL.GL_DEPTH_BUFFER_BIT);
camera(width/2,height/2,+520,width/2,height/2,0,0,1,0); 
rect(width, height, -100, -20);
fill(255);
textSize(10);
text("Score   " + score, width-90, height-5);
popStyle();

As you can see this also required me to redirect the camera to a ‘birds eye view’ again.
The Android version

In the timescale we had I would say that the android version of the game is not as developed as I would like it to be. However there is a completed version on the way which might possibly be found at:

http://www.projectfireball.com/wp-content/uploads/

Hopefully by the time you have read this there will also be a new WordPress blog post for the android version.

Sounds generative walls.

Version 2

A World Of Creation

Welcome to jaboston.com, my online portfolio.

I am currently a student on the goldsmiths creative computing course and this is some of the material which I have created in my time at Goldsmiths.

Please feel free to explore the wonders of my work using the tabs at the side or above.