After a great deal of brainstorming about how we could use augmented reality to transform one’s surroundings into some kind of music creation system, my teammates and I decided that we wanted to be able to attach looping sounds to the objects around us. This system was exciting for everyone on the team — musicians and non-musicians alike — because it would allow users of any skill level to create music in the form of a spatial audio experience.

While the rest of our wonderful teammates collaborated on designing the UI and UX, James and I were the small but mighty software development team. I handled most of the interactions and created the Unity scenes, while James was the mastermind behind the object detection system. I also assisted Marina with getting her super cool audio visualization to appear correctly. She was the one who designed and coded the visualization; all I did was handle some minor debugging and add it into the sound object control menu.

Click here to watch the demo video.

If we continue this project in the future, we have quite a few features we would like to add, such as uploading custom sounds, using non-looping sounds, and recording and exporting one’s musical creations.