Sonic Kayak sound mapping workshop

“You all really put a heart felt thanks from me back into humanity, I really miss canoeing and will now keep doing it. It felt incredibly ground breaking what you were trying to achieve with endless possibilities canoeing or kayaking or cycling or embedded into urban realm.”

“I just wanted to thank you all for the lovely day and wonderful experience I had yesterday at the quarry. I have got rather grey in my thoughts recently and you all replaced the colour for me.”

Last year we were contacted by the accessible kayaking organisation Access Lizard Adventure to see whether the Sonic Kayak could be used to help people with visual impairments kayak independently. We were lucky to receive some funding from Smartline to start working on this together. Since then, we’ve done quite a bit of R&D on the system to make it safer for people without full sight (and therefore also safer for everyone else), including removing most of the cabling, using waterproof bluetooth speakers, and redesigning the kit so it sits out of the way behind the kayaker. One of the other major bits of work has been upgrading the satellite navigation unit for one that is more accurate, using GPS (the US satellite system), GLONASS (Russian satellites), and Galileo (the new European satellites).

The Sonic Kayaks have several layers of sound:

• Temperature sensors on the kayak are sonified in real time, the pitch rises in warmer water and vice versa.
• Hydrophones relay underwater sounds from boats, water and wildlife.
• Sounds can be placed as zones on a map, and the map installed on the Sonic Kayaks so that the sounds are played automatically once the kayak enters a zone, triggered by GPS.

This approach to sound and sonification has remained the same since the project started, and comes from merging the sound art aspects of the original Sonic Bikes with environmental citizen science (you can read more about this here). As part of the process of adapting the system, last week we held a workshop for people with visual impairments to start thinking about how to use sound for augmenting kayaking and facilitating greater independence.

We had 5 people with visual impairments, together with some of their partners, Access Lizard Adventure, researchers from the European Centre for Environment and Human Health, and of course sound artist Kaffe Matthews who pioneered the Sonic Bike project originally (huge thanks to FEAST who provided extra funding to bring Kaffe over from Berlin). The workshop was held at Trevassack Lake – a disused quarry which is about to be developed by the Helford River Children’s Sailing Trust as an inclusive water sports centre. As an aside, this was the first fully off-grid digitally-based workshop we have done, which means we are now equipped with a big solar setup for future events, and the fabulous Kat Roberts is now experienced in off-grid catering.

One of the most important aspects of the workshop was simply giving people with visual impairments a chance to go kayaking in a safe environment. Claire Eatock and Matt from Access Lizard Adventure are pros at this, taking people out on tandem kayaks, carefully and thoughtfully guiding them, and making adhoc adjustments like taping a pen onto a paddle to help with achieving the correct orientation of the paddle in the hands.

“I have never been on a kayak before, it was a new experience for me. The kayak on the shore is dead but when floating it becomes alive, the leveled, manmade areas of the shore created a strong contrast with the calm waters of the lake and natural surroundings. It was like moving from the utilitarian to the emotional and spiritual. Kayaking introduces the risk and uncertainty which provides the framework for the experience. The splash and drops of water from my paddles created an initial chaos of sound for a while but the hiss of water from the bows spoke of certainty and direction. Paddles dipping and lifting from the water created a rhythm conducive to a meditation.”

The GPS saga

The main concept being tested was whether we could use sounds placed on a map as warnings, for example drawing a zone around the boundary of a lake so that a sound would play before the kayaker hit the edge. We have always been nervous about this because GPS is notoriously unreliable, and if it is to be used for safety, this is obviously not ideal. The day before the event we went to the site to test the GPS, as we were concerned that the cliffs surrounding the lake might cause problems – we drew a zone around a small square car park to test the system, and it worked perfectly and with very high accuracy. All seemed positive. The first group to go out in the morning of the workshop seemed to be having some problems – we weren’t out on the boats with them but we think that there were issues with the boundary zone sounds not turning off once the paddler had left the area. The test map was designed as a single boundary zone with looping sounds – we believe we may be able to fix this problem by instead making lots of small zones all around the boundary, each with a one-shot sound associated with it.

The later groups to go out on the kayaks had even less luck – the sounds seemed to be either not triggering at all or to be stuck on continuously. We were frantically trying to troubleshoot in the background, resetting the systems and looking at the data being recorded – all three GPS units had started outputting gibberish at the same time. One participant who was ex-military said that the jets that had been flying over earlier in the day would mean that the nearby military base would have scrambled the GPS locally.

With more investigation, we thought it looked like the GPS units had flipped into a proprietary Ublox (the company that makes the chips) binary format. Since we had rebooted the systems so many times at the site, we assumed this flip was permanent – by confusingly once we were back at our studio and turned the Sonic Kayak systems back on again, everything was working perfectly. We are not sure whether the new GPS units that we are using might be problematic in some way, and whether the issue could be easily fixed by going back to our previous (less accurate) technology. Our job now is to see if we can get the GPS to break in the same way – if we can do that then we’ll be in a good position to work on a fix. You can see the ongoing troubleshooting process on this wiki.

If we can’t get the GPS to be reliable, then we have two options: 1. Ditch the idea of using sound for safety zones and instead use it as augmentation of the environment, back closer to the original Sonic Bike approach. 2. Ditch the GPS and instead use wifi triangulation for determining the location of the kayak – this would require us setting up infrastructure at the site rather than relying on external satellite infrastructure, which would be no bad thing. Switching to wifi triangulation would be a major new job, but could be worth it if Trevassack were to be a long-term Sonic Kayaking site.

“I thought of the quarry when it had been a working one. I wonder how deep it is. I could see in my thoughts a wagon and four sweating horses being whipped up the slope from the quarry bottom with massive stones on the wagon bound for the construction of a great house. I could also see, at a different time, a smaller cart with just one pony and loaded with smaller stones used to repair a cottage or wall. Overhead yet another jet roared and I was struck by the juxtaposition of the modern weapon of war overhead and the image of the single man below loading stone in an easy going and timeless manner. I thought of the blasting of the quarry wall to bring down more stone. The quarry men who had gone away to the First World War and again herd the blasting of explosives and this time it would be the collapsing of trench walls instead of a quarry face. I wondered if the quarry men did not return from the war and this was why the quarry was closed and flooded. Only the water on which we were floating lay between my image of First World War trenches and the roar of the Great War bird which screeched above.”

Sound mapping

As well as trying out the existing kit, we worked with the participants to think about how sound could be better used on the Sonic Kayaks. We built a tactile map so that people could feel their way around the geography of the site, which was very well received – in hindsight we should have brought this out right at the start of the day to help people orientate themselves, as it was more successful than verbal descriptions. The map immediately resulted in an understanding of an initially unknown space, guiding people’s fingers around to show ‘you are here’.

The tactile map also meant that we could add zones for the sounds (cable ties cut and glued on – top tip from a participant was that Wikisticks are good for this), so people could learn what to expect beforehand. This is very different to sighted participants who are often happy to explore and discover – with the participants with visual umpairments, knowing what to expect is key. In future we could consider making a map that automatically previews the sounds as the person moves their hands around the map (for example using an approach similar to the one used by Makey Makey).

New ideas for sonic mapping included:

• Use of clock face to represent a “schematic map” (like a metro map) that indicates navigational waypoints around the island, possibly connecting to rocks.
• Navigation was seen as more interesting/important than warning areas.
• Chaining together locations using sound, to help navigate.
• Using memory of songs or poems to indicate order - again familiarity and reinforcement of understanding from other senses is emphasised over surprise.

It might be that the focus on independent kayaking is a bit of a red herring. Participants didn’t necessarily seem to want to be able to kayak independently, one said they prefer being in a tandem because then there’s someone to have a chat with.

“I believe that the sonic navigation should have two sections. One, a final warning, when getting very near the bank; perhaps five yards off. This should be automatically operated as a warning. The second navigational section should be either voice or switch operated and used to indicate the position on the lake by using the clock system to indicate how far the kayak is from a named area of the shore such as the floating pontoon, the islands or the beach. This should facilitate the creation of the image of the area in the mind of a visually impaired person.”

Sound generation

We brought along lots of sound-making equipment, the most useful of which was the Arturia MicroBrute analogue synthesiser which has no screen, and is purely tactile with a keyboard, knobs and sliders. This could be simply explained and resulted in immediate sound material made by the participants. This was restricted to tone generation, filtering and envelopes but gave us a way to articulate discussions we were having on things like useful sounds for echolocation.

Kaffe and some of the participants also recorded sounds while out paddling on the water, which can now be processed to use, and one of the participants wrote a poem that they are now recording which can be added as a zone in future tests.

Discovery of the environment via sensors (like the temperature sensors) was discussed, but it was more important not to impede hearing – for example one participant spoke of being able to tell the difference between tree species by the sounds the leaves make. On a similar note, participants spoke of wanting to be able to switch sounds off, or switch between different sounds, for example when they know they are in a safe spot they might want no sound at all so they can listen to the environment around them.

“The sound shadows from the cliffs were very noticeable although not threatening like taller cliffs and the sound of the wind in the trees helped with orientation. Although I could hear the silent shadow from the cliffs it was always a bit of a shock to put my hand out and feel the warm rock and course grass.”

What next

Over the next month there are two more sessions planned with Access Lizard Adventure and with some of the same participants. Our immediate job is to get the GPS working and figure out a fix to stop it going haywire again. Once this is done, we will start editing some of the new sounds that the participants generated, and place these on a map for testing. It seems that we should make one map that is aimed at navigation (perhaps clock-based around the lake), and another that is a composition of sounds and music with no navigation or safety aims.

Trevassack, a reflection on freedom and losing sight - by participant Zoe Partington:

"A shimmer of metallic silver blue 

dancing on the iridescent, impermanent, distorted liquid surface. 

a shifting shaft of solar light 

Captivating yet never ending, 

a transient emptiness flowing, trailing into eternity 

While for a fleeting moment, a deep breath is drained of life, 

Metamorphosing from a vibrant, rocky, textured edged expanse, 

whirling into black liquorice oily depths. 

The ever-changing refracting, bouncing light 

Glistening, in time ticking, with the warm wind, 

Blistering, a salty taste, with a cocktail of chaotic charm 

Among the tranquil chilly, lapping shores. 

Echoes of turbine blades synchronise with each stroke of the paddle.

A humming scape harnesses a visual paradox"