Enabling blind and partially sighted people to engage with visual data

Existing solutions to help blind and partially sighted people experience information – such as Braille – are not suitable for adequately relaying graphs, maps and other visual data. PhD student Albert Higgins is working on a tactile pin display with interactive audio to help solve this problem.

Solutions such as Braille have a notable positive impact on blind and partially sighted people, enabling them to read written information. But maps, images, graphs and other spatial information is much more difficult to get across to users through existing technologies. At present, solutions for this include audio descriptions, embossed paper, or 3D printed models – but there are issues with each of these. Audio descriptions don’t adequately impart spatial concepts like embossed paper can. But embossed paper is expensive, and once a visual concept is printed on it, it cannot be changed. 3D printed models are interactive and relay information very well, but they take several hours to print, and cannot be modified once they are printed. For data visualisations, there are often changes to datasets, and these cannot be reflected using existing methods.

As such, there is a need for a device that can display visual data to blind and partially sighted people, with the ability to change what is displayed easily.

Making visual information accessible

In response to this, PhD student Albert Higgins is focusing his PhD research on creating a pin display that can solve this problem.

Higgins started his research looking at navigation, which is a key issue for blind and partially sighted people when it comes to engaging with visual content. He then started exploring a range of different multimodal shape displays, which have different actuators in each block and rise up and down in response to information being fed to it. Through this work, Higgins discovered the importance of having different types of tactile sensations to meaningfully depict visual data. Shape, texture, heat conductivity and compressibility all play a role in how visual data comes across through touch.

As a result, Higgins created a tactile pin display with actuators that include all of these aspects, as well as audio elements. In addition, he has been working on hand and gesture tracking, as well as making the audio aspect interactive, to create a much richer experience for blind and partially sighted people. Information can be relayed to the device via open source APIs and scripts that Higgins has developed, that the device then renders on the display.

Higgins tested some of these ideas through a weather forecasting application. He created a 2.5D model of the UK and added different weather forecasts to it. As users moved their fingers across the model, they could hear different weather forecasts, such as the sound of rain and thunder – or nothing when fine weather was forecast. Users who tested this device said it was intuitive and offered a rich experience.

Next steps

Now that the initial adaption has been performed, Higgins will explore it’s use for specific applications using co-design and iterative development with blind and partially sighted people. A promising area is its application for games, or using a gamified approach to help users more quickly develop familiarity with the device.

Higgins did an undergraduate degree in chemistry, before working as an environmental consultant and completing a Master’s in Computer Science. Experiences while volunteering in Germany to help with the Syrian refugee crisis in 2014-15, helped him appreciate the importance of both design and engineering. This led to him returning to London to gain a second Master’s in Human Computer Interaction (HCI), to help design more effective applications and products. He worked as a user researcher before joining the GDI Hub to do his PhD. “Through my volunteering experience, I realised there wasn't really a lack of engineering talent. There’s a lack of design talent,” Higgins said. His experience at UCL Computer Science gave him some vital skills to make a difference in this field. “I’m passionate about computer vision and assistive technology. I want people who are blind or partially sighted to experience information without needing any help, in a way which is comparable to someone who's sighted.”