Bankart Surgery Release

Created for Fundamental Surgery, this virtual reality simulation of a Bankart procedure creates pioneering medical orthopaedic training with a haptically-enhanced arthroscopic shoulder procedure.

Here you see an arthroscope being manoeuvred using a haptic arm with full kinesthetic capabilities.

By enabling haptic feedback when drilling, hammering and more, the simulation provides a new level of immersion for orthopaedic procedures - enhancing surgical skills acquisition through realistic rehearsal.

A video of the simulation in action can be found here.

Our Patent for Haptic feedback in Virtual Reality

The patent details how we use a combination of worn sensors and machine vision for pose estimation, before translating user interactions into force feedback. This is simulated using finger actuators and electro-muscular stimulation - a culmination of over 3 years of research and development!

The patent we filed in February 2018 has been published! Having been through numerous searches and legal processes, Valkyrie Industries Haptic feedback technology is now recognised and protected by the UK Intellectual Propery Office. We’ll be taking it global next, watch this space..

The full patent can be found here, it’s amazing that the design goals from early research in 2017 align so well with the Alpha Devkit a full 3 years later, even as our experience grows.

Evonik, Essen

We travelled to Germany to demonstrate the Valkyrie Alpha devkit

Live demonstrations can be nerve-wracking, always a novel experience for those involved!

I travelled to Germany with Dr Ivan Isakov to demonstrate the Valkyrie Alpha devkit and showcase it’s potential for applications in industrial workplaces. The setting was magnificent; Evonik hosted the trade conference in the Zollverein, a disused coal mine outside of Essen. The UNESCO site is beautifully preserved with industrial architecture expressing the functional ideals of the New Objectivity and Bauhaus movements.

This was a fitting locale for exploring technological advancements in industry, surrounded by the machinery of a bygone era we heard talks from several experts discussing the ramifications of ‘Industry 4.0’. Our very own Ivan gave a talk on the benefits of an XR enabled workplace, introducing our tech as a means of extending the capabilities of skilled technicians.

I learned a lot from the trip, not least that the opportunity for XR in Industry is enormous. There was genuine excitement around our kit, however it is also clear that changes in practice happen very slowly in a sector where innovation is expensive. We also realised that we need to do more work on sizing; some of the more muscular attendees struggled to fit into the Alpha!

I’m looking forward to further exploring XR in industry. Our hosts were keen to find some future collaboration, and I’d be delighted to return to the dramatic architectural vistas of Westphalia.

Writing software for haptics

Over the past year and a half we have had to create a software suite that connects, controls and calibrates a completely novel style of haptics, and I’ve learned a lot along the way…

1)      Hardware is hard! I’ve written a separate post about this but hardware requires several software components for communication, so planning interprocess communication through APIs is paramount.

2)      Asynchronous processes are king. The various microcontrollers the software components run on will all have wildly varying capabilities – you don’t need the latest multicore processor to change a few actuators. The ever-decreasing power and scale of controllers as they reach the periphery of an actuator network mirrors the keystones of ubicomp: Pads, Tabs and Dust. As with any networked device, these need to function independently; however they must also act as responsively as possible, bringing me on to…

3)      Latency! Similar to the visual response from XR, your body recognises even slight delays in physical stimuli which can be very jarring. Keeping packet sizes small, minimising the number of nodes from input to outputs and using non-blocking network processes are all crucial to succeeding.

4)      Literal moving parts: This initially caught me off guard, but physical state changes in actuators need to be considered when building the software control systems for haptics. This introduces a new question into the control paradigm; should the local controllers have set routines or should the hub have full control? With set routines, the packet sizes shrink and the controller can better manage the input queue, allowing actuators to complete cycles to better track physical states - important when you don’t have sensors to do so. This comes at the cost of flexibility and creativity for the haptics experience designer, as the actuator performance is hard coded. In the end, we took something of a hybrid approach, which worked well with our more dynamic linear actuators.

5)      User calibration. I have left probably the most important area until last: tailoring the haptic experience to account for an individuals’ neurophysiology to elevate the immersion to a whole new level. Thresholding each actuator according to user preference creates a parametric model for their sensory input. A huge part of the software and GUI was dedicated to this, creating a unique profile to ensure the users comfort and enjoyment!

Valkyrie in Hong Kong

Having completed the presitigious Brinc Incubator in Hong Kong, Valkyrie returned to demonstrate the wearable Alpha devkit for our patented touch technology and pitched at Rise HK.

Creative Summit Sweden

Hosted in the continuous daylight of Skelleftea, northern Sweden, the Creative Summit is a terrific event mixing technology, design and culture with a variety of speakers. I was invited to talk about Haptics and to give a live demo of Valkyrie’s touch technology.

Robotics in VR with the European Space Agency

We undertook a project to create gesture based control for robotic arms, to be used on a return mission to Mars!

Valkyrie Alpha, wearable haptics

The Valkyrie Alpha devkit is here!

Studies in EMS with Southbank University

Examining correlations between lefting weights and the effects of electromuscular stimulation with Doctors Katia Mileva and Ivan Isakov.

Dubai Expo 2020

Immersive + Santiago Calatrava

Tokyo Stadium

Working with Marc Winklhofer and Patrik Schumacher from Zaha Hadid Architects, this project attempted to visualise the scope of the planned Tokyo Stadium for the 2020 Olympic Games.

The project has particular resonance, in part because it was never realised but mainly due to the untimely passing of the visionary architect Zaha Hadid.

Machine Analysis of Architectural Style - Masters Thesis

A Comparison of linguistic definitions with spatial analyses of Andrea Palladio & Frank Lloyd-Wright.

My masters thesis for UCL inestigated machine inference of architectural styles, using plans from two seminal architects of quite different epochs.

Screen Play

Second Semester Project - UCL

Lorem Ipsum

Press Play London

A workshop hosted by the interactive architecture lab, project development over a weekend.

The Interactive Orchestra is a collection of rapidly prototyped instruments which generate midi files from body gestures. These midi files are then sent to a laptop to generate a musical score.

Hacky Bird

“Whatever you do, don’t make another flappy bird clone”

Hackathon project at UCL.

We built a version of flappy bird in VR, with users flapping their arms to fly! Participating over two days in the UCL main quad, it won the most entertaining project.

Kinesic

My First Semester masters studio project involved creating wearable devices to learn human gestures. Initially working with Myo armband, the device was eventually extended using IMUs to increase the users range of movement.

The Myo uses electromyography (EMG) to measure muscular loads and therefore predict hand gestures. Whilst the factory software allows for four pre-assigned gestures, a more detailed profile could be built using machine learning to better predict finger positions.