4i: Immersive Interaction Design For Indie Developers With Interactive Machine Learning

Funder:

The Engineering and Physical Sciences Research Council (EPSRC)

Value:

£497,829

Collaborators:

Marco Gillies Goldsmiths, University of London

Rebecca Fiebrinck Goldsmiths, University of London

Phoenix Perry, University of the Arts London

Bruno Martelli

CU Project Team

Co-I Ruth Gibson

Project Objectives

This project aims to support independent developers and artists in designing movement and body based interaction for Virtual Reality and immersive media, by building tools that allow designing by moving via Interactive Machine Learning. By creating better tools and working processes developers, we aim to enable those developers to create better movement interaction for players, audiences and end users.

Objective 1:

Enable the creation of a new generation of movement interfaces for immersive experiences by small scale creative teams, independent developers, artists and others

Objective 2:

Create easy to use, immersive tools, based on interactive machine learning, for movement interaction design, in order to help achieve Objective 1.

Objective 3:

Develop design methodologies and workflows for movement interaction that centre on movement.

Objective 4:

Use participatory design to understand the needs of our target users (small scale creative teams, independent developers, and artists) and ensure that the tools from Objective 2 and methods of Objective 3 meet those needs.

Objective 5:

Better understand the process and challenges of applying Interactive Machine Learning to this domain in order to inform future research in this area and to suggest future uses. This will be done through evaluating the creation and use of our interactive machine learning tools.

Impact

The major beneficiaries of this project will be developers of Immersive Media (VR, AR and Mixed Reality experiences such as location based installations). They will benefit from having better tools and techniques for designing interaction, making it easier to develop more compelling forms of interaction that focus on users' body movements rather than on traditional button/joystick. This will not only enable them to make their experiences better but potentially to create new forms of experience, leading to new markets.

This project particularly focuses on small, independent developers, artists, and underrepresented groups within games and digital media. These SME developers are key both to the economics of the industry (over 95% of UK game developers are SMEs) and also to the cultural strength and diversity. Unlike major companies, they do not have access to the resources and specialist expertise current approaches to complex movement interaction and for the use of machine learning. It is therefore particularly important for them to have to types of usable and rapid tools that we intend to develop in this project.

This project will directly work with participants in these demographics in our hackathons. Our participants will be the first major beneficiaries, with immediate access to the software and training in its use. The software will be directly disseminated to developers as a plugin to a popular development platform such as Unity or Unreal Engine. This will make it easy for them to integrate it into their work. We will also develop a range of online training and support materials, including video, blog posts and MOOCs.

The result will be important economic impacts. The games industry is an important part of the UK economy (£4.19bn in 2015) and VR is the fastest growing sector of the UK entertainment industry (predicted £1.2bn by 2022). Supporting indie developers in creating new and innovative forms of immersive interaction in a cost effective way will support the general health of the immersive industry in the UK, and therefore economy in general.

There are also important cultural benefits. The tools we are developing will not only make it easier to create existing forms of interaction, but to create new forms of experience. This will help develop the cultural and artistic depth and diversity of immersive media, particularly if the tools are in the hands of creators out of the mainstream of games development, such as fine artists, women and other underrepresented groups (key participants in this project). This will help foster a rich and diverse culture within the new media of the 21st century, greatly benefiting the audiences for this media, opening them to new audiences and strengthening UK culture in general.

While the major impacts of this project are in immersive media, a secondary objective is to improve the usability of machine learning in general. ML is one of the most important technologies of the 21st century and has applications in a huge range of areas from medicine to advertising and finance to robotics. The impacts of machine learning are potentially huge in many sectors, however, it is still considered an obscure and difficult to use technology. Making it more usable will increase this impact by making the technology accessible to more people and sectors. Machine Learning could be used for tasks that it would not be used for otherwise because people working in those areas do not have machine learning expertise. Importantly, it may also help mitigate some of the negatives of the technology by expanding the scope of who can use it, therefore democratising machine learning. These impacts will be more indirect since the tools we develop will be specific to immersive media, however, we will disseminate general ML results through academic publications for researchers and experts and blog posts for broader public of ML practitioners and software developers.