Things in context
"The Internet at large has made us feel closer and further away, depending on the lens we're using."
The feeling of physical and emotional closeness that we used to enjoy towards people close to us, has, to an extent, been disrupted by the Internet. Connections are made between people of shared interests, visions, opinions and thoughts. This transformation of how we perceive proximity has made one concept all the more important: context.
Context is what defines the world – or worlds – that we live in. What is useful and important in one scenario, would be irrelevant or even dangerous in another. It is something that is vital to disciplines concerning academia as well as those in more creative fields, as the recontextualisation of something can make it interesting in a way that would never have been the case before.
It is also a thread that runs through much of the work of product and interaction designer Alexandra Deschamps-Sonsino. Her background in product design and technology led to the creation of Tinker, which started as a shop based on the popular Arduino platform. It grew to become a studio with a rich mix of people from HCI, academia, innovation studies, interaction design, and electronics engineering. Projects from the studio included work with the BBC on the future of remote controls, to a campaign with Dare and Sony Ericsson, where a Twitter hashtag was linked to a physical installation. As Deschamps-Sonsino observes, "That space got very interesting for agencies in particular, as everyone had a clear idea of what a strategy and digital campaign looked like, but suddently this opened up possibilities for that campaign to exist outside of the digital realm, Facebook and Twitter." The economic downturn resulted in Tinker closing late in 2010, with its founder now working in a freelance capacity, looking at how technologies such as RFID and QR codes are used, and could be used, in everyday life. Such technologies are now becoming more overt – witness the many poster ads now featuring QR codes – but also more covert. Chromaroma is given as an example of a simple recontextualisation of technology: changing TFL's Oyster device from a travel card, to a token in a play experience.
As smartphones start to dominate, it is the physicality of something like the QR code that add both value and ease of use to information gathering. From the perspective of interaction, "...it's a lot easier to explain what technology can do in a screenless world, than it was when everybody had laptops, and had to learn how to use word processing and clunky pieces of software." Visual metaphors feel more natural than clicking a particular mouse button, or to remember a URL (and it will be interesting to see whether URLs are largely replaced by technologies such as the QR code in coming years). By changing metaphors to those which are more natural and more based on a visual experience, the interaction is smoother and more natural – and, let's not forget, easier for novices to pick up and use.
Games in the real world
These more visual means of interaction, however, have already been vocabularised. Think of it as the reverse effect of long-time PC users trying to get their head around a Mac. Deschamps-Sonsino gives the example of Microsoft Surface, whose zoom function required two fingers from both hands, pulling apart from each other. Unfortunately, demonstration devices resulted in confusion among the public, who were already used to Apple's pinch/pull method of zooming – resulting in Microsoft having to cater for it. Where the new frontier lies is in gesture-based gaming, and Deschamps-Sonsino starts with the story of the Wii controller. "That was a little revolution - that you didn't have to sit there with your controller where both of your hands were busy; you could sit on the couch and do nothing. There could be another way. This is a third area - where there is nothing there. The right games and social context are to play a bigger role than before, because if you're in your living room and it's packed with kids on a Saturday afternoon, it's still OK to have a PS3 controller and just play a game on the television. To bounce around in the living room becomes a little more embarrassing - because it's unclear, unless everybody's involved, what it is that you're doing. You're either fooling around, or you're having to say that you're playing a game on the TV on your own, because the difference between fooling around and playing a game becomes invisible." This makes the social context and implications of gesture-based gaming extremely important: because users are not actually holding a controller any more, there is currently no widely-understood visual signifier to suggest that a game is being played. Perhaps, over the next few years, people will start to rediscover the usefulness of the curtains at their front room window.
While this new visual language currently makes it difficult to understand what is happening, it also presents opportunities in terms of friction. Scenarios might change based on how the player is behaving; a level could become more difficult if the user is not moving enough – and if they are straining too much, that interaction could slow down. As Deschamps-Sonsino says, these "Tiny bits of interaction can become sensitive to their environment, and sensitive to people - and how people respond to things."
There is a further, and perhaps rather overlooked, point in gesture-based gaming: the cultural perception of "Invited space". People speaking very loudly on a mobile phone feels uncomfortable in one culture, and acceptable in another. The same culturally-specific approach could be true of a broader shift to non-physical interaction. "Every single country will have their ways of understanding what's happening, and their ways of dealing with it, accepting it, fighting against it. Switching on your phone's speaker while playing music on the bus is something that you would accept kids to do, but you would be completely bewildered if someone else did that. The things you're expected not to do as you grow older, change all the time. So, while the kids have a boom-box mobile phone, will they accept that from the kids in 10 years' time - because they will all be about voice-based control: yelling at things, instead of having to move at all."
Computers and blues
In addition to her freelance work, Deschamps-Sonsino is also an ambassador for LIREC, a European Union-funded research project on how we could live with digital and interactive companions. The project concerns both the physical and emotional implications of such a setting.
The project is clearly exciting on a personal level. "It's pushing the barriers of what product design has to offer. It's AI, it's software design, it's hardware design, it's psychology, it's mythology, it's looking at animal behaviour... and what that means for us. For me, it's interesting to mix with professionals in that field, because they are looking at it from a very different perspective. There are all academics on the project except myself, so I am looking at it with years of experience in entrepreneurial design, and design in industry, and saying 'wow', it's full of potential - potential that I don't think we've realised."
One of the experiments is to look at battery life, and to extend it as much as possible, while giving the robot massive capabilities in terms of movement. It is currently very difficult to offer a robot that can, for example, answer a phone within three rings from across the room, and to avoid a significant impact on its battery. This physicality is clearly intertwined with how the robot behaves, which, if it were human, would change based on the mood of itself and in reaction to others.
"A lot of what makes companions likeable and for us to develop relationships with, is because they don't obey us all the time. Dogs, for example, try to establish what their leeway is, and how much they can ignore your commands. There's constantly a little bit of friction, and that's the nature of human relationships. If human relationships were always about getting along with each other, then we would live in a very boring world. Robots don't understand that. There's no understanding right now of developing robots that disagree, because that would mean understanding where the context was for disagreement, where that context lie, and whether it was alright for you to disagree when it wasn't. That's a part of Artificial Intelligence that is in development, but is nowhere near how we respond to situations. We are incredibly quick in our responses, which makes us very unforgiving towards robots, which are very slow."
Another key aspect is the context of people. "How does a robot know that I am responding in a particular way, and how can it evaluate whether I am in the mood to know what it has to say? So, one of the robots in the project handles face recognition, and tries to analyse emotions. It then tries to track that emotion for a period of time. If it sees that I am displaying a sad face for more than a few minutes, it will establish that I am sad, and then not loiter. That is something that we do every day; we walk into the room and go 'Oh, he's in a bad mood' and we do it almost immediately. We do it almost immediately, and don't need to think about it too much." These small shards of emotion are, of course, very important in daily life. Vocal pitch, the holding of the body, and the speed of walking are just three facets that we are implicly aware of all the time – in that we do not have to verbally acknowledge them to others – but are extremely complex for a robot to understand, and therefore very challenging for the project to address. As Deschamps-Sonsino acknowledges, it's easier for software companions, who can read data such as login records to build up a picture of human understanding – the user might know less about a product if their session duration was low, for example. She is relieved about the fact that we have moved on from the ubiquitous Clippy, Microsoft's Office 97 device that aimed to help, but did not understand the context of a letter – whether a love note or a formal letter of complaint.
A multitude of things
Much of Deschamps-Sonsino's work is loosely based on an increasingly-popular concept, the "Internet of things" - physical devices that can respond to network data. The concept has been around for years, but has only recent started to gain traction, due to the increasing volume of appliances that can be networked. One of the more well-known – and therefore hackneyed – examples of the "Internet of things" is the networked fridge. It's at this point that Deschamps-Sonsino elaborates on what the term means, and its often misused defintion as something "large" and overt – as the networked fridge is.
"The answer to the question 'Is there any milk left?' has always been no. So, go and pick up a pint, because it's not going to ruin your life if you have more milk than you should. Instead of inventing strange scenarios in terms of how to live our life at home or elsewhere, just find tiny ways of tweaking things. It's about tweaking; it's not about complete integration." The term "Internet of things" is seen as being one which melts, embeds itself into our lives, in the way that QR codes – for example – are starting to be seen and used without a great degree of social disruption. The term is certainly, in Deschamps-Sonsino's view, better than Ubiquitous Computing and Physical Computing, terms which she sees as effectively dead. Where the term will gain true meaning is with the mobile, with examples of the "Internet of things" being very much phone-centred. Its portability and immediacy allows for a potential of access that larger devices simply cannot handle. "We reach out to things to understand the world. You touch things when you're a child."
An example of such phone-enabled interaction is Stickychat, a play platform for children, which uses QR codes. The QR code could, for example, be on the back of a teddy bear, and play a mother's message when it is scanned. Ultimately, whether through NFC or RFID, the mobile will start unlocking information about our cities, whereabouts, and health. The impact is to create "Tiny behaviours" - such as connecting Nike+ data to the Tesco shopping list, to vary a week's diet – rather than the networked fridge's "Over-the-top intelligence. No-one thinks about Nike+ as the Internet of things, but that's totally what that is." As with gestural interaction, mobiles also help us to reappropriate our sense of the physical – to walk around a city, and to discover it in certain contexts. As Deschamps-Sonsino observes, it is to have a 3D experience from information.
An always-on life also requires a much stronger filter. "That's really strong. It's becoming such a strong social signal. 'I switch my phone off for you' is almost like a love letter. It's a sign of dedication beyond what you would have done 20 years ago. Now you have to switch off all of these channels, so when you do that... it's an interesting behaviour. So, how we respond to your chair telling you that you've gained weight this week, and a text message going 'You're doing well', is different from me getting a text message saying 'Your weight this week is this, and your target is this' - it's all about terminology, subtleties, how much that information is there to bug you, and you should be able to tweak it to whatever level you want, but should never feel like it's intruding."
We're getting better at finding information in order to forget more quickly, because so much happens.
This filter also results in our rediscovery of the art of how to forget – the ability to selectively and instinctively "drop" certain factual information, in the knowledge that it can be retrieved later by other means. "We're getting better at finding information in order to forget more quickly, because so much happens. 'I saw a tweet that someone said, and I can't remember what it was, but I know that person said it. So I'm going to that person's feed to look up what they're talking about. I can't remember what the final point was, but I know how to get there.' We're much better at wayfinding through paths of information. My favourite articles are those that I can't find, but I know how to Google. I can find it on the first page; it's good enough, and is as much memory as I want to have - whether the whole world around is speaking to us, depends on what we want to hear about. We'll play DJ to all of this information."
What binds us together now is much richer, and is much more about the relationships to people that we know. Deschamps-Sonsino's interactions, both personally and physically, are all about the familiar – developing or preserving shared experiences. "We might not share passports, but share enough for us to create a little net, but not a web. Just a little net."
Alexandra will be delivering a keynote talk at Digital Belonging, an event from NearLabs taking place on Saturday 5 March at Soundings in London. For more information, visit the page on Eventbrite.