Content

tagged: teaching

Design for Human Interaction

The PhD scholarship I got had some perks before starting the research process. For instance, a six-month internship, and offers of courses across Queen Mary. One of those in particular led to a lightbulb moment for me, and it informed much of my PhD. In a bid to pay this forward I taught this course on completing my PhD, for the 2017/8 academic year.

Design for Human Interaction

Technology has the potential to transform human communication. It changes not just where and when we communicate it also changes how we communicate. If we understand the basic mechanisms that underpin effective communication we can develop technologies that enable richer, more expressive forms of human-interaction. This research led course explores these issues by introducing the social science of human interaction. We explore how we can use this to understand how people use new communication technologies – such as video conferencing, facebook, twitter, virtual environments – and their impact on human interaction. The course covers interaction at a range of scales from face-to-face conversations through to the social dynamics of live audiences. In each case we explore how people exploit the communicative resources available to them, such as speech, gesture, touch and body orientation and then ask how we could augment these resources to enable more effective and engaging interactions.

The course isn’t trivial. Conceived by my PhD supervisor Pat Healey, it’s research-based, with a hard commitment to the idea that to design for something, you have to understand it first. This sounds fine, until the other hard commitment: as people as much as researchers, we can only rely on what’s externally manifest. That scrubs out most design thinking I’d known – no ascribing intent here. That there were methodologically secure ways to design that sensitively address human needs while in effect treating people as unknowable black boxes was a revelation to me.

Convening this course isn’t trivial either. The institutional expectations of the students are diverse: they’re drawn from Psychology and Computer Science, span final year undergrad and postgrad, plus PhD candidates audit. The methodological commitments of ethnomethodology and psycholinguistics are clearly counter to the students expectations, whether from a design background (me), or having taken the interaction design path through CompSci. Same for the Psychology students, though I have less visibility there.

  • 12 Lectures
  • 60+ Students
  • 3 Courseworks
  • 1 Exam
  • OLE, administration, pastoral, etc.

Image taken from the Online Learning Environment’s lecture recording. I’m talking about how designing for human interaction can involve programming a robot to interact socially, i.e. have the nuance to deliver jokes to a live audience.

See also: teaching diary posts, talk diary posts.

For posterity here’s the main coursework brief I set. It marks the period, when social VR could in principle be no different to interacting face-to-face, but the limitations of the nascent technologies provide many a wrinkle –

Human-human interaction in facebook spaces

Introduction

The world's largest social network launched a 'social VR' platform this year, tagged 'VR is better with friends'. You are to apply the learnings of this module to identify and critique the ways in which facebook spaces supports human interaction.

Marketing page for facebook spaces: https://www.facebook.com/spaces

As VR equipment is not widespread, your primary source will be video documentation rather than first-hand experience. The main document is a demonstration video created by Ars Technica as part of their coverage of the launch. You may choose to draw on other materials, such as other videos of facebook spaces in use, or reports of the technical capabilities of the supporting VR hardware (Oculus Rift CV1).

The video is part of this article – https://arstechnica.co.uk/gaming/2017/04/facebooks-first-vr-app-surprises-lets-us-collaborate-and-be-juvenile/

The video has been extracted from the article and is available to download – https://www.dropbox.com/s/6u1m3fn3sx7a7ou/Facebook%20Spaces%20-%20A%20social%20VR%20demo%20-%20Ars%20Technica.mp4

Note 1: a pertinent specification of the VR hardware is field-of-view. There does not seem to be any official documentation; various sources say c. 110°, an authoritative source appears to be http://doc-ok.org/?p=1414

Note 2: the Ars Technica video contains some crass content. I would prefer to set a video without. While searching for a useful exploration of the technology for our analysis, I chose this nonetheless as of the videos I found, this felt the least staged (i.e. natural dialogue, frank appraisal). If you wish to base your analysis on another video, you may do so.

Task

Provide a write-up of no more than 1,750 words, comprising three sections, as follows. 

1. Describe the salient features that facilitate joint activities for people using facebook spaces. Start with two people present (i.e. 00:35 in the Ars Technica video; ignore any joining/leaving procedures). To get you started, and in no particular order, you will want to cover: mouth animation, shared space, infographic above avatar, etc.  Use annotated screenshots and refer to timecode as necessary. 

2. Compare these features to being really co-present, i.e. face-to-face. You will want to consider i. what cues of interaction are possible in facebook spaces that are not possible face-to-face, ii. what cues of interaction are not possible in facebook spaces that are possible face-to-face, iii. whether facebook spaces might limit or mis-represent cues of interaction that, prima facie, are equivalent in both. 

3. Consider Clark and Brennan's argument that grounding changes with the medium, an except of which follows.

By the principle of least collaborative effort, people should try to ground with as little combined effort as needed. But what takes effort changes dramatically with the communication medium. The techniques available in one medium may not be available in another, and even when a technique is available, it may cost more in one medium than in the other. Our prediction is straightforward: People should ground with those techniques available in a medium that lead to the least collaborative effort.

— Clark and Brennan (1991, p140)

Based upon your comparison of features in the previous section, provide an analysis of one of their 'Costs of Grounding' for this implementation of social VR (e.g. start-up costs; speaker change costs; display costs, etc.)

Note: you have limited words and time. The key word in this assignment is salient. Focus your write-up on analysing a few features well; identify what is interesting, rather than aiming to be exhaustive.

Note: submit in PDF format. This will preserve your layout, diagrams and formatting generally.

References

H. H. Clark and S. E. Brennan, “Grounding in communication,” in Perspectives on socially shared cognition, L. B. Resnick, J. M. Levine, and S. D. Teasley, Eds. American Psychological Association, 1991, pp. 127–149.

project | 2017

dorkbot / wipeout

given the opportunity had come about via dorkbot, only felt right to present back what i’d been up to. a good “who is this toby”, as much as anything else.

diary | 09 apr 2024 | tagged: cardboard wipeout · pervasive media studio · teaching · dorkbot

watershed » cardboard mario kart

martin had also worked a little magic, digging out some stage lights and delivering on our parting conversation from the day before:

toby – “the gopro camera will stream over wifi, but the latency is too slow to drive the car using it. the fpv drone racers must have a modern solution for this, but back in the day, those lipstick analogue spy cams would be perfect for this”
martin – “hmm… i wonder… let me go rummage”

diary | 05 apr 2024 | tagged: cardboard wipeout · pervasive media studio · teaching · embedded

watershed » cardboard mario kart

tony, the producer, kept the track up for the next day’s monthly first friday networking drinks. that gave me a chance to try my my kids’ rc car, which was… quite something. suffice to say, the £75 UDIRC car demolishes the track, scrambling up the slopes and jumping clear off the bridge. i reckon we could do a loop-the-loop with it.

diary | 05 apr 2024 | tagged: cardboard wipeout · pervasive media studio · teaching · embedded

watershed » cardboard mario kart

love the championship cup that came out of the first day. that’s either a very canny child, or the hand of a parent to e.g. find that dragon on some cardboard and line it up behind.

diary | 04 apr 2024 | tagged: cardboard wipeout · pervasive media studio · teaching · embedded

watershed » cardboard mario kart

thanks to tony for making this happen, and ably facilitating on both days. amazing feedback from the families throughout, to the point of asking if i could do it in their schools. i’ll take that as a win, and… yes, maybe?

diary | 04 apr 2024 | tagged: cardboard wipeout · pervasive media studio · teaching · embedded

watershed » cardboard mario kart

having named one of the group challenges as “the banana” – after mario kart’s well-known banana that can spin you out – it was a happy moment to see tbe carboard + microswitch engineering had been appropriately set-dressed.

diary | 04 apr 2024 | tagged: cardboard wipeout · pervasive media studio · teaching · embedded

watershed » cardboard mario kart

and here’s the second day’s lot, making their track. so different!

diary | 04 apr 2024 | tagged: cardboard wipeout · pervasive media studio · teaching

watershed » cardboard mario kart

the three-hour format was to run twice. and here’s the first day’s lot, racing their track.

(well, half of them are out of shot, but you get the idea)

original photo credit: shamil ahmed

diary | 03 apr 2024 | tagged: cardboard wipeout · pervasive media studio · teaching

watershed » cardboard mario kart

making the micro:bits do what we wanted was half of the “creative tech”, as they need to be part of our physical game. and with me, “let’s hook them up to the track” quickly turns to some fetishising the clicks of different switches. any excuse to quote kenneth grange’s small pleasures.

original photo credit: shamil ahmed

diary | 03 apr 2024 | tagged: cardboard wipeout · pervasive media studio · teaching

watershed » cardboard mario kart

micro:bits come loaded with a demo program that’s great for taking them out of their retail box, plugging in a battery and passing them around the group.

but later, in smaller break-out sessions, the question becomes: we’ve seen what the people who make these boards make it do, but that’s not what we need for our game. how do we make it do what we want it to do.

this 8-14 year old workshop skewed younger both days, so a lot of the wonder was the kids simply selecting an icon in scratch on my computer, and then seeing that light up on the board’s 5x5 LED matrix.

but also, like here, sometimes the wonder was seeing something i’d prepared earlier, and grokking a piece of the “grown up” code that animated the LED strips.

original photo credit: shamil ahmed

diary | 03 apr 2024 | tagged: cardboard wipeout · pervasive media studio · teaching

watershed » cardboard mario kart

when i dreamt up cardboard wipeout, it was for a two-day, 10 hour slot. it was appropriate to go “here’s your empty shipping container, let’s figure out how to fill it”. but a three-hour workshop is quite different, and so: blank canvas, like before? or… and if “or”, then what? the decision was to prime the space with a section of track: a figure-of-eight is a classic use of limited space, it gets you a bridge, it makes you consider how walls might work, and is a good demo for efficient use of a cardboard box. well, three. quite elegant, i think.

diary | 02 apr 2024 | tagged: cardboard wipeout · pervasive media studio · teaching · embedded

watershed » cardboard mario kart

i went to dorkbot bristol, and good things happened. one of which came from a producer from the watershed’s pervasive media studio standing up and saying “does anyone have a creative tech workshop? i’m looking to do something with families this easter.”

i do, said so, and lo: here i am hacking a fresh set of argos’s finest cheapest radio control cars. two notches later, it’s as if the designers of the toy meant for the micro:bit board to go here.

the only problem is that staff at watershed figured that while wipeout might be where the idea of the workshop came from, mario kart was the better hook to sell the idea to the public. they are of course right, and from the point of view of the workshop, they are identical games: tracks with power ups and downs.

diary | 28 mar 2024 | tagged: cardboard wipeout · pervasive media studio · teaching · embedded · dorkbot

contains art » cardboard wipeout

…and did i have fun? sheesh!

photo credit: jon @ contains art

diary | 19 jan 2019 | tagged: cardboard wipeout · contains art · teaching

contains art » cardboard wipeout

the brief was computer game. so, let’s make this thing an immersive experience. you enter the container, the ‘press play to start’ message is flashing, music booming…

with a micro:bit – and it’s radio-link to the track and car – connected to a laptop, a quick bit of patching i’d done the day before got some videos and coundown sounds playing in response to the “wait, 3, 2, 1, start, finish, wait” coming down the USB cable. add projector and PA, and boom!

we’d done it. ten teenagers, ten hours. quite the achievement, and time enough to play with it!

diary | 19 jan 2019 | tagged: cardboard wipeout · contains art · teaching

contains art » cardboard wipeout

so how does it look? and how does it look with a computer-driving-game first-person-view!? could you stream that camera feed to a phone hanging in front of someone’s eyes?

now i know that wifi streaming from a go-pro isn’t going have the low-latency required for actually driving it, but that’s just a spur to research what might… fpv racing drones seems to have this solved.

bonus: this photo was taken by one of the teenagers. it’s a great one.

diary | 19 jan 2019 | tagged: cardboard wipeout · contains art · teaching

contains art » cardboard wipeout

track hacked, onto task three: the car.

with micro:bit in there, parasitising it’s power off two of the three car batteries, we could disable the motor until the go! of the countdown, have different team logos lit up on the roof, send accellerometer data back to a computer to make futuristic driving sounds… or, most simply, give us a button to start the countdown. feels right having it on the car.

with the limited time, this is actually one i made earlier; the removable roof and fit of the micro:bit was a bit of a gift. within the workshop, we experimented with fitting cameras to the back-up car…

diary | 19 jan 2019 | tagged: cardboard wipeout · contains art · teaching · embedded

contains art » cardboard wipeout

lights programmed with a countdown and chase animation i’d made the day before, onto the next task: fit them round the track.

diary | 19 jan 2019 | tagged: cardboard wipeout · contains art · teaching · embedded

contains art » cardboard wipeout

with the track made, time to add the digital magic and make it a game.

first up: LED strips with an extra wire… a programmable wire. and here is a micro:bit, essentially the guts of a phone made so we can control and play with things. with a bit of drag-and-drop in a web browser, we can make patterns down the strip and animate them.

but we need two strips, with a controller each, to go round our track. so how to animate them together? the micro:bit has radio, they can talk to each other and say “now!”. this is also how we’re going to make the whole game work, with more of these boards talking to each other.

photo credit: jon @ contains art

diary | 19 jan 2019 | tagged: cardboard wipeout · contains art · teaching · embedded

contains art » cardboard wipeout

to watchet, for the first of my two workshops getting teenagers to do creative-tech stuff. twee for me, but west somerset has the lowest social mobility in the whole of the UK and watchet is the most deprived ward in the district. so kinda serious.

the brief was gaming, and one idea led to wipeout-with-cardboard. one shipping container, two dirt-cheap radio-controlled cars and a village’s worth of post-christmas cardboard later, we had a track. admittedly more cardboard than tech at the moment, but this route is surely more engaging than coding 101.

aaaand… wipeout? i challenge anyone to find a better cultural artefact from 1996 than the opening sequence of wipeout 2097: the designers republic, future sound of london, it’s a coming together of all the arts.

diary | 05 jan 2019 | tagged: cardboard wipeout · contains art · teaching

la » live in live cinema at imap

a day in a coffee shop making a major revision to the about the live in live cinema talk, and then back to USC to give it as a IMAP seminar. the host was holly willis – former editor of the amazing res magazine amongst other things – and somebody who not only is published on digital cinema and live cinema, but whose definition of live cinema is often embraced: “real-time mixing of images and sound for an audience, where the sounds and images no longer exist in a fixed and finished form but evolve as they occur, and the artist’s role becomes performative…” Holly Willis (Afterimage July/Aug 2009, Vol 37, No 1, p. 11/). my talk is not about definitions however, its about exploiting the liveness, and what that could mean.

also found the adage that everything can change in two blocks is indeed true, routing from the recommended coffee shop that turned out to be closed to one that was open put me through some streets i’d prefer not to walk down, let alone with a bag packed with my digital life. the no-healthcare-mashed-up-bodies of the homeless seemingly kettled / corralled to then be routed around: it isn’t a phenomenon unique to america, but its the one that alienates me the most.

diary | 25 apr 2011 | tagged: live in live cinema · research · talk · teaching · live cinema

la » d-fuse workshop

as part of the rhythms and visions event, a day of workshops was organised. los angeles visual artists - lava had the morning, and covered the past and present of ‘visual music’ works. mike, matthias and i had the afternoon, which we nailed the four hours precisely with a tour through the d-fuse oeuvre and a journey through our particle production process. the latter was my main contribution, and its a tricky balance to give: lots of really cool stuff – shooting, taking crops, building abstracting effects – but with what can reduce down to a sea of noodles and buttons. pretty happy though, good feedback that the thread was there and it all tied up: people got it.

diary | 23 apr 2011 | tagged: teaching · talk · video-out · vdmx · quartz composer · dfuse · vj

paralympics » yeast slides, yeast film

although it meant i had to forgo the celebratory beers-in-the-sun after the showcase, i managed to get the digimedia showcase onto the plasma screens at the reception - so we got to show some work as well as the film that yeast facilitated documenting the whole forum. that film really did the job: well done james.

diary | 11 sep 2008 | tagged: *spark · vj · teaching · beijing paralympics · yeast

paralympics » 100 slides for the showcase

here’s the presentation in pdf form that was built-up during the workshop, finished by me and shown to all at the final showcase. i’m quite proud of it, and watching your computer turn pin-yin into chinese characters for the first time is pretty cool.

diary | 11 sep 2008 | tagged: *spark · vj · teaching · beijing paralympics · yeast | downloads: yeast-yap-dmshowcase-web.pdf

paralympics » there we are

while excessive taking group photos seems is definitely endemic here, its nice to see all the group in one.

diary | 10 sep 2008 | tagged: *spark · vj · teaching · beijing paralympics

paralympics » “kitchen”

the other play, caught in the moment.

diary | 10 sep 2008 | tagged: *spark · vj · teaching · beijing paralympics

paralympics » 10 word play to stop motion

steve with the final day’s project: watch a 10-word play developed by another workgroup, and turn it into an animation. this is the moment when they realise that stop-motion is a long-haul game…

diary | 10 sep 2008 | tagged: *spark · vj · teaching · beijing paralympics

paralympics » yes you can (#23 / 240)

no arms but want to play the piano? yes you can.

diary | 09 sep 2008 | tagged: *spark · vj · teaching · beijing paralympics

paralympics » workshop

and so it starts. 27 “young advocates” drawn from china, the uk and beyond, of mixed abilities, ranging from learning difficulties to physical impairment, representing those who’ve achived great things in their communities back home to those picked out of special schools… and in some cases all in those in the one person.
yeast culture ran the film making workshop and the digital media workshop, and i was a leader for the latter. in what could have just been a get media-savvy and learn powerpoint thread, we instead delved into the power of the image; communication beyond photos and text. and they learnt photoshop, and when to abandon photoshop in favour of print-outs, scissors and card.
here is kelly during our (ahem) ‘monkey see, monkey do’ afternoon of photoshop tutorials.

diary | 07 sep 2008 | tagged: *spark · vj · teaching · beijing paralympics · yeast

paralympics » spectacular: check

spectacular, it was. the paralympics opening ceremony might not have had quite the lavish and epic reach of the olympics that i saw broadcast, but it did have soul - and watching a wheelchair bound athlete pull himself up a rope to the top of the stadium to light the olympic flame was something both awe-inspiring and humbling: when he paused to get his composure back 50 meters up and still a little of the war to go, i’d wager there were 90,999 other people also experiencing a deep moment of reflection.

diary | 06 sep 2008 | tagged: *spark · vj · photo · teaching · beijing paralympics

paralympics » ticket, check

kelly and steve just having got their tickets - we’re going to the paralympic opening ceremony, school trip style.

diary | 05 sep 2008 | tagged: *spark · vj · teaching · beijing paralympics

paralympics » workshop room (spacious)

30 desks… not quite filling the room. not your average digital media workshop venue, this.

diary | 04 sep 2008 | tagged: *spark · vj · teaching · beijing paralympics

paralympics » beijing airport / british council

…which is (to oversimplify) a british council project working with young people, hosted as part of the paralympics here in beijing.
young advocates programme: http://www.yaponline.org/index.jsp
yap youth forum in beijing: http://www.yaponline.org/youth-forum.jsp
oh - and just when you’d appreciated the scale of the beijing airport terminal, you take a shuttle train out and under a few more identical and identically sized terminals, eventually walk out into what you think is the outside, just to find yourself in another cavernous hall. scale… scale…

diary | 04 sep 2008 | tagged: *spark · vj · teaching · beijing paralympics · yeast

bbc blast » goodbye pavillion

the pavillion is a day, park, outside thing. strange to see it misty at night as we walked away, project over.

diary | 30 aug 2008 | tagged: *spark · vj · bbc blast @ serpentine · teaching

bbc blast » final filming

and on performance day, the sun comes out. aaaah.

diary | 30 aug 2008 | tagged: *spark · vj · bbc blast @ serpentine · teaching · yeast

bbc blast » tapes, eh

i’d happily forgotten about things like this in an age of solid state. managed to rescue what was needed, but the last minute stress quite apart from the glitches wasn’t welcome at all.

diary | 28 aug 2008 | tagged: *spark · vj · bbc blast @ serpentine · teaching

bbc blast » aaah, the dlr

instant supercool visuals for young people #1: ride the dlr, apply mirror effect. ref cityscan to see it taken the highest level, and arrive as the piece that started it all.

diary | 21 aug 2008 | tagged: *spark · vj · bbc blast @ serpentine · teaching

bbc blast » dance + vjs

not just playing with a live camera feed and kaos pad, but shaping the dance through the control of graphic elements: a simple cross can become a character, a circle shape the movement

diary | 20 aug 2008 | tagged: *spark · quartz composer · vj · bbc blast @ serpentine · teaching

bbc blast » vj taster sessions

working as part of yeast culture for bbc blast on a two week project for young people that encompassed film, vjing, soundscape, spoken word and dance culminating in a performance at this years summer pavillion at the serpentine gallery. frank gehry, no less.

diary | 18 aug 2008 | tagged: *spark · vj · bbc blast @ serpentine · teaching · yeast

µ:avit2008 » quartz composer workshop

gave a complete overview to quartz composer from a vjs perspective, and got some really nice feedback. photo by todd thille.

the basics

  1. getting beyond quicktime clips: what qc can do for you
    ie. dancing abstract 3d a la toneburst and memo
    ie. animating text files in your vj sets
    ie. making dynamic installations

  2. quartz composer 101
    your first patch
    dynamic controls
    the key idea of time
    common pitfalls

applying it and getting advanced

  1. qc and vdmx
    qc as dynamic clips
    qc as filters
    qc as rendering stacks

  2. shaders
    image kernels
    glsl

  3. 3rd party plug-ins

diary | 24 may 2008 | tagged: *spark · quartz composer · avit-vj-network · vj · teaching · visual berlin

name » desktop snapshot

guess what we were up to. if only leopard and reactivision would play nicely together: the qc osc receiver doesn’t seem to be receiving what it should, and its beyond me to port the tiger hacked-up plug-in to an official api leopard one. that is something that will hopefully change after christmas, when i finally embrace cocoa and the live cinema interface.

diary | 01 nov 2007 | tagged: *spark · quartz composer · vj · teaching · name · novak

name » quartz composer 101

straight from a marina in italy to a biscuit factory in newcastle, home of the most proper name. they had organised three days “mentoring” aka professional development funded by the region, and i was the mentor. first up was quartz composer 101. so glad we waited for leopard to debut, what a change: its actually sane now!

diary | 31 oct 2007 | tagged: *spark · quartz composer · vj · teaching · name · novak