Brains and Machines

Driving by wire

I'm a pretty good navigator, but it's still a pain (and sometimes dangerous) to drive around places you don't know and read maps at the same time. So, when I was in going to the mid-West and California on business in September I decided to try satnav. Though this technology has been around for decades, it had never seemed to me worth paying for. For this trip, however, my schedule was so tight and my exhaustion level was clearly going to be so high that I thought it might be a one way of taking the pressure off. It really was.

The system I used was simple: you called up a call center and told them the addresses of all the places you wanted to go (you could do this all at once or as needed). Then you just picked the address you wanted from the address book and, after acquiring your position, the phone would show you (on its tiny screen) and tell you (using a computer voice) which way to go when.

If you're a Mapquest cheapskate as I've been for the last few years, you can think of the added value you get like this. Not only do you get directions at will while in your car, but you have someone read them out to you as you go along. More crucially, they'll download new directions if you take a detour: whether for a pit stop, a wrong turn, or a blocked road.

Of course, the system has it's limitations. Sometimes it can't read the GPS, or connect to the server, or needs a reboot. Sometimes it can be slow at figuring out where you are and it's directions are too late. Sometimes, as with all these systems, it's just plain wrong: the road it takes you to just doesn't connect any more. Plus, I would occasionally have to turn down NPR in order to listen out for what to do next.

But the system completely changed the way I looked at my trip. I was completely relaxed about getting from A to B in a way I've never been before. I knew that, as long as I left enough time, I'd get there: no more driving around in circles for hours trying to catch glimpses of my maps as I passed under streetlights.

As someone who doesn't have to drive in London—and felt oppressed by driving when I lived in the Pacific Northwest, the Boston area, and the Bay area—this technology made me feel calm, relaxed, and confident. Kind of the same way the Internet makes me feel confident about finding train times, looking up words, or getting in touch with people.

I'm also excited about the added value of other services that you can get on the more sophisticated systems. It would have been great if the thing could have told me where the nearest gas station was, the most well-reviewed diner on the next hour of my route, or the next decent coffee shop. I could almost conceive of driving becoming a pleasure. Except for one thing. I've yet to hear of a satnav system that could guide you to an open parking space. 

 

Photo

My travelling companion in September was a little java-based satnav system that ran on a mobile phone. Photocredit: Mikal from Canberrra. http://www.stillhq.com/

Flying blind

Of all the gizmos I've tried in my recent research, the tactile suit was the most exciting. Pilots can become disoriented because of poor or confusing visual feedback conflicting with other cues, a phenomenon that accounts for a significant proportion (up to 10%) of air accidents. The tactile suit is intended to help by letting the pilot feel their orientation through their body rather than trusting their vestibular (inner ear) or visual systems.

The version I used consisted of a set of three vibrating buzzers (tactors) on each arm and four each on the chest and back. (Additional leg tactors—shown on the screen in the picture—had been disabled in the version I used). It works as follows. As the plane tilts to the right, the right wrist starts to vibrate as it tilts more, the elbow comes in too, then the shoulder. As the plane tilts forwards, the top two chest tactors come in, then the lower as the dive becomes more extreme. The back tactors become active when the plane starts to climb.

Apart from commercial flights, I have had quite limited experience of flying as a passenger and almost no experience of being a pilot (in a simulator or otherwise). I was not, therefore, in any way surprised that I was, well, pretty crap at trying to use some of the Operator Performance Lab flight simulators. However, this was not true while using the tactile suit. When I could feel my orientation, I was much better able to steer. So much so that the researchers were able to put me into an awkward and dangerous orientation (one that most pilots have great difficulty finding their way out of) and I was able to find my way to level flying easily. With my eyes shut.

What was particularly impressive about this is that it's not what this particular suit was designed for: it was meant to warn the pilot, not offer feedback for control. However, Captain Angus Rupert who invented the original idea, almost ten years ago, has now developed the technology to the point where pilots really can use it to tell which way is up... Field trials show pilot improvement whether visual conditions are good or poor.

My personal experience was that the device allows you to 'feel' space in a way that is otherwise impossible, but that nevertheless feels both incredibly natural and physical. This physicality is, I think, the key to the success of the technology.

Photos

Top: Seeing the activation of tactors on screen as I feel them through the suit.

Bottom: Flying blind, using the tactors as the only cue to feel my way to level flying.

A hundred pixels is much better than none

We're so used to seeing megapixels these days that we don't appreciate how even a few can make the difference between 'seeing' and being blind. I recently got the chance to try a tongue display—a device that literally displays images to the tongue using small currents—at the University of Wisconsin at Madison and at Wicab Inc.. If you experimented when you were a kid (!) it's a little like putting a 9V battery on your tongue: or rather, 100 9V batteries.

At the university, I used the display to follow a path set by a computer. Arrows and other symbols or pictograms were 'shown' to me via my tongue to tell me what to do, and I carried out the instructions using the joystick. I could only tell how well I had done (pretty well!) by moving some windows to look at the path I had taken after the fact. Although it may look like I'm staring intently at the screen in the picture, I actually can't see anything on it that would help me with the task.

As impressive as this was, using the display to show me my immediate environment was much more interesting. Here, the display is connected to a camera that you put on with glasses that cover your eyes. After an initial training period looking at white lines and shapes against a black background, I was allowed to wander around the Wicab offices and an empty warehouse space. With my head continually scanning back and forth and up and down—a natural response when looking at the world with such a narrow field of view, and at such low resolution—I was able to walk around without bumping into walls (more or less), could follow a path defined by lines on the ground, and could even see objects floating in space.

Apparently I did better than most people on my first try, which my trainer put down to my having done the navigation demo the day before. Even with that caveat, I was very surprised at how much I got out of it. Three things struck me the most. First after just a few minutes the experience starts to feel very 'visual': when I look back on it, I don't remember my tongue tingling, but rather the pictures  displayed as if I had seen them through my eyes. Second, I had been told that, after training, people could tell the difference between a baseball and a tennis ball. Although I didn't get to that point, by the end of the demonstration I was convinced that this was true.

Finally, before I used the device, I was skeptical: surely 100 pixels worth of image really wasn't worth the effort, even to someone blind. I stand corrected.

Photos

Top: Shown is the Tongue Display Unit version 1. The saliva in our mouths helps it to make excellent electrical contact.

Middle: Navigating around a virtual maze by accepting instructions from the display. I'm 'looking' at the pictures inside my head that are generated by the tongue display: the maze is hidden by another window on the screen.

Bottom: Walking around the Wicab warehouse, staying on the path defined by the lines on the floor.

Probably the best job in the world

Tom Schnell (I just love nominative determinism) is a pilot and a flight instructor who runs a research lab where he gets to grapple with fundamental issues related to human perception and behavior. As Director of the Operator Performance Laboratory at the University of Iowa his job is to understand how pilots react while performing tasks in different sets of circumstances, which he does by hooking them up to many different sensors, including an electroencephalography (EEG) net to measure brain activity. (Will say more on this in a later blog.)

Though there’s no doubt that the team love their work and have a lot of fun doing it, it’s a difficult way to do research. For one thing, you have to think about the weight of all the various computers, amplifiers, cameras and other experimental equipment. OK, its not as bad as if you were sending the kit into space, but it’s an important factor.

Another issue is power. When the plane is on the ground, without the engine running, it cannot feed the data acquisition and analysis gear: a cable has to be brought in from the lab during pre-flight preparations. Then, when they’re ready to fly, they have to switch over from one power supply to the other without interruption. If this goes wrong (as it did on my flight), you have to set up again from scratch. And, in case you’re wondering, those power-strips that can bridge interruptions to home and office electricity supply are not an option. Capacitors, a crucial component to many of these systems, have an unfortunate tendency to blow up and/or catch fire. Not something you want to risk in an enclosed space where evacuation is difficult or impossible.

There’s cost too. Every experimental run takes the time of two pilots, plus the insurance, maintenance, and fuelling of the aircraft.

If your job is to help acquire the data, there’s another issue as well: and that’s air sickness. Sitting at the back, trying to read information from screens and sheets, keep track of data, and perform timings, affords little time to concentrate on the important things: like trying to keep your breakfast down: especially during wing overs and alternating rolls.

On my ride-along on Friday, this was supposed to be my job, though I was advised that I should put my stomach before the work. I was grateful they let me off the hook in advance: sitting at the back of the plane, you don’t get the view that allows your visual and vestibular (balance/orientation) system to agree with each other. Your brain doesn’t understand what’s happening, so you feel sick.  To compensate, I tried to sway with the plane, almost as though I was riding a bicycle, and this did help (my breakfast remained intact throughout). Unfortunately, however it took 100% of my concentration just to mitigate my ‘stomach awareness’.

The good news is that the guy who usually does the job has a cast-iron constitution. With Tom and the others from the lab, he gets to fly and experiment and analyze and study and, ultimately, learn how our cognitive systems respond to the world. Now that’s a great job.

Photos

Top: Karl, a pilot and experimental subject, prepares to have his mind read by EEG, as researcher Greg makes sure the electrodes make good contact with the skin. (Don’t ask what that expression on my face is all about, I’ve no idea…)

Middle: Greg (back to camera) performs pre-flight preparations as Tom warns me of the dangers of air sickness.

Bottom: I prepare to do my small part of the experiment: little do I know that my stomach will make that impossible. Fortunately, the minimal data I am supposed to record can be picked up from the video playback.