Adventures in wearable computing

A little over a month ago I said goodbye to my Lenovo Thinkpad x61t. I had placed it on my knee and as the case flexed slightly, it turned off. Repeatedly. I LOVED that machine. The stylus was held together with hot glue, the screen hinge had cracked so the screen no longer stayed up-right, I had replaced the fan a year previous, sometimes the computer wouldn't start with a DMA warning, there was a crack along the back from where I had dropped it once. But I didn't want to say goodbye.

When I looked for a new laptop, I couldn't find anything that suited me. After having such a perfect laptop, I found it hard to bring myself to put down money for something I would be less than 100% satisfied with. In the time that I owned the x61t laptops don't seem to have improved much. They may have added a few hours battery life but they have no more RAM than they ever had. I briefly looked at the Samsung laptops with Superbright displays. The screens on those things are truly a step up. However, I've been feeling more and more strongly that laptops are not the future.

I once heard a wisdom that guitars are social instruments and violins are asocial instruments. The reasoning being that the guitar sits on your lap leaving your face free for communication, while the violin covers your face. I think that laptops are asocial instruments like violins. They create a wall between you and your surroundings. Laptops are also unergonomic, you have to bend your neck to look down at the screen or use an external keyboard.

Finally, they are less portable than they could be. To leave your workspace you have to fold your laptop up, likely unplug it. You cannot use a laptop outside due to sunlight.

The wearable display as a laptop replacement

I do most of my work in vim. Firefox is mostly a distraction for me. I do have a few GUI programs that I run; umlet, keepassx, xfce notes. I don't need a lot of processing power, and when I do, I usually need bandwidth as well. Even now, I delegate the really heavy lifting (like testing Docker) to the "cloud".

That's why my first dive into wearable computing involved a Raspberry Pi. I plugged a pair of Vuzix VR920s into a Raspberry, spent a week at the sewing machine getting a pair of datahands to dangle comfortably at my side:

front view

(Click on images to enlarge)

back view

This worked.

However, there were a number of issues with this setup which led me to abandon it:

  1. The Vuzixes provide an uncomfortable, bright, flickering, blurry, and distorted image. They are also quite heavy on the nose.

  2. My wrists were sore from folding around the thick datahand modules.

  3. I couldn't sit down and was tired all the time. It made my back feel a bit stiff.

  4. When powered from battery, my Raspberry Pi killed an SD card every 2 days. When the SD card was dead, it could not be fixed.

The Epson Moverio BT-200

After failure with the datahands and Vuzixes, I felt that a more expensive setup might be worth a shot. These glasses are supposed to replace my laptop, right? So I can spend at least a thousand dollars on doing so.

Portrait

Unboxing the Epson Moverio BT-200

The Moverio display

When I first put on the glasses, I was afraid that there was some kind of optical flaw. I saw the "Moverio" logo sliding up the screen, but ghosted. Five or six transparent layers followed the letters, each less opaque than the last. I shifted and wiggled the glasses and finally the image slid into clarity. Then it slid out of clarity again. Reading the text on the screen was HARD. After a while playing with the glasses I realized that the optics have the following properties:

Glasses top

Unfortunately, my eyes are too close together to use the Epson Moverio BT-200. In order to get one eye centered behind the mirror, I must move the glasses so far over to the side that the other eye doesn't see any image at all. This was very frustrating for me. I tried every combination of tilts and angles, various distances etc. Nothing worked. Without being able to change the interpupil distance of the mirrors it is truly hopeless. I wore the glasses for just one day, and I've started writing this review on my laptop the next. My left eye is still slightly sore from the strain of trying to use the glasses. There is a company that sells prescription lenses for the Moverios, however, as far as I can tell, they don't move the mirrors, they don't list a price, and their nearest optician is in West Germany. I am returning the glasses for lack of any better remedy.

One last note about the display. I had imagined myself writing a "real life ad-blocker" for the display. This would not be possible. The viewing angle is far too narrow. What you see is a floating rectangle in front of your vision. Looking around that rectangle is trivial and I found myself looking down or up to do common tasks. Despite their advertisements, this is in no way shape or form any sort of augmented reality!

Software

The Epson Moverio has Android 4.0 without the Google play store. When you turn on the device, you are greeted by a splash screen pointing you to a video. You have to click through the splash screen in order to start using the device.

Once the "computer" has started up you see a blue wallpaper of Athens(or some other Greek ruin, not sure). The wallpaper moves with your head, but slowly, with a 3-4 second delay, and not to the same degree as your head moves. It is rather unpleasant, but easy to turn off in the settings.

I, of course, immediately installed F-droid and Connect Bot. I had bought a Lenovo Thinkpad Bluetooth Keyboard to connect to my Moverio. Everything went smoothly as I had planned. SSH worked pretty flawlessly around the house.

However, I began to realize how much details matter. Modifier keys didn't work as they should, I couldn't swap the Ctrl and Capslock keys. Indeed, without rooting the device or buying some piece of third party software, it seems it is impossible to remap the keyboard in Android at all!

I thought that I would get used to this small frustration. I did not.

The purpose of a thin client is to be as thin as possible, to act as an interface to your main computer. Unfortunately, Android with Connect Bot fails at this.

Optics, not electronics, are what hold wearable displays back

Lenses of the Vusix 920s

Lenses of the Moverio BT200s

It is the lenses that make up the bulk of both the Vuzix 920s and the Moverio BT200. Indeed, the electronics have been shrunk to the point where they are almost inconsequential.

Vuzix electronics

The Vuzix works by a very simple mechanism. A tiny LCD screen, too small to see the pixels of even with a magnifying glass, WAY more dense than "Retina".

Vuzix screen

Then, a giant, heavy, double lens magnifying glass allows you to see the screen. No magic here folks. You're really just cramming a tiny LCD screen up right next to your eye ball.

Here is a side by side comparison, so you can see how inconsequential the electronics are next to the lenses.

Vuzix comparison

The BT-200's are even more extreme, as the electronics have gotten even smaller, while the lenses are about the same size. However, I have hope that this can be remedied. Epson did not seem to put much effort into minimizing lens size, and there are large swaths of seemingly unnecessary plastic around the edges. It is more difficult than it seems at first though, because the edges of the thick lenses block one's vision, cause weird glare and reflexions. If they were to make the lenses smaller, and cut out that "unnecessary" plastic, then those distortions and glare would block more central parts of the user's vision.

After having seen how important optics are for these glasses, I would have to say that it makes more sense for companies like Nikon and Carl Zeiss to make video glasses than an electronics firm.

More about interpupil distance

I still haven't given up on video glasses. Far from it. After seeing the quality of the image coming out of the Moverios, I KNOW these glasses are the future. So I've started shopping again.

I'm not finding much, but one promising product I did find is the ST1080 from Silicon Micro. At first sight, they seem to be a dream come true for me. Similar technology to the Moverios with an adjustable IPD! However, looking closer, I learn that the IPD adjustment is 60-70mm. My IPD is 57mm. How can it be that I've been left out? Time to do some research.

First off, it's important to note that IPD issues apply only to binocular glasses. Monocular displays such as the Google glass do not suffer from this issue.

Table of video glass IPDs
Model IPD
Epson Moverio BT-200 Not made public(non-adjustable)
ST1080 60-70mm(adjustable)
Zeiss Cinemizer 59-69mm(non-adjustable)
Oculus rift 65mm(read the article!)
Sony HMZ-T3W Not made public(not adjustable?)
Fatshark Dominator HD 57-73mm(adjustable)
Fatshark Attitude 59-69mm(adjustable)
Vuzix Wrap 1200dx adjustable but exact specifications not public
Chineese no-brand sold on ebay 58-66mm(adjustable)
Avegant glyph(not yet available) 52-75mm(adjustable)

I found a wonderful, and freely available, overview of IPDs published by the Cambridge Computer Laboratory.

On page 3 they have a graph of IPDs in the general population.

IPDs in the adult population

IPDs in the adult population

At first glance, Silicon Micro seems insane for having shifted their IPD range so high. They're missing out on about a third of the population with that range!

However, as we move down to page 4 the situation becomes clarified.

IPDs male

IPDs male

The Silicon Micro displays are aimed at men.

IPDs female

IPDs female

It's not just me that is being left behind, somewhere around 15% of all men and nearly half of all women are being left out on this technological revolution!

The strange thing is that making the IPD adjustable, or more narrow, is not a technical limitation in the glasses that I have experience with. When I took apart the Vuzix VR920s, it became clear that there was plenty of room for the lenses to be closer together. On the EPSON Moverios, it is also clear that the mirrors can be MUCH closer together. The Oculus Rift appears to be less flexible in its design, the eye cups are large and thus cannot be crammed closer than 60mm, however the other projects seem to have little excuse.

After my battles with the IPD problems, I tried dealing with them in two ways. I tried modifying the Vuzix glasses to use on one eye, and I tried strapping a mini-projector to my head.

Vuzix cyclops front

Vuzix cyclops side

The first method kind of worked, but after a while my brain got really tired, and it caused eye pain. I kept on having the trouble that my brain would start ignoring one side of my vision. Not exactly a pleasant feeling. I would generally consider this experiment to be a failure. It did not allow me to do any meaningful work.

The projector was almost successful.

Lying in bed with a projector strapped to my head

I literally just attached an Acer C20 pico projector to a ski hat and plugged it into a Raspberry Pi. I used the same wireless bluetooth keyboard as I had used with the Moverios. This worked pretty well, so long as it was dark enough, and I could find a patch of white wall (or in the case of the photo, ceiling) to project upon. However, it was not practical and I have dropped this setup. One thing that did become clear though, is that low resolution is a problem. This projector has a native resolution of 854x480 and it was hard to meaningfully get two terminal windows side by side on it.

The myth about social skills

When a person smells, talks too loudly, or leans too close to their conversation partner, their partner provides them with (typically non-verbal) negative feedback. The myth about social skills says that a socially skilled person perceives social feedback and is able to dynamically and automatically respond to it. Perhaps they will wash more, talk less loudly, lean less close. All this will lead to a more pleasant conversation experience for both parties. It is true that most people do have an abstract sixth sense which encompasses social feedback and societal norms. It is also true that most people dynamically and automatically respond to this feedback. However, it is false that this constitutes social skill.

A socially skilled person perceives social feedback and societal norms and they are also capable of dynamically responding to these inputs. They do not, however, do so automatically and unconsciously. As much as people feel that social norms represent a kind of moral mandate, society is very capable of being wrong. Social pressures often coerce people into doing stupid things and evil things. Doing stupid things or evil things because you think society wants you to is not a social skill.

Once it becomes possible to use head mounted displays for long periods of time without excessive eye strain, it will be stupid to continue using traditional laptops. There are many benefits to HMDs, especially the transparent ones:

Unfortunately, most, if not all articles about video glasses devote a significant portion of their discussion to whether the glasses look dorky/nerdy or not. While society has provided us with some negative feedback when it comes to HMDs, it is stupid to care. There is no moral issue at stake here. There is no harm that HMDs can cause to those who do not wear them.

Augmented reality and racism

While HMDs face stigma, overcoming that stigma may do more for the world than simply making people more accepting of advanced technologies. It is possible that augmented reality could reduce other kinds of stigma and prejudice as well. I once heard an acquaintance jokingly remark that if augmented reality ever takes over, everyone will augment the human appearance of others, so that everyone looks like a naked version of Nicole Kidman. Actually, this isn't a bad thing. We have so many prejudices about people's skin color, age, gender, height... that everyone looking the same might actually make the world a better place. The online world of Second Life was in its heyday sometimes regarded as the end of stereotypes and preconception. There were many people black and white, ugly and fat who donning an avatar felt for the first time free from their physical appearance. What if we could bring that sense of freedom to the "real" world? Would that help us aim towards a fairer and more equal world? If we weren't to judge people by their "birth" appearance, would we see more deeply into their true personality? If people were able to choose avatars of their own in the real world, would that not change the game of looks from one of stereotype and stigma to one of identity and creativity? But again, what if every Pecola Breedlove could choose to have blue eyes? What would that say about Black identity? Would that really forward the cause of equality, or would it lead to a tyranny where you really truly had to be an apparent white and male to survive?

Conclusion

For now, I have given up on video glasses and augmented reality. I'll have to wait for a pair of glasses that don't give me a headache. I'd be willing to pay $2-3 000 for them, but they have to work, they have to have a clear image, and they cannot come with a warning that I should take a break every 30 minutes and cease use if I feel like throwing up.