More Surface secrets – you'll wish you were here

this app was REALLY cool
There were still a few surprises up Microsoft’s sleeve today at this little conference I’m at. They’re talking about the SDK, testing the surface with robotic arms, and they demoed some totally freaking awesome apps they’ve got running in launch or alpha state. Believe me, you do wish you were here. Lots more pictures inside.

First, we saw a really nice photo app that’s going to be shipping with the units. It’s actually not that hard to set up, apparently — we had a developer show us how to throw together a pretty complex photo-and-tag-viewing program for the Surface in just a few minutes. The SDK is pumped up with all kinds of shortcuts and things that allow developers to take advantage of, say, the ability to slide an object around and have it stop based on friction or what objects it hits. This is great for the photo app, where you can move photos in and out of a little bubble or bar and toss them back in easily. Sorting and stacking the pictures felt very natural, like just sorting your usual 3x5s, except in this case some were videos, some you could expand to enormous size, and you could organize them automatically by metadata, category, and so on.

Next we saw probably one of the coolest things I’ve seen in a while. Essentially a geotagged photos viewing application, it had Virtual Earth (very fun to play with, as I did without permission for a bit) and spread across the world were little photos. The photos were placed on the map exactly where they were taken and they could be selected by hand gestures, or filtered by other means so that only, say, panoramic photos taken in 2007 would show up. I know I have a lot of those. Geotagging has always been kind of “meh” for me, but this made it accessible, purposeful, and entertaining. And it was something a developer had been working on in his spare time.

They then took us down to the labs, where we got to see how they stress-test these mothers. After all, there are so many freaky combinations of touches, drags, and rotations that when combined with all the stuff they’re trying to make instantly accessible, bugs are certain to crop up. So since they can’t have 1,000 monkeys tapping on 100 units until they’ve exhausted every possiblity, they use an emulator to simulate multiple fingertips; it looks pretty freaky having all these ghost touches working over their demo environment.

They also have a robotic arm named Petey. It’s kind of like a soft little jackhammer that has a bunch of preset tests that they can run to check whether a configuration is up to date — drag latency and that sort of thing. It was fun to watch in action, although to be honest I was kind of hoping for something more android-y, with fingertips and everything. I get the feeling the people involved really want to make this thing exactly like their vision and are willing to go to kind of ridiculous lengths to make sure it’s not going to crap out on some grandma and ruin her night.

The developer we talked to built this app that, using pretty simple coding and in just a couple minutes, would read the tag on the back of this physical picture, fetch the metadata from the flickr account it was from originally, and display it all side by side with the other tagged, categorized pictures that were in fact not physically real. Seriously people, this was like some science fiction tech here but it’s real and it actually works very well.

One of the last things they showed us were some research facilities where they test out the Surface on unsuspecting Microserfs. They record a ton of information from each testing session and it’s displayed thusly:

That was more or less the end of the ride. It’s hard to tell from the pictures (and even in video it’s hard to see just how dynamic these applications are and how well they work), but this technology is really impressive. I’m a little disappointed that it’s mainly going to be doing grunt work like hotel concierge stuff and parlor tricks until it really gets a big roll-out, but I firmly believe that once a few people get their hands on these things, they’re going to want them everywhere. I asked about seeing more of their little games and pet projects but we were out of time. They’re running down lots of possibilities, they said, but for now they actually need to put out a product so they can continue to make these things better and better. I’ll link you all to some video as soon as the other guys I was with get it live.