Preamble: I wrote this 6 months ago as a post-mortem on a project I did over a year ago. I  have held off on publishing because I continue to feel ambivalent about it. As a technologist, I love the possibilities of where this is heading.  On the other hand, I continue to struggle with my own feelings about privacy and social networking. In the last few weeks, Google’s Project Glass has once again been making news. For better or for worse, the time is now.

…has not been made. Yet. It is a never-produced social-mobile network based on facial-recognition software. This post is a post-mortem on an idea pursued and ultimately abandoned by two veteran entrepreneurs. What I hope is that the pre-pre-production process we engaged with Ipvoo is useful, despite our eventual decision not to continue. More, I think Ipvoo raises important questions around the nature of technology and privacy. These question are not only current and relevant, but indeed, more and more unavoidable. We chose not to pursue Ipvoo for a combination of technical and moral reasons. I am, however, concerned that the kind of facial recognition/social media interaction represented by Ipvoo is coming. Indeed, Google has already patented some of the technology that might be used for something like Ipvoo. I think of this type of social media network as a Pandora’s Box, tempting, but luckily still made out of reach by contemporary technological barriers. What will happen when, and if, we lose those barriers? What Ipvoo tells me is that we need to think about the issues raised by the future of facial-recognition software; we need to think about these issues now, before we find ourselves having already opened the box.

But first, what is Ipvoo?

Like me, game designer/producer Matt Mihaly is a veteran of several start-ups. Ipvoo emerged from a discussion we had about what kind of new social-mobile network we could create. We came up with a social network that combines facial recognition software with something that looks like a twitter feed. A solid, simple idea. We explored it for about three weeks, asking: What was the idea? How would one launch it? Is the technology there and how would it work? What are some use cases? How would we monetize it?

Below, I discuss what we found and why we decided not to go forward.

The concept:

1. I, the user, take a picture of a person with my phone; I have the option to type in a comment. This is the entirety of the user interface.
2. This image gets date, time and location stamped; then, uploaded to a server.
3. I get a response that is a list of every picture in the system (complete with comments, dates, and locations) that has been taken of this person.

Powerful in it’s simplicity.

So how might this work in the real world? I see someone interesting, either friend or stranger. I take their picture with my phone.

I have the option to write a comment: “Phoebe loves Play-Doh!” or “look, it’s Bon Jovi!” I submit the photo. In return, I get back a stream of images. This stream consists of every single picture this person has been in, and any accompanying comments: “Phoebe loves doggies!” “Phoebe loves chocolate cake!” or “Isn’t that guy famous?” “Cut your hair!” Very simple and very fun.

What if you want to check and see if you are in this social network? You just take a picture of yourself—in return you get every picture taken and comment posted about you. Of course, you are now in the system….. ’cause you just took a picture of yourself.

Adding a Facebook Connect option would allow users to include an existing library of tagged photos for themselves and their friends.

We enjoyed thinking about how we could pitch this:

Imagine going to a room full of people sitting around a conference table, and saying “Everyone gather around the conference table for a minute—say cheese!—Ok, you are now all in my new social network. Would you like to see what I just said about you? Go ahead take a picture of yourself and find out.”

If I was previously linked to at least one of these people through Facebook Connect then more then just the demo input content would be available.

At this point, we realized that our little theoretical service here was a social network with no opt-in, opt-out option. This was something that made us extremely squeamish.  We spent a few days talking about how to make it so people could opt-out and still keep things simple. The best we could come up with was that, to opt-out, people would have to actually make an account where they explicitly state: “I do not wish to be a part of this.” Alternatively, one could opt-in and regulate what images and comments are available. Neither of these are ideal solutions, as they place the burden of maintaining privacy on the individual. (This eventually became one of the many roadblocks that killed the project.)

We also quickly realized that all of the things that we would have to do to make it so users can regulate their privacy would make this a much larger and more complicated project. Matt said, “This is powerful in its simple form, we shouldn’t be working hard to make it less powerful.” In the end, we came to the conclusion that the best launch would be one that was simplest. User controls, opting in or out, all of that could be added later.

Next, we turned our attention to how a successful launch might be engineered.

The launch should be as simple as possible, featuring only the core idea. Start here, and iterate. In terms of building the thing, this seems like the most obvious way to proceed. Launch the core service and then figure out what needs to be added. Research showed us that while there are people talking about facial recognition, there doesn’t seem to be anyone who has taken it and run in the manner we were proposing here.

We felt that launching Ipvoo, a facial-recognition purely mobile social network (no accounts, no web access, no opt-out setting) would be quite controversial……and, consequently, advantageous. Controversy is publicity right? And controversy would help in the face of the biggest problem at launch: density.

We walked through the following scenario: I hear about this. I am the first person to get it. I start walking around taking pictures of people. I take one. I get no info about the person. I take another. I get no info about the person. A new user is probably only willing to submit images a certain number of times before it begins to feel futile.  What we want is for a user to get a response by the 3rd or 4th upload. That’s how we trigger the “A-ha! I see how this works” moment, and more importantly, the “this is fun, let’s do more,” moment. But if there are no images in the system yet…It’s a problem faced by all new social media platforms: how do you get a critical mass of users?

One small solution for this would be to rely on Facebook Connect to seed the system with images people have tagged in their photos. But Facebook Connect doesn’t have the power or coverage we would need to truly become viable. Still, we felt that Facebook did, indirectly have the solution.

We started by looking at how Facebook launched. When Facebook started, they targeted specific populations. In their case, they went college-by-college, translating in-place, real-world communities of friends into virtual communities of “friends.” Because Ipvoo is mobile, it would be an activity people do at locations. Could we do promotional activity that targets locations? Maybe we could do low-cost poster advertisements in specific places: public festivals, concerts, college campuses. These posters could have games and activities embedded in them that introduce Ipvoo. The people in the posters could be famous, or invented characters; basically we could guarantee an interesting interaction occurred. By targeting specific geographic regions we could try and achieve local usage densities. To tie this together, we could roll out the network on a region-by-region basis. Combine this with allowing people to connect with Facebook and other social media. At a conference, for example take a picture of someone and it pulls up his or her LinkedIn profile. Together, these efforts begin to solve the density problem. Yes, user acquisition would be a fight (it always is). We felt, however, that we had enough strategies for user engagement such that early adopters would turn into long-term users.

Our next question centered on technology. Is the technology for Ipvoo ready and available?

In theory, the Ipvoo system should be comprised of already existing technology. Initially, we were excited to see that there seemed to be a lot of it available for this project.  We found Face.com quickly (which has since been purchased by Facebook). There were other services available as well. We set out to test the available tech.

We began with the then-in-place Facebook recognition software. We uploaded 40 pictures of 3 different celebrities. We found that the FB software had trouble matching almost any of the images. FB couldn’t even identify a number of straight-on headshots of Brad Pitt. We realized we had an identification problem.

As I indicated in my introduction, facial recognition software is just not that good yet. It may get better in a year or two, or ten, or never at all. When I was a kid, I was sure we would have flying cars by now. On the other hand, I also thought we might have Jetson-style video-phones. And we do. We don’t know when and if the technology might be available for an Ipvoo-style project. 

We came to three ways we could solve this problem:

1. Do better facial recognition. Rather then going with an off-the-shelf facial recognition package, we could work with a team to develop a better, custom program.
2. Use other factors to solve the question of identity. For example, we could use location and timing services to help narrow the field of possible faces in an image. If we see that a photo was taken of a person in a coffee shop in Mill Valley at 9 am on a Tuesday, it is likely that a similar photo taken in the same place the following week is actually a match. On the other hand, a photo taken in San Francisco is unlikely to match a photo taken a ½ hour later in Tokyo.
3. User input. When, as an Ipvoo user, I upload a picture, I am then prompted to respond to three similar pictures “Is this person A the same as this person B?” etc. In theory, this would then begin to build the database such that these prompts would become less and less necessary.

Given our team expertise, building better facial recognition software is unlikely; the other solutions, while less-then-optimal, were enough to keep our conversation going for one more week.

We moved on to our final questions: What are the uses cases for Ipvoo? And, how might we monetize it?

Lots of social networks don’t monetize right out of the gate. Indeed, infamously, Twitter has yet to find a way to successfully make money.  We could accept that monetization may not be our driving objective here. Still, the question of use cases applies more broadly.

With Ipvoo, we assumed that our first users would simply be using the network for fun and socialization. We also felt that once the service was up, users would discover uses we hadn’t thought of.  We knew that we would want to watch for those use-patterns and move quickly to support them. One feature that seemed to make sense was the idea that a user could enable a sub-network: a series of posted pictures with comments that I could then share with a select group of people. We filed that under “not-for-launch” and moved along.

We also identified some use-cases specific to this particular technology. I’ve already mentioned the conference attendee who accesses a colleagues’ LinkedIn profile with a quick click of the camera.  Here’s a few more:

• A maître d’ at a fine restaurant (or for that matter anyone in the service industry) takes a surreptitious snapshot of a customer. This identifies the customer for them and allows them to pull up information from previous visits. The maître d’ can then greet them by name, take them directly to their favorite table, and serenade them with Abba songs (if that’s indeed what this customer prefers).
• At a wine tasting, people buy 10 tastes with no tickets. Instead, the servers could record each taste with a quick picture. These pictures, and comments about them become part of the fun.
• Security at a fair or concert could identify troublemakers. They could flag someone who has already been asked to change their behavior. One warning, then you are out, buddy.

This last example demonstrates why, for once and for all, Matt and I decided against pursuing Ipvoo. Once again, this particular social network has no real opt-out setting. These days, by Sunday morning, anyone I met on Saturday night has already checked out my Facebook page, liked a few photos, and maybe even reposted some old posts. I’m already not happy with the way that Facebook blurs the line between public and private. Ipvoo takes it a step further. With Ipvoo, anyone who has any kind of access to a photo might easily have access to all of a person’s online photos. Conventional notions of privacy are effectively obliterated.

In the end, we decided not to pursue Ipvoo. Not only is it technologically dubious, it feels ethically unsound.

Be sure, however, that once the technological barriers are down, someone is going to create some kind of mobile-social network based on facial recognition software. Indeed, Googles’ Project Glass and the iPhone/Android app Scene Tap  both pose similar challenges to contemporary notions of privacy, while seeming poised to take advantage of (eventual) facial-recognition software.  So take this as a warning. This is what is coming. How are we, as a society going to deal with this?