Streams: Layers and Data

This past week TechCrunch jumped on the stream meme. Erick Schonfeld published an article that generated some noise, citing a recent post by John Borthwick.

Playing with Facebook this week got me thinking more about the role of data within streams. Specifically, I have a few follow-up thoughts to my last post on the value and consequences of real-time information. At a basic level, streams are flows of objects, with each object containing unique metadata. However, this metadata can be abstracted to create parallel streams layered one on top of the other. I think we’re about to see a rise of what I’d call ‘data abstraction layers.’

Facebook’s stream already provides a glimpse into what an abstraction layer might look like. Individual inputs enter the stream, but they often have significantly more value when threaded together to produce a synthesis unrecognizable if only viewed separately. Think about the assumptions/conclusions that can be drawn and delivered from disparate data: the sum is often truly greater than the parts. In a minor way Facebook already has created a data abstraction layer delivered in the form of Facebook Notifications: birthdays and relationship updates are sent to users as synthesized data, not as individuals inputs. Stating, “Six of your friends liked this linkâ€� is a synthesis of six friends individually clicking “like.â€�

With relatively little effort, Facebook could take this a step further. Many of my friends constantly change their profiles and thankfully Facebook doesn’t alert me about each of these changes. Yet, if my friend Sally adds a new favorite movie and that movie overlaps with one listed on my profile, Facebook could easily deliver the synthesis: “Sam: you and Sally have a favorite movie in common!â€� Another example: the fact that I publish a status update geo-tagged ‘Austin, TX’ could ping my Facebook friends living in Austin, “Sam Huleatt is traveling is Austin. You should look him up!â€� These are just a couple examples of synthesized data acting as a layer (another stream!) sitting on top of my Facebook stream of individual inputs.

As a last thought — these parallel stream layers could be also be exponential; each subsequent layer produces its own synthesis of the data inputs and patterns residing underneath it.

Hmm.

Tagged as: data, Erick Schonfeld, facebook, John Borthwick, metadata, streaming, streams, techcrunch, twitter