Interesting Data Algorithms and Dream-e

When I was in London a few weeks ago, I met up and stayed with friends who developed an unusual web application on iPhone and Android.

 

ccc
Originally, we met up on my last trip, in late September 2011, when I attended a Japanese Tea Ceremony at their place.  At the time, I was first introduced, I saw a prototype of the Dream-e application, which just launched on the ITunes  and Android platforms yesterday.

When in London last month, I also attended the Dream Cafe, which took place near Covent Garden, London, the session I went to was focused on the first dream of the new year.   But I didn’t understand the Dream application till I spent a few days talking with Adi, saw the programming and mind behind it, and some of the other applications he worked on, over the years.

In spending the time there, I was interested in showing the various applications of Social Media Analytics that I have used (pretty much all of them) but found what he had to show me was actually much more interesting.   It got me thinking, also, that many of the issues in Social Media Analytics that I regularly write about, both in my blog, at AllAnalytics.com and in my book, are not the only issues – there is also a real need to have intelligent algorithms that find interesting patterns in the data stream, patterns that people are usually unable to isolate out of the “river of news”.

The applications that Adi  showed me were more in the nature of engineering, one project showed the results of black box aircraft data which looked for abnormal data patterns in take offs and landings; another involved finding sequences of patterns in electrical impulses stimulating electric paper type displays that led up to the Amazon Kindle.  It seemed the original prototypes of that technology had a problem with echo images that made the technology unusable (images too fuzzy) till the problem was solved.  Adi wrote a genetic engineering algorithm that led to solving the image echo problem.   A few other projects led to similar insights, with the key factor being the intelligent algorithm that made the data more valuable and usable.

Many of those algorithms and heuristics  aren’t really used enough, or smart enough, to help up make sense of social data – which I felt was where the opportunity for social platforms now lies (collection of data will always be somewhat imperfect, but what makes the data useful that we collect is the ability to look for the unusual patterns, the things that are most important for us to know).

Enter the Dream-e application.    When I asked Adi about Dream-e, he mentioned the algorithm he used will work with any dream – and so I tried it with a couple of dreams, and it seemed to work pretty well for me – it looked deceptively simple, but I suspect, isn’t simple at all – and sometimes what seems simple is the hardest of all to pull off.

Screenshots of Dream-e:

    

First, after downloading the application you start by launching it, working on a dream you already saved, or entering a new one.

 

Typically, in the first version of this application, you type the dream as you recall it – I found writing a long dream might make it harder to decide what to process in it, so staying with a short paragraph is probably better, but whatever works for you should also work with Dream-e.  In this case, the image above shows a dream (covered, I think in the video I embedded) that included a tiger, etc.

 

There are a couple of options of what you can do with a dream once you save it – those are shown above, and usually the choices cover most of the possibilities, and you can analyze any dream in about 10-20 minutes, depending on how deeply you want to explore the dream.

    

Once done, you get the analysis of your dream which you can also share in Facebook and Twitter; the interpretation is more for major themes, so you don’t need to worry about there being anything too personal in the interpretation – but you can edit the text, in any case, before posting it.

Certainly, the application presentation can be improved, and will be in the future, but it’s the algorithm behind it that I find fascinating, when you enter your dream, the analysis pieces together different aspects and makes suggestions to you on what to work on (the old “you” vs. the “new you”) that are way beyond simply echoing back your own imagery and text.

How does this tie into Analytics?  

Dream-e, with its deceptive simplicity, and the fact that it focuses on giving you what you really need, what is important for someone who uses it to know, which is what any good analytical tool should do, allows someone to analyze any dream at a fairly deep level.  That’s pretty hard to do.

Most platforms, in general, are designed to be good at only a few things.  Why?   Because their founders and investors need to succeed, and they are told to focus on just a few things that customers will be interested in having or need – and be exceptional at it.

Certainly that was the advice I heard at the UltraLight Startups the other night in NYC – but that advice also makes it really hard for businesses and individuals who buy and consume information and processes, creating the “mess” most businesses now face with big data, and a bunch of non-interoperable applications, they were all designed to just one or two things well – but often never fit anyone’s particular needs (esp in Social Media Analytics).

I could give examples of that if readers really want – and perhaps I will some time soon.

 

The same mind that programmed Dream-e is behind some of the most sophisticated algorithms I’ve seen, the very kind that are missing from Social Analytics platforms – even the most popular ones (especially the most popular ones).    Funny thing, in Web Analytics, Google had done some work in Google Analytics with Event Prediction that was considered fairly innovative, but we have yet to see anything much in Social, save perhaps Recorded Future and Glide Intelligence (which was incidentally, sold to NASDAQ a few months ago – talking about where real value lies).

With Glide – look at the bottom line of the timeline.  And, btw, Glide Intelligence actually generates a timeline similar to that, mostly automatically, based on an algorithm that uses Viterbi logic – i wrote about it in Chapter 4 of #smabook.

A Viterbi logic decoder reads online content and compares it with a set of “probable” interpretations. The interpretation with the “least error” is selected and used. This logic is trainable within systems such as Glide Intelligence and Crimson Hexagon.

I don’t know for sure if Crimson Hexagon does use Viterbi Logic, though, come to think of it, and it might be an entirely different algorithm – but the value of Crimson Hexagon is in the Algorithm, that’s for sure, which founder, Gary King (who is working with the UN on a Big Data initiative called the UN Global Pulse) is quoted about in the New York Times article today on Big Data.

Getting back to algorithms, look below at what happened to Glide Technologies – it was all about the algorithm – I know the guy who wrote the Viterbi logic decoder algorithm – and it make Glide Technologies do something with data automation that got NASDAQ very interested.

….


With Adi Andrei’s gift as a engineer, I’m talking about what is essence of a platform, what gives it intelligence – that is the algorithm, and it may well be that only a couple hundred people on the planet could create and adapt algorithms in the way he did, the example of Glide Intelligence should be an example of what happens when an when you combine data with the right algorithm.

Download the Dream-e application on the ITunes  and Android and try it for yourself.

 

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>