Big Data mined for voter sentiment

No face will ever be lost in the crowd these days, thanks to Big Data. This observation was proven again in the aftermath of the Boston Marathon bombing.

As part of the safety preparations for the marathon, CCTV cameras were strategically placed along the route of the run. As it happened, a team of IBM experts were on hand to help zero in on the suspects with the use of video analytics.

The IBM program is designed to comb through terabytes of unstructured data like CCTV images. It immediately went to work looking for people carrying backpacks and matched photos with names. Soon enough, they had both face and name.

Many of us think of Big Data as only numbers, letters and words. Actually, images are also part of Big Data that can be mined for information. That’s what made a difference in the aftermath of the Boston Marathon bombing.

Big Data can either be structured or unstructured data. The list of the top taxpayers is an example of structured data. Our Tweets and Facebook posts can be seen as unstructured data.

The system they had for the Boston Marathon is like the security component of IBM’s Smarter Cities program now being used by Davao City. I understand they have CCTVs all over Davao that’s connected to a control center.

Business ( Article MRec ), pagematch: 1, sectionmatch: 1

Video feeds, police blotters and similar sources of data are being captured all the time. With the use of analytics, data from all these sources are at the fingertips of the local city officials and law enforcers. IBM Analytics has given them the means to make sense of and use those data.

The same is true with social media… Facebook, Twitter among others. People constantly pour out their hearts, their views, their preferences on social media. All that data, unstructured as they are, can be mined to understand public sentiment on issues.

That is precisely what ABS-CBN’s HALALAN 2013 coverage is doing now with the help of IBM Analytics. Using a proprietary program of Big Blue, ABS-CBN is trying to understand how candidates and issues are being discussed in social media.

What they are doing is not a survey. And unlike a survey, the data they are getting from the “crawlers” IBM has unleashed on the web is real time. They can find out where the social media noise is coming from and the intensity. They can also separate the grain from the chaff… they can tell what is genuine social media sentiment from what are mere noise from PR professionals working for candidates.

But how valid are the findings from such an effort? There are those who say that social media is merely the playground of those who have access to computers and the Internet and therefore rather elitist. That’s not an accurate view.

Social media is not as elitist as one would suspect. Cheap access to computers is now available in internet cafes nationwide. With many families having a relative or two working abroad, Pinoys at all levels of society have become rather computer and internet savvy. There is immense value in understanding social media noise.

I am told that at least 1 in 3 Filipinos now have access to the internet, and with 80 percent-95 percent of these netizens using social media. There are supposedly close to 30 million Pinoys on Facebook and some 10 million on Twitter.

Indeed, social media is changing the way Pinoys communicate. This includes how we convey our feelings and opinions about candidates and issues during elections.

What citizens and candidates say or do via social media–through Twitter or Facebook– have a significant impact. A photo or video uploaded on the internet that goes viral and is picked up by TV and radio can make or break a candidate. A tweet or a Facebook post shared by hundreds or thousands can change voters’ sentiments toward a candidate.

The growing popularity of smart phones is also revolutionizing how Pinoys use social media. I recall someone saying in one meeting I attended at ABS-CBN that for our news website, there is a noticeable shift of readers who are now accessing it through their smart phones rather than via a computer.

Thus, with new media and traditional media being shared instantly, literally at the fingertips of millions using smart devices, more citizens now have more power than ever before to move voter sentiment. Their ability to access and share data, socially, with viral speed is the digital version of people power.

The idea to mine Big Data to provide depth for our election coverage came from Gabby Lopez. I remember the management meeting where he expressed admiration for the work of Nate Silver in mining data so accurately as to predict an Obama re-election even when some experienced pundits thought Obama was finished.

We toyed with the idea of replicating that work here but couldn’t because we don’t have the rich reservoir of publicly available data that the US has. I lamented this situation at one IBM conference I attended and Charles Manuel, an IBM Singapore official I was talking to told me I was wrong. Good work can be done using unstructured data Mr. Manuel said, pointing out they have done it in Singapore.

I was not impressed… I told them Singapore’s politics is boringly uncomplicated compared to ours. But Mariels Almeda Winhoffer, the Filipina who is now country manager here brought in Lim Wee Tee, the IBM Singapore guy who worked on their Singapore election project to brief me.

Well… I got the impression that what they have is something like what Google is doing with data search we are familiar with but with a lot more power to do analysis. Mariels then had Chris Howard from IBM’s Sydney office to fly here to talk to me and make a presentation to ABS-CBN News led by Ging Reyes and Mark Lopez, ABS-CBN’s CIO. Chris worked with the groundbreaking Obama campaign which successfully used Big Data and social media to win the election.

Like all newsmen, we were naturally skeptical. It didn’t help that technology is somewhat alien and intimidating to many of us who were journalists when the newsrooms still used teletype machines. But Mariels was eager to show what Smart Analytics can do and offered to work with us anyway and prove their capability.

Being active social media participants ourselves, Ging and I and the rest of her news operations executives were also eager to make sense of social media activity in the 2013 mid-term elections. So we decided to take Mariels on and ABS-CBN Integrated News and Current Affairs (INCA) tied up with IBM Philippines to utilize its deep analytics expertise and patented tools to analyze public interactions on social media. Besides, we like the pioneering spirit of being the first news operation in the country to try this groundbreaking add-on to our election coverage.

We have now started to tap into publicly available information posted on Twitter and Facebook in an attempt to measure the online pulse regarding Halalan 2013. With use of a social media tracker, we are getting a good idea of who among the candidates are being talked about by netizens. We are also seeing if these public engagements are mostly positive, negative or neutral.

By analyzing social media feeds, we have an ear on the “chatter of reality.” However, we recognize the limitations of the system. The IBM guys told me this is also the first time they had to build a new dictionary to cover Tagalog, Taglish, Cebuano words.

IBM is also working on ways to make the cloud program more powerful and analyze larger data sets. They are increasing the accuracy of the insight the program is capable of so that maybe it can in the future mirror the offline world.

Mesmerizing as the IBM Analytics tool is, we view this effort to understand social media as something that’s not really about the tool, but about the message. Used properly, it should help strengthen our democracy, making vox populi come alive and have its sentiment expressed and received by the powers that be in real time… better than any House of Representatives can.

Check it out in the Halalan 2013 website: http://halalan2013.abs-cbnnews.com/battlefield/home.html and let us know what you think. It is work in progress… it is not a survey… it is an attempt to make sense of all the chatter… all for the good of our democracy. –Boo Chanco (The Philippine Star)