Bookmark and Share

Research: Intelligence and Corporate Intelligence / Media Analysis / Industry Monitoring / Brand Mining / Public Perception / Social Media / Event Mining

I am the chief architect of the largest open source intelligence project in academia, leveraging commercial, governmental, and declassified intelligence products to produce global databases of human activity across multiple disciplines over time. One project involves cataloging all "societal stability" events (riots, assassinations, protests, etc) across the world from 1946 to present using tens of millions of local news reports captured and translated from the local presses of each nation. I was also the chief architect of the NCSA VIAS project, one of the early "web-scale" industry monitoring systems. I am founder of the Carbon Capture Report, the premier global climate change news analytics service. I have worked extensively on a wide range of corporate intelligence, event mining, and media monitoring initiatives, especially issues such as trend mining and public perception.

MEDIA ANALYSIS / INDUSTRY MONITORING / BRAND MINING / PUBLIC PERCEPTION

I have worked extensively on a wide array of media analysis, brand mining, industry monitoring, and public perception problems, including numerous "grand challenge" problems.

Carbon Capture Report

One of my current signature initiatives is the Carbon Capture Report, a global news and social media monitoring and public perception analytics platform for studying global patterns in climate change coverage. The system continually monitors the mainstream and social media spheres, including the blogosphere, Twitter, YouTube, and social bookmarking sites, in realtime, performing "crowdsourcing", advanced geographic intelligence analysis, data mining and analytics, and compiling autonomous bibliographies of all people and companies mentioned in the news and their relationships. The core underlying technology is easily adapted to other industries or topical areas and a wide range of more sophisticated analytical services are available.

VIAS

I was the chief architect and technical director of the NCSA VIAS project. VIAS is an award-winning domain-specific information retrieval, archival, and processing system. It has been at the forefront of a number of Fortune 50 corporate intelligence initiatives and deployed extensively in data mining and intelligence applications for corporate, academic, and governmental clients over the better part of a decade. A VIAS Dynamic Knowledge Repository (DKR) is seeded with a set of topic words, URLs, and other information that define the operational parameters of its topical area. VIAS then monitors all known mailing lists, USENET groups, available wirefeeds, and other streaming sources, as well as autonomously crawls the web using a federation of directed web crawlers to collect information on this topic. All information is screened for relevance against the topical parameter space and then undergoes a series of metadata extraction and production processes, where filters identify people, company, and organization names, bibliographic references, and acronym resolutions, together with higher-order processing such as summary abstract generation. An advanced analytics and visualization suite offers a range of features based on this generated metadata, such as temporal entity graphs and relationship mining.

SOCIAL MEDIA

I've worked extensively on the intersection of the social and mainstream media spheres, including diffusion across geographic and typographic borders. I've developed a large array of robust fully automated analytical tools that have been applied in a number of large intelligence and media monitoring initiatives, including advanced crowdsourcing platforms.

EVENT MINING

I am the chief architect of the Social Political Economic Event Database (SPEED) at the University of Illinois Cline Center for Democracy, an effort to compile a global event database covering 172 countries from 1946-present. The resulting database will consist of all major human events in those categories occurring anywhere on earth in the past 60 years, offering an unprecedented view into the underpinnings of societal evolution. More than 1,200 variables are captured for each event, including date and location to city or landmark resolution. A combination of machine and human processes are used to automate much of the heavy lifting of event coding, while still allowing for the sophistication of interpretation afforded by human analysts. The combined event dataset takes the form a vast network of human activity, with each event being a node and all 1,200 variables acting as links connecting those events. Predictive analytics and "scenario modeling" can ultimately use this event network to forecast sets of potential outcomes for emerging conflict across the globe.

WHITEPAPERS