social media monitoring

Audience Intelligence Formula – Late 2017

SUMMARY: The formula for best-practice audience intelligence work is dynamic, due to rapidly advancing technologies and practices. We also have to take into consideration federal and nation-state regulations which are constantly in flux. As of December 2017, this is a formula for producing a fully enriched audience based on the beginning point of email, social handle, or a conversation snippet from social media, blogs, forums or comment threads on news sites. I might add that the acceleration of AI (Artificial Intelligence) and ML (Machine Learning) is rapidly condensing the steps below. APIs (Application Programming Interface) are being woven together to produce highly sophisticated machinery that blends outputs from various data sources.



STEP 1: Interview the client to determine goals and deliverables. This is important because we want to understand the client’s business and his specific goals related to the research. This is the time to see eye-to-eye with the client and really get to know his/her business from the inside-out.


STEP 2: Perform tests using the software below to discover opportunities related to the client’s desires/needs. Most social monitoring/aud intel solutions provide a way to quickly preview “the universe” related to a specific research request. This step is important as part of putting together the proposal for the client.


STEP 3: Design the audience intelligence study. Gain approval from the client, sign contracts, receive funding from the client. This is important because the client will want to look over the exact type of deliverable he/she will be receiving, as well as hearing a bit about your approach in putting together the final “data lake” of prospects and, of course, how the insights will be presented.


STEP 4: HOW DO PEOPLE DESCRIBE THEMSELVES IN SOCIAL BIOS: We gather a set of social handles where our keywords are in the bio. This from, Affinio, Brandwatch and PeoplePattern primarily, but can also be from using in LinkedIn, along with many other sources herein un-named. This is important because self-description by individuals is rather verbose these days..and this is helpful to us in finding our targets. And, where self-description is not verbose, there are clues through company/press/3rd-party descriptions of specific individuals who work for/play at/eat at/drink at/shop at/you-name-it at the locations/venues/places we are studying.


STEP 5: WHAT DO PEOPLE TALK ABOUT IN SOCIAL MEDIA/BLOGS/FORUMS: We gather a set of conversation snippets related to our keywords from Brandwatch, Sysomos, Crimson Hexagon and/or Meltwater. This is important because what someone says about an activity/product/service/location/you-name-it contributes towards our understanding of the consumer/customer/competitor’s consumer/customer. In addition, these conversation snippets from social media, forums, blogs, comment threads on news sites form a body of data that we can segment into specific topic groups. These specific topic groups can then be used to form a point of view/set of insights on the target we are studying.


STEP 6: SEGMENTING STEP BY MACHINE – TOPIC MODELING APPLIED TO BIOS & CONVERSATION SNIPPETS: We use Converseon’s Conversus tool to perform topic modeling and separate bios and conversation snippets into discreet topics. This is important because this speeds up the analysis of the bios and separates out the bios that matter to us the most. No solution in the world is more accurate and complete in segmenting bios and conversation snippets into discrete topic groups than Converseon’s Conversus. This solution is pure magic and the analysts at Converseon who are using Conversus are second-to-none in their expertise at building out superior insights based upon the use of their in-house solution. It is during this stage where an analyst begins to really gain deep insight into a sets of bios and sets of conversation snippets.


STEP 7: SEGMENTING STEP BY ANALYST:  Now, these analysts wade into the output from Converseon’s Conversus and identify the Topic where our targets are present (and any other discovered target – unknown unknowns). This vital step by humans helps us know which Topic groups in Conversus are populated by our target. This is important work that can be recursive, whereby the analyst segments the data using Conversus, reads through the results, and then segments again to refine even more deeply. I might add that this particular step is where the machine will eventually outstrip the analyst. That outstripping of the analyst will take some time to come along, though. For now, on this step, the human continues to be the last mile.


STEP 8: RE-STITCHING AFTER SEGMENTING STEPS: Reunify the bios from the “right” Topics in Conversus with the handles in the original source data sheet. This is important because we want to have the correct bio next the correct social handle. We also stitch handles and bios next to discovered conversation snippets at this stage. We find that working in Jupyter notebooks using Python is one of the handiest ways for our teams to work together efficiently on this step.


STEP 9: PEOLE PATTERN AUDIENCE INSIGHTS STEP: Upload Twitter AND/OR Instagram handles into PeoplePattern for deeper enrichment of Interests, Location, Age, Persona. This is important because we find out more about each person and we also delineate between Individuals and Organizations. In addition, we move a step closer through PeoplePattern to verifying the “real people”. Finally, we gain insight into the Persona types, Interest groups and lots of other useful info.


STEP 10: FULL CONTACT STEP: Use the Full Contact API to append add’l social handles. The value in this particular step is discovering a full name related to the handle AND additional social handles. We also gain bios from various social media handles, thus bulking out our story about an individual.


STEP 11: PIPL STEP: We use the PIPL API to gather Email, Phone, Address, other social handles, age, and many other bits of info on individuals. This is important because we will gain additional important information about the individuals that verifies they are “real people”.


STEP 12: CRYSTAL STEP: We use the Crystal Knows API to enrich the profiles with DISC personality type, personality overview, messaging guidance, selling guidance. This is important because then we are guiding our client on how best to market, advertise and sell to this individual. We can then group individuals by DISC type, if desired.


STEP 13: CLEAN UP ROWS FOR COMPLETION: Stitch together the results from the various APIs and then filter for complete rows. Again, we use Jupyter notebooks and Python for this work, as the teamwork and efficiency is vastly improved. This stage is important because we want every row to have every cell filled with correct & complete data.


STEP 14: EXACT DATA STEP: If we want to add an extra step for verification at this point, we run the Exact Data enrichment on the emails/names discovered. This is important because this extra step adds validity to the claim that our audience is full of “real people”.


STEP 15: SPOTRIGHT STEP: Upload the social handles into Spotright to gain enrichment of offline Axciom data, such as buying/purchasing styles, net worth, income level, political/religious affiliations, housing info, household complexion info, brand preferences of a specific group uploaded and much much more.


STEP 16: INSIGHTS, METHODOLOGIES & RECOMMENDATIONS PDF: Create 3-5 page Summary PDF with insights related to the research. This is important because brand leadership will now have a set of insights about the discovered individuals (our initial data lake of prospects) and, importantly, we can recommend further research steps for successive work together.


Who is the actual human we are targeting?

We begin by identifying every single person who is online. We do this by proving which social handles are connected to real people and which are simply junk accounts. The way to prove if a social handle is connected to a real person is to append extra data next to a social handle, such as other social handles, emails, addresses, phone numbers, the social handles of family members, job titles/bios from social sites (LinkedIn, Facebook, Twitter, Instagram, Vimeo). This is important because we want to analyze real human beings and their behaviors. The reason we want to study real people is to make accurate predictions about human behavior and the outcomes of behavior in different contexts. When we can predict behavior more accurately, then we are able to influence individuals, which is one of the ultimate end games of marketing, advertising and pubic relations.

When we have a healthy set of columns with this kind of information, then we can move on to using tools that study all of the content posted by an individual, as well as the way the individual describes himself/herself in bios on various sites. We can also study posts about individuals, some of which may include video footage, images, interviews, and sites that denote achievement. This set of software studies all of this content about an individual and classifies him/her as a specific persona type, along with the interests this individual focuses on. This is important because we want to know more about the type of person each individual is and what influences his/her behavior.

The next set of software we use derives insight from offline data, such as the data from credit bureaus, credit card companies, direct marketing companies, catalogue marketing companies, club membership research, background checks, etc. When we blend offline data with online data, we are able to demonstrate with more confidence the brand preferences, purchasing-buying styles, and many other classifiers related to an individual. This is important because online conversations, posts, and self-description in bios do not always give us enough to deeply understand the behavior of individuals. The blending of offline and online data results in a more complete portrait of the individual human being.

In short, we are able to more precisely influence what an individual human being will do in the future. For a brand, this long-term influence is very important as this will be how loyalty and sales are ensured. There are, of course, deeper goals for other organizations, such as governments, religious groups and media groups. These groups are often interested in wholesale culture change, particularly in enemy/competitive territory. The action of changing another culture is a top long term priority of groups that have been around far longer than the Unilevers and P&Gs of the world.

A Formula for Better Insights on Consumers & Their Emotions


THE PROBLEM & QUESTION: The Head of Insight at a major CPG brand is trying to figure out why brand equity is falling. A CEO is wondering why sales are dropping. A Senior Analyst is assigned by the Head of Insight with answering these questions.

THE FORMULA FOR ANSWERING THE QUESTION: The Senior Analyst decides to work with social data to discover the reason for falling brand equity. She comes up with the following formula:

A focus group of individual consumers (with a full dossier on each one) + these consumers’ emotions (at the individual level) about a brand or category topic (machine classifies millions of conversation snippets by emotion) + macro insights on each “Emotion Group”, based on offline purchase data AND all known interests of individuals in that group.

THE RESULT: When Heads of Insight/Brand Managers/C-Suite execs can zoom from macro insights about consumers INTO individual level dossiers making up an “Emotion Group” (a group of people expressing a specific emotion about a product/service), they will gain a clearer understanding of the intent and actions of these consumers. They will see when a consumer says, “I hate this product because…” or “I love this product because…”. They will see what characteristics entire groups of consumers who love or hate a specific product share.

THE IDEAL FINAL SCENARIO: Head of Insight to C-Suite execs, “I can see groups of consumers who are excited/disgusted with our product, the statement each consumer made about our product, and dossier-level insight into each individual making up these groups. I can then compare these groups with groups of consumers who are excited/disgusted with our competitor’s product in a nice neat single screen. Now I know why our brand equity is falling and I can make evidence-backed recommendations about our next move.”

The Advent of Machine Learning as related to Social Listening

The advent of #MachineLearning as related to social listening precipitates a lot of honest-to-God hard work during a transition from human-led to machine-led insights. This will happen because many humans will be needed to create custom attributes and custom classifiers for the machine to use in the new upcoming #AI driven insights industry. Humans teaching the machine to think like we do, to analyze like we do, to understand nuance, culture and the evolution of language (keep in mind, social tech experts, I am referring to a truly global machine analyzing the languages of the world…not just English).

Since there are many gaps in social data, we face an uphill in pairing other data sources to the snippets we get from public sources of conversations. The key to a complete “AI Social Data Intelligence Machine” will be sources such as Axciom, census data, background check sources, private networks like Doximity/A Small World and finally, if possible, the nexus of data from Facebook and Amazon. This particular nexus is a kind of holy grail that the leading machine intelligence experts will eventually open to brand leadership. But previous to such events, we have a grueling journey as analysts, insights researchers and programmers to “teach the machine how to be a human”.

One of the most exciting branches of this “teaching” will be a self-aware machine, an artificial mind that will begin to offer recommendations…and then may decide to act upon these recommendations.

Social Intelligence of The Future — Focus on shortening the distance between the consumer and the brand

These are questions that we can be answering using insights from social data.

What can social data tell us about our consumer’s actions? What is he/she doing on a daily and hourly basis?

What experiences are our consumers having?

When our consumer turns away from a major brand, what is he/she turning towards?

What is our consumer experimenting with? Can we experiment with him/her?

How can we help our consumer as he/she faces so many choices?

How can we help our consumer determine what sources of information are valid?

Are we discovered in the midst, even at the core, of our consumer’s trusted sources of information? (influencers)

What does it mean to be seen as a basic utility by our consumer?

What does it mean to be the answer to our consumer’s “short term” needs/decisions/desires?

How can we help our consumer with small actions on an every day basis?

How can we be more pragmatic vs ideological as a brand? How can we be the answer for our consumer’s pragmatic questions and needs?

How can we help our consumer as he/she evaluates and re-evaluates his/her decisions about the smallest things? Where can we appear during that consideration phase?

How can we be there when our consumer acts impulsively? Where and when does he/she act impulsively?

How does our consumer’s “operating system” work? How can we “hack” or “patch” into his/her operating system?

How can we create activity that fits within our customer’s existing behavior (based on lots of small data points)?