market intelligence

Audience Intelligence Formula – Late 2017

SUMMARY: The formula for best-practice audience intelligence work is dynamic, due to rapidly advancing technologies and practices. We also have to take into consideration federal and nation-state regulations which are constantly in flux. As of December 2017, this is a formula for producing a fully enriched audience based on the beginning point of email, social handle, or a conversation snippet from social media, blogs, forums or comment threads on news sites. I might add that the acceleration of AI (Artificial Intelligence) and ML (Machine Learning) is rapidly condensing the steps below. APIs (Application Programming Interface) are being woven together to produce highly sophisticated machinery that blends outputs from various data sources.



STEP 1: Interview the client to determine goals and deliverables. This is important because we want to understand the client’s business and his specific goals related to the research. This is the time to see eye-to-eye with the client and really get to know his/her business from the inside-out.


STEP 2: Perform tests using the software below to discover opportunities related to the client’s desires/needs. Most social monitoring/aud intel solutions provide a way to quickly preview “the universe” related to a specific research request. This step is important as part of putting together the proposal for the client.


STEP 3: Design the audience intelligence study. Gain approval from the client, sign contracts, receive funding from the client. This is important because the client will want to look over the exact type of deliverable he/she will be receiving, as well as hearing a bit about your approach in putting together the final “data lake” of prospects and, of course, how the insights will be presented.


STEP 4: HOW DO PEOPLE DESCRIBE THEMSELVES IN SOCIAL BIOS: We gather a set of social handles where our keywords are in the bio. This from, Affinio, Brandwatch and PeoplePattern primarily, but can also be from using in LinkedIn, along with many other sources herein un-named. This is important because self-description by individuals is rather verbose these days..and this is helpful to us in finding our targets. And, where self-description is not verbose, there are clues through company/press/3rd-party descriptions of specific individuals who work for/play at/eat at/drink at/shop at/you-name-it at the locations/venues/places we are studying.


STEP 5: WHAT DO PEOPLE TALK ABOUT IN SOCIAL MEDIA/BLOGS/FORUMS: We gather a set of conversation snippets related to our keywords from Brandwatch, Sysomos, Crimson Hexagon and/or Meltwater. This is important because what someone says about an activity/product/service/location/you-name-it contributes towards our understanding of the consumer/customer/competitor’s consumer/customer. In addition, these conversation snippets from social media, forums, blogs, comment threads on news sites form a body of data that we can segment into specific topic groups. These specific topic groups can then be used to form a point of view/set of insights on the target we are studying.


STEP 6: SEGMENTING STEP BY MACHINE – TOPIC MODELING APPLIED TO BIOS & CONVERSATION SNIPPETS: We use Converseon’s Conversus tool to perform topic modeling and separate bios and conversation snippets into discreet topics. This is important because this speeds up the analysis of the bios and separates out the bios that matter to us the most. No solution in the world is more accurate and complete in segmenting bios and conversation snippets into discrete topic groups than Converseon’s Conversus. This solution is pure magic and the analysts at Converseon who are using Conversus are second-to-none in their expertise at building out superior insights based upon the use of their in-house solution. It is during this stage where an analyst begins to really gain deep insight into a sets of bios and sets of conversation snippets.


STEP 7: SEGMENTING STEP BY ANALYST:  Now, these analysts wade into the output from Converseon’s Conversus and identify the Topic where our targets are present (and any other discovered target – unknown unknowns). This vital step by humans helps us know which Topic groups in Conversus are populated by our target. This is important work that can be recursive, whereby the analyst segments the data using Conversus, reads through the results, and then segments again to refine even more deeply. I might add that this particular step is where the machine will eventually outstrip the analyst. That outstripping of the analyst will take some time to come along, though. For now, on this step, the human continues to be the last mile.


STEP 8: RE-STITCHING AFTER SEGMENTING STEPS: Reunify the bios from the “right” Topics in Conversus with the handles in the original source data sheet. This is important because we want to have the correct bio next the correct social handle. We also stitch handles and bios next to discovered conversation snippets at this stage. We find that working in Jupyter notebooks using Python is one of the handiest ways for our teams to work together efficiently on this step.


STEP 9: PEOLE PATTERN AUDIENCE INSIGHTS STEP: Upload Twitter AND/OR Instagram handles into PeoplePattern for deeper enrichment of Interests, Location, Age, Persona. This is important because we find out more about each person and we also delineate between Individuals and Organizations. In addition, we move a step closer through PeoplePattern to verifying the “real people”. Finally, we gain insight into the Persona types, Interest groups and lots of other useful info.


STEP 10: FULL CONTACT STEP: Use the Full Contact API to append add’l social handles. The value in this particular step is discovering a full name related to the handle AND additional social handles. We also gain bios from various social media handles, thus bulking out our story about an individual.


STEP 11: PIPL STEP: We use the PIPL API to gather Email, Phone, Address, other social handles, age, and many other bits of info on individuals. This is important because we will gain additional important information about the individuals that verifies they are “real people”.


STEP 12: CRYSTAL STEP: We use the Crystal Knows API to enrich the profiles with DISC personality type, personality overview, messaging guidance, selling guidance. This is important because then we are guiding our client on how best to market, advertise and sell to this individual. We can then group individuals by DISC type, if desired.


STEP 13: CLEAN UP ROWS FOR COMPLETION: Stitch together the results from the various APIs and then filter for complete rows. Again, we use Jupyter notebooks and Python for this work, as the teamwork and efficiency is vastly improved. This stage is important because we want every row to have every cell filled with correct & complete data.


STEP 14: EXACT DATA STEP: If we want to add an extra step for verification at this point, we run the Exact Data enrichment on the emails/names discovered. This is important because this extra step adds validity to the claim that our audience is full of “real people”.


STEP 15: SPOTRIGHT STEP: Upload the social handles into Spotright to gain enrichment of offline Axciom data, such as buying/purchasing styles, net worth, income level, political/religious affiliations, housing info, household complexion info, brand preferences of a specific group uploaded and much much more.


STEP 16: INSIGHTS, METHODOLOGIES & RECOMMENDATIONS PDF: Create 3-5 page Summary PDF with insights related to the research. This is important because brand leadership will now have a set of insights about the discovered individuals (our initial data lake of prospects) and, importantly, we can recommend further research steps for successive work together.


How Analysts Deliver Meaningful Actions to Stakeholders

The following post was inspired by a recent day of meetings at a major FMCG (Fast-Moving Consumer Goods) retailer in the United States. The post focuses on the use of market-related insights to deliver action-inducing stories to brand leadership and technical staff within the corporation.

THE “DEEP” ANALYST – A DEFINITION: A “deep” analyst chooses a psychological theme for analyzing a massive amount of data. For example, a “deep” analyst could apply an understanding of Jungian psychology to his/her analysis, focusing on classical Jungian therapeutic definitions as a guide for data segmentation. Such an analyst would seek to answer classical Jungian questions about an individual or group through his/her work with the data — for example, what “shadow” or unconscious elements of our corporation now animate our workforce? A corporate leader would want to know about these “hidden” animators of daily workforce behavior to better align manager actions and gain more precise results from staff.

(Summary: A researcher can study customer behavior and then tell corporate leadership what is secretly controlling staff and consumer actions.)

THE VALUE OF THE “DEEP” APPROACH: A significant value in the “deep” psychological approach to data is a wealth of short, yet powerful, anecdotes derived from research and “strung” along the path of a classical inner journey. When applied to a corporation, such stories offer tremendous value to stakeholders in making better decisions about a host of internal and external issues. As an example, what would happen if a major corporation studied Joseph Campbell’s Hero’s Journey, discovered where on that journey specific business units currently “walked”, and was better able to make decisions based upon this metaphor?

(Summary: A researcher can study customer behavior and then tell stories to corporate leadership as a means to inspiring specific helpful actions.)

THE ACTIONS OF A “DEEP” ANALYST WITHIN THE CORPORATION: The first action such a “deep” analyst takes within a corporate entity is to reveal the archetype at the brand core: that force in the unconscious of the company that is animating its stakeholders. The second action is to reveal who is consciously engaged with that archetype. The third is to influence, change and enhance the magnetism of the brand’s core so that customers and stakeholders are better served by a more inspired brand leadership.

(Summary: A researcher is curious about the main story of a brand or company. He/she is also interested in who else is aware of this story and what they do about this story on a regular basis. Finally, the researcher takes action to grow awareness of this story so that more people in a corporation will be helped/inspired and do a better job.)

RESULT #1 – INSPIRATION FOR TEAM LEADERS FROM INSIGHTS: Leadership will take insights gleaned by a “deep” analyst and engage more effectively with the core of the brand. A corporate leader must be steeped in the living and numinous entity that pulses at the center of the brand. Such devotion by the leader inspires stakeholders, merchant partners, employees and customers. You can tell when a corporate leader has become fully possessed by his/her brand…just take a look at Branson, Hsieh and Bezos. These are beings who enter regularly into the “fire” that burns at the core of their respective brands, emerging with powerful inspiration, drive and leadership for the entire corporate entity. The right set of stories derived through insights will ideally lift the “uninspired” leader to a new level of excitement about his/her business unit. In such cases, a single metaphor derived from insight can give birth to a greater level of passion in the leader and lift his/her staff up to a higher level of performance as a result.

RESULT #2 – WHAT A “DEEP” INSIGHT REVEALS: A deep insight reveals what animates a corporate entity, identifies the lenses within leadership upon company events and is a story of consequences (both “good” and “bad”). A deep insight inspires a bevy of possible actions and conceives strategies that will best serve current initiatives, stakeholders and customers. A deep insight will chart a path for brand leadership in winning on all levels. A deep insight contains the voices of a Critic AND a Creative, doing their duet. In this song, the Critic refines the Creative’s gift and the Creative ends up delivering happiness to the Critic: a win-win. Key: A deep insight provides a host of windows for times when doors seem to be closing.

THE FMCG EXAMPLE – HOW INSIGHTS BECOME PRACTICAL AND ACTION-INDUCING: Let us take, for instance, a leading FMCG entity with one foot in “flesh-retail” and one foot in the digital realm. This entity has an opportunity to weave flesh and digital with such elegance, such precision, that a customer hardly has to think when being served by one or the other. Deep insights will deliver tech so deeply embedded into the customer experience that the tech disappears…the customer makes his/her use of the tech a daily, even hourly, action.

Let us say, for instance, that this FMCG entity has created an app that delivers daily personalized discounts based on the customer’s past purchases, current financial “reality” and publicly expressed wishes. What a win for everyone! The customer wins because every time he/she steps into the retail or digital outlet, a discount on his/her faves is given (plus a whole lot more, in the ideal scenario). The resulting love affair influences the customer’s friends to participate and the merchants serving this FMCG are glowing with happiness as sales go up. The resulting community is a truly potent entity, able to extend seemingly un-ending generosity within itself and to its greater community. Everyone wants a piece of the action in such a scenario.

SUMMARY: An analyst focused on “deep” insights will evangelize the power of such intelligence throughout the corporate entity, throughout the brand organism. Such an analyst will educate every silo on how to use insights from intelligence for daily wins: within internal focus groups, (solving previously “un-solveable” issues), within customer focus groups, (solving those daily headaches that slow down the purchase cycle), within competitive situations, (revealing collaborative possibilities). A deep insight is the “A-ha!”, the “Eureka!”…and with the depth of data currently available, the market intelligence analyst working within a corporate entity has abundant and daily opportunities to make this exclamation and deliver powerful actions to every silo, every leader, every employee and, ultimately, to every customer.

Social Business Intelligence Advance #4

The business intelligence solution that marketers want is the following: a cloud-based service that is comprehensively analyzing in real-time networks of regular customers AND matching these customers (and their friends) to upcoming deals specific to past purchases. For instance, if Amazon were to fully integrate its data with Facebook’s data, a profound level of matching would become possible. Senior leadership within major FMCG and retail entities should be actively cultivating technical vendors and in-house technicians to achieve this type of customer intelligence solution.

Research leads to freedom

FREEDOM: Many individuals want to break free from corporate walls, familial boundaries and cultural structures. This, in fact, is the story of progress and also of experiencing depth. There are many systems of thought that offer paths to freedom. If one considers the interior of his/her psyche as akin to a corporate structure, then we can go further with a few metaphors that may be of help in finding freedom. Before getting into this though, it is vital to know that freedom comes from taking 100% responsibility for one’s own actions within a given context. This singular truth is the key to freedom. The intelligence gathering process for business and government provides leadership with clarity, security and power. And the best kind of power emerges from individuals who have taken 100% responsibility for the process of discovery and subsequent strategy.

INTELLIGENCE IS MAINSTREAM NOW: For our current article, we will take the market intelligence community and its function as a means to discovering freedom via responsibility. Market intelligence used to be a rarified action, not frequently discussed amongst co-workers on your average coffee break. In fact, in many traditional settings this is still true. But there are many forward-thinking corporations, communities and individuals who understand the power of deep intelligence on any given topic or interest. The market intelligence (MI) process has actually become more widespread in the corporate world due to the advent of social media monitoring tools like Radian6, BrandWatch, Sysomos and PeopleBrowsr. Even very basic Twitter, Facebook, Pinterest and Google searches provide the average man and woman a point of view previously unknown. For example, one may type keywords related to a specific interest into the search field at a social network and discover people nearby who share that interest. A new “in-the-flesh” experience becomes possible through such location-based searches related to an interest.

THE FIREFIGHTER AND FUTURIST: The advent of readily available intelligence brings humanity as a whole to an entirely new level of responsibility. Answers to most questions are at everyone’s fingertips. And this experience of responsibility translates into freedom for the corporation, the community and the individual. The process of going from a passive “bored at work” search for one’s passion to standing up from the chair and diving into a completely new life may be described as going from the “Firefighter” to the “Futurist”. To explain: the Firefighter is approaching an interest on an ad hoc basis, reacting to outside stimulus and “sitting at the station” waiting for an emergency. The Futurist is deeply engaged on both a global and local level, using intelligence to drive a process of rewarding, pro-active discovery. The Firefighter has no focus other than the emergency he reacts to…the Futurist is immersed in a broad, deep and future-oriented journey that draws in topics, people and places outside the immediately relevant environment. To be clear yet again on this comparison: the Futurist is creating the reality that the Firefighter often reacts to.

The Futurist has realized that reality may be defined ahead of the curve of fate or karma. While the consequences of an action take place within the boundaries of a specific world, the Futurist has moved on and discovered a completely novel experience, devoid of the content and context wherein the previous consequence was generated. He/she may then transform “the past” via valuable learnings, cultural understandings, and innovative approaches discovered “in the future”. A leading solution helping Futurists do this now is Recorded Future.

RESEARCH LEADS TO FREEDOM: The deepest importance of Market Intelligence has to do with creativity. When one discovers the complexion of an eco-system surrounding a specific interest, then he/she may tweak the elements of that eco-system and create what he/she wants. This is why researchers and business leaders must work hand-in-hand to shape our world. And this is also why it is inexcusable for any individual to say he/she cannot find a path to a dream. Even the simplest location-based search around an interest will place one in the company of others who have thrown themselves with passion into achieving very similar dreams to one’s own. Try this today, using the search fields in the various major social networks. Go deeper with social business intelligence tools.

A brief description of Social Business Intelligence

WHAT IS SOCIAL BUSINESS INTELLIGENCE: The basic concept behind the term “social intelligence” is to derive customer, competitive and market intelligence via data scanned in social networks, such as Twitter, Facebook and LinkedIn.

THE NET RESULT OF A SOCIAL BUSINESS INTELLIGENCE PROJECT (WHAT YOU GET FROM A S.I. PROJECT): The value of a social intelligence project is manifold, including the following benefits:

• discovery of warm leads related to psychographics and demographics from a customer’s current database.

• discovery of current trends and developments in a market sector that could lead to product/service innovation.

• discovery of a competitor’s activities that may aid one’s sales efforts.

• discovery of what your customers talk about, leading to product/service innovation and changes in marketing/sales strategy/tactics.

• discovery of new regions where your product/service is being discussed and your competitors are making money.

• discovery of upcoming events where you could generate awareness and sales.

• discovery of new pools of customers in digital networks. You may not have been aware of these pools of customers and their interests.

• discovery of conversations that social marketers and community managers can enter and utilize for higher brand awareness and sales.

• discovery of vendors and employees via professional networks like LinkedIn or the European Women’s Professional Network.


WHO: Gathering social intelligence is best performed by someone trained in research. Although a great amount of data may be aggregated automatically during a social intelligence project, it takes a human being to segment and make sense of this data and create insights related to the data.

HOW: There are many tools used to gather social data. Some of the best are expensive and require additional training to master. One of the best ways for a novice to begin is to use Twitter and LinkedIn. After setting up accounts in both social properties, use the free tool Listorious to research lists of influencers surrounding specific keywords in Twitter. Be sure to sign up for an Executive Account in LinkedIn so you can use all of the features in the Advanced Search tool.

WHICH TOOLS: More advanced social intelligence tools include Radian6, Sysomos, BrandWatch, PeopleBrowsr, Crimson Hexagon, Recorded Future and Trackur. A comprehensive list of these and other social intelligence tools may be found here.