Communication Multimodality

Ray Birdwhistell was an anthropologist who founded the field of kinesics, the study of body movements. In his influential Kinesics and Context, he argues that human communication is more than just the verbal aspect of it:

We cannot investigate communication by isolating and measuring one channel, the acoustic. Communication, upon investigation, appears to be a system which makes use of the channels of all of the sensory modalities. By this model, communication is a continuous process utilizing the various channels and the combinations of them as appropriate to the particular situation. If we think of Channel 1 as being the audio-acoustic (vocal) channel, Channel 2 as the kinesthetic-visual channel, Channel 3 would be the odor-producing-olfactory channel, Channel 4 would be the tactile and so on. Thus, while no single channel is in constant use, one or more channels are always in operation. Communication is the term which I apply to this continuous process.

communication channels.PNG

The channels act to provide redundancy, to reinforce each other, and to provide additional context and depth:

Redundancy makes the contents of messages available to a greater portion of the population than would be possible if only one modality were utilized to teach, learn, store, transmit, or structure experience. Multichannel reinforcement makes it possible for a far wider range within the population to become part of and to contribute to the conventional understandings of the community than if we were a species with only a single-channel lexical storehouse.

Birdwhistell developed a system of kinegraphs to annotate body movements captured in the interviews he had filmed.


Transcribed interviews would look like this:


Insights: Understanding How Things Work

There are many definitions of what an insight is, and a few useful ones. An insight can be a novel revelation that leads to unexpected ideas. An insight can also be a perception of a unique opportunity that exists in the marketplace at a particular time. Finally, an insight is an understanding of how things work — the causal rules that govern a particular system.

We need to understand these causal rules in order to make tools that are effective within the system:

Scientists have noted that animals that exhibit higher intelligence have two notable descriptive differences. First, these animals are capable of using tools. Second, they tend to live in complex social structures.

One simple model for the process of intelligence in the creation and use of tools is illustrated in the upper row of boxes in [the chart]. Here, the first step is to make sense of reality. By making sense of reality, simple causal relationships can be understood. This understanding of causality, in turn, allows for prediction and finally the use of tools.

The chain, while simple, is extremely powerful. It has led to ravens using simple tools, to early man creating stone weapons and wheels, and eventually to modern man creating the Large Hadron Collider.

The Sense-Making Process and Its Side Effects, from  Systems Thinking In Business  by Rich Jolly

The Sense-Making Process and Its Side Effects, from Systems Thinking In Business by Rich Jolly

Long-Term Memory Systems

“Memory is not a single entity but consists of several separate entities that depend on different brain systems. The key distinction is between the capacity for conscious recollection of facts and events (declarative memory) and a heterogeneous collection of nonconscious learning capacities (nondeclarative memory) that are expressed through performance and that do not afford access to any conscious memory content.” (Squire and Zola, 1996)

“A taxonomy of long-term memory systems together with specific brain structures involved in each system.”

“A taxonomy of long-term memory systems together with specific brain structures involved in each system.”

What Should We Make?


When you can make almost anything, how do you know what to make?

A Havas study found that “84% of people expect brands to produce content. Yet 60% said the content brands currently create is poor, irrelevant, or fails to deliver”.

How do you do better that that?

Start with clearing the hurdles of Possible, Desirable, and Effective.

Possible. Can we do it? Do we have the right expertise, the organizational will and resources, is what we want to do within the limits of what's on-brand, and is it legal?

Desirable. Do people want it? Will it make their day a little better? Will it make them feel anything? Is it about them, too, or is it only about you?

Effective. Not everything you can make will work equally hard to further your business goals. What will people remember about your brand and for how long will the memory last?

Yes, Surveys Can Predict Behavior

Surveys deliver valid and useful information if they are done well and for the right reasons. Specifically, surveys are a good instrument for predicting near-term consumer preferences, although the accuracy degrades as the forecast horizon expands.

Every once in a while, you’ll see someone in the avant garde marketing circles bashing surveys with antivaxxer enthusiasm. For example, Philip Graves, "a consumer behaviour consultant, author and speaker,"  takes a dim view of market research surveys in his 2013 book Consumerology. Graves writes that "attempts to use market research as a forecasting tool are notoriously unreliable, and yet the practice continues."

Read More

I Miss My Old Media

I miss all the news that fit to print -- not all the news, and pseudo-news, and churnalism, and press releases published verbatim, and gossip, and updates to gossip, and galleries, and listicles  that drive  just one more page view.

I miss editors who say no.

I miss reading Playboy -- or anything -- for the articles.

I miss cutting things out to save them for later.

I miss ads that sell hard from a full spread and feel good about it;  ads that don't stalk you, and nag you, and creep you out.

I miss hearing from my friends once a year and spending all night catching up and telling them how much their kids have grown since I'd last seen them because the last I'd seen them was a year ago.

I miss organically yellowed pictures.

I miss the way old film cameras used to smell.

I miss having just one TV remote on my couch.

I miss turning the dial on my radio, and hearing crackling, and static, and then catching a faint song that sounds like it's played thousands of miles away because it is.

I miss songs on the radio being selected by a human and not a playlist algorithm.

I miss knobs, and buttons, and dials, and switches.

I miss running to my mailbox and finding a handwritten letter.

Polling In The Dark


How do you survey people in a dark movie theater?

CinemaScore conducts exit polls in theaters by asking movie goers to pull back tabs on the ballot, the design of which has remained mostly the same over the past 35 years.

CinemaScore tabulates the results and reports each movie's letter grade. Only 19 movies in the company's history got an F. The score is not a simple average:

CinemaScore has an algorithm,” [founder and president Ed] Mintz explains. “A long time ago, we tweaked and analyzed until we came up with what we thought to be the absolute right system. Obviously I can’t share that. That’s the McDonald’s secret sauce,” he laughs. “But if you have 100 ballots, even if you divided it evenly, and had 20 As, 20 Bs, 20 Cs, 20 Ds, 20 Fs — in school, that’s a C. In our curve, it’s a lot worse; a B in school is more equivalent to a C in our terms. When you start getting Bs with CinemaScore, it affects the algorithm and curve a lot harder than it does in school. If you have 20 percent Cs, 20 percent Ds, 20 percent Fs — imagine how bad that is.” (

The results are used to estimate word of mouth and multiples— the overall gross in relation to the opening weekend.


The Effect of Incomplete Logos on Brand Perception

incomplete logos.png

A study has found that companies with typographic logos that are intentionally missing or blanked out parts of the characters (think IBM) are perceived as less trustworthy but more innovative.  "The former influence is tied to the logo's perceived clarity, while the latter influence is tied to its perceived interestingness."

Consumers with a prevention focus (?) have an overall unfavorable attitude towards the firms with incomplete logos.

What Does Your Typeface Taste Like?

In "The Taste of Typeface" paper: 

"Participants matched rounder typefaces with the word “sweet,” while matching more angular typefaces with the taste words “bitter,” “salty,” and “sour.”

Why would people match tastes and typefaces varying in their roundness and angularity? The more that an individual likes a taste, the more they will choose a round shape to match it to, and the less they like it, the more they will tend to associate the taste with an angular shape instead." 

Logos of Powerful Brands Should Be Placed Higher

Consumers prefer brands with a high standing and influence in the marketplace more when the logo is featured high on the packaging rather than low (source).

They prefer less powerful brands more when the brand logo is featured low rather than high.  "The underlying mechanism for this shift in preference is a fluency effect (?) derived from consumers intuitively linking the concept of power with height."

What about "fake it till you make it?"

"There is the possibility that managers may choose to place their logo high to signal power even when such a strategy does not match their brand’s true category standing. Although valid, the current research suggests that when category standing is known by the consumer, this strategy may not work."


When to Test Your Prototype

Rough drafts are the staple of the creative business. Art directors sketch and storyboard. Interaction designers mock and prototype.  

How soon should you put the half-baked idea in front of people?

This IDEO designer says the sooner the better: “Instead of starting with our usual process where we gather inspiration, we thought: What if we share a prototype we are really excited about with the client right away?”

On the other hand, Alan Cooper cautions in a Twitter thread:

“Prototyping and testing is not interaction design. It helps, but it isn’t user centered. It’s designer centered. Prototyping & testing has one huge virtue: it makes management happy because it never rocks the boat. It never requires big changes. It always keeps the designer in command. It’s very ego gratifying.

When you put an artifact in front of a user, you instantly shut down an infinity of good ideas, avenues of thought, opportunities to create.” 

The first prototype you show becomes the reference point and sets the direction for everything that follows.


30 Reasons Why People Use Phones in the Store

When designing a mobile strategy for your store shoppers, consider that there is not one but at least 30 different reasons why people are on their phones. 



1. Find a store that carries the merchandise they need using search and navigation apps
2. Go to the store to check out the merchandise they are planning to buy online (aka showrooming)
3. Post their impressions, pictures and videos of the merchandise while shopping
4. Alert others about the rare or discounted merchandise they found
5. Bring in detailed product information to the store instead of having their shopping process guided by the sales staff (in auto dealerships, for example)


6. Find their way to the store using a GPS app
7. Take a picture of the car to remember later where it’s parked
8. Use the phone as a shopping list
9. Receive updates for their shopping list from someone else (a friend, a spouse)
10. Arrange their shopping list in a way that minimizes their shopping time (and any accidental exposure to unplanned merchandise)
11. Use their smartphone to entertain young children to shop in peace
12. Use the smartphone to kill time while waiting for someone else to finish shopping
13. Receive offers from competing business while shopping at a store or navigating at a mall
14. "Bookmark" merchandise by taking pictures for future reference
15. Monitor things back home through a video feed and shop in peace
16. Check for important work emails, which before would have required being physically present 


17. Use their smartphone’s front camera as a mirror (when trying on glasses, for example)
18. Send a picture of the item to someone else for confirmation of correct selection (are these the tampons you need, honey?), or for approval (you look great in those)
19. Look up ratings and reviews for an item they are considering
20. Look up dimensions and other specifications for an item
21. Look up post-purchase information:  additional costs (insurance or add-ons, for example), as well as usage, care, or assembly instructions
22. Take and send pictures of the merchandise  they are considering to friends and receive their friends’ feedback
23. Look at pictures taken earlier for reference (pictures of a room, for example, when selecting a carpet)
24. Compare the item's price against the store’s own prices online. Compare the price against other online and  brick-and-mortar stores
25. Look up price change history for the item
26. Look for online coupons for the store they are already in
27. Like something, then look up its availability at competitors or at more convenient locations


27. Carry loyalty cards with them at all times on their phone
28. Look at the phone during the check-out process and not at the shelves, driving the sales of the impulse-purchase merchandise down
29. Abandon the purchase and buy the same item online in order to save money or to avoid standing in line or having to carry it
30. Pay with the phone


Where are things going in the next 10 years? Will phones detect our our subconscious reactions to the merchandise and make suggestions?  Will voice assistants sternly suggest we pay a lower price elsewhere?  Will they prevent us from overindulging and buying two tubs of ice-cream instead of one? Will we see floating AR-enabled arrows pointing towards the shelves that have the items from our shopping list?   

How Likely Is Probably?

When you are asking research participants to estimate how likely something is to happen,  there are several pitfalls to watch out for.  One of them is the lack of consensus about the meaning of words.

From Psychology of Intelligence Analysis published by the CIA:

“The table [above] shows the results of an experiment with 23 NATO military officers accustomed to reading intelligence reports. They were given a number of sentences such as: “It is highly unlikely that...”

All the sentences were the same except that the verbal expressions of probability changed. The officers were asked what percentage probability they would attribute to each statement if they read it in an intelligence report. Each dot in the table represents one officer’s probability assignment. While there was broad consensus about the meaning of “better than even,” there was a wide disparity in interpretation of other probability expressions.



Inspired by Sherman's experiment, a Reddit user u/zonination polled fellow redditors with a similar questionnaire, asking them to assign numerical probabilities to different words, and mapped the results.

When you ask someone how likely they are to purchase your product and they answer "unlikely", it could mean a less-than-5% chance, or a greater-than-35% chance. For someone else, "probably" could mean less than even odds.

If you are asking about purchase intent in a survey, be careful how you present answer options, since the probability implied in their order could conflict with what people usually understand each word to mean.  If "probably", on average, means a higher probability than "likely", it could be confusing for respondents to see the options in this order:

  • Definitely

  • Likely

  • Probably

  • About even

  • ...


Sherman Kent, the original experiment's author, talks about what sparked his interest in how intelligence professionals communicate different probabilities:

“A few days after the estimate [of Soviet actions in Yugoslavia] appeared, I was in informal conversation with the Policy Planning Staff's chairman. We spoke of Yugoslavia and the estimate. Suddenly he said, "By the way, what did you people mean by the expression `serious possibility'? What kind of odds did you have in mind?" I told him that my personal estimate was on the dark side, namely, that the odds were around 65 to 35 in favor of an attack. He was somewhat jolted by this; he and his colleagues had read "serious possibility" to mean odds very considerably lower.”


Your author, skating away from the puck.

Your author, skating away from the puck.

Blogging is like vinyl — it just sounds better. 

Between 2004 and 2011, I edited the Advertising Lab  blog about technology and its future in advertising.

Eventually, writing about tomorrow began feeling like a way to avoid having to deal with today.  If everyone skates to where the puck is going, who will be left minding the puck now?  

This new journal is about today.