Chapter 4: Exploiting perceptual vulnerabilities

Before a human can reason about information, they first have to perceive it. With so much of our lives being online and on-screen, it is useful to consider how visual perception works.

Although it’s tempting to consider healthy human eyes as perfect high-definition cameras, they are actually very different.¹ In fact, much of what we visually perceive is fabricated by the perceptual systems in the brain, and our eyes provide highly incomplete information. For example, the human eye has a physical blind spot at the back, where the optic nerve connects the eyeball to the brain. People with normal vision do not see any blind spot, despite it being present at all times. It’s filled in by our visual cortex.² In simple terms, the human visual perception system guesses at what should be present and fabricates it. Or in other words, our brains are making it up as we go along.

In the same way, the middle of the retina contains sensors called cones that enable us to see in colour. Around the periphery of the retina we mainly have a different type of receptor called rods, which provide non-colour vision, and work better in low light. Yet we do not perceive any variation in the colour of what we see from the centre to the periphery of our field of vision. The human visual cortex does an enormous amount of ‘guesswork’ to fill in the gaps, making an inconsistent data source seem utterly full colour and high definition.

What’s more, when a person with normal vision looks at something, they usually perceive a steady, fixed scene. However, the human eyeball typically moves around a great deal. When we read something or scan our surroundings, our eyes rapidly flick from here to there and back again, taking in all kinds of pieces of information in addition to what we’re focused on. The fast movements are called saccades, and they last somewhere between 20 and 200 milliseconds. They’re interspersed by fixations, when the eye stops, briefly, for 50 to 600 milliseconds. Yet we don’t get motion sickness from it – we don’t notice it at all.

To summarise, what you ‘see’ as a human is not reality, but an internal representation of reality involving imperfect sensors (our eyes) and an enormous amount of internal processing that uses amazing guesswork to fill in the gaps. This means that the entire visual system is exploitable, making it easy to hide things. In other words, camouflage.

A famous example of camouflage in nature is the lime hawk-moth (Mimas tiliae),³ which has evolved to blend into its environment using colours and visual contrast to mimic its background and to create false edges that disrupt its shape, thereby avoiding visual detection by predators when positioned on a lime tree or similar vegetation.⁴

With apps and websites, exploitative designers frequently use similar techniques, by manipulating text colour contrast and size.

The interesting thing about colour contrast is that it is straightforward to calculate.⁵ You capture the hex codes for the text foreground and background colours, then enter the values into a calculator tool.⁶ There is an internationally recognised standard for minimum colour contrast: the W3C’s Web Content Accessibility Guidelines (WCAG 2.1). It has three levels. The middle level, ‘AA’, is widely recognised as the baseline to aim for.⁷ This means you can use a colour contrast calculator tool to instantly work out if a piece of text on a page meets the baseline recommended level for colour contrast.

One trick to watch out for is differences in text contrast on a page. If most of the text on a page is relatively high-contrast and one bit of text is relatively low-contrast, this could make readers less likely to notice or pay attention to the lower-contrast text, even if it is AA-level compliant. Readers often interpret colour contrast as a signal of what they should read versus what they can safely ignore (i.e. ‘this pale grey text can’t be very important – if it was, they’d have made it more prominent’).

One of the first cases I worked on as an expert witness was Arena vs Intuit Inc.⁸ In 2019, a law firm called Stueve Siegel Hanson approached me and asked me to review the account creation and sign-in process relating to Intuit’s TurboTax products. A screenshot of the sign-up page is shown below. See if you can spot any issues, based on what we’ve talked about in this section.

You might not realise it by looking at the screenshot in the figure above, but if you clicked the ‘Create Account’ button, you would be agreeing to binding arbitration. In other words, you would be unable to take Intuit to court. To find the information about arbitration, you are expected to notice and read the text below the big button (‘By clicking Create Account…’).

In my analysis I found that the colour contrast of that text was lower than most of the other text on the page, and the font size was smaller too. I can’t write too much since a good deal of my report is confidential, but the key point here is that the judge agreed with this analysis. To quote:

‘…both the notice and the hyperlinks therein are in the lightest font on the entire sign-in screen […] The Court finds that a reasonable consumer would be less likely to notice text in a significantly fainter font than other text on the same page.’

—Charles R Breyer, United States District Judge, 12 March 2020

During the case, Intuit were required to disclose their analytics data, which showed that less than 0.55% of users actually clicked on the relevant hyperlink during a four-month period in 2019.⁹ This is compelling evidence, and also a reminder that internal company data or documents can end up being revealed in a lawsuit, and it’s sometimes the job of an expert witness to suggest what to ask for.

To summarise, small low-contrast text is an effective way to hide content on a page, and to prevent users from subsequently comprehending it and making an informed decision. Like many other exploitative strategies, manipulation of perception may be illegal in some jurisdictions depending on the way you use it.¹⁰

Another more brazen approach to exploiting perception is to remove something entirely from the user’s perceptual field. If you don’t want people to comprehend something, you just don’t show it on the page, and you put it behind links or buttons that allude to something else. This is a very common practice in cookie consent dialogs – where the first thing the user sees gives no clue that there may be a button somewhere that allows them to reject all forms of tracking. In 2020, Nouwens et al. carried out a research study to measure the impact of this.¹¹ Forty participants took part in an online field experiment. Results showed that removing the ‘Reject all’ button from the first page of a consent pop-up increased consent by up to 23 percentage points.

In another study, the UK government’s Behavioural Insights Team (BIT, aka ‘the nudge unit’) worked with an Australian government department to improve payment rates for fines, debts and taxes.¹² They sent two different letters to 48,445 people. In one letter, they featured a large red ‘pay now’ stamp on the letter, shown below. The other letter didn’t have this stamp.¹³

They found that the letter with the ‘pay now’ stamp delivered a 3.1 percentage point increase (14.7% payment rate without the ‘pay now’ stamp; 17.8% payment rate with it).¹⁴ We can look at that figure the other way around: the letter without the ‘pay now’ stamp delivered far fewer payments. So the simple act of removing a call to action is very effective. If someone doesn’t perceive something – such as the need to act, or a reason to do something – they are less likely to think about it. If they don’t think about it, it won’t influence their decision-making.

There are other ways to manipulate perception that I should mention before we move on to the next type of strategy. Most commonly, exploitative designers employ clutter and noise – subverting common graphic design principles like white space, repetition, alignment and proximity. (If you’re not familiar with these principles, any introductory textbook can provide you with a beginners guide.¹⁵) This serves to create a sort of smokescreen, making elements of the page harder to see, read and scan: playing into the exploitation of expectations strategy and the exploitation of vulnerabilities in comprehension strategy.