At times, creative and data are seen as being at odds. Some designers view data as a potential barrier to their creativity and feel their designs shouldn’t be judged solely by data. Three years ago a flashpoint in the “design vs. data” debate happened when the top visual designer at Google, Douglas Bowman, left the company after being frustrated with its oppressive, data-driven approach. On his personal blog, he shared his perception that data eventually had become “a crutch for every decision, paralyzing the company and preventing it from making any daring design decisions . . . I won’t miss a design philosophy that lives or dies strictly by the sword of data.”

It was interesting to see the reaction in the design community to the whole data-driven design debate. As designers Lukas Mathis and Tom Chi noted, a key difference between art and design is constraints. Whereas artists have relative freedom to do what they want, designers must create a solution that works within certain parameters. An unknown source stated, “Design is the art of gradually applying constraints until only one solution remains.” There’s a false notion that data kills creativity. I believe creativity actually flourishes with restrictions. One of these key constraints is the performance of your design. For example, does your design help more people to accomplish their online tasks? Does it drive more online conversions for your business? If the effectiveness of your design matters, then measurement and data are essential.

Design is not just what it looks like and feels like. Design is how it works.
Steve Jobs, Co-founder of Apple

Author and designer Luke Stevens blamed Google’s broken decision-making culture for Bowman’s challenges and disagreed with Bowman’s position. He felt web design should live or die by the sword of data, and that designers shouldn’t view the data as some abstract thing but “a person like you or me trying to do something, and succeeding or failing.” He went on to say, “For perhaps the first time in history we can accurately measure all interactions with a piece of design. If it can be measured, it can be improved. And each of those improvements represents helping someone do something a little more successfully.” If more web designers fully appreciated what online data represented (i.e., attributes, interests, and behaviors of real people), then I reckon more would to seek to wield the sword of data in their craft.

Different Degrees of Data Adoption

Some people might take the position that you just need a healthy balance between using your intuition/gut and focusing on data. It’s wrong to think of intuition and data usage as being on opposite ends of some continuum, and it’s all about finding equilibrium between the two points. It’s more about understanding how attuned your intuition is to what really works. Sometimes you’re not going to have data to lean on. In those cases you’ll need trust your experience and instincts to put the best foot forward and then re-calibrate as needed once you have data. While I do think there are different levels of data adoption, it’s definitely not about finding the middle ground between intuition and data.

I’ve found that the term “data-driven” is a loaded term for some people, and I don’t want semantics to stand in the way of changing opinions and behaviors in this area. By data-driven I mean data is being actively consulted for information and insights that can shape and inform a design. Some people prefer using “data-informed,” but I believe it really represents a weaker, more passive stance on data. On the surface, data-informed and data-driven individuals act the same way when the data supports their intuition. However, what happens when the data goes against your intuition? That’s where the two approaches diverge.

If you’re data-informed, you’ll always side with your intuition over whatever the data tells you. You’ll only use the data if it’s useful in advancing your position or supporting your decision. If it strays from your intuition, then it is no longer useful and discarded. However, if you’re data-driven, you’re more likely to trust the data even when your intuition disagrees. You view these situations as an opportunity to calibrate your intuition and learn. Some people misinterpret data-driven as “data-controlled” where humans lose all control of decision making. Intuition and data should operate as checks and balances. I also believe you can be data-driven and still decide to follow your intuition from time to time. However, in these intuition-over-data situations, you’re fully aware of the potential consequences and prepared to closely monitor the results of your design decision and re-calibrate as needed.

There’s another key mindset difference between data-informed and data-driven. If you’re data-informed then you view data as nice to know; whereas, if you’re data-driven, you view data as need to know—it’s essential to your role. It’s a subtle but important difference. In addition, being data-driven requires effort. A great deal of discipline and diligence goes into seeking out data throughout the design process. It really bothers you when you don’t have the data you need. If you don’t like the term “data-driven,” you might prefer Joshua Porter’s concept of metrics-driven design. Regardless of your semantic preference, consider how using data in your web design can have a positive effect on you, your organization/client, and its online visitors.

Qualitative and Quantitative Data

If you’re interested in data-driven web design, there are two main types of data at your disposal: qualitative and quantitative data. Qualitative data focuses on people’s expressions of their opinions, feelings, and behaviors. When analyzing qualitative data, you’re interpreting the meaning of the words that were shared. Common forms of qualitative data include usability testing, focus groups, and open-ended feedback from online surveys. In contrast, quantitative data focuses on numbers or statistics that are collected through web analytics, A/B testing, or customer satisfaction scores. Many web designers are more comfortable with qualitative data than they are with quantitative data. This needs to change, especially if you’re using neither type.

Each form of data has inherent advantages and disadvantages. Qualitative data can answer the elusive “why” and provide valuable context. Quantitative data helps to clarify who, what, when, where, and how. On the negative side, qualitative data is highly subjective and limited in scope to small samples (e.g., three people in the focus group said they hate the blue color so we can’t do it in blue). Quantitative data lacks contextual detail (e.g., I know most visitors are abandoning on a specific step in the process, but I don’t know why). Whenever possible it’s beneficial to leverage both data types as they can complement each other. For digital initiatives, which are highly measurable, you can access rich levels of quantitative data that can shape and inform your designs. If you’re only using qualitative data, you’re missing out on the vast majority of the available insights that can be gleaned from online data.

Here are some examples of the types of quantitative data and reports you can leverage in your web design:

Referral sources: Where are your visitors coming from? How are they finding the website?
– Referral domain or URL
– Marketing campaign or channel
– Search keywords

Experience parameters: How are they experiencing the site?
– Device type
– Browser type
– Screen resolution

Content popularity: What content are they interested in? What are they looking for?
– Articles
– Videos
– Downloads
– Site sections
– Internal search terms

Engagement: How compelling or engaging is your site?
– New / Return visitors
– Return frequency
– Landing page bounce rates
– Time spent on site or time spent on specific pages
– Page views per visit

Task completion: What are they able to accomplish? Where are they running into problems?
– Goal / Conversion funnels
– Key navigation flows / Page pathing
– Page heatmaps
– Attrition or exit points
– Page load performance
– Micro-conversions (downloads, trials, demo views, product views, etc.)
– Macro-conversions (orders, leads, subscriptions, applications, etc.)

Overall success: How successful is the site, app, or campaign?
– Revenue
– Macro-conversions (see above)
– Conversion rates on key macro-conversions
– Marketing costs, ad spend, etc.

Customer satisfaction: How happy are your customers with your company? How satisfied are they with your website?
– Net promoter scores
– Customer satisfaction scores

Optimization: What influences or contributes to online success?
– A/B testing
– Multivariate testing

With the help of this data, you have an enriched understanding of your target audience and how they’re interacting with your designs.

Data is a two-edged sword

You can be a victim of the sword of data, or you can wield it to your advantage. Data is a two-edged sword that can both challenge and empower you as a designer.

One side of the data sword challenges designers. Data sets a higher standard for web design—one that is objective and not open to interpretation. It holds web designers more accountable for the effectiveness of their designs, which may make some uncomfortable. Data will present challenges that will require you to solve unexpected problems and iterate on your designs. As organizations become increasingly data-driven and place more emphasis on success metrics, you can choose to ignore the data—but no longer without consequences. Just as companies are evolving in their use of data, web designers must adapt and embrace data to increase the efficacy of their designs.

The other edge of the data sword empowers designers. With greater insight into your visitors and their unique behaviors, you are better equipped to design for their needs. If you are designing without data or fail to understand the results of your designs, you’re also passing up the opportunity to learn and improve. Data will validate or invalidate your assumptions, and it can refine and sharpen your design approach over time. In addition, the metrics can be a common language between web designers and business groups. Armed with data, designers can better defend their creative decisions and elevate the focus away from dueling opinions of the business users, executives, and the creative team. It can speed up decision making and gain buy-in from key stakeholders by adding a sense of urgency (e.g., this problem is costing us $$$ each day we wait). Data can also generate more visibility for the impact your design work is having on your organization.

There’s a difference between holding a sword and wielding it. You can grasp the sword of data awkwardly and not use it effectively. Alternatively, you can wield it with skill and ease in your web design. Just like with real swords, you’re not going to master the sword of data without practice so don’t become frustrated in the beginning. With time you can learn to master its power in your design efforts.

In my next post, I’ll share how data can be integrated into web design processes to make it more data-driven.

‍