They have an online shop that sells a wide variety of products from books, games and coffee mugs to the more exotic and expensive, such as artwork and even electric bikes. NZZ offers two models of the latter, which sell for the equivalent of between 3,600 and nearly 4,075 euros.

But back to that paywall. The 242-year-old, German-language news publisher is well regarded for their high-quality, trusted journalism, and requiring users to register became a top priority for them several years ago. For the past few years, NZZ has been running a number of tests and experiments where different readers get different subscription journeys based on their initial behaviour on the site after they’ve registered.

As with the experiments themselves, there have been ongoing developments internally as well.

For example, initially, the testing was largely done and owned by NZZ’s marketing department, but expanded to included staff from editorial, from product, data and technology. Then the pandemic hit in 2020, which forced further changes.

‘We had to discover our baseline again’

“In order to meaningfully move this forward, and create experiences that would positively affect our readers, we had to figure a way to work together,” NZZ’s Nashua Gallagher told participants at WAN-IFRA’s Digital Media Europe conference in Oslo.

Gallagher is Head of Customer Journey Management at NZZ, and previously worked at the FT and The Economist.

“We focused on audience habits,” she said. “We realised that it didn’t really matter what we thought we knew, things had changed so significantly in 2020 that we had to do a deep dive into how users were behaving then and compare it to previous years. … we had to discover our baseline again.”

NZZ then began a need-finding exercise that lasted for three months. After learning some basic things about their audience, they needed to decide what they wanted to test.

“That took another couple of months,” Gallagher said. “And then the technical implementation of that. Then finally we thought ‘I don’t think we can do this in one test. We need to actually design an experiment of six months and learn as we go and optimise as we go as well.’ “

This process takes a full year, Gallagher said.

“We have to give the experiment time to run,” she said. “We need to understand cohort behaviour for a longer period of time. We need those cohorts to reach significance. And we also need to protect the revenue of our business.”

60 percent of conversions happening in the first week

In the first quarter of 2021, NZZ began a need-finding exercise that looked at four key areas, including “fast subscribers,” or those who sign up very quickly.

“We found this really fascinating insight where, of all the people who register and then eventually subscribe, 60 percent of them did so in their first week, and actually a majority of that 60 percent did so in their first day,” Gallagher said.

The next insight was about the 60-day cohort, she said, meaning the first 60 days after a person registers.

“No matter what we did, we couldn’t get away from the first two months being super important,” Gallagher said. “So important in fact that if the user didn’t subscribe on that first day or week, another 30 percent will subscribe within the first 60 days.”

They also found out they have a large number of users who are basically “one-and-dones,” signing up to read only an article or two, and then disappearing.

“They’ll register with a fake address or their real address, and just never come back,” she said.

Since some of this testing was going on during the early stages of the pandemic, they also discovered some readers who literally just came to them for information about COVID. Some would consume large amounts of content relating to that topic, but would not read or watch anything else.

“The interesting thing for me was this was post-subscription,” Gallagher said.

She added that NZZ then ran a few tests where they tried to get those subscribers to read a bit more broadly, but to no avail.

Testing their propensity model

They used further experiments to test their propensity model, which had been developed in-house, and uses more than 80 data points, but they hadn’t really tested it since 2018.

“We ran a bunch of cohort tests from September last year to March 13th of this year, where we looked at each cohort and we bookended them in periods of 60 days,” Gallagher said. “There were lots of internal discussions on the role of metering, so I thought, OK, let’s just test it. Let’s have one group where we don’t give them a paywall for the first two weeks completely, just nothing. Then we set the meter at 10 for the next two weeks, which means that effectively for a whole month, they don’t see a paywall.”

They also ran a few things on targeting outlier reading habits, she said. For example, if people like those COVID-only content consumers mentioned earlier, were reading very narrowly, could NZZ change their onboarding journey to get them to read a little more broadly?

NZZ also decided to repeat the analysis they did in Q1 2021 in Q1 2022.

“Fast subscribers remained prevalent, perhaps not surprisingly because we had such extraordinary news events in between,” Gallagher said.

NZZ also identified the key conversion zones, and learned that the first 60 days after someone registers are definitely important, and their onboarding process runs for the same amount of time. 

In addition, she said, they have been able to identify clear zones from when conversion takes place across all groups.

“The groups that had unlimited metering as well as metering of two or a metering of four, all of them converted best in Week One, and then again, oddly, in Week Five,” Gallagher said.

“We also saw 12 percent of users who read consistently during that 60 day period, who had limited paywall exposure, who were really highly engaged, but they never converted,” she said.

And then finally 45 percent of users were the previously mentioned “one-and-done’s.”

What’s next? More experimentation

Looking ahead, Gallagher said NZZ will continue to experiment.

“We’ll also run monitoring groups so that we don’t run into the same issues three years from now,” she said.

She added that they’ll keep looking at what the registration journey is, what it should be, what it shouldn’t be, and also at what they can do to better reach casual readers, but most importantly, they know things will continue to develop and change.

“What we are doing today could be completely different two years from now, and I think that’s how it should be as readers mature and have different expectations,” she said. “And as market changes constantly, the best thing that we can do is to ensure that we have a set up that can flex to whatever we need it to do.”

Main image by jarmoluk, via Pixabay.

The post How long-running experiments help Switzerland’s NZZ see patterns among subscribers appeared first on WAN-IFRA.