Evaluating web site activity is tricky business. Because web activity varies constantly, the challenge is to separate everyday random variation from the non-random changes due to marketing programs, web site design, or search engine rank.

Failing to recognize the sources of variation can result in one of two errors:

• Random variation in activity is "explained" by a change in marketing or the web site when no "real" change in activity occurred.
• A real change in activity is ignored.

Either of the two errors wastes resources and business opportunities.

Enter Statistical Process Control. SPC charts were specifically invented to separate the two sources of variation. While control charts have been most commonly used in manufacturing processes, they "work" with virtually any time series. Additionally, they are easy to understand and use.

On a control chart, upper and lower control limits are automatically calculated to separate common variation from special variation. Points inside the control limits are due to common variation. Points outside the limits are due to special causes.

To make the charts more sensitive to small, sustained shifts in level, "pattern rules" are also used. An example rule is, "Eight consecutive points above the center line signals a special cause."

If a shift in level is identified, a separate set of control limits can be applied to pre- and post-shift data.

As an example, web site traffic can be analyzed by charting the weekly number of web sessions on a company web site.  Every week, the count of sessions where three or more pages were hit is charted.  Our SPC software product, NWA Quality Analyst, automatically extracts the data from a SQL Server database of web logs and charts the data. All the user has to do is enter the desired date range.

The control chart shown in Figure 1 displays the web sessions for January 30, 2000 through April 29, 2002.

Figure 1

ENLARGE IMAGE

The chart reveals the following:

• Out of control points around the holidays can be ignored; they occur every year.
• A minor change was made to web site navigation in mid-May of 2001 ("Article Button"). There was an immediate increase in (three or more page) sessions. We think visitors are finding the articles through a search engine and are now more likely to look at the rest of the site. We believe some single page sessions have turned into multi-page sessions.
• Beginning January 2002, there was a jump of over 600 sessions per week. A 'Google Ad Select' began in early March and yielded 170 sessions per week, but that doesn't explain the entire increase.
• The chart is very sensitive; it's easy to measure the effect of a marketing campaign. We can detect a one or two week change of 1000 sessions, or a sustained shift (five to eight weeks) of 200 sessions.

As shown in Figure 2, the hit rate on web site news articles (individual hits, not sessions) shows the effect of the web site navigation change more dramatically. There was an increase of 1000 article hits per week after the "Article Button" was added. Since the start of February there was also an upward shift in article hits. The unusually high rates during June and August 2000 were due to external technical factors.

Figure 2

ENLARGE IMAGE

Quality Analyst software can analyze web log data in other ways. The most popular 25 articles from January 27 to April 27 appear below in Figure 3.

Figure 3

ENLARGE IMAGE

As we can see, the most popular article is a Petrochemicals article.

Figure 4 is a control chart of the petrochemicals article hits. The article is old (1999), but the number of hits increased in February 2002 from 200 per week to about 300 per week. Is the petrochemicals sector looking up?

Figure 4

ENLARGE IMAGE

About the Author

David Shearer is Technical Director at Northwest Analytical where he's responsible for the statistical and analytical features of NWA's products. He is also a primary instructor for the company's SPC seminars and provides statistical support to customers. Dave is also certified for the operation of public firework displays. He can be reached at info@nwasoft.com.

This article was first published in the October 2003 issue of the Telesian Technology newsletter,  News & Notes - What's Working in Marketing & e-Business, at www.telesian.com.

back to top