Mozfest 2005: Outline for Progress

While the first 25 million downloads of Firefox cannot be discounted as mere luck, there's some of us who wish to guarantee at least another 25 million and extend Mozilla's success to a viable long-term strategy, especially with recent reports indicating Firefox's slowing rate of growth.

Knowns & Unknowns

As the last line of persuasion in getting downloads and hence raising marketshare, the website makes up the pivotal system crucial to Firefox's continued success. Despite the fact that it's the ultimate destination for every single potential Firefox enduser, we do not feel that the website for its truly unique role has been given enough importance, as demonstrated by the two big unknowns:

While marketshare is periodically reported, it is not an efficient day-to-day indicator by which the website's content may be updated or improved.

Lack of Evidence

In the midst of the euphoria of reaching almost ten percent marketshare, there's just one problem: the millions of downloads so far have no bearing on the potential millions who did not download. Just because this problem is unnoticeable at the present time does not mean it cannot exist. There might be millions who never bothered to download Firefox because they were not convinced. In fact, the unknown nature of this problem is the very cause for concern.

The deafening silence of this leak threatens to damage Mozilla in two ways. Firstly, millions of users might not be downloading Firefox because of unknown usability problems with the website. Secondly and less obviously, these users will form a perception of Mozilla and its products as somehow inferior to their Microsoft alternatives (Internet Explorer and Outlook). As hardened perceptions are difficult to reverse, these users will be virtually impossible to reach later. There's absolutely no need to lose downloads and at the same time alienate users.

Unfortunately, the bigger Firefox gets, the leak grows proportionally. But where there is fear, uncertainty, and doubt, the solution lies in not ignoring the problem or ignoring our calls to remedy it. The paragraphs below provide an outline to determine the size of the problem and the rectify it.

Experiment

The first order of business would be to figure out the actual ratio of Internet Explorer users who download Firefox. We call this the Internet Explorer Download Rate (IEDR). While this refers to only downloads and doesn't measure if the user continues on to use Firefox, the IEDR is the only unit which can tell of the success rate of the current website, in a quick and objective manner. (As far as the more elusive and more important marketshare goes, it's safe to assume that the chance of person using Firefox who downloaded it is infinitely higher versus someone who did not.)

The procedure for measuring the IEDR is very simple:

  1. Tag a small sample group of IE users who visit Mozilla.org with cookies.
  2. Register these users in a database.
  3. If the tagged IE user downloads Firefox, increment the counter.

The counter's number over the total number of cookies passed out would give the IEDR.

Cause

As the IEDR will most certainly fall short of unbeatable highs, the website needs improvement. For example, if we assume for now that the current IEDR is 70%, it's not impossible to conceive of higher rates, which could easily translate into millions of downloads.

Many would see that the website is not crucial and is just another cog in the overall strategy. If we are to believe that not every person who is referred to Mozilla.org can be convinced to download Firefox, what's the reason: is IE better in any way than Firefox? Since Firefox is hands down better, we do not see why it's impossible to come up with a design and content to match Firefox's superiority over Internet Explorer.

Design Contest

To encourage a development of a new website strategy, Mozilla.org needs to announce a fair and open competition to change the current website. It is important to note that the final products of this contest would amount to not just stylesheet changes, but entirely new designs with new content that focuses on endusers precisely. Also, these new designs would prove their merit under an objective test. The contest should unfold in three parts:

Phase I: Template

Any team or individual which includes all the listed requirements in a single-page design intended to serve as the frontpage of Mozilla.org (as a template for the rest of the phases) should be eligible to compete in Phase II:

The qualifying designs would be showcased to the community. Each project could then recruit volunteers to help the team to complete Phase II.

Phase II: Content

All the designs would now be up filled with actual content. Thinking along the lines of meeting the enduser experience of at least the current site, to qualify to move on to Phase III, each proposed site would need:

While Phase I could attract dozens of entries, Phase II would limit the rest of the contest to those with dedication. Only the truly serious would spend time working out the content for an entire site. As for the last item on the list above, teams should once again be reminded that Mozilla.org is looking for more than a superficial change.

Phase III: Split Run

To determine the IEDR of these proposed sites, the experimental procedure would have to be slightly modified:

  1. Tag a small sample group of IE users who visit Mozilla.org with cookies.
  2. Register these users in a database.
  3. Redirect an equal number of IE users to each design, including the current one which would act as the control variable, while making sure that these IE users will always get the design first redirected to.
  4. If the tagged IE user downloads Firefox, increment the counter.

In practice, if there's 9 designs plus the original, not all designs would get 10% of all visitors each. Under this scheme, the large majority of IE users would not be involuntarily participating in this experiment, as each design only needs a few thousand users for an accurate sample. To be even fairer, it is our recommendation to make each day's stats publicly viewable.

This experiment could run for a week or two. Teams could then make slight changes to see if such small problems caused any big losses. After a few runs, it would become very obvious which ideas work and which don't.

To some, the guidelines of the contest might seem very lax, allowing for too many entries to be tested; however, we could not disagree anymore. There's a single reason not to expect any more than a dozen teams to compete in Phase III: Few groups of people are going to spend a considerable about of time on something that's risky. (Also, a "first-come-first-first" clause could guarantee only a certain number of designs maximum.)

Note: These are proposed ideas or procedures. Should they be accepted, the Mozilla Foundation would responsible for conducting this entire community-building contest. They would determine what counts as a win, how the improvement is implemented, and for how long each phase runs.

Case Scenarios

The worst case scenario of a design contest would be a stronger community, new ideas, and a few cool CSS templates, with a few thousand downloads going to waste. Considering the potential of the winning design(s) is in the millions, the risk is almost non-existent while the payoff is high.

However, if we just raise the IEDR by just a meager 10%, the end result is 10% more downloads, which translates to millions over the long run.

Authors/Contributors

Andkon. Foxtrot. Abarnes. Andrew Heckford. James Houdek. Alex Marte. Dr. Eric Livingston. David C. Paternostro.

Additional Info

An analogy originally posted on SFX:

Imagine you leave cookies in a room. You then sit in the corner and turn off the lights, expecting people to take these cookies. You hear some noise, knowing that people are in the room, but you can't exactly tell how many. When you turn back on the lights, you know how many cookies were taken (and some people even tell you how really good they were). However, you do not know how many people did not take the cookies and more importantly why they did not.

The Mozilla website is like the dark room, while Firefox is the cookies. We do know the download numbers (and even the general marketshare). However, regardless of your opinion of the current website, it's not known how many people are unconvinced and do not download Firefox. As with the cookies example, we also have no clue why they don't. Unless we want to delude ourselves into thinking that every single person who comes to Mozilla.org downloads Firefox, there's every reason to figure out how many people are not downloading Firefox.