 Semphonic BlogsThe Basics of Measuring Site Funnels
Reporting on Forms and Conversion ProcessesPart Nine in Series on Form Abandonment and Online Process Measurement I seriously considered blogging about the collapse of the Microsoft-Yahoo deal but in the end, I decided it wasn’t really worthwhile. I don’t think I have much to add beyond what is, for the most part, conventional wisdom. On the whole I thought the deal probably made sense (at least for Yahoo). Management isn’t wrong to seek the best deal possible or even to desire to continue running the company themselves. But managers in a public company have a real responsibility to consider such deals fairly – perhaps even more than fairly to counter the natural bias of self-interest - and not simply do their best to sabotage them. And while it is interesting, even occasionally gripping, watching these corporate battles unfold, I am usually relieved to return to the world of our craft. The concerns of craft are always meaningful; more meaningful to the craftsman than the ultimately rather uninteresting business of who owns what. So I’m going to finish up my series on Forms measurement with a discussion of reporting KPIs that I hope will wrap up the series in a cohesive fashion. In the first post in this series, I talked about the role for behavioral analysis in the development and improvement of Forms-based processes. This included a discussion of design and usability, and how CEM tools like Tealeaf and web analytics tools could extend the improvement of Forms on into their operational lifecycle. In the next post, I considered what kinds of process issues you actually study with web analytics and argued that the most prevalent web analytic measure (step drop-off) is not actually very useful. This post laid the ground-work for subsequent posts on measuring the qualification level of visitors to a process, measuring the need and success for selling INSIDE the process, and measuring and optimizing completer (thank you) pages. My last three posts in the series have been on key extensions to the techniques of Form and process measurement including discussions of visitor segmentation, tracking process errorsand warnings and the implications of multi-session process usage. At Semphonic, our focus in reporting has increasingly been on “systems” based reporting. The goal in a systems-based report is not simply to capture the state of a key variable (KPI) but to capture the relationship between variables so that the underlying business system is fully described. For forms and processes, I think of the reporting system as involving several distinct systems that can be combined at a high-level into a global system. The most important systems are typically these:
The pre-qualification system should measure the variables that tend to illuminate differences in process conversion that have nothing to do with changes to the actual process. Key measures in the pre-qualification system that should be modeled include:
The marketing system attempts to measure how well the process works from a sales perspective. The marketing tasks in most processes include providing re-assurance, reminding about sales benefits, surfacing necessary information, and re-engaging upon completion. Key measures that should be modeled in the marketing system for a process include:
In the operational system, you are measuring how well the form is accomplishing its basic task of collecting and processing information. The key measures to model in the operational system are:
Combining the high-level outputs of each reporting system into a global process report should capture the key changes in each system and function of the process. Understanding how qualification levels varied can help protect site managers from misinterpreting changes in conversion rate. Understanding the difference between marketing abandonment and friction can help focus on the right type of change to actually help solve the problem. By combining these factors into a high-level “analytic” report, you’ll have transitioned your thinking about conversion processes and web analytics from the sterile and often misleading focus on “step fall-out” to a deep understanding of the factors that actually matter in improving or draining your site’s process performance. And not only will you have gained a much better understanding, that understanding will have deepened your thinking about the types of actions that will make a difference. You can spend – and waste – endless amounts of time streamlining a process whose real problems may lie within the marketing system. And, of course, you could add more and more sales material to a process and achieve nothing but increasing losses due to friction. Only thoughtful, system-based measurement can insure that your process meets the true, real-world needs of your customers. Posted by Gary Angel 5/04/08 Go to Blog Titles. Forms Measurement and Multi-Session BehaviorIn this, my penultimate post on the topic of analyzing Forms and Conversion Processes, I’m going to tackle the issues and measurement surrounding processes that are abandoned and then resumed in subsequent sessions. Multi-session form behavior greatly complicates the task of analysis – not least because most web analytic tools do a very poor job with any form of multi-session analysis. In the next installment I’m going to wrap up with a discussion of Form KPIs and reporting. Not every form or process exhibits much multi-session behavior. It’s rare to see multi-session behavior on short, simple forms. These may be abandoned immediately and then tried again, but I generally think of multi-session behavior as belonging to the set of cases where a visitor fills out part of the form, abandons the session, and then resumes later on. Understanding multi-session form/process behavior can help answer quite a few different questions including:
The right place to start is figuring out if – for a form/process – multi-session behavior actually matters. Intuitively, you can expect that any long form will exhibit a fair amount of multi-session behavior. But the extent is quite variable and often driven by the type of information required of the user. If a process requires a user to provide things like bank routing numbers, license numbers, tax-id numbers, etc. then a significant amount of multi-session behavior is almost inevitable. I remarked that most tools make it quite difficult to understand multi-session Forms behavior. But the one thing you can usually get quite easily is the extent to which such behavior exists. The easiest way is to simply generate a report for each page of the process that includes the views, visits and unique visitors. The visits to visitors ratio here will provide you with an immediate read on the extent to which multi-session behavior actually exists. Make sure (of course) that you aren't looking at a number like Daily Uniques or an addition of Daily Uniques. If significant multi-session behavior does exist, you’ll want to focus on the VISITOR abandonment rate as the true measure of form/process conversion. For the entire Form, that rate is again, quite trivial. It is simply the Unique Visitor Count for Form Complete divided by the Unique Visitor Count for Form Start. This rate may be significantly better than a visit-based measure of form conversion whenever multi-session behavior exists. This will also implicitly answer the question of whether funnels or fall-out tools can be used to track form processes. These tools are almost all visit-based. So if you have significant multi-session behavior, they will present a very misleading view of actual form performance. In addition, many such tools simply will not work correctly if your Form contains Save-and-Continue functionality. Save-and-Continue functionality can be a big boon to usability. Chances are, good usability testing will have already shown whether or not it is important. But if you have significant multi-session behavior and can see that abandonment is occurring well into the Form (particularly by time spent), then a strong case for Save-and-Continue functionality can almost always be made. At this point, however, we have left the realm of easily accessible KPIs. To understand when/where Form Abandoners who later-returned left the process is a non-trivial endeavor. You can’t simply look at Exit Rates by Page because you don’t know whether the exits are for returnees. Instead, you have to be able to segment your Form visitors based on the number of visits that contain the Form. Depending on your segmentation tools, you might also choose to look at a simple segment like "Form Completers." If you have a segment of Form Completers, you can assume that any Form/Process Page Exit (other than complete) was followed by a return visit. Using this, you can pinpoint which pages in the process caused visitors to abandon and how long they spent on the process (on average) before they abandoned. With these two pieces of information, you can probably make a better decision about the utility of a Save-and-Continue capability (or, possibly, a reorganization of the Form). It’s also interesting to consider directional abandonment for multi-session Form Completers. Using that same Form Completer segment, you can look at Next Pages for each step of the Form. When those pages are outside the Form Process, you can see what types of information or re-assurance visitors who ended up completing might have been looking for. This ability to pinpoint where multi-session completers abandoned the process is the best behavioral tool for deciding whether or not there are special informational requirements driving abandonment and whether or not the Form is adequately conveying those requirements. By testing up-front Form guidance, you can measure the impact on multi-session abandonment points and see if you’ve made a difference. It’s a lot easier to see if you’ve made a difference with this functional approach than if you need to move the needle significantly on end-point conversion. Many changes that make a Form better, faster and easier for the user will have only a very small impact on final conversion. That doesn’t make them worthless. One final piece of learning from multi-session behavior involves 2nd time abandonment points. If you find that Form users are abandoning in the same place on return visits (attempts to define user names on popular sites can have this issue), then you’ve certainly identified a high-friction area of a process. If you’re losing significant numbers of visitors committed enough to return for a second try, then you need to re-think your approach to the Form. Multi-Session behavior – like nearly all of my other Form/Process topics – has a real impact on the reporting you do. Ignore it, and you can significantly misread the real conversion rate of your Forms and the true potential for improvement. In my last post on Form/Process measurement, I’ll tackle reporting on processes with a systems approach (analytic reporting) and how the various points raised in the previous posts surface new metrics that are rarely used in reporting on Forms and make it clear how others (like Form Conversion Rate) can be easily misinterpreted. Posted by Gary Angel 4/27/08 Go to Blog Titles. Never Make the Same Mistake TwicePart Seven in a Series on Analyzing Conversion Funnels and Form Abandonment Most formal processes of continuous improvement rely heavily on the simple idea that if you made a mistake you should figure out why and try to eliminate the possibility of it happening again. In “Part Seven” of this series on measuring form processes I’m going to talk about extending that idea into web forms processes. I’ve looked at and measured many online processes over the years. And it’s safe to say that in almost every case errors inside the form were invisible to the web analytics solution. This used to be for the good reason that there was no way to report such errors but this is no longer true. With modern tag-based systems, you can almost always pass information about process errors to custom variables in the web analytics tool. It doesn’t happen very much for two reasons: it takes extra work when building the process and most people erroneously assume that such errors are uncommon. I get the extra work problem. That’s always a disadvantage. You can save yourself a lot of effort and get yourself a very powerful tool for tackling continuous improvement in complex form processes by acquiring a CEM tool like Tealeaf. But Tealeaf-caliber tools aren’t for everyone and there is much that can be done even within the more limited constraints of a traditional web analytics tool. Nor is the amount of effort here so burdensome as to make the work a difficult call when it comes to return on investment. As to whether forms processes errors are uncommon, I can only say that my experience suggests that while errors and problems are not everywhere neither are they an endangered species. So what you should do to track errors? The most important thing is that you should have a measurement standard for any form process. This standard should embody a few simple ideas: every time an error message is returned to the user the web analytics solution should know about it; every time a client-side field fails validation, the web analytics solution should know about it; and every time a popup help or error message is displayed to the user…you guessed it, the web analytics solution should know about it. There are two fairly straightforward strategies for achieving this. You can send custom variable encodings to the web analytics tool whenever such an event occurs or you can save them and pass them all at once whenever the form is abandoned or continued. Either way, you’ll typically want to report to the web analytics tool a variable that contains all of the following: The Page Name of the Process : The Type of Action : Field of Action : The Message or Description 'Page Name' is obvious. By including it in the variable encoding, you can make it easier to report on error messages and understand when an error message is getting triggered (many applications have errors that are redundant across pages). The 'Type of Action' describes things like “Client-Side Validation Failure.” The 'Field of Action' describes the field of the form (where appropriate) that triggered the call. The 'Message or Description' passes back the Error Message or additional descriptive text about the event.With this style of encoding, an error on an entry-field in a form might result in a web analytics variable being populated with: “Shipping Information – Step 3 : Field Validation Error : Ship State : Not Entered” Collecting every error in a process in this fashion greatly extends the utility of web analytics conversion analysis. In addition to the obvious fact that you’ll have true visibility in the number of errors that are occurring, you’ll get a host of useful additional information including:
This is valuable stuff. High step or field error rates may indicate true problems in the form process. But they can also potentially identify places where additional help or explanation may be required. They can even be used to help target form step length appropriately - putting too many fields in a form step can result in high error rates and increased frustration. When you remember that these conversion processes are almost always the highest-value places on your site, it’s not hard to see why understanding potential visitor problems in these segments is critical. Everything that I’ve said so far applies – emphatically – to Ajax and Flash forms as well. With more sophisticated programmatic forms, pretty much every part of the measurement must be hand-built anyway. So there really is no excuse to not trap and report on errors as well. One special note – when you report custom variables as a special call in most web analytic applications, you may be sending an additional page request. You should take pains to make sure that if you are doing this, that the page request is either isolated from the rest of your system or accounted for in your reporting. You’re not likely to ever get a web process down to the vanishingly small error rates common with sophisticated manufacturers. They simply have more control over their environment. But this very fact makes it all the more essential that you constantly track and report on what happens inside an online form – especially when it comes to errors. I have two more posts in this series – one on analyzing multi-session behavior with respect to form processes and a final post covering form KPIs based on this whole series. Posted by Gary Angel 3/23/08 Go to Blog Titles. Form - Know thy VisitorPart Six in a Series on Analyzing Conversion Funnels and Form Abandonment Visitor segmentation is a fundamental tool in virtually every web analysis. Take almost technique I’ve ever talked about and it can (and should) be applied within the context of a visitor segmentation. The analysis of conversion processes is no different. Every aspect of Form Performance is heavily tied to the type of visitors using it. And neglecting to analyze Form Conversion by visitor segment will leave a big part of the picture colorless. One of the drawbacks to talking about Visitor Segmentation (though I’m going to devote attention to this in the coming weeks) is that most good visitor segments are highly business and site specific. That makes it challenging to talk about them in an interesting but generic fashion. There are some pieces of most visitor segmentation schemes that are commonly of value in Conversion Process analysis, and I’m going to focus on them. But you’ll still find it worthwhile to look at every aspect of Forms abandonment and Conversion Process KPIs in terms of your business-specific segmentations. Here are some of the common uses for more generic visitor segmentation in doing funnel analysis: Isolating Friction in Forms vs. Lack of Sales Reinforcement Form Friction and Lack of Sales Reinforcement both manifest themselves in terms of abandonment. But you can often isolate them and make their real effect much clearer by focusing on the behavior of visitors who are highly-likely to be committed to your brand (and vice versa). This group will include returning customers and visitors who self-select immediately into your conversion process. You can also isolate committed users based on the extent of previous site behavior (especially for media and social sites analyzing subscription or registration processes). Visitor segments in these segments are committed to completing the process and need little sales reinforcement. Analyzing the form friction for this group will establish the true baseline. Looking at the less-committed visitor segment should provide much better and more informative numbers about lack of sales-reinforcement and directional abandonment. As with so much of web analytics, visitor segmentation is less interesting if you simply take conversion rates and slice them by types. You’ve just learned your most-committed visitors are least likely to abandon your process. Hurrah. It’s in the context of an analytic purpose that visitor segmentation is truly interesting. First-Timer Form Friction This is the most basic of all visitor segmentations and while it only applies to certain types of process, it's often surprisingly interesting. If your processes are almost all one-time types of efforts (buy life insurance, generate a lead for a business-to-business purchase, register on a media site) then this won’t be fruitful segmentation. But if you have processes that visitors may repeat more than once (making a purchase, doing a stock trade, loading a video, etc.), isolating the performance of 1st Time Process users is nearly always informative. Repeat process visitors have higher completion rates, faster avg. page times, and different patterns of directional abandonment. If your process gets a substantial mix of first and repeat users, then the averages for nearly Form KPI will be misleading. Avg. Page Time will be a meaningless muddle of fast repeat users and slower first time users. Possible performance and abandonment issues can easily get lost – particularly in cases where a majority of form users are experienced. Form Navigation and Option Selection Many conversion processes have alternative navigation strategies and different types of options built into them. Understanding the impact of these options (as well as the take-up rates for Form options) is heavily dependent on visitor segmentation. Both of the generic segmentation strategies I’ve already talked about matter here (new vs. experienced visitors and committed vs. uncommitted) but even more important are your business-specific interest segments. These segments will have dramatically different take-up rates for up-sells or cross-sells presented in the conversion process. Some segments perform better with up-sells/cross-sells and others without. If you evaluate Form Performance “on average” you will significantly mis-optimize the conversion process for key business segments.Completer Routing My last post dealt with the importance of holding on to the visitor or messaging the visitor at the completion of a conversion process. It’s often the case that your messaging strategy should be tailored to your key business segments. As with Option Selection, generic segments may also be interesting but are less likely to provide strong analytic differentiation. Carrying visitor segmentation all the way through the conversion process into the analysis of routing from Completer’s will improve your understanding of what types of actions might come next. The importance of applying visitor segmentation isn’t limited to deep-dive analysis. I’ve written recently about the value of applying analytic models to reporting, and I think conversion processes particularly benefit from this. If you are reporting the classic funnel analysis without segmentation, then you’re mis-reporting on the conversion process system. You’re leaving out key variables (visitor types entering the Form) and making it all too likely that report consumers will misuse and misunderstand changes in your KPIs. If process conversion rates have improved because of a change in the mix of visitor types entering the Form, it is absolutely essential that you identify the cause – not simply the existence - of that change!Posted by Gary Angel 2/17/08 Go to Blog Titles. Finishing the FormPart V in a Series on Measuring Form Abandonment I’m returning to my series on Form Abandonment and in this post I’ll talk about what happens AFTER you seal the deal. Because what you do after the deal is often even important than what you did before. Nearly every Forms process ends with a Thank You page – in the Functional lexicon we call that page the Completer. And it has traditionally been one of the least used and least interesting pages on a site. The primary role of the Thank You page is to be something like a hand-shake: a formal recognition that the deal is done. This formal recognition was especially important in the early days of the web because order processes were so frequently buggy and broken. People wanted to make sure they had really finished successfully. Even with today’s much more reliable order processes, however, the Completer page is an important part of the process. It marks a psychological boundary between activities. You are now done with X – you can leave or go on to Y. And that sentence above gets to the gist of the problem. Because most companies just assume that after a Completer Page you will leave. It’s an assumption borne out by the real-world fact that most Completer Pages have very high exit rates. It’s often a bad assumption though – and even if it’s true it doesn’t mean the Completer should page should be nothing more than a barren Thank You. You just completed some form of significant process with a visitor. You sold them something. Signed them up for something. Registered them. Isn’t there something you’d like them to know beyond “you’re done”? Part of what makes this opportunity so attractive is that the real-estate on the Thank You page is wide-open. It doesn’t require much space to complete the Thank You. There aren’t too many web pages with as much open real-estate, with no down-side to distracting the visitor, and with a proven high-qualified viewership. What can you do with that space? Quite a lot. Since the visitor has just completed a discrete activity, they are probably open to some route guidance. And every Completer page really should include some suggested routes. We use the percentage of controlled routes from a Completer Page as the PRIMARY measure of success when evaluating their performance. And because the visitor has just done something real on your site, you almost always have powerful targeting cues for what to suggest. In addition to suggested routes, however, this is a great time to do some brand messaging. And it can also be an excellent time to collect voice-of-customer information. To some extent, how you use this real-estate will depend on the type of site and the type of process just completed. If you are lead-generation site and a visitor just downloaded a white paper (or equivalent), you’ve just gained dramatic new knowledge about their level and type of interest. Use it! Lead form Completer Pages are especially likely to be lousy – and yet they are also much more likely than eCommerce Completer Pages to retain customers if used properly. If your customers have just registered for your site or a specific site service, don’t just say thanks. Give them an immediate routing to some new stuff they can/should do. This first entrée into your gated system deserves careful guidance. The more content you can open the visitor up to and the more you can direct them to the right areas, the more likely they are to fall into a valuable usage pattern on your site. No one should underestimate how quickly visitors fall into specific patterns of behavior with respect to a site. And once they’ve established a pattern, it can be really difficult to increase or change their usage. For eCommerce sites, the Completer page is more challenging. You’ve probably already offered up-sells during the process. And there’s a pretty good chance that a visitor won’t want to re-engage in a buying process. Instead, this might be a place to push for partner sales (the way travel sites do), provide support and branding messaging, or push for some additional level of engagement with the site (notifications, registrations, etc.). Measurement of Completer Pages is straightforward. To convince a business that there is an opportunity here, we typically measure the existing routing performance of the page. How many exits? How many returns to the homepage? How many searches? Companies are frequently surprised to learn that 25-35% of their visitors aren’t exiting from these pages. And when they see how un-productive are the routes those visitors are taking, they usually get the message. Once the page has been re-designed, we look for changes in the immediate routing behavior – particularly the percentage of visits that take one of the suggested routes. In general, we want 70% or more of the visitors who do route to take a suggested route. We also want to see some decline in the Exit rate for the page. For pages that opt for more of a branding approach, the measurement should focus on repeat visitor rates. It’s far from slam dunk to get a noticeable uptick in repeat visits, but it is possible. I’ve talked and blogged about Completer pages before. But I still see many sites taking little or no advantage of the real-estate on these pages. It’s a surprisingly common problem – one with an easy fix – and with a fairly high ROI. That’s a pretty compelling combination. And it’s why you really should think carefully about designing and measuring the Completer Page of any forms process. The Completer Page is the first step you take with a customer after they say “I do.” Don’t waste your honeymoon with just a hand-shake. Posted by Gary Angel 2/10/08 Go to Blog Titles. Selling Inside the FormIn the first installment of this series on Forms, I talked about the critical role of usability testing in the development of good Form processes on the web. Even after a Forms process roles out, a lot of the focus of analysis tends to be on its basic operational performance. Is a Form Page broken in some instances (usually because of untested field edits)? How long does it take users to traverse the Form? What error messages get triggered during Form completion? These are all important questions and they tend to get a lot of focus. But one of the key real-world aspects of Forms behavior – and something that you absolutely CANNOT test in usability labs – is whether the Forms are doing a good job selling the product. Forms aren’t supposed to sell the product, are they? As usual, I'm going to answer, "It depends." There is a significant school of design thought that emphasizes paring away every aspect of the Form that isn’t absolutely necessary to get the job done. It seems logical. An almost inevitable consequence of some of the principles I elaborated earlier: that every Form element introduces at least some small amount of friction. Reduce friction and you improve Form performance. This focus on operational perfection, however, isn’t always ideal. Particularly for longer, more complex Forms, the issue is NOT clear cut. As friction builds up during the process, visitors who aren’t totally committed to the purchase may drop off. Adding Form elements that reinforce or build the commitment to the sale may have a positive impact on the process despite the addition of some Form friction. You can’t test this in usability. Testers aren’t actually buying the product. So no amount of usability testing can ever tell you how much - or whether -additional sales reinforcement during the Forms process is necessary. Are there any behavioral cues that can help? There are. But this is also a place where testing will likely be the ultimate source of answers. If you’re looking to make a case for testing (which can be expensive in Forms processes) or just to decide if sales reinforcement might be in order, then you should focus on a measure we call directional abandonment. When users exit a Forms process, their manner of exit can be very informative. There are really three types of Form Exit: exited the site, exited to the rest of the site but did nothing of interest (returned to home page and exited or backed-out to the page prior to Form Start and exited), and existed to the rest of the site and looked at something. This last category of exit is what we call directional abandonment. And it’s extremely informative about what types of issues or concerns may be arising during the moments before a sale. Don’t expect Directional Abandonment to be an enormous percentage of exits. For most sites, process exits are mostly site exits. But even modest levels of directional abandonment can be very significant and can provide cues about the psychology underlying the much larger number of site exits. There are different ways to look at Directional Abandonment. It’s a web analytics task that is complicated by the inability of pretty much all WA tools to isolate the behavior of visitors AFTER an event. In most cases, you’ll begin with a Next Pages analysis. I believe this is most fruitful on a step by step basis. Different points within the Form will often yield different types of Exit behaviors. If you want to be fancy, start by building segments of visitors based on their deepest penetration into the Form. So if you have a five-step process, build a segment of visitors who reached Step 4 but not Step 5; then a segment of visitors who reached Step 3 but not Step 4; and so on. Building segments in this fashion let’s you compensate for the fact that visitors will often slide backward in the Form before exiting. Using these segments, look at Next Pages from each step of the Form. Throw out site exits and Form Pages, and you have a list of destinations. I usually classify these by Functional Types. So I’d look at how visitors left for Engager pages (like the Home Page), Router Pages (by subject), and Convincer Pages. Within the Convincer Pages, I’d want to look at how Exit To rates compared to Viewed and Entered From rates. I’m looking to find a pattern that suggests a specific Page(s) represents the area of primary concern. Don’t forget to always look at Search Keyword behaviors as well. As I’ve written about before, Internal Search keywords can be a powerful tool for understanding what visitors are looking for or care about at any specific point in a site visit. Of course, Next Pages doesn’t capture a significant chunk of interesting behavior for Forms exits. It’s commonly the case that navigation outside the Form is limited to a few pages (such as the Home Page). So you need to be able to do path analysis from each page of the Form Process based on the “depth of penetration” segments. In some other types of analysis, I’d suggest that you use an affinity measure. But that won’t work here. Affinity analysis (viewed in same session) has no conception of sequence. And it’s far too likely that visitors will have viewed the Convincer Pages BEFORE entering the Form for this analysis to be meaningful. So you’ll have to use pathing to establish how many times visitors exited each Form Step and went to any Convincer Page and the rate at which they subsequently viewed each specific Convincer page. By doing this, you’ll often be able to isolate the page(s) that are most of interest to Form abandoners. You may be wondering why I don’t suggest just asking visitors what’s wrong. It’s not a bad strategy. But neither is it a replacement for behavioral analysis. This is a notoriously tricky place to capture good data. And even if you can get enough visitors to respond, it's difficult to get the right questions in place to really understand the behavior. It’s certainly not a bad idea to ask “Why did you choose not to buy?” But don’t expect the visitor to answer intelligibly. Buyers often won’t tell you – “I needed to be persuaded about the value one last time.” How much (if any) reinforcement is necessary is not a question that can ever be settled in the abstract. It doesn’t even have a single right answer for a site process – since changing the mix of visitors into a process can change the optimal process design. It has been my experience, though, that many longer and more complicated forms processes can benefit from sales reinforcement. Done well, the frictional cost of reminding the user why going ahead is worthwhile can be very minimal. I've learned through bitter experience that in the consulting business, it’s never a good idea to assume the job is yours until the paperwork is actually signed. Forgetting to reinforce the sale (if there’s an opportunity) during the “paperwork” stage has cost us more than one sale. The same is true on the web. I would say that you can take it easy once you’ve got the sale, but that isn’t really true either. Because in my next post I’m going to re-visit a topic I’ve talked about several times in the past: how to think about the last step in a process – the Thank You page. Posted by Gary Angel 1/30/08 Go to Blog Titles. A Funny Thing Happened on the Way to the FormPart III on Measuring Conversion Funnels in Web Analtyics I have a few quick notes of things I wanted to mention before I dive into more discussion of Form Abandonment and Conversion Funnels. With the new year kicking off I have a whole raft of interesting speaking gigs coming up. This Thursday I’m going to be back in San Diego speaking at SemDirector’s InflectionPoint 2008 Conference. I’ll be on a panel discussing “Measuring Success in Search Interactive.” I’m definitely looking forward to the Conference. The next week, I’m going to be talking web analytics (in an introductory fashion – a stretch for me) at the AAN Web Publishing 2008 Conference. In February, I’ll be at the eXl Pharma Conference on Search Engine Marketing (talking about structuring your SEM program), The Web Managers Roundtable in Washington DC (talking about Analytic Reporting!!) and then the SMX West conference (Measuring SEM programs). That’s three conferences in three days in three cities. Boy am I (not) looking forward to that! So let’s talk about Form Abandonment. The main point of my last post was that looking at step drop-off is very far from being the main focus of good conversion funnel analysis. Today, I want to start talking about some of the other things that you should be looking at that may, in the end, be more fruitful. I decided to start off with thinking about what happened before a visitor entered the Form. First things first, after all! A conversion process is very far from being an island unto itself. And, to a surprising extent, you have control over when visitors reach a Forms process. The placement, sizing, wording and aggressiveness of “Calls-to-Action” on a web site make a big difference on when and from where visitors reach the actual process. In our Functional Methodology, we make the distinction between many different types of pages in the sales cycle. "Convincer" pages are designed to win the customer. But they often are more focused on providing the necessary marketing information than on making the request for the sale. That’s why many sites also have a distinct class of pages we call "Closers." Their function is to ask for the sale. Naturally, some sites also blend these functions. And some sites have various types of Convincer pages – some focused on more informational ("Informers") or explanatory ("Explainers") functions. There is a significant school of thought in web design that emphasizes the importance of the Call-to-Action. I actually tend to belong to that school of thought. But it CAN be overdone. When your site turns into nothing but a big Call-to-Action, you may be losing more visitors than you’re winning. For many sites – especially in complex areas like Financial Services – the sales cycle cannot be shortened too much. You need to realize that driving visitors too quickly into a conversion funnel will raise drop-off rates and, at some point, provide diminishing marginal returns. It is a purely empirical issue when Calls-to-Action impair instead of enhance the efficiency of your site. All of this just underscores the point made last time – Form Performance is not independent of the rest of site. By adding aggressive Calls-to-Action, you will almost certainly cause a decline in funnel conversion. Whether that decline offsets the increase in form starts is left to measurement. I call the degree to which visitors are ready to enter the conversion funnel their pre-qualification rate. If you dump PPC or Ad Banner visitors directly into a Conversion Funnel, your pre-qualification rate is likely to very low. But as with almost any web statistic, no single pre-qualification rate is either inherently good or bad. To measure issues with pre-qualification, you need to track the rate of Form Success against a set of variables that track what happened before entering the Form. Some of the most commonly useful variables to look at include: # of pages viewed before entering the Form, # of visits before entering the Form, Time on Site before entering the Form, # of Pages and Time Spent on Convincer/Closer Pages before entering the Form (Functionalism provides a nice way to help you focus on how much time visitors spent on the pages that, presumably, matter) and the immediate page before entering the Form. As part of this analysis, I’ll typically carve out the group of visitors who enter the site on the Form and the group of visitors whose first site action (after Entry) is to go to the Conversion Process. All other visitors form the third distinct visitor segment. To measure pre-qualification, you should collect the form start and form completion rates by each of the potentially useful variables listed above for each segment. You’ll be looking for places where there are significant drives to the conversion process that have markedly better or worse rates of Form Completion. In general, you’ll see very much what you’d expect. The more stuff (and the more sales stuff) that visitors looked at, the higher their level of qualification. But since there is drop-off at each step in a web-site, this increasingly level of qualification needs to balanced against fewer form starts. Dramatically different levels of pre-qualification between relatively close steps often indicate places where you’ve been too aggressive in calling for the sale. On the other hand, where little difference exists between the various steps, you may not be using calls-to-action aggressively enough. Most sites will find that they have a group of visitors who arrive on the site “Ready to Buy.” This group will proceed directly to the conversion funnel (that’s why we always try to segment them into a separate group) – and it’s why it’s almost always a good idea to have reasonable direct access to conversion from your landing pages. If you don't segment out this group of visitors, you risk messing up the rest of the analysis. These visitors will make it appear as if the less content visitors view, the more effective the site. Most sites will also find that they have a segment of visitors willing to buy but needing some amount of convincing. These buyers need to be routed to the appropriate place on the site – and the Convincer pages need to contain a good balance of content and Calls-to-Action. For these pages, you’ll be looking to see if there is a clear sweet-spot (content that visitors must view to be pre-qualified - and the amount of time/pages before pre-qualification reaches a reasonable level). If you can find a sweet-spot, you’ll want to tune your site design to guide visitors to the right content in the right number of pages and THEN aggressively call for the sale. This means that the behavior of visitors in your conversion process can tell you a lot about how to structure your site effectively. Strong differences in pre-qualification rates (held constant by referring source) can help pinpoint the type and amount of content your “willing to buy” visitors need. That’s a vital piece of information in thinking about your overall site structure. By tuning the navigational flow of your site and the placement and nature of Calls-to-Action, you can significantly improve Form performance. Without ever touching your Form! Posted by Gary Angel 1/20/08 Go to Blog Titles. Abandonment: It’s Not All about the StepThe classic web analytics funnel analysis is simple indeed. The key metric is the percentage of times the process is abandoned on each step. The implicit assumption is that steps with the highest abandonment rates are the biggest problem. When you actually do this type of measurement, the most common finding is that the conversion funnel step abandonment rates look like a big U. The first step of a form process often has a high-abandonment rate, followed by a series of steps with small and relatively similar abandonment rates, concluding with a final confirmation step that also has a high-abandonment rate. Different design philosophies and implementations will produce different curves, but this is the pattern I’d describe as “classic.” Given this common pattern (or the existence of any common pattern other than an equal rates), is it meaningful to suggest that the step with the highest exit rate is reflective of a problem? Not really. That would only be true if the natural forms abandonment model was that each step had an equal chance of abandonment. Once you discard that assumption, it's obvious that no particular level of step abandonment is positive evidence that the step is somehow broken or worse than any other step. Thinking the issue through, it should also be obvious that no particular level (or even change) in step abandonment is necessarily evidence of a problem. This is a similar point to one I’ve made before many times about reporting. Just as no one Conversion Rate (or even an improvement in Conversion Rate) is positively good, a step abandonment rate must be viewed within a larger context. Let me give you an example. Suppose you have form process with four steps and you produce a report like this: Has the form gotten worse? Maybe, but perhaps not. Indeed, with no physical change in the process the real-world odds are heavily against such a conclusion. Far more likely is that the shape and quality of traffic into the form has changed for the worse. If you just started a major PPC effort, it may be that you are driving far more, and somewhat less qualified, traffic than before. Form performance will reflect that. This point is especially critical to understand if you change your conversion process AND change your site design at the same time. This combination may produce worse Step Conversion rates even with a Form that has been signficantly improved - a fact that is almost always missed or mis-interpreted. I believe this illustrates two critical points. First, a step abandonment rate taken as a single fact means nothing. Like so many individual metrics, it is dangerous and misleading when used as a single point of decision-making. Second, measured form performance is very dependent on exogenous factors. This should give pause to anyone who thinks that the basic process of measuring conversion processes is to find the steps with high abandonment and fix them! I’m going to step back for a moment and layout some more general principles for thinking about measuring conversion processes. It seems to me that in building up a system of measurement, we have to a bit of a theory about form-based processes on the web. Here are some basic rules (some purely theoretical some based on actual experience) I think might make up such a theory:
I believe these principles are sound, though probably not even close to exhaustive. Still, they produce some fairly straightforward analytic consequences:
It should be obvious from this that the basic measurement task in a conversion process is not to simply identify step completion rates. From the above discussion, it should be clear that a high abandonment rate doesn’t imply failure. On the other hand, it is a purely empirical point whether or not such steps are easier or more difficult to improve than lower friction steps. Based on our real-world experience, I’d argue that the weight of empirical evidence is, in fact, in the contrary direction. It’s often easier to improve conversion processes by focusing on relatively low-friction steps. A similar lesson applies to field-based abandonment. While the field focus (the place where the cursor was when the user left the Form) on abandonment is sometimes significant, it is quite often simply the first (or last) field on the form. Neither is particularly meaningful. So while looking at field-based abandonment can occasionally be helpful, it is hardly the slam-dunk analysis that people unpracticed in web analytics often expect. In this post, I focused mainly on the wrong way to approach conversion process analysis. In my next post, I’ll take up some of what I believe are the real analytic tasks around form measurement! Posted by Gary Angel 1/13/08 Go to Blog Titles. Form Abandonment – The Analysis of Key Web Site FunnelsIn my first post of the year, I talked about the importance of having a detailed project oriented plan around measurement. But what kind of projects actually go into a measurement plan? Obviously, the exact makeup of a plan is highly client-specific. In the past couple of months I’ve written up measurement projects as diverse as a study of how visitors interested in 2008 election stories could be engaged with a broader news site to an Analytic Report on the key drivers of Newsletter Subscriptions. But there are some projects that go into nearly every company’s plans. And one of the most ubiquitous is the study of key funnels on a site. For a long time, this supposedly simple analysis was THE bellwether web analytics project. It was the analysis that was most often used to justify web analytics and it was the analysis I most often heard vendors and consultants talk about when they trotted out examples of multi-million dollar returns. All of which is kind of unfortunate, because it’s also one of the more misunderstood analytic tasks. The common folk wisdom (as embodied in most expert "best practices") on how to think about and measure Funnel Abandonment is deeply flawed. Combine this with unrealistically high expectations (based on largely anecdotal opinion formed in years past when most site conversion funnels WERE really bad) and a strong tendency to do this analysis as a first project and you have an excellent recipe for failure. That being said, Funnel Analysis is important. It may the single most important piece of many governmental form-based sites. Funnel Analysis is a vital part of any eCommerce site. It is a central analysis for service-based Financial Services sites in Insurance, Banking and Brokerage. And even for content and community sites, there are nearly always key funnels that are fairly important in terms of overall site performance. So I thought I’d tackle a short, analysis focused series on the basics of measuring site funnels. I’ll begin the true analytic part of the series in my next post – when I’ll take a close look at Form Abandonment by Step – the classic conversion funnel analysis. Following that, I’m going to cover some of the key behavioral questions you can answer about key funnels and how to tackle each. That will probably cover three or even four posts since there is a fair amount of analytic work around these areas. Finally, I’ll cover Funnel Analysis KPIs (since this is an area where analysis should almost always drive to reporting) and also some special cases within conversion analysis. But before I dive into that, I think it’s necessary to give a bit of background on where web analytics fits in the overall process of creating and optimizing conversion processes on a site. This is definitely an area where web analytics is just one piece of a larger puzzle. There are at least three critical steps in creating good funnel processes on a site: developing and applying good design principles, doing real usability testing and, finally, doing behavioral analysis. I might even be inclined to add a fourth – doing CEM-based analysis driven by customer support and web analytic issues. Each of these is absolutely essential to building truly world-class conversion or form-based experiences. When you start any forms-based process, you’ll start with business requirements welded to good design principles. The truth is that both behavioral analysis and usability analysis work much better when a Form is relatively well-designed to begin with. It can be nearly impossible to converge on a highly optimized design when the basics of the design are a mess. Critics of multivariate testing often make the very legitimate point that testing may converge on a local optimum and miss much larger opportunities. That's a very fair point – and a poorly conceived overall design can make this ever so likely when it comes to funnel analysis. Your basic design needs to be responsive to questions like:
Once you’ve built your initial test process, the next essential ingredient is true usability testing. Generally, this involves turning loose test users who are asked to complete the specific processes and are closely monitored while doing so. Usability testing is in no respect redundant with the behavioral analysis you do in web analytics. It provides you with completely different types of information. First, usability testing has the huge of advantage that you can do it BEFORE rolling out a process to your real customers. That’s nice. Second, usability testing provides a much better view into potentially crippling navigational issues. You might discover these problems with web analysis (but you might not) – and, in any case, you’ve discovered them long after you really should have if you aren't doing usability tests. Because usability testing lets you study and discuss problems with testers, you can often immediately identify the source of difficulties and resolve them. This is by no means a slam dunk type of activity with web analytics. Usability testing focuses on answering questions like:
Usability testing of thoughtfully designed forms will pretty much insure that your funnel processes are solidly acceptable. So what do you need behavioral analysis for? Behavioral analysis (web analytics) can take your funnel processes one-step beyond solidly acceptable into the truly outstanding category. Because the behavioral analysis you get from web analytics generally focuses on what happens with REAL customers. The elements of re-assurance, persuasion, timing, and friction that go beyond basic design and are not captured in the artificial labs of testing can make a very big difference in actual performance. So the real questions of funnel analysis in web analytics tend to somewhat different than many people think – since the questions they often start out with should already have been answered in usability and design. Instead, I think questions like these tend to come to the fore:
In the next couple posts, I’ll talk about how to answer each of these behavioral questions with web measurement. How about the CEM tools I mentioned earlier – do they fit into this picture? We at Semphonic haven’t done much in this area (hoping to remedy that), but I think there is a fairly compelling case for CEM if your site has complicated conversion processes. CEM tools let analysts re-create sessions with considerably more surrounding detail – they are like a cross between usability tests and web analytics. Usability tests retain the advantage of occurring before rollout and of allowing conversation with the user. Analytics retains the advantage of looking at aggregate real behavior to identify trends and problems. But CEM tools – especially when driven by problems discovered in web analytics or in customer support – can provide analysts with the unique ability to re-experience real-world problem sessions. Particularly for more complex funnel processes, this might be a critical fourth leg on which to stand. Put these four steps together, and you will definitely be on the road to having world-class conversion funnels on your site. I think that’s enough background for now. I must say, I'm excited to be tackling a true deep-dive analytics topic again. I haven’t done that since the massive SEM Analytics series I wrapped up last fall. There’s a surprising amount of interesting analysis around conversion funnels and I’m looking forward to this series! In the next post, I’ll take a look at Form Abandonment and talk about why I think it is so frequently misunderstood. Posted by Gary Angel 1/06/08 Go to Blog Titles. |
|
 |
|
 |
 |
| Site Map | Privacy Policy | Email us or Call 1-800-763-2821 | Copyright © 2008 Semphonic All rights reserved. |