Audrey Rasizer's Blog

A self-proclaimed geek and expert in statistics, Audrey Rasizer is a senior analyst for Autonomy Interwoven and specializes in Website optimization-analytics. She joined Autonomy Interwoven as part of the Optimost acquisition in 2007, and has been a key member of the team for almost four years. Audrey’s blog will focus on helping marketers understand the basics of multivariable testing. She will share some interesting insights that she’s come across while trying to optimize the world. Audrey works in the London Autonomy office, and enjoys dancing and eating candy corn. She graduated from Cornell University with a degree in Psychology. Connect with Audrey on LinkedIn.

Orange you glad you’re testing?

April 22, 2009 @ 9:45 am by Audrey Rasizer

Shortly after the release of the new design, a mob of irritated, angry oj drinkers swarmed the internet to fight the rebranding. This eventually led Tropicana to revert the cartons back to their old look. I hate to say it but the Tropicana re-branding decision was probably a result of the classic HiPPO phenomenon (the Highest Paid Person’s Opinion… See Seth Rosenblatt’s Blog on “Don’t Just Watch the Hippo Swim Faster“). If the new logo had tested before the release, then the deep emotional bonds people held with their orange juice cartons would have become apparent, and disaster never would have occurred.

 

No Comments | Category: Multivariable Optimization

Recession-proof your website! 3 Quick and Easy Tips

March 18, 2009 @ 12:05 am by Audrey Rasizer

Times are tough, but don’t lose hope! Here are three quick and easy tips for optimizing your site during the recession:

 

• 1. Keep the font simple and easy to read. The other day, a colleague of mine passed around an interesting article from Scientific American. In a study at the University of Michigan, all students were given the same instructions for a regular exercise routine. In one group, they used this simple font (Arial font). In the other group, they used this fancy font (Brush font). Those who were given the instructions in the easy to read font were much more open to the prospect of exercising, and were more willing to make exercise part of their day. Moral of the story: use simple font. People will be more likely to do what you want them to do because they will think it’s less effortful

 

• 2. Emphasize price cuts, sales, free shipping, etc. This is a no-brainer, but a lot of times websites have so much information and details about the product that the excellent price or free shipping deals are being overshadowed. In fact, during a recent experiment on a product page, markedly more people were adding items to their basket when the info links and the ‘read reviews’ links were removed from the product description.

 

• 3. Use familiar concepts. There’s a reason why chocolate manufacturers have reported (sugar) high profits during the recession. People turn to familiar vices in times of stress. Center headline messaging about something that hits home. Don’t throw extraneous and often confusing language at potential customers. Rather, just present the familiar offer and make the call-to-action button prominent.

No Comments | Category: Online Marketing

Yes We Can… Optimize Email Campaigns

February 12, 2009 @ 2:16 pm by Audrey Rasizer

With budgets being cut left and right, it is no surprise that email campaigns will be at the forefront of every marketer’s mind during the economic downfall. Email campaigns, especially to the house list, are cheap and effective, having one of the highest returns on investment.

According to a survey done by Marketing Sherpa in partnership with Ad:Tech and Babcock & Jenkins, of  247 Marketing Organizations with over $250MM in revenue, email was found to be the key player in generating and nurturing leads.

Here is a chart by Marketing Sherpa…

But first a little explanation of the chart:

• The bubbles show the relative importance of specific marketing tactics in lead generation and nurturing.
• The bubble size represents the percentage of marketers who use each tactic.
• The x-axis shows the percentage of marketers who said a tactic makes a major contribution to lead generation.
• The y-axis shows the percentage of markets who said a tactic makes a major contribution to lead nurturing.

 

Notice how House Email stands alone in the upper right portion of the graph. This means marketers say that it has made the highest contribution to both lead generation and marketing.

Give Your Email Campaigns a Stimulus Plan: Optimize them.

Running multi-variate experiments on email campaigns prove to be very lucrative. Just recently we ran an experiment on an email campaign for a large bank. The goal of the email was to get people to extend their commitment with the bank-a very important goal in these economic times. More than eight aspects of the email were tested including imagery, headline messaging, body copy, link position, and submit button. Amongst other things, shortening the body copy, revamping the submit button and changing the main imagery contributed to increased clickthru activity by 90.7%. Needless to say, this was a grand slam for the bank. 

Conclusion

Email campaigns are good, but optimized email campaigns will be the best investment you make, especially given the state of current world finances.

No Comments | Category: Multivariable Optimization | Online Marketing

Is the Price Right? The Importance of Price Point Testing

January 7, 2009 @ 1:36 pm by Audrey Rasizer

Do you know how to price your products or services to generate the highest revenues? Are you overcharging/undercharging? You will never know unless you test.

In a recent study at CalTech, subjects were asked to sample 5 different bottles of Cabernet Sauvignon and rate their taste preferences.  However in reality, they were presented just 3 wines, since 2 of the wines were shown twice, once with the true price, and again with a fake price.  They presented a $5 wine as costing $5 and $45, as well as a $90 bottle costing $90 and $10. The remaining bottle had an accurate price.  Subjects consistently give higher ratings to the more ‘expensive’ wines.  Brain scans actually showed greater neural activity in the medial orbitofrontal cortex-  the area of the brain,  believed to encode pleasure from, when presented with the more ‘expensive’ wines. The study suggests that people may actually experience identical products as being better by just sticking a higher price on it. In other words, the price of an item can affect the way we value it.

What does this mean for you?

Determining the right price for your product is both an art and a science.

A person’s perception of the product will determine the ‘right’ price, but also the price plays a part in determining how the product is perceived. It is a causal loop. Inside the loop lies the most lucrative price point. The only way to find it is to test.

Recent Testing

Recent testing proved very profitable for America’s Test Kitchen. Their reason for testing was two-fold. Firstly, they were interested in knowing the best pricing point for both their one year and monthly subscription options. Secondly, they were curious to see whether removing the monthly subscription option would generate more revenue by forcing interested prospects to opt only for the one year subscription. This would also reduce future costs on sending out renewal forms and such.

It was found that raising the subscription prices up by almost $10 had a significantly positive impact on revenue per visitor. Considering the lifetime value of a subscriber, this is particularly true in this case since the subscription could potentially continue for years.

Bottom Line: Your intuition is wrong

Intuition fails when trying to evaluate the optimal price point for your product, subscription, or services. Every business is unique and testing is the only way to determine the ideal pricing structure that will generate the most revenue for your business.

1 comment | Category: Multivariable Optimization | Online Marketing

Top 5 Trends for Optimization in 2009

December 4, 2008 @ 11:32 am by Audrey Rasizer

2008 may not have been the year for improving the value of your home.  But it was the year for improving your website, as website optimization became an integral part of online marketing for many leading online destinations.  The website optimization momentum will continue with full force in 2009.

Here are the top trends for optimizing your website for 2009.

1) Adaptive Targeting

People visit websites for all sorts of reasons and enter through many different paths. It is about getting the right message to the right person at the right time. This is what targeting is all about. The idea has been around for some time now, but the problem with most website targeting solutions is their inflexible approach. The set of creative alternatives has always been limited, as has been the data collected about the customer.  That limit has finally been lifted. Website targeting has evolved to what is now known as Adaptive Targeting.  Adaptive Targeting uses multivariate testing in conjunction with a range of new, more enhanced targeting techniques and data types, to discover new segments fast and test unlimited combinations of content. It mines all major categories of visitor attributes: context, geography, time, demographic, behavioral, visitor entry, and profile. For example, a New York handbag store with a special weekend promotion could deliver targeted headline messages specifically to females aged 30-35 years, with an income level of  $150k+,  living in Manhattan, who typed in “Gucci” on google.com and clicked on an organic link on Friday evening. Wow!

2) Mobile Device Targeted Content Delivery

Partly due to the release of Apple Inc’s iphone, the smartphone market is skyrocketing. It is only a matter of time before every cell phone will be a smartphone, having PC-like functionality. While people use their smartphones for a wide variety of purposes, from banking to watching videos, the most frequently used function is browsing the internet. This opens up the door for mobile device targeting. 2009 will be the year in which mobile device content delivery gets an extra close look.  Making sure your company’s website is available and optimized on all formats is a key component to success. Remember that a person using their phone to go to an airline site is looking for quick flight information, not looking to book a vacation three months from now. It is important to deliver the appropriate content to them.

3) Sales Funnel Optimization

Fundamental to the success of all e-commerce sites is perfecting the sales funnel process. Beginning by capturing the attention of prospects and then effectively converting as many as possible into buyers has always been a huge challenge. But how many steps should this funnel be? Is it better to have 3 shorter form pages or 1 longer form? Advanced testing technology in 2009 brings us a simple, easy way to test this and find the optimal funnel that will convert as many visitors as possible.

4) AJAX Applications and Widgets Optimization

In 2009, wise marketers will capitalize on the continual rise of AJAX applications and embedded flash objects (known as ‘widgets’). Both of these increase responsiveness and interactivity of Web pages, while also providing unprecedented access to unique target audiences. When top social networking site Facebook.com opened the gates to third-party widget developers last year, it started a widget avalanche.  Making sure your AJAX application or widget is optimized for speed, functionality, and usability will be huge factors in your success and making your application stand out from the crowd.

5) Optimizing Customer Engagement

Nearly 123 million people in the US viewed 7.2 billion videos online in one month as reported by ComScore. With broadband penetrating the world and more people than ever watching videos online, it is essential to optimize on customer engagement.  Doing so means making more money through ads. New 2009 testing techniques allow us to easily optimize on key performance indicators that measure customer engagement such as time spent watching a video, time spent on a webpage, or page views. More customer engagement means more ad visibility, and this will result in more money in your pocket.

So there you have it-the top trends for optimization of 2009.  It’s imperative to keep up with the optimization momentum and constantly improve your website. If not, your home might not be the only thing losing value!

2 comments | Category: Mobility | Multivariable Optimization | Online Marketing | Targeting & Engagement

What’s the time, Mr. Wolf?

November 5, 2008 @ 3:08 pm by Audrey Rasizer

Ever wonder how to run a simple multivariate experiment on your way to work?

Well, here’s how:

Step 1:  Identify your Control.

The control will be you asking, ‘What’s the time?’ (with a neutral facial expression) to strangers on the way to work.

Step 2:  Identify your Key Performance Indicator and Goal.

Key Performance Indicator will be the percentage of people who give you the correct time (out of the people asked).

The goal will be to increase the KPI by manipulating three variables and finding the most effective combination.

Step 3: Identify your Variables and Values.

The variables manipulated are: Politeness, Urgency, and Facial expression.

The values are listed under each variable below. For example, variable A_Politeness, value 1 puts ‘Excuse me’ in front of the control version, hence it would be ‘Excuse me. What’s the time?’. Variable B_Urgency, Value 1 adds ‘I need to know in under 15 seconds’ after the control making it ‘What’s the time? I need to know in under 15 seconds’. Variable C_Facial Expression just manipulates facial expression. When Value 1 is used, you must put on a big happy smile. When Value 2 is used, make a sad frown. 

Step 4: Make an orthogonal, balanced, multivariable Design Martrix.

We are doing a full factorial design. Here are all the possible combinations of the above variables and values:

 

 

Step 5: Implement your Experiment

Over the course of the next few days on your way to work, start asking strangers the above combinations. Record how many people you’ve asked (A) and how many people gave you the correct time (B). Divide B by A and multiply by 100. This will give you the Unique Action Rate of People Answered (Your KPI).

Note: Remember to ask the strangers all the combinations randomly and evenly. In other words, the same amount of strangers should hear Combination #1 as Combination #10. (Don’t skip combinations!). Also, ask enough people for the results to be statistically significant. 

Step 6: Analyze your Results

Perform in-depth multiple regression analysis. Identify the impact for each value, the confidence level and the interval.

See which combination gave you the highest response rate. Check to see if that combination is supported by your analysis. Pick your winner.

Advanced analysis:

Try day-parting and week-parting. See how time of day and day of week impact response. Perhaps one combination works best in the mornings when people aren’t so friendly, but another one is optimal during lunch hour when people tend to be more relaxed.

Try Persona Targeting. See if one combination works better on adults vs children or male vs. female.

The possibilities for testing are endless! Don’t be afraid to let your imagination run wild. You will be surprised to discover what things work and don’t work.

4 comments | Category: Multivariable Optimization | Online Marketing | Targeting & Engagement

I’d Like Some Ice Cream with a Little Optimization on Top Please

September 30, 2008 @ 1:15 pm by Audrey Rasizer

Ice cream is one of my favorite foods. I eat a lot of it. I don’t buy the tanks of Edy’s from the grocery, but rather, I like going to the ice cream shop and getting a cone.  Maybe it brings back childhood memories and I get all warm and fuzzy inside. I’m not sure. Anyway, there happens to be an infamous Tasti D-lite just steps away from my Manhattan apt that I like to frequent. I go in. I tell the guy what kind of ice cream I want on my cone.  He makes it and gives it to me and then asks me for money. Read more

4 comments | Category: Multivariable Optimization | Online Marketing

The Dating Game And Prehistoric A/B Testing

August 26, 2008 @ 5:00 am by Audrey Rasizer

She’s beautiful. He’s standing close to her at the bar. They make eye contact. He smiles. Conversation starts about the weather. After 10 minutes of light chatting, he asks for her number. She says ‘no’, turns, and flees. He’s unsuccessful. Read more

5 comments | Category: Multivariable Optimization | Online Marketing | Web Content Management