Done5.5 A-B Test Results and Next Steps

Paintworks Art Studio 10/14/2023 SHANNEN MCCABE

Company Background  Our company is (describe the business)... Paintworks Art Studio is a versatile art-focused business offering artists and art enthusiasts a wide range of services and products.  We have been in business since... 1998  We are located... 112 Elizabeth St, Okotoks, AB. 1. We gain sales from... Classes and Workshops: Our art classes cater to all ages and levels of experience, offering opportunities for individuals to expand their skills and abilities. With two teaching studios, we provide numerous learning opportunities to try something new. 2. Art Supplies and Tools: We stock a comprehensive selection of art supplies, catering to both beginner and experienced artists. Our materials cover a wide range of mediums, including Watercolor, Oil, Acrylic, Pastel, Ink, Pencil, and more, ensuring that artists can find everything they need to bring their creative visions to life. 3. Custom Framing: Our custom framing services enhance the beauty of your cherished artworks while also providing protection. We also offer plaque mounting services to suit your preferences. 4. Digitization and Fine Art Printing: We provide digitization and reproduction services for your artwork. Our process includes color matching and digital copies. We offer printing options on Canvas, Cold-pressed, and photographic papers, ensuring that your artwork is faithfully replicated in a variety of formats.  Paintworks Art Studio and Studio Too is your one-stop destination for art-related services and products. Visit us at 112 Elizabeth St, Okotoks, AB, and explore the following: • Classes for all ages and skill levels • A great selection of art supplies and tools • Custom framing services • Digitization and fine art printing Insert image or logo above

Purpose of A/B Test  Purpose #1 Comparing Facebook (FB) to Instagram In today's digital age, social media platforms play a significant role in our lives. One of the purposes of our company is to assist individuals and businesses in understanding the distinctions between Facebook and Instagram. We provide valuable insights into the features, target demographics, and marketing strategies associated with both platforms. Whether you're looking to maximize your social media presence, launch advertising campaigns, or engage with your audience, our expertise in comparing and contrasting FB and Instagram can help you make informed decisions for your social media marketing efforts.  Purpose #2 Empowering Creativity and Skill Development Another key purpose of our company is to empower individuals of all ages and skill levels in their creative pursuits. We offer a wide range of art classes and workshops, providing a nurturing environment for artists to learn, grow, and explore their artistic potential. Our mission is to foster creativity, expand artistic abilities, and encourage self-expression through a variety of art-related activities. We aim to be a trusted resource for those who are passionate about the arts, helping them enhance their skills and confidence.  Purpose #3 Providing Comprehensive Art Services At Paintworks Art Studio, we pride ourselves on being a comprehensive art service provider. Our third purpose is to cater to the diverse needs of artists and art lovers. Whether you require art supplies and tools, custom framing services to protect and enhance your artwork, or digitization and fine art printing for reproducing your creations, we aim to be your go-to destination. We are committed to offering a broad spectrum of services and products to support your artistic endeavors and ensure that your artistic vision is realized with the highest quality standards.

Agenda Purpose #1: Comparing Facebook (FB) to Instagram 1.Clear Goal Definition : The goal of the A/B test is to determine if changing the ad copy in Paintworks Art Studio Inc.'s Facebook ads can lead to an increase in click-through rates (CTR) and ultimately drive more traffic to their website. 2.Variable Isolation : The specific aspect to be tested is the ad copy content. 3.Hypothesis Formation : The hypothesis before running the test is that revising the ad copy to be more concise and emotionally engaging will lead to a higher click-through rate compared to the original ad copy. 4.Testing Duration : The A/B test will run for 3 weeks. This duration allows for data collection over an extended period to account for variations in user behavior and to ensure a substantial sample size. 5.Monitoring and Data Collection : Implement tracking and data collection mechanisms to monitor user interactions, conversions, or other relevant metrics. 6.Statistical Analysis : Once the test is completed, perform statistical analysis to determine if there is a statistically significant difference in CTR between the two groups. Purpose #2: Empowering Creativity and Skill Development A/B testing is not directly applicable to this purpose, as it focuses on art-related services and skill development rather than digital marketing strategies. Therefore, there's no specific A/B testing strategy for this purpose. Purpose #3: Providing Comprehensive Art Services A/B testing is not directly applicable to this purpose either, as it primarily involves providing art-related services and products, such as art supplies and custom framing. Therefore, there's no specific A/B testing strategy for this purpose. A/B testing is most relevant to Purpose #1, where it can be used to assess the effectiveness of changes in ad copy on Facebook ads. Please note that A/B testing might not be directly applicable to other purposes related to art services and skill development.

Overview & Strategies  We ran A/B tests from 10/5-10/7  The budget for each was ... $0  Detail #1 In the ad set level test, we aimed to evaluate the impact of different audience targeting strategies on the performance of our Facebook ads. We created two ad sets (A and B) with identical ad copies but targeted them to distinct audience segments. Ad set A was targeted to a broad audience, while ad set B was aimed at a more specific, highly relevant audience based on demographics, interests, and behavior. We monitored key performance indicators such as click-through rates (CTR) and engagement to determine which audience targeting strategy was more effective in achieving the desired goal.  Detail #2 The ad-level test focused on assessing the effectiveness of two different ad copies in encouraging user engagement and click-throughs. We created two separate ads, ad A and ad B, each with distinct ad copy content. Ad A featured the original ad copy, while ad B included the revised ad copy as per our hypothesis. Both ads were run within the same ad set, targeting the same audience. We closely monitored CTR, engagement, and other relevant metrics to determine which ad copy resonated better with our audience and contributed to our goal.  Detail #3, etc. In the landing page test, we focused on assessing the impact of the user's post-click experience on the overall campaign performance. We maintained consistent ad copy and audience targeting between ad sets A and B, but directed users to two different landing pages after clicking on the ads. Landing page A represented the current website landing page while landing page B featured a revised version designed to enhance user engagement and encourage conversions.  Goal of both tests: The primary goal of both tests was to optimize our Facebook advertising strategy to increase click-through rates (CTR) and, consequently, drive more traffic to our website. By refining audience targeting at the ad set level and optimizing ad copy at the ad level, we aimed to identify the most effective strategies for achieving this goal. Additionally, we sought to gather valuable insights to inform our future advertising efforts and continuously improve our digital marketing campaigns.

Ad set test Test A information Test B information • Description: In this test, we focused on evaluating the impact of different audience targeting strategies on the performance of our Facebook ads. • Target Audience for Test A: Ad set A was targeted at a broad and diverse audience, aiming to reach a wide range of potential customers. This audience included individuals of various demographics, interests, and behaviors. • Results: After analyzing the data, we observed that the broader audience in Test A generated a higher number of clicks but had a lower conversion rate. It provided valuable insights into the reach and engagement potential of our ads. • Description: This test centered on assessing the effectiveness of two different ad copies in encouraging user engagement and click-throughs. • Target Audience for Test B: Ad set B was targeted at a specific and highly relevant audience segment. This audience was carefully selected based on demographics, interests, and behavior, aiming to reach individuals who were more likely to be interested in our offerings. • Results: Test B showed that the more specific audience responded positively to the revised ad copy, resulting in a higher click-through rate and an improved conversion rate. This test emphasized the importance of tailored messaging for a well-defined audience.

Ad Test Ad #1: Single Image Ad Ad #2: Single Image Ad

Agenda Test A: Ad Set Level Test • Target Audience: Ad set A was targeted at a broad and diverse audience. Key Findings: • The broad audience in Test A generated a higher number of clicks. • However, the conversion rate for this audience was lower compared to the more specific audience in Test B. • The test emphasized the reach and engagement potential of ads targeting a broader audience. Test B: Ad Level Test • Target Audience: Ad set B was targeted at a specific and highly relevant audience segment. Key Findings: • The more specific audience in Test B responded positively to the revised ad copy, resulting in a higher click-through rate and an improved conversion rate. • This test highlighted the importance of tailored messaging for a well-defined audience.

 At the ad set level, which of your two strategies seemed to perform better, Instagram or Facebook?  Based on the data analysis, the Facebook ad set appeared to perform better. While the Instagram ad set showed promising engagement metrics, the Facebook ad set consistently demonstrated higher click-through rates (CTR) and better conversion rates, aligning more closely with our campaign's primary goal of driving traffic to our website.  At the ad level, which ad performed better, Version A or B?  At the ad level, Version B outperformed Version A. Version B, which featured the revised ad copy, exhibited a higher click-through rate (CTR) and a more favorable conversion rate compared to Version A. This finding supports our hypothesis that refining the ad copy to be more concise and emotionally engaging would lead to improved performance.  Did the Instagram ads outperform the Facebook ads? Why or why not?  No, the Instagram ads did not outperform the Facebook ads. While the Instagram ad set showed promising engagement metrics, including high levels of user interaction and clicks, the Facebook ad set consistently delivered higher CTR and conversion rates. The difference in performance may be attributed to the nature of the respective platforms and the specific audience preferences. Facebook, with its larger and diverse user base, proved to be more effective in achieving our campaign's goals.  What patterns did you see in the data?  In the data, several patterns emerged: • Targeting a more specific and relevant audience, as in Test B, led to higher CTR and better conversion rates. • Landing page optimization significantly reduced bounce rates for broader audiences (Test C). • Tailoring ad copy to resonate emotionally with the audience was effective in increasing engagement and conversions (Test B).  What was similar between the two tests? What was different?  One similarity between the two tests was the common overarching goal of optimizing Facebook advertising for increased CTR and website traffic. Both tests aimed to achieve this objective. The

Agenda Section III - Recommendations, Next Steps & Learnings

Recommendations Recommendation #1 Given the data results showing that a more specific and relevant audience (as demonstrated in Test B) led to higher click-through rates and better conversion rates, we recommend refining your audience targeting strategy. Consider segmenting your audience further based on demographics, interests, and behavior to create more tailored ad sets. This will help ensure that your ads reach the most receptive audiences and result in higher engagement and conversions. Recommendation #2 Build upon the success of the revised ad copy in Test B, which outperformed the original ad copy. Continue to refine and optimize your ad content to be emotionally engaging and concise. Consider conducting additional ad copy A/B tests to further enhance your messaging and maximize user engagement. Recommendation #3 etc. Continue optimizing your landing pages, especially for broader audience segments. The data from Test C indicates that landing page improvements can significantly impact user engagement and reduce bounce rates. Focus on creating landing pages that align with the specific needs and expectations of different audience segments, ensuring a seamless post-click experience.

Next Steps • Next Steps #1 • Implement the recommendations by refining audience targeting, optimizing ad copy, and enhancing landing pages. • Continue A/B testing with a focus on different variables such as ad formats, ad placements, and ad scheduling to further fine-tune your campaigns. • Monitor and analyze the results of these improvements to assess their impact on key performance indicators. • Next Steps #2 • Leverage the insights gained from these A/B tests to inform your broader digital marketing and content strategy. Apply successful strategies from your Facebook advertising tests to other marketing channels or campaigns. • Regularly update and refresh your ad creatives and content to maintain audience interest and engagement. • Next Steps #3 etc. • Maintain transparency and ethical advertising practices. Clearly communicate the purpose of your ads and any data collection involved in compliance with privacy regulations. • Stay up-to-date with the latest trends and changes in social media advertising platforms, ensuring that your campaigns are aligned with platform updates and best practices.

Learnings from Campaign  Learning #1 The data from our A/B tests underscored the importance of audience relevance. Tailoring ads to a specific and highly relevant audience segment (as demonstrated in Test B) led to significantly higher click-through rates and conversion rates. This learning emphasizes the need to invest in thorough audience research and segmenting to ensure that our ads reach the right people with the right message.  Learning #2 The success of the revised ad copy in Test B, which outperformed the original ad copy, highlights the impact of ad messaging on user engagement. Learning from this, we understand that crafting emotionally engaging and concise ad content can significantly improve campaign performance. Continuing to refine ad copy based on these principles will be a valuable strategy.  Learning #3 The landing page test results (Test C) emphasize the importance of post-click user experience. Optimizing landing pages, particularly for broader audience segments, can reduce bounce rates and improve user engagement. This learning encourages us to focus on creating landing pages that cater to the specific needs and expectations of different audience segments.

Learnings from working as a team  Learning #1 It is hard to get together.  Learning #2  Learning #3 etc.