Shogun ‑ AB Testing for Shopify

Shogun ‑ AB Testing

A/B testing for pages, themes, templates, price testing & more
Developer
Shogun Labs, Inc.
View on App Store
Shogun ‑ AB Testing
Rating
4.1
Feedback
37
Reviews
Chart
#1,870
Shopify
Reviews over time
New reviews per month across the app's lifetime.
App ranking
Overall position in the Shopify App Store.
Reviews summary

The reviews for the Shopify App are mixed, with both positive and negative feedback. Positive aspects include easy A/B testing, great customer support, and helpful features like pricing testing and new landing page testing. However, negative themes include issues with theme publication, uneven traffic allocation, bugs in A/B testing, and problems with product detail testing. Some users experienced disruptions to their store themes and order losses due to these issues.

Time spent using app
How long merchants keep the app installed.
Less than an hour 0%
Less than a day 5%
Less than a month 49%
Less than a year 46%
More than a year 0%
All reviews
1 / 5 Share
July 7, 2026

WORST APP EVER - MY PRODUCT PHOTOS LAGGED AND DID NOT WORK / THE PRICE EVEN AFTER UN INSTALLING KEPT CHANGING . Subject: Re: Toothy - pricing test issue Hi Sweetan, Thanks for looking into this. Before reinstalling or sharing collaborator access, I want to share what I've found on my end, because I think it points to the actual cause. After uninstalling Shogun, I found that Shogun had written code directly into my theme files rather than only using app embeds/blocks. Specifically: 1. A snippet called `snippets/product-media-gallery.liquid` had Shogun-authored Liquid code reading from metafields under the `app--214825828353` namespace (`product-pricing`, `product-variant-pricing`, `test-media-1` through `test-media-4`). This code used `{% paginate %}` on values that aren't valid paginateable objects, which caused a "Liquid error: Array is not paginateable" on my live product pages. 2. Separately, I found a file called `snippets/shogun-optimizer.liquid`, labeled in its own comments as "Auto-generated by Shogun-AB." This file contains embedded, minified JavaScript that appears to run content-type detection across templates and includes a `cartCreate` GraphQL mutation. This file was still present and apparently still active on my storefront after I had already uninstalled the app. 3. The pricing metafields under `app--214825828353` also still exist on my products, since metafields persist independently of whether the app is installed. This is very likely why the price kept changing even after uninstall — the leftover Liquid/JS was still reading that stored data. Given this, before I reinstall anything or share collaborator access, I'd like to understand: - Does reinstalling the app actually remove or replace `shogun-optimizer.liquid` and the leftover metafields, or does it just resume using them? - Is there a clean uninstall process that removes all injected snippets and metafields, rather than leaving them orphaned in the theme? - Can you confirm whether the `cartCreate` mutation in that script ever affects the price actually charged at checkout, separate from what's displayed on the page? I want to be sure no customers were charged incorrectly during the test. I'd rather resolve this by removing the leftover code and metafields directly, since I already have a good picture of what was left behind. If there's a specific reason full collaborator access is necessary beyond what I've described above, let me know and I'm happy to discuss it further. Thanks,

Store Using app Total reviews Average rating
Toothy About 2 months 1 1.0
Developer Reply

---UPDATE - July 13, 2026--- Hey there, Hamad. I just wanted to check if you received the last update and if that has helped. I am also reaching out via the email thread, and we'd appreciate it if you could reply back to us. Thanks for your patience and understanding. Sweetan | Client Services Shogun ----------------------------------------------- ---UPDATE - July 8, 2026--- Hi Hamad, Thank you again for taking the time to investigate this so thoroughly. We shared your findings with our engineering team, and I wanted to address each of the points you raised. Regarding `snippets/shogun-optimizer.liquid`, you're correct that this file can remain in the theme after the app is uninstalled. Once the app is uninstalled, Shopify immediately invalidates our API tokens, which means we're no longer able to modify the theme or remove files as part of a cleanup process. That said, the optimizer includes a safeguard where, if it hasn't received a refreshed configuration for approximately 24 hours, it automatically stands down. This is intended to prevent visitors from continuing to receive experiments after the app has been uninstalled. With respect to `snippets/product-media-gallery.liquid`, this is an existing theme snippet that's responsible for rendering your product image gallery. During an AI Theme Modification, this is one of the files that is commonly updated so that Shogun can swap in the appropriate media for products participating in a test. The behavior you described- reading the media metafields and displaying the corresponding images for the active variation is expected. You also asked whether a syntax error could have been introduced. While our AI-generated modifications go through validation, no automated process is perfect, and it's possible for an issue to slip through in certain scenarios. This is why we recommend reviewing the modified theme and verifying that key pages are functioning as expected before publishing it. We'd be happy to review your theme with you or help rerun the theme modification if necessary to check this better. Regarding the product metafields, our expectation is generally that Shopify manages app-owned data on uninstall. However, we've seen reports from other developers that, in some cases, app-owned metafields can remain after an app is removed due to Shopify platform behavior. To answer your remaining questions: * Reinstalling the app **does not** automatically restart or resume paused tests. It performs the necessary setup, but any previously paused optimization remains paused unless you explicitly start it again. * The `cartCreate` mutation does **not** modify the price charged at checkout. It simply passes the selected variant to the cart. It has no effect on the final checkout pricing. * The cart transform function is managed by Shopify and is automatically removed when the app is uninstalled, so it is no longer active after uninstall. It is possible you were seeing the cached version on your end. We really appreciate the effort you've put into documenting what you found. Collaborator access would still be valuable because it would allow our engineering team to inspect the current state of your theme and configuration directly, verify whether there are any lingering issues specific to your store, and determine whether there's anything we can improve on our side. If you're open to it, we'd be happy to continue the investigation together. Thanks again for your detailed report, and I look forward to hearing from you. Sweetan Shogun Support ------------------------------------------- Hi Hamad, we're sorry to hear about your experience. I see you reinstalled our app on July 2, ran the test for a few hours, stopped the test, and then uninstalled the app. We weren't able to locate any support conversations under the email address associated with the account. If you contacted us using a different email address, please let us know; we'd be happy to look into that correspondence. That should help us to analyze this better for you. Regarding the pricing, when a pricing test is started, paused, or stopped, Shogun updates the related data to ensure pricing remains consistent across your storefront. That said, we'd like to investigate your specific experience thoroughly to understand what happened. I've reached out to you via email to gather a few additional details, and I'll also be sending a Shopify Collaborator access request so our technical team can investigate. We'd really appreciate your reply so we can get to the bottom of this and work toward a resolution. Thanks. Sweetan | Client Services Shogun

5 / 5 Share
June 25, 2026

Store Using app Total reviews Average rating
Sistaco™ USA About 1 year 1 5.0
Developer Reply

Hey there, thank you so much for taking the time to leave us a review. Please let us know if there is anything we can help you with. Sweetan | Client Services Shogun

1 / 5 Share
June 23, 2026

it f*cked up a live theme. Now I have to figure out what to do in the middle of the day while traffic enter the broken theme.

Store Using app Total reviews Average rating
Overlott About 2 hours 1 1.0
Developer Reply

Hey there, Kevin. We are sorry to hear about the issues that have led to this experience and review of our app. We want to understand the issue better so that we can analyze and help you better. When we checked the internal logs, we noticed that our AB testing app was installed and the Pro plan's trial was selected. You then upgraded the free trial to our Unlimited plan in order to run the Price test. You ran the theme modification, which is required to run a price test. It looks like the theme modification was completed correctly. We then notice that you left a review, and then deleted the test that you were working on. Could you please help us to understand what issue you were noticing on the live theme? I also don't see any support outreach, as our team is more than happy to help. Our product team has reached out to you via email, and I am reaching out in a separate email thread to gather some additional information and gain some clarity so we can help. We would greatly appreciate it if you could reply to us. Thanks for your support and understanding Sweetan | Client Services Shogun

4 / 5 Share
June 12, 2026

Store Using app Total reviews Average rating
Rituera 25 days 1 4.0
Developer Reply

Hi there, thanks for sharing your feedback and leaving us a review. We would love to hear further about the experience and what we can do differently. Thanks. Sweetan | Client Services Shogun

1 / 5 Share
June 3, 2026

Store Using app Total reviews Average rating
TallacLiquors 5 months 1 1.0
Developer Reply

Hey there, Manjit. I apologize for any inconvenience that has led you to leave this review. I wanted to check if there was an issue that you were facing with our AB app. I see our app was uninstalled back in April, April 30, and I also could not find any support conversations under your email address, which would help to understand any issues. This review was left today, a month after the app was uninstalled, and we wonder what went wrong. I was checking the logs and I could not see any AB tests that were set. Could you please help us with the issue that you are/were facing? I am reaching out via email to understand this further so that we can assist you. We will look forward to hearing back from you. Sweetan | Client Services Shogun

Sign up to receive AppNavigator updates

Find out when we add keyword-monitoring, review RSS feeds and more.