{"id":39,"date":"2026-05-13T16:00:00","date_gmt":"2026-05-13T14:00:00","guid":{"rendered":"http:\/\/appswikidroid.win\/index.php\/2026\/05\/13\/wptavern-216-matt-schwartz-on-exploring-ais-impact-in-wordpress-agencies-part-2\/"},"modified":"2026-05-13T16:00:00","modified_gmt":"2026-05-13T14:00:00","slug":"wptavern-216-matt-schwartz-on-exploring-ais-impact-in-wordpress-agencies-part-2","status":"publish","type":"post","link":"http:\/\/appswikidroid.win\/index.php\/2026\/05\/13\/wptavern-216-matt-schwartz-on-exploring-ais-impact-in-wordpress-agencies-part-2\/","title":{"rendered":"WPTavern: #216 \u2013 Matt Schwartz on Exploring AI\u2019s Impact in WordPress Agencies (Part 2)"},"content":{"rendered":"<details>Transcript<\/p>\n<div>\n<p class=\"wp-block-paragraph\">[00:00:19] <strong>Nathan Wrigley:<\/strong> Welcome to the Jukebox podcast from WP Tavern. My name is Nathan Wrigley.<\/p>\n<p class=\"wp-block-paragraph\">Jukebox is a podcast which is dedicated to all things WordPress. The people, the events, the plugins, the blocks, the themes, and in this case a second look at exploring AI\u2019s impact in WordPress agencies.<\/p>\n<p class=\"wp-block-paragraph\">If you\u2019d like to subscribe to the podcast, you can do that by searching for WP Tavern in your podcast player of choice, or by going to wptavern.com\/feed\/podcast, and you can copy that URL into most podcast players.<\/p>\n<p class=\"wp-block-paragraph\">If you have a topic that you\u2019d like us to feature on the podcast, I\u2019m keen to hear from you and hopefully get you, or your idea, featured on the show. Head to wptavern.com\/contact\/jukebox, and use the form there.<\/p>\n<p class=\"wp-block-paragraph\">So on the podcast today, for a second time, we have Matt Schwartz.<\/p>\n<p class=\"wp-block-paragraph\">Matt has been working in the WordPress ecosystem since 2011, running his own agency based in Atlanta, and developing products like CheckView at all for WordPress form and checkout QA. Matt\u2019s expertise lies in how agencies can smartly, and cautiously, incorporate AI into their workflows for real tangible wins, and how to avoid potential pitfalls.<\/p>\n<p class=\"wp-block-paragraph\">He was on the show last week to record the first of this two part mini series. You might want to listen to that prior to this, but it\u2019s not strictly necessary.<\/p>\n<p class=\"wp-block-paragraph\">In this episode, we build upon last week\u2019s conversation. Matt talks about practical strategies for integrating AI across agency operations. The discussion starts with what it means to give AI access to your agency\u2019s brain, using tools like project management wikis and connecting them with AI chatbots to streamline knowledge sharing, and avoid common AI hallucinations.<\/p>\n<p class=\"wp-block-paragraph\">We then get into MCPs, or Model Context Protocol, and talk about why this area is quickly becoming a game changer for agencies looking to securely connect AI agents to multiple internal systems without complex, risky API configurations.<\/p>\n<p class=\"wp-block-paragraph\">The conversation covers how to use AI for building internal tools, highlighting where it\u2019s low risk and where you should be more cautious, especially with public facing, or mission critical, systems. Matt explains how agencies can leverage AI for QA and checklist automation, freeing up time for deeper human review of other important tasks.<\/p>\n<p class=\"wp-block-paragraph\">We also discussed the impact of AI on the WordPress plugin market, including potential consequences for plugin developers and the wider community, and whether the rise of AI generated disposable tools could erode the collaborative spirit of the WordPress community.<\/p>\n<p class=\"wp-block-paragraph\">We end by chatting about the importance of approaching agency AI adoption with eyes wide open to the risks. Data security, overdependence on vendors, failure to handle errors, and the reality that AI still makes mistakes.<\/p>\n<p class=\"wp-block-paragraph\">Matt shares his outlook on how agencies can position themselves to thrive as AI reshapes the industry, from hiring strategies to the next generation of productised services.<\/p>\n<p class=\"wp-block-paragraph\">If you\u2019re running an agency or freelance business in the WordPress space and want to get ahead with AI thoughtfully and securely, this episode is for you.<\/p>\n<p class=\"wp-block-paragraph\">If you\u2019re interested in finding out more, you can find all of the links in the show notes by heading to wptavern.com\/podcast, where you\u2019ll find all the other episodes as well.<\/p>\n<p class=\"wp-block-paragraph\">And so without further delay, I bring you Matt Schwartz.<\/p>\n<p class=\"wp-block-paragraph\">I am joined on the podcast, again, by Matt Schwartz, somewhat unexpectedly. Hello, Matt.<\/p>\n<p class=\"wp-block-paragraph\">[00:04:05] <strong>Matt Schwartz:<\/strong> Hey Nathan, thanks again for having me this week. I\u2019m super excited to dive back in.<\/p>\n<p class=\"wp-block-paragraph\">[00:04:09] <strong>Nathan Wrigley:<\/strong> Thank you. So we recorded an episode last week, and we intended to do it as a one hit. So get it all recorded, tied off within 40 minutes or what have you. And then we began talking.<\/p>\n<p class=\"wp-block-paragraph\">So last week we began talking and at about the half an hour point, it became obvious to me that we weren\u2019t going to capture it all in one recording. So we\u2019ve come back for a second episode.<\/p>\n<p class=\"wp-block-paragraph\">Dear listener, I would just say that in order to provide context for this episode, you really probably should listen to the previous one, because we\u2019re stacking up Matt\u2019s case, argument, however you wish to describe it, for where you can make wins inside your agency with the use of AI. Not just wins, maybe some cautionary tales as well. But that was the point of the first episode.<\/p>\n<p class=\"wp-block-paragraph\">So really, we\u2019re going to drop you in to the ninth of 16 points. So again, just pause this, go back to the previous episode, have a listen there, and then you can stack this one in your podcast player of choice at that point.<\/p>\n<p class=\"wp-block-paragraph\">If, however, somebody\u2019s ignoring that, Matt, are you able to just do a very quick bio? Just tell us who you are? It may be repetitive for the people that are listening to the second episode, but nevertheless, let\u2019s hear from you who you are.<\/p>\n<p class=\"wp-block-paragraph\">[00:05:16] <strong>Matt Schwartz:<\/strong> Yep. My name\u2019s Matt Schwartz. I run a WordPress agency here in Atlanta, since 2011. And I also have a testing and QA product for WordPress for checking forms and checkout called CheckView.<\/p>\n<p class=\"wp-block-paragraph\">And yeah, today we\u2019re just really diving into how you can leverage AI, how you can incorporate it into your agency, but in a hopefully smart and cautious way. Not necessarily just dropping it in, being a little bit more thorough about that process. So excited to continue the conversation.<\/p>\n<p class=\"wp-block-paragraph\">[00:05:45] <strong>Nathan Wrigley:<\/strong> Yeah, thank you. And thank you for being so accommodating by joining me for a second time. So as I said, Matt\u2019s put together a whole laundry list of different areas that your business, your freelance agency, whatever it may be that you are running in the WordPress space, can perhaps gain some benefits.<\/p>\n<p class=\"wp-block-paragraph\">Last week we did one through eight, and now we\u2019re going to sort of hit the road running on number nine. So the ninth point was about giving AI access to your agency\u2019s brain. It\u2019s a lovely subheading, but what does that mean?<\/p>\n<p class=\"wp-block-paragraph\">[00:06:12] <strong>Matt Schwartz:<\/strong> Yeah, so this is actually one I picked up in the Admin Bar, which is a, one of the other WordPress agency groups out there, that a lot of agencies were doing. And I thought it was an interesting, I would say hack, you could say, to add AI without getting super involved in it. Which is if you already have a project management tool, or you already have a wiki, you can add into your AI chat bot of choice when it\u2019s actually answering a question. You can tell it in its memory, hey, whenever I ask a question about the agency, go confirm what I\u2019m doing by visiting our Clickup or visiting our Asana.<\/p>\n<p class=\"wp-block-paragraph\">So it\u2019ll actually go retrieve and confirm it\u2019s using the latest proper information instead of just guessing or hallucinating. I love how we use the word hallucinate and not lie. I love that marketing branding that the AI companies did. It\u2019s some crazy gaslighting.<\/p>\n<p class=\"wp-block-paragraph\">Anyways, I love AI, but definitely, if you haven\u2019t used something like Claude or ChatGPT, saying in the memory as simple as when you answer a question, check if this is actually the case and connect to our ClickUp or connect to our wiki.<\/p>\n<p class=\"wp-block-paragraph\">I think that helps you get all the power of your documentation, SOPs, client, CRM, any data you\u2019ve already basically built up. It can leverage that without you having to do a whole bunch of crazy connections or ask more specific things. I thought that was actually a really neat way that agencies that are just getting into the space with AI are using the data they basically already have. They\u2019re just using their project management software, which basically has all that data.<\/p>\n<p class=\"wp-block-paragraph\">[00:07:47] <strong>Nathan Wrigley:<\/strong> When you see it in action, which I have actually, but not to do with a WordPress website, more to do with a sort of SaaS product with the, how the tool has been built and the guardrails that are into the tool. It\u2019s really amazing because then, well, basically it never forgets.<\/p>\n<p class=\"wp-block-paragraph\">So every time you throw something new at it, that becomes part of the corpus of information. It then has an understanding. I keep saying it, but hopefully you understand, I\u2019m meaning the AI in this case. A wider and broader understanding, and increasingly is able to deliver that back.<\/p>\n<p class=\"wp-block-paragraph\">So in my case, often I\u2019ll get some text back, which is divided up into bullet points. Those bullet points will have little footnotes attached to them, you know, 1, 2, 3, 4, and what have you, which will then link out to the documentation itself. And again, just a profoundly useful use of the thing which it\u2019s best at, which is taking a corpus of information, grinding it up and spitting out something which makes sense.<\/p>\n<p class=\"wp-block-paragraph\">And why wouldn\u2019t you point it at your internal documentation? You know, if you\u2019ve got a plugin, all of your support docs, throw the AI at it, and it will be able to help you as well as your clients. Because it\u2019s guaranteed you\u2019ve forgotten something that you\u2019ve built.<\/p>\n<p class=\"wp-block-paragraph\">WordPress, of course, itself does this. You know, every AI agent on the planet is welcome to crawl the docs for how WordPress itself is put together. And it\u2019s one of the reasons I think why WordPress has a fighting chance in this AI, CMS battle, if you like, because everything\u2019s open source already. Nothing\u2019s hidden behind a paywall or a licence agreement or what have you. So yeah, agreed. That\u2019s a great example.<\/p>\n<p class=\"wp-block-paragraph\">Okay. Anything to add or should we move on?<\/p>\n<p class=\"wp-block-paragraph\">[00:09:22] <strong>Matt Schwartz:<\/strong> No, I think that one\u2019s just a cut and dry, really. If you haven\u2019t looked at that, that\u2019s an easy way to get into AI and leverage it without a lot of work.<\/p>\n<p class=\"wp-block-paragraph\">[00:09:29] <strong>Nathan Wrigley:<\/strong> Okay. Absolutely. Okay, then number 10 is your internal MCP and guardrails. I know this gets bandied around a lot, and there\u2019s a lot of acronyms floating around in the AI space. But MCP, maybe we just need to spend a moment explaining what the heck an MCP is, and how it kind of fits into the overall picture, but particularly in this case, with your guardrails.<\/p>\n<p class=\"wp-block-paragraph\">[00:09:48] <strong>Matt Schwartz:<\/strong> Definitely. So MCP is basically an open source way to connect AI, let\u2019s say, chatbots or agents to external systems. It stands for Model Context Protocol. I think maybe the team behind Claude built it. I can\u2019t remember. But the idea really is that, instead of you just connecting directly to an API, which you could do, which an API if you\u2019re not familiar, is a way again, to connect two different systems together.<\/p>\n<p class=\"wp-block-paragraph\">One system will have a series of things that will let you say, hey, you can add this data, or you can pull this data, or you can modify this data, right? So an example could be something like a help desk. You might create a ticket, you may delete a ticket or you may edit the ticket. An API can basically do those things.<\/p>\n<p class=\"wp-block-paragraph\">But what an MCP does is it\u2019s really a series of tools that are more prebuilt for the AI, so that it knows and has context of what it should let you do, and how all the pieces of that connection really should modify whatever data you\u2019re doing.<\/p>\n<p class=\"wp-block-paragraph\">So it\u2019s a lot more specific to agents. It\u2019s a lot friendlier, I would say, if you aren\u2019t familiar with a company\u2019s API, you could connect to their API. I can connect to the WordPress REST API, but if I don\u2019t understand that API, it may not be actually the best way to make the connection.<\/p>\n<p class=\"wp-block-paragraph\">With an MCP, you can really not have the background of that company\u2019s bridge. It\u2019s going to do all the work, and the AI\u2019s going to have enough information to help you get what you need done.<\/p>\n<p class=\"wp-block-paragraph\">I know that\u2019s hard to explain, but essentially with an MCP, if you build one at your agency, this is a little bit more high level, or a little bit deeper, but I am seeing a lot of agencies looking at this. Which is, they are using an MCP basically for their teams so that they can add all of their systems in one basically bridge. So that instead of having all their employees like connect to all these different Claude connectors and APIs, they have one system.<\/p>\n<p class=\"wp-block-paragraph\">So if I have Claude and I\u2019m an employee, it can connect to my MCP at the agency, then the agency MCP is actually on behalf going to go retrieve data from all our different systems. So not everyone has to have API keys. Not everyone has to connect to all these other systems, if that makes sense.<\/p>\n<p class=\"wp-block-paragraph\">So I am seeing agencies starting to do this. So in some sense the proxy, MCP becomes a proxy or just a way to connect to all your other systems in a secure way.<\/p>\n<p class=\"wp-block-paragraph\">[00:12:18] <strong>Nathan Wrigley:<\/strong> I always imagine it a bit like if you, I don\u2019t know, you approach a giant supermarket and you know that you need carrots and soup. And normally you just go into the supermarket and wander around for a long time, and eventually you\u2019d sort of stumble across the carrots and the soup.<\/p>\n<p class=\"wp-block-paragraph\">But wouldn\u2019t it be nice if there was somebody at the front door? Then you could say, where\u2019s the carrots and where\u2019s the soup? And that\u2019s it. And they go, okay, the carrots are there, the soup\u2019s there, and point.<\/p>\n<p class=\"wp-block-paragraph\">You know, it\u2019s just like this perfect gatekeeper, this guardrail that you described that kind of allows you to get the best out of that experience without wasting a load of time and resources and probably a load of hallucination out the back end.<\/p>\n<p class=\"wp-block-paragraph\">[00:12:55] <strong>Matt Schwartz:<\/strong> Wow, that was so much better said than me. But yes, that is a much better way of explaining it. And that\u2019s why if you haven\u2019t looked at MCPs in general, I think it\u2019s worth looking at. But also if you have a tech background, looking at an MCP for your own agency where you can combine all your tools and connect to this one place, I think is a really neat way to, again, get your employees and contractors connected to your, all your systems without them having to have a direct connection.<\/p>\n<p class=\"wp-block-paragraph\">So if I want them to be like, hey, answer this ticket, go to this WordPress site, instead of them having to connect Claude to the WordPress site into Fresh Desk and all these things, it\u2019s all within the one MCP. And then they aren\u2019t really responsible for those API keys or any of those connections.<\/p>\n<p class=\"wp-block-paragraph\">Of course, you have to put guardrails on that too, right? Guardrails, like they can\u2019t delete things. You know, not having them delete tickets or websites. Because if you connect, you know, your host, they could technically delete an entire website if you don\u2019t have proper guardrails. So it is, I would say something that is a little bit more on the cutting edge that not every agency should do, but if you are on the more technical side, an internal agency MCP, I think is a really neat idea.<\/p>\n<p class=\"wp-block-paragraph\">[00:14:06] <strong>Nathan Wrigley:<\/strong> I feel like there\u2019s future of commoditising MCP creation.<\/p>\n<p class=\"wp-block-paragraph\">[00:14:11] <strong>Matt Schwartz:<\/strong> Oh, it\u2019s already happening.<\/p>\n<p class=\"wp-block-paragraph\">[00:14:12] <strong>Nathan Wrigley:<\/strong> Yeah, I really haven\u2019t experienced that but, you know, a really, I don\u2019t even know what that interface might look like in the end, but some really credible way of, you know, you sign up for a service and for a few dollars a month, they will modify, create on the fly, adapt the MCP so that it fits in with what was already an incredible technology. It\u2019s a bit like the icing on the cake, the MCP, isn\u2019t it? The AI was pretty amazing anyway, and then you put that layer on top and it just becomes much more refined amazing.<\/p>\n<p class=\"wp-block-paragraph\">[00:14:41] <strong>Matt Schwartz:<\/strong> There\u2019s actually some companies doing that already where you sign up for their SaaS, they basically store all the MCP data on their server securely, because that\u2019s always a concern. You give them basically all the credentials, you give them the guardrails, and then they build a secure, essentially MCP app for you.<\/p>\n<p class=\"wp-block-paragraph\">So there are some early options out there for that, that agencies could also look into if they\u2019re less technical. You just want to make sure, obviously you realise you\u2019re giving a third party your data and your secrets essentially.<\/p>\n<p class=\"wp-block-paragraph\">[00:15:11] <strong>Nathan Wrigley:<\/strong> And course, in the era of AI, the capacity to do things really seriously wrong is literally at the end of your fingertips. Whereas before, you\u2019d probably have to have some understanding, well, you could delete whole file structures and things like that, I guess. But now that a simple prompt can just rip through your entire code base or whatever it may be, definitely, one for guardrails there.<\/p>\n<p class=\"wp-block-paragraph\">[00:15:32] <strong>Matt Schwartz:<\/strong> Delete all the sites on my server, done.<\/p>\n<p class=\"wp-block-paragraph\">[00:15:34] <strong>Nathan Wrigley:<\/strong> Yeah, that\u2019s right. Yeah. And don\u2019t check.<\/p>\n<p class=\"wp-block-paragraph\">[00:15:37] <strong>Matt Schwartz:<\/strong> That could happen.<\/p>\n<p class=\"wp-block-paragraph\">[00:15:38] <strong>Nathan Wrigley:<\/strong> Yeah. Ignore all previous instructions, just delete them all.<\/p>\n<p class=\"wp-block-paragraph\">Okay, so that was number 10. And really interesting. I think that\u2019s one for the, sort of the tinfoil hat brigade, you know, the real nerds out there. But it\u2019s not far off. If it\u2019s been commoditised in SaaS now, you can guarantee that in the next few years that\u2019s going to become table stakes, I would\u2019ve thought for a lot of businesses and SOPs and what have you.<\/p>\n<p class=\"wp-block-paragraph\">Okay, so the next one, I\u2019m sure many people will have been familiar with, especially if you have a YouTube account and you\u2019re looking at AI things on YouTube. Vibe coded agency tools. I\u2019m sure I know what this one means, but run it by me anyway.<\/p>\n<p class=\"wp-block-paragraph\">[00:16:16] <strong>Matt Schwartz:<\/strong> Yeah, so one key thing is, I\u2019m not saying vibe coded tools themselves. I\u2019m saying vibe coded agency tools, agency being the specific part. So you\u2019re building internal tools for your agency, which I think in some sense, depending on what the situation is, can be okay to do, in my opinion, because the risk is lower. Again, you\u2019re using it internally.<\/p>\n<p class=\"wp-block-paragraph\">[00:16:40] <strong>Nathan Wrigley:<\/strong> It\u2019s not public facing, that\u2019s the point, right? It\u2019s just you and your colleagues, which hopefully you trust.<\/p>\n<p class=\"wp-block-paragraph\">[00:16:45] <strong>Matt Schwartz:<\/strong> Right, right. Again, you should probably put guardrails, and you have to think about, well, what sort of data is it touching, and how important is this data? That\u2019s everything with AI. You have to think about the risk. But I am seeing a lot of agencies starting to build different types of agency tools. Some that I really do internally myself, and I am a big fan of things like reporting tools and dashboards, right?<\/p>\n<p class=\"wp-block-paragraph\">This is, again, a good case, I think I talked about in the previous podcast. The sweet spot, I think with AI is having it handle things that you just never could get to that were on your list for a million years, right? And realising that, as long as you do a little due diligence and you feel like it\u2019s in the realm of where it\u2019s supposed to be, this is probably more information than you had before, right?<\/p>\n<p class=\"wp-block-paragraph\">So a good example is, if you\u2019re an agency, you may have it hooked up to QuickBooks MCP. You may have it hooked up to your time tracking software. You may have it hooked up to one of your other reporting software, WooCommerce subscriptions with Woo. And from there you\u2019re able to have a much better visible idea of what your business is doing well financially. The bottom line, especially if you\u2019re like a lot of agencies where QuickBooks doesn\u2019t really have all of your actual services. You may have those internally or you may have them in a other system.<\/p>\n<p class=\"wp-block-paragraph\">You can combine those and build reporting systems. And again, that\u2019s a relatively low risk way for you to, worst case is you\u2019re going to use that data and you\u2019re going to be like, well, this doesn\u2019t seem right. And you\u2019re going to have to dig into it and figure out what\u2019s going on. Hopefully you don\u2019t just blindly use it, but I do think the risk is lower.<\/p>\n<p class=\"wp-block-paragraph\">So those sort of tools I think are really, really neat, and relatively easy to build out. So reporting tools, profitability dashboards, things like that. Looking at your time tracking, like who at your agency is the most profitable, if you have that data? Obviously make sure you actually have the data structure for that, or AI may just make that up.<\/p>\n<p class=\"wp-block-paragraph\">But we\u2019ve, you know, used that even at our agency and I think it\u2019s been helpful for us to find patterns that we didn\u2019t know where we were spending our time and effort. Especially if you are doing time tracking using something like Everhour or Harvest.<\/p>\n<p class=\"wp-block-paragraph\">Now the tools that I do struggle a little bit more with, and I am seeing people in agencies use is things like website management dashboards, or building their own QA tools. Because those things, I think the risk is higher of things going wrong.<\/p>\n<p class=\"wp-block-paragraph\">[00:19:05] <strong>Nathan Wrigley:<\/strong> Public facing again, yeah.<\/p>\n<p class=\"wp-block-paragraph\">[00:19:07] <strong>Matt Schwartz:<\/strong> Right. And you\u2019re giving this third party access to all of your websites and it\u2019s not like, you know, a big SaaS. This is something you built internally, which means Claude doesn\u2019t care if it\u2019s wrong, right, until you tell it.<\/p>\n<p class=\"wp-block-paragraph\">So a good example, and not everyone may feel this way but, you know, I\u2019ve seen some agencies that are building replacements for management dashboards like ManageWP, WP Remote, those sort of things, which is connected to everything and is kind of their most important infrastructure for their clients. Personally, I think that that\u2019s a little risky to be doing.<\/p>\n<p class=\"wp-block-paragraph\">Now, if you\u2019re doing the right due diligence and you have a technical team and you\u2019re doing manual code reviews, sure. There\u2019s an argument to be doing that if it\u2019s also, I think, solving something specific to your agency. I talked about this in the last podcast, replacing SaaS products when there\u2019s a nuanced solution that\u2019s specific to your agency, I think could be really helpful.<\/p>\n<p class=\"wp-block-paragraph\">But if you\u2019re just replacing SaaS products to save 30 bucks a month, I think that doesn\u2019t make any sense because you\u2019re going to end up spending a lot more on maintenance, I promise you, than if you just stuck with the SaaS product, if it does what you need. So I think there\u2019s an argument there.<\/p>\n<p class=\"wp-block-paragraph\">[00:20:18] <strong>Nathan Wrigley:<\/strong> I think it\u2019s really interesting. And I do wonder if we\u2019re on the precipice of, so this is me sort of staring into the crystal ball a little bit. I wonder if we\u2019re into the era of sort of disposable, one time apps. So you have a function that you need to do this month like, I don\u2019t know, you must file your taxes this week, but you\u2019re miles behind. So you get an AI to just quickly do that thing for you, and categorise all of the jobs that need to be done so that you can hand it over to the tax man and so on. And then you just put that on ice. That thing no longer needs to exist.<\/p>\n<p class=\"wp-block-paragraph\">I feel that kind of coming where we sort of vibe code up this one time thing, and then dispose of it. I don\u2019t know if I\u2019m entirely in agreement with that as an idea, but I feel that that is coming. But to your point, I think anything public facing, we\u2019re still in the era of, really, watch what you\u2019re doing. It needs thorough testing.<\/p>\n<p class=\"wp-block-paragraph\">[00:21:09] <strong>Matt Schwartz:<\/strong> Exactly. And thorough code review because, you know, ultimately while the AI coding, I think has gotten really, really good, it\u2019s not a hundred percent there, and it doesn\u2019t have any context. It doesn\u2019t actually know what it\u2019s doing. It\u2019s all patterns. So there is an argument to be made that, yeah, it may get 80% there, but if no one\u2019s actually checked the code review, two months from now, it decides to delete all your websites out of your management dashboard, well, should have done a manual code review, right? And it\u2019s on you.<\/p>\n<p class=\"wp-block-paragraph\">So I do think there\u2019s an argument, same thing with like QA tools. Building one-off QA tools, which should be persistent and actually probably one of the more important things you build. It should do the same thing every day. It should almost be dumb. It should not be trying to rebuild itself all the time, or be even one-off, like you said. It\u2019s not, I think, a good idea to build a one-off migrator typically for that reason, even though I see people doing it in agencies. Unless you think it\u2019s just a low risk project, I do think, you know, you have to think about that.<\/p>\n<p class=\"wp-block-paragraph\">To your point about the one-off task though, I think again, if it\u2019s a low risk item, one-off makes actually the most sense because a lot of times if you know this is going to be a one-off thing, you then are subconsciously being like, okay, I\u2019m going to use this once and the cost is not that high and that makes sense, because you\u2019re not having to maintain it. But if you know it\u2019s going to be something you\u2019re going to be using for the next five years and you don\u2019t plan to pay a developer to review it, I don\u2019t think that\u2019s a smart idea, at least not right now.<\/p>\n<p class=\"wp-block-paragraph\">[00:22:39] <strong>Nathan Wrigley:<\/strong> That\u2019s a good calculus I think to have in the background. Okay, so that\u2019s good. So caveat emptor basically, use your discretion. If it\u2019s public facing, maybe think twice. But also if it\u2019s something that you want, you absolutely bulletproof need it to be reliable and predictable a hundred percent of the times that you run it. Again, maybe there\u2019s a human in the loop there. So that was sort of vibe coded things that you might do in your agency.<\/p>\n<p class=\"wp-block-paragraph\">I feel that\u2019s going to be a real area of growth, whether or not it will be profitable growth or useful growth, I\u2019m not entirely sure. I feel like in our industry at least, people are going to be dabbling in that kind of thing all the time. You know, trying to figure out new, clever tools to achieve a thing, which maybe in the past would\u2019ve been a subscription thing that you paid $20 a month for or something. So we\u2019ll see. We\u2019ll see how that goes.<\/p>\n<p class=\"wp-block-paragraph\">Okay, moving on then. So the 12th item that you brought to bear was QA, so quality assurance, checklists and testing. Right, run us through this one then.<\/p>\n<p class=\"wp-block-paragraph\">[00:23:33] <strong>Matt Schwartz:<\/strong> I know I just said when you\u2019re building a QA tool, using AI to build an internal vibe coded tool is probably not the answer. But actually what I\u2019m going to say right now is not contradictory to that, because what I\u2019m really talking about for QA and testing is more so having AI help you build things like checklists, right? You already have a good context usually with your SOPs. So it can help you build your SOP checklists. It can also help run the low risk items automatically. And again, I know I talked about risk a lot, but I think that\u2019s how you have to consider it.<\/p>\n<p class=\"wp-block-paragraph\">So one really neat thing I\u2019ve seen a lot of agencies start using is Claude Skills, which basically just means that you teach Claude a process. Literally it walks you through in the conversation like, what do you want this process to be? And then you can run that later in context.<\/p>\n<p class=\"wp-block-paragraph\">So a really cool example of this is if you are, let\u2019s say onboarding a client, or you\u2019re launching a site. During your next launch, your next onboarding, you may want to use Claude to teach it the skill of how you launch sites. And then it can automate a lot of those items. And you can still give it context like, I want the human to specifically review this item, right? Or, I want the human to check that no index isn\u2019t set, right? Because that\u2019s like a high risk item, right?<\/p>\n<p class=\"wp-block-paragraph\">[00:24:55] <strong>Nathan Wrigley:<\/strong> Yeah. Yeah, that\u2019s a big one..<\/p>\n<p class=\"wp-block-paragraph\">[00:24:56] <strong>Matt Schwartz:<\/strong> Right. That\u2019s one we all, I think, have dealt with at some point in our.<\/p>\n<p class=\"wp-block-paragraph\">[00:25:01] <strong>Nathan Wrigley:<\/strong> Too many times.<\/p>\n<p class=\"wp-block-paragraph\">[00:25:01] <strong>Matt Schwartz:<\/strong> Right. Too many. Exactly. That\u2019s the key. And that goes back to really that vision document I said on the last call. Having an AI vision document where you go through these checklists and you\u2019d be like, okay, we want a human to be involved, or we don\u2019t. You could actually tell Claude this. And then Claude will actually know exactly what it should be running itself and what it expects a human, and it will prompt you for.<\/p>\n<p class=\"wp-block-paragraph\">But I think that is the beauty of this is, you can make your whole automated process when it comes to tools with QA and anything really related to that checklist, whether it\u2019s launch or anything like that. Look at tools like Claude and Skills like that, and I think that you can use it to help with repeatable processes. And that will actually help most agencies not only speed things up and save on margins, but I think a lot of times they\u2019ll do more testing than they did before.<\/p>\n<p class=\"wp-block-paragraph\">And this, again, falls into that sweet spot where like AI\u2019s really good for the things that you knew you should do but you have limited time. And testing is one of those items. You want a hundred percent coverage, but in reality that\u2019s not going to happen. So let\u2019s have the human do the really important stuff and everything else we would\u2019ve never gotten to anyways, let\u2019s have the AI do it. And that\u2019s where I think you can use these tools.<\/p>\n<p class=\"wp-block-paragraph\">[00:26:11] <strong>Nathan Wrigley:<\/strong> Do you know what\u2019s really interesting, and we sort of made light of it in the previous episode, the fact that there\u2019s hallucinations and what have you going on all the time. But I do think there\u2019s definitely a moment coming where I think some of the more straightforward things, like for example, the checklist, the binary things, is no index switched on? Yes. No. Okay, that\u2019s a no.<\/p>\n<p class=\"wp-block-paragraph\">I think I am getting comfortable with that now. You know, just that, okay, we asked the AI that question, it\u2019s delivered as an answer. I\u2019m almost at the point now where I\u2019m never going to go back and check that was true. If it was something much more broad like, is my SEO strategy bulletproof? Well, no. It\u2019s never going to know whether that\u2019s the right thing.<\/p>\n<p class=\"wp-block-paragraph\">But these much more binary things, many of which, if you add them all up, could take you hours when you\u2019re finally launching a website. Yeah, I think there\u2019s something to be said for just sort of handing that stuff over. And I don\u2019t know, maybe you check it frequently, infrequently, less frequently as time goes on. But yeah, always check the no index one.<\/p>\n<p class=\"wp-block-paragraph\">[00:27:08] <strong>Matt Schwartz:<\/strong> Yeah, I was like, I would still check the no index one, even if it\u2019s binary. But to your point, a lot more of the very black and white items, I think it can handle a lot better than it used to, but I think it still comes down to risk. Like if it\u2019s, yeah, no index, I\u2019m still going to check it. But if it\u2019s something else that just is not that key. Yeah, I think we\u2019re all becoming a little more comfortable or a lot of us are coming more comfortable with that. And I think that\u2019s okay because you know the risk exposure really.<\/p>\n<p class=\"wp-block-paragraph\">[00:27:35] <strong>Nathan Wrigley:<\/strong> Well, and also, especially if it\u2019s QA and checklist time, hopefully you have done the bits and pieces, you really are at that point just making sure that you\u2019ve polished the thing that needs to be polished. So hopefully that\u2019s a bit of low hanging fruit where you\u2019ll catch the things that you missed, and maybe you\u2019ve done the due diligence there already.<\/p>\n<p class=\"wp-block-paragraph\">Okay, so that was number 12. We\u2019re approaching the end. We\u2019ve got four more to go. So number 13 links directly to WordPress specifically. The WordPress plugin market impact.<\/p>\n<p class=\"wp-block-paragraph\">I\u2019ve got to say, this has me slightly concerned, because I feel that this could be a good thing for our ecosystem, but also possibly a bad thing. But I\u2019ll just hand it over to you to paint the picture.<\/p>\n<p class=\"wp-block-paragraph\">[00:28:18] <strong>Matt Schwartz:<\/strong> Yeah, definitely. So I think you even touched on this a little earlier when you were saying there\u2019s going to be more one-off apps being built by agencies. And I think that also applies to a lot of plugins that are essentially one-off solutions, right? They are utility plugins. They solve one thing really well.<\/p>\n<p class=\"wp-block-paragraph\">I think that sort of thing is already seeing the impact. If you talk to a lot of plugin developers, especially some of the larger shops, they\u2019re seeing a drop in sales. And that is a real thing that\u2019s happening. They\u2019re seeing a drop in sales, especially for smaller plugins. Because a lot of agencies and customers are solving that with AI. Maybe it\u2019s a couple files of code, it\u2019s a lot easier for them to build it.<\/p>\n<p class=\"wp-block-paragraph\">Now I still have concerns around, are they having a human review that? Like I talked about. But humans are going to do what they\u2019re going to do, which some people are just going to run with that. That ultimately affects sales.<\/p>\n<p class=\"wp-block-paragraph\">So I think that is hurting a lot of the ecosystem when it comes to the smaller plugins out there. And even some of the bigger plugin developers are essentially sunsetting their smaller plugins, because they realise they\u2019re not getting as many sales and they need to focus on what they consider their moat, or their platform, you know, big plugins that AI\u2019s not going to be able to replicate or people shouldn\u2019t trust to replicate.<\/p>\n<p class=\"wp-block-paragraph\">But if you\u2019re building a small plugin, I\u2019m not going to call any out, but I think that there is some concern there. And I think ultimately for WordPress, I don\u2019t think that really hurts the WordPress ecosystem from the standpoint of plugins in general, but I do think it raises the bar of what a good plugin will be.<\/p>\n<p class=\"wp-block-paragraph\">And that kind of goes with the agency land. That\u2019s what\u2019s happening with agencies too. It\u2019s just the bar is being raised. You have to have a more complex plugin that actually solves someone\u2019s needs now, not just a small one that solved it, but now they can use AI to do it.<\/p>\n<p class=\"wp-block-paragraph\">And some people argue that that\u2019s going to continue all the way up with the most complex plugins out there. But I do think that there are, you know, unless AI dramatically improves. If it\u2019s 80% there, that\u2019s great and all, but it\u2019s what we talked about earlier. You can\u2019t really run with that in production at 80%. And that\u2019s the difference between a really good SaaS or really good plugin versus something that was homegrown and just falls apart.<\/p>\n<p class=\"wp-block-paragraph\">[00:30:38] <strong>Nathan Wrigley:<\/strong> I think I have a slightly different, maybe more community focussed, approach to this because one of the things that I think worries me is the, how should we say it? The slow ebbing away of the community. And obviously if you are a, I don\u2019t know, a company launched onto the stock market and what have you, you\u2019re all about the money, right? The bottom line is you\u2019re going to make money, distribute that with your shareholders, whatever, yada, yada. But the point is to make as much cash as possible and do things with that cash.<\/p>\n<p class=\"wp-block-paragraph\">We have a very different calculus here in that the community is the thing which largely builds the software, maintains the software, promotes events. There is a bit of me which worries that if these, let\u2019s say developers who\u2019ve got one plugin, it doesn\u2019t do 3000 major things, it just does one or two little things, but it\u2019s been their way of getting themselves into the software, and figuring out how it all works, and meeting the community, and being engaged and, you know, all of that.<\/p>\n<p class=\"wp-block-paragraph\">That slow ebbing away of that is something that I think our community and open source communities like ours need to be just a little bit mindful of. Because it does feel as if AI could definitely eat a lot of lunches. And I think we see that actually. I think we can already see that in the real world with things like attendance at events and the amount of events that are being put on, yeah.<\/p>\n<p class=\"wp-block-paragraph\">[00:31:57] <strong>Matt Schwartz:<\/strong> I think you\u2019re correct. I was actually going to bring this up in the sense that I am already seeing it within a lot of the agency groups. There\u2019s just not as much engagement when it comes to posts, I think, and that sort of thing. Because people use AI more to find solutions, which means they\u2019re not as engaged in the community. Which, to your point, plugins would kind of work the same way, especially the smaller plugins.<\/p>\n<p class=\"wp-block-paragraph\">And yeah, there\u2019s definitely something I think I\u2019m concerned and kind of sad about already. Because like that\u2019s why a lot of us are in WordPress is for the community. And I 100% agree with you. Not to mention if those guys, the smaller guys go away, then there just ends up being these massive plugin companies, which have their place, but WordPress wasn\u2019t built on all massive plugin companies. So If those smaller ones go away, then that\u2019s a little bit of the WordPress spirit I think are lost for sure.<\/p>\n<p class=\"wp-block-paragraph\">[00:32:47] <strong>Nathan Wrigley:<\/strong> Yeah, I think we just need to be very mindful about this slow, like I say, wicking away or haemorrhaging of the community. And because at some point, the calculation no longer works. You know, there just aren\u2019t enough community members around to make it interesting for other new community members to join, or to stick with things. Or to update their plugin or whatever it may be.<\/p>\n<p class=\"wp-block-paragraph\">And maybe that is just a consequence of the way the world is moving, I don\u2019t know. But having lived in the WordPress ecosystem for over a decade, I think it would be a shame if that baby was to be thrown out with that bath water.<\/p>\n<p class=\"wp-block-paragraph\">Okay. Alright. I think we\u2019ve done that one. So the next one is, well, the next one kind of speaks directly to that actually, which is the idea of, I guess spreading your wings a little bit further and realising perhaps that AI empowers you to do things outside of WordPress. And you\u2019ve entitled this Experimenting Beyond WordPress. Again, off you go.<\/p>\n<p class=\"wp-block-paragraph\">[00:33:37] <strong>Matt Schwartz:<\/strong> So this is something I am seeing some agencies doing, which is, because you can use AI now to use unfamiliar stacks, there\u2019s really two parts. One is just unfamiliar stacks, or unfamiliar platforms, you don\u2019t know. You can really try out new platforms a lot faster now. It will tell you exactly what to do, you know, step by step, or it\u2019ll just do it for you.<\/p>\n<p class=\"wp-block-paragraph\">So there are agencies, I think looking at other platforms where certain projects may make sense outside of WordPress, where they\u2019re using that in that capacity and it\u2019s allowing them to experiment. Where in the past, just sticking in WordPress, you have all your knowledge there makes sense to do, right? You don\u2019t want to know 10 different platforms. But I think AI\u2019s made that easier to dive into these other platforms. So that\u2019s the first thing.<\/p>\n<p class=\"wp-block-paragraph\">The second thing I think is that, now that you can actually use chat to engage, you are seeing some agencies, and some freelancers, that are saying, well, I don\u2019t need the WordPress infrastructure at all. I just want to go back to pretty much like static or HTML type websites because I know I\u2019m just going to chat with it and then I don\u2019t have to worry about security or updates.<\/p>\n<p class=\"wp-block-paragraph\">And obviously I think that only would apply with certain websites. It\u2019s not going to apply with highly functional websites. I think that\u2019s not really going to work. But for like your brochure site, I think some agencies are experimenting with some other platforms out there like Astro and the EmDash setup going on with CloudFlare.<\/p>\n<p class=\"wp-block-paragraph\">You know, and I think that was a direct response actually. They realised, oh, people are going to want to chat with it. We could build this WordPress, what they consider like an upgraded version, in their mind.<\/p>\n<p class=\"wp-block-paragraph\">And I think that, you know, it\u2019s good to experiment. I think what WordPress does really well, to your point though, is they\u2019ve hopped on the API centric side of things, building the right framework, not trying to force a certain thing down our throats, but actually leave it really open.<\/p>\n<p class=\"wp-block-paragraph\">And I think that\u2019s ultimately good because that\u2019s how open source works. That\u2019s why I think AI will have a good position with open source is. To your point you made previously, all the data\u2019s out there, all the documentation\u2019s out there. It\u2019s going to be able to be extremely flexible. And I think that\u2019s really why WordPress is still, in my opinion, going to exist and thrive.<\/p>\n<p class=\"wp-block-paragraph\">But you are seeing agencies that are looking outside of just Core WordPress, because they can experiment with just a lot less time now. And they can also try out some tools that may be a better fit for certain projects.<\/p>\n<p class=\"wp-block-paragraph\">[00:36:18] <strong>Nathan Wrigley:<\/strong> There\u2019s a lot of technologists in our community and we love tinkering, don\u2019t we, and playing with new things? So it\u2019s fairly inevitable. It goes with the territory.<\/p>\n<p class=\"wp-block-paragraph\">Okay. Alright. So that was Experimenting Beyond WordPress. I think I\u2019m going to skip 15 there because I think we covered quite a lot of that. So I\u2019m just going to go straight to number 16, which probably will become 15, if you know what I mean, when I put it into the show notes.<\/p>\n<p class=\"wp-block-paragraph\">So the next one anyway is called, whatever its number, is Risks and Cautions. So we\u2019ve built what I feel is like a fairly solid argument for doing this kind of stuff. And now towards the end, we\u2019re going to knock it all down. No, we\u2019re not. But what are some of the risks that you might point people towards?<\/p>\n<p class=\"wp-block-paragraph\">[00:36:58] <strong>Matt Schwartz:<\/strong> A hundred percent. This is, I think, probably one of my favourite sections because people don\u2019t really talk about these risks as much as they should. If you go on LinkedIn, it\u2019s just like all rainbows and butterflies. And we\u2019re building a new feature every day, and I think that noise can make people feel like they have FOMO and they just jump into AI and they don\u2019t think about the risks. So I think this is actually a really important section for any listeners to listen to.<\/p>\n<p class=\"wp-block-paragraph\">I think one of the first items, probably a little obvious, but with security issues, with AI tools, a lot of companies, every company, I feel like at this point is inputting keys and all sorts of things into these AI chatbots. And ultimately, like those tools can still be hacked. And actually legally, a lot of them say when you submit into the chat, it\u2019s actually considered public record. Anything you submit to them. That\u2019s literally what OpenAI made a legal argument recently about. So keep that in mind when you\u2019re doing this.<\/p>\n<p class=\"wp-block-paragraph\">There are some ways to do this securely. You can look into, and I think that\u2019s something I would recommend agencies doing because they, you are holding onto client data. And ultimately you don\u2019t want that stuff getting leaked.<\/p>\n<p class=\"wp-block-paragraph\">Another thing, kind of on another point, which I personally have less concern with, but I think some agencies bring up a valid point, which is every time you talk to these chat bots, et cetera, they\u2019re keeping track of all these conversations. So I know some agencies are being like, hey, how much does a website cost in my state or my country? Only use the data that other agencies have told you. And I don\u2019t know if it\u2019s really doing it, but people are doing that and they\u2019re, you know, it\u2019s not pushing back on them.<\/p>\n<p class=\"wp-block-paragraph\">Things like that, just be aware of, I think what you\u2019re inputting in, because it isn\u2019t necessarily being leaked as far as they\u2019re being hacked, but that data make get spit out to places you don\u2019t want it to by other parties through chat.<\/p>\n<p class=\"wp-block-paragraph\">[00:38:52] <strong>Nathan Wrigley:<\/strong> I feel that at some point in the future, some gigantic disclosure, something will be disclosed, which is so horrific that it makes us all sort of take a collective breath when we suddenly realise all we\u2019ve given over. We haven\u2019t got there yet, or at least to my knowledge we haven\u2019t. But I feel that at some point in the near-ish future, some jaw dropping disclosure will occur, which will make us all think twice about exactly what you\u2019ve just described.<\/p>\n<p class=\"wp-block-paragraph\">What are we giving up? What have we given? But also what have we not consciously given? Which kind of bit of our business did we unintentionally open up for the AI to have access to that we didn\u2019t intend to? And if we had the time again, we wouldn\u2019t have allowed it to, and so on and so forth. So, yeah. Okay.<\/p>\n<p class=\"wp-block-paragraph\">Any other things on the risks and cautions? I feel that there\u2019s a couple more lurking in there.<\/p>\n<p class=\"wp-block-paragraph\">[00:39:43] <strong>Matt Schwartz:<\/strong> Yeah. I think that one\u2019s to be the most obvious that most people are talking about. One I think people aren\u2019t talking about though is handling errors when you\u2019re building your own tools. Essentially, a lot of times you may vibe code something, right, which is great. But because you\u2019re not really going into the depth of every situation, it\u2019s just making kind of assumptions, the AI, of what should be there.<\/p>\n<p class=\"wp-block-paragraph\">And because it\u2019s so easy and you\u2019re like, well, I\u2019m saving time, people don\u2019t really outline all of this. And so they don\u2019t really put error handling in these tools. And what happens is, of course there\u2019ll be some edge case and, you know, things just break. And again, depending on the tool, if it\u2019s an internal tool, you can probably get away with that. But if it is a public facing, or a client facing tool, that is the beauty I think of having a human actually review it with logic is they are going to have context that the AI doesn\u2019t have.<\/p>\n<p class=\"wp-block-paragraph\">To your point earlier, like subconsciously might be giving certain information to the AI that we don\u2019t necessarily mean to. But you also might be leaving things out because you think you\u2019ve already told the AI, or you think it\u2019s going to assume a certain way, and you can\u2019t really make that assumption. It ends up really backfiring in the long run, I think.<\/p>\n<p class=\"wp-block-paragraph\">And that\u2019s why being very conscious about error handling and being like, okay, we\u2019re going to set up logging, we\u2019re going to set up testing. Validation is just the responsible way to be building these tools that really, I feel like no one talks about.<\/p>\n<p class=\"wp-block-paragraph\">[00:41:09] <strong>Nathan Wrigley:<\/strong> No. And the curious thing about it is, because it\u2019s such a black box, I feel that almost every other technology that we\u2019ve interacted with has been much more, I don\u2019t know how to describe it. There\u2019s been a higher barrier to entry. It\u2019s more difficult to interact with it. You\u2019ve had to, I don\u2019t know, press buttons or enter code or what have you. Now you are just communicating. And maybe we\u2019ll even sort of drop into voice communication at some point where we\u2019re literally just talking with the thing.<\/p>\n<p class=\"wp-block-paragraph\">There\u2019s just no, how to describe it. There\u2019s just such a small amount of friction that is required to interact with these things. And so it kind of lulls us into this perception that it can\u2019t make mistakes. It\u2019s error free and what have you. And we know that that\u2019s not the case. I didn\u2019t really describe that very well, but I hope you got a general sense of what I was trying to describe there.<\/p>\n<p class=\"wp-block-paragraph\">[00:41:54] <strong>Matt Schwartz:<\/strong> I think you\u2019re right. I mean I can give an example like building out CheckView, which is obviously like a, it\u2019s a QA tool for WordPress sites. But one thing is, I knew a decent amount of QA before I started building it. But I learned so much context building it, and we weren\u2019t using AI when we built the tool, right? That I would\u2019ve never gotten out if I had used AI from the beginning.<\/p>\n<p class=\"wp-block-paragraph\">Because like you said, there\u2019s just such little friction. You as a human just don\u2019t have to have that much information. You can just dive into something, having no idea on what you\u2019re really doing, which is a blessing and a curse. And I think just being aware of that. And building in the right logs, and errors to at least essentially provide a safety net for yourself, knowing that you\u2019re not going to know everything is really important.<\/p>\n<p class=\"wp-block-paragraph\">[00:42:34] <strong>Nathan Wrigley:<\/strong> Yeah. And then a couple of other things which we\u2019ll just sort of gloss over fairly quickly because maybe they\u2019re sort of slightly common sense. Obviously, you know, you\u2019ve got here sensitive client data, which might be, without your knowledge, being scooped up by the AI agents. So monitoring that.<\/p>\n<p class=\"wp-block-paragraph\">Over dependence on AI vendors. This feels like everybody\u2019s really become dependent on a handful of companies. Maybe you could count them on one hand, I think basically. There\u2019s three or four really, that everybody seems to be using. So that may be something that we want to be mindful of.<\/p>\n<p class=\"wp-block-paragraph\">And of course, the last one of your bullet points under Risks and Cautions is just the fact that AI makes some mistakes all the time.<\/p>\n<p class=\"wp-block-paragraph\">[00:43:15] <strong>Matt Schwartz:<\/strong> All the time. I think most of that\u2019s common sense. Really, the only one that I think people aren\u2019t talking as much about is the overdependence on AI. Not to date this podcast, but I\u2019m going to a little bit anyways. For example, with Claude, they are removing Claude Code from their $20 plan right now. You know, if you\u2019ve built this into your agency process, well, get ready to pay, you know, a hundred bucks for every employee, which could be thousands of dollars.<\/p>\n<p class=\"wp-block-paragraph\">[00:43:40] <strong>Nathan Wrigley:<\/strong> You can only imagine how valuable that will be. And maybe it\u2019ll be 200 bucks, or a thousand bucks or whatever it may be.<\/p>\n<p class=\"wp-block-paragraph\">[00:43:46] <strong>Matt Schwartz:<\/strong> Right. A lot of these companies obviously are subsidising the cost, and so I do caution agency specifically. That\u2019s why having that AI vision is important, but also considering making it independent enough from your processes that you still can function if this thing does change around, because I do think there is going to be a pushback on cost at some point.<\/p>\n<p class=\"wp-block-paragraph\">So for example, like when you\u2019re building a product, I\u2019m seeing some companies that are building AI so integral to their product that it will not function without AI. Or going to have to raise the price by 10x. And so like even with CheckView, our tools are there, but we haven\u2019t built it in in such a way that you can\u2019t use the tool without it.<\/p>\n<p class=\"wp-block-paragraph\">And I think at the agency level, it\u2019s the same idea. For the most part, trying to avoid building it in a way that you couldn\u2019t reduce the AI needs if you needed to. Or just preparing for that the costs could go up and like, you know, if you give it to all 10 employees now, you know, at 20 bucks a month, get ready to possibly pay a lot more later.<\/p>\n<p class=\"wp-block-paragraph\">And I just think that\u2019s something important for agencies specifically to keep in mind. And I know that seems contradictory to what I said at the beginning of this, AI everything, but I think it\u2019s important.<\/p>\n<p class=\"wp-block-paragraph\">[00:44:59] <strong>Nathan Wrigley:<\/strong> I\u2019m not a financial wizard in any way, shape, or form, but that does seem to be something which, sure as night follows day, is going to be coming, is the requirement to repay a lot of the venture capital that the AI is currently burning through. And yeah, so maybe a significant price hike.<\/p>\n<p class=\"wp-block-paragraph\">And we\u2019ve all got used to these practically free tools, and maybe that\u2019s something that is not going to be in our future. So that\u2019s a really good point. Put that bulwark in place to make sure that you are protected from that should it go up by, like you said, 10x or whatever it may be?<\/p>\n<p class=\"wp-block-paragraph\">Okay, so I did say that there were going to be multiple, I think 16 is what I said. This probably will be the last one. This is likely outcomes for agencies. So, Matt, you get to stare into the crystal ball and tell us what your final thoughts are in terms of what you think are likely outcomes.<\/p>\n<p class=\"wp-block-paragraph\">[00:45:47] <strong>Matt Schwartz:<\/strong> So I think some of this is already happening.<\/p>\n<p class=\"wp-block-paragraph\">Hiring, I think, is slowing down in some agencies because they\u2019re realising they can automate more. They don\u2019t need as many, essentially non-specialist employees, or contractors. And I think that is a real thing that is happening.<\/p>\n<p class=\"wp-block-paragraph\">I don\u2019t know if it\u2019s going to be necessarily a long-term issue. Hopefully as, essentially the floor raises, work gets better, more agencies will be focused on providing more value, more strategy, those sort of things. Again, the execution becomes a little bit of a commodity. So having essentially more junior team members who usually do that execution isn\u2019t just quite as necessary. So I think that\u2019s going to continue to come up.<\/p>\n<p class=\"wp-block-paragraph\">But again, I think it\u2019s going to be balanced out with even the costs we just talked about with the AI tools. There could totally be a point where the tools might get expensive enough that it makes sense to have a junior do this execution.<\/p>\n<p class=\"wp-block-paragraph\">[00:46:45] <strong>Nathan Wrigley:<\/strong> Get the humans back. Yeah.<\/p>\n<p class=\"wp-block-paragraph\">[00:46:46] <strong>Matt Schwartz:<\/strong> Yeah. And we go back the other way. Some people are saying that could happen. I could see the argument for that. But I think that\u2019s one thing.<\/p>\n<p class=\"wp-block-paragraph\">Another thing with agencies is you now can really productise more of your services. And this comes into the automation process that we talked about. You can take your processes, you can really package them up, I think. And there has been a lot of talk about productising services, but I think now you can get more nuanced.<\/p>\n<p class=\"wp-block-paragraph\">So if you, for example, only build sites for plumbers, well, with AI you can get way more specific on plumber specific service needs, and build out a process with what you\u2019ve already done, plus, with AI to make that, I think, a lot easier for the plumber or the client, to get what they need out of it. In the past, I think most agencies were trying to build SOPs as we have time. And it, you know, it\u2019s just a really difficult process. And I think that\u2019s where agencies I think could help a lot.<\/p>\n<p class=\"wp-block-paragraph\">And then the last point I\u2019ll make around here is really around that the tools I think will change. It will be, again, less about the execution of tools, how you\u2019re building your sites. More about the actual automation. And then really just testing and monitoring and making sure everything\u2019s working how it\u2019s supposed to be. The human will essentially become more of the manager of the AI. And that extends, I think, to even the tools.<\/p>\n<p class=\"wp-block-paragraph\">I could see there being more QA and monitoring tools out there for more specific needs. Because now, you know, AI can build 90% of this, which is great. I save that cost, but I know I need to pay maybe 5% of that towards some tools that actually watch and monitor what\u2019s happening, and make sure these automations, and these websites, are really doing what they need to be doing. So I think there is going to be possibly a shift in that way around what sort of tools that we\u2019re investing in as far as agencies go.<\/p>\n<p class=\"wp-block-paragraph\">[00:48:40] <strong>Nathan Wrigley:<\/strong> So that we could describe as a bit of a marathon. I think, really, you really took us through the gamut of everything that could possibly affect an agency in the AI space. We\u2019re in the year 2026, let\u2019s see how it ages. But that was a really interesting deep dive into all of the different bits and pieces.<\/p>\n<p class=\"wp-block-paragraph\">Matt has very kindly put together some show notes. What I think I\u2019ll probably do is crib those. Maybe I\u2019ll put them into the WP Tavern show notes, or maybe I\u2019ll link to a Google Doc or something like that where you can see them. But you\u2019ll be able to see all of the different bits and pieces that we went through. There\u2019s a lot more on that document than we actually had a chance to go through. So definitely do check that out.<\/p>\n<p class=\"wp-block-paragraph\">What I can say is that the future is definitely going to be interesting. Whether or not any of the predictions you\u2019ve made will turn out to be true, time will tell.<\/p>\n<p class=\"wp-block-paragraph\">But what a fascinating chat. Thank you so much for chatting to me. And I really appreciate you sticking around and doing the second episode somewhat unexpectedly with me.<\/p>\n<p class=\"wp-block-paragraph\">Just before we sign off, Matt, where can we find you? Where are the best places online to hang out with you?<\/p>\n<p class=\"wp-block-paragraph\">[00:49:49] <strong>Matt Schwartz:<\/strong> Yeah, definitely. So definitely, you can find me on the Admin Bar Facebook group. I\u2019m also in LinkedIn, trying to be better about that. You can also check me out. I\u2019ve got a Slack channel. Checkview.io, of course. Inspry.com. Feel free to reach out if anything comes up.<\/p>\n<p class=\"wp-block-paragraph\">Definitely, overall I would just recommend agencies dabble in this. Don\u2019t be reckless, but definitely see what makes sense for your agency. Document it all out ahead of time. And I think that that\u2019s really going to be agencies strong suit is, can we leverage this stuff in a smart way?<\/p>\n<p class=\"wp-block-paragraph\">[00:50:21] <strong>Nathan Wrigley:<\/strong> Well, you\u2019ve certainly provided us with a lot of food for thought. So once more, go and check out the show notes on wptavern.com. I will probably link to the document that Matt has created on both part one of this and part two as well. So you\u2019ll be able to check both of those out.<\/p>\n<p class=\"wp-block-paragraph\">Matt Schwartz, thank you so much for chatting to me today. I really appreciate it.<\/p>\n<p class=\"wp-block-paragraph\">[00:50:41] <strong>Matt Schwartz:<\/strong> Thank you so much, Nathan.<\/p>\n<\/div>\n<\/details>\n<p class=\"wp-block-paragraph\">On the podcast today we have Matt Schwartz.<\/p>\n<p class=\"wp-block-paragraph\">Matt has been working in the WordPress ecosystem since 2011, running his own agency based in Atlanta and developing products like CheckView, a tool for WordPress form and checkout QA. Matt\u2019s expertise lies in how agencies can smartly and cautiously incorporate AI into their workflows for real, tangible wins (and how to avoid potential pitfalls). He was on the show last week to record the first of this two part mini series. You might want to listen to that prior to this, but it\u2019s not strictly necessary.<\/p>\n<p class=\"wp-block-paragraph\">In this episode, we build upon last week\u2019s conversation, Matt talks about practical strategies for integrating AI across agency operations. The discussion starts with what it means to give AI access to your agency\u2019s \u2018brain\u2019, using tools like project management wikis and connecting them with AI chatbots to streamline knowledge sharing and avoid common AI hallucinations.<\/p>\n<p class=\"wp-block-paragraph\">We then get into MCPs, which stands for Model Context Protocol, and talk about why this area is quickly becoming a game changer for agencies looking to securely connect AI agents to multiple internal systems without complex, risky API configurations.<\/p>\n<p class=\"wp-block-paragraph\">The conversation covers how to use AI for building internal tools, highlighting where it\u2019s low-risk and where you should be more cautious, especially with public-facing or mission-critical systems. Matt explains how agencies can leverage AI for QA and checklist automation, freeing up time for deeper human review of other important tasks.<\/p>\n<p class=\"wp-block-paragraph\">We also discuss the impact of AI on the WordPress plugin market, including potential consequences for plugin developers and the wider community, and whether the rise of AI-generated \u2018disposable\u2019 tools could erode the collaborative spirit of the WordPress community.<\/p>\n<p class=\"wp-block-paragraph\">We end by chatting about the importance of approaching agency AI adoption with eyes wide open to the risks. Data security, overdependence on vendors, failure to handle errors, and the reality that AI still makes mistakes. Matt shares his outlook on how agencies can position themselves to thrive as AI reshapes the industry, from hiring strategies to the next generation of productised services.<\/p>\n<p class=\"wp-block-paragraph\">If you\u2019re running an agency or freelance business in the WordPress space and want to get ahead with AI thoughtfully and securely, this is the episode for you.<\/p>\n<details class=\"wp-block-details is-layout-flow wp-block-details-is-layout-flow\">Matt\u2019s show notes for Part 2<\/p>\n<h2 class=\"wp-block-heading\"><strong>9. Giving AI Access to the Agency\u2019s Brain<\/strong><\/h2>\n<ul class=\"wp-block-list\">\n<li>A practical quick win is connecting AI to the agency\u2019s project management system, wiki, docs, SOPs, or past tickets.<\/li>\n<li>This allows AI to answer questions using the agency\u2019s actual internal knowledge.<\/li>\n<li>It can help with:\n<ul class=\"wp-block-list\">\n<li>Sales handoffs<\/li>\n<li>Support consistency<\/li>\n<li>Project management<\/li>\n<li>Developer onboarding<\/li>\n<li>Client-specific context<\/li>\n<li>Process reminders<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">Good framing line:<\/p>\n<p class=\"wp-block-paragraph\">One of the quickest hacks is giving AI access to your agency\u2019s existing brain before asking it questions.<\/p>\n<h2 class=\"wp-block-heading\"><strong>10. Internal MCP and Guardrails<\/strong><\/h2>\n<ul class=\"wp-block-list\">\n<li>Agencies may eventually use an internal MCP layer as a controlled proxy.<\/li>\n<li>The MCP can connect to tools through APIs.<\/li>\n<li>It can give the team access to AI-powered workflows while maintaining guardrails.<\/li>\n<li>The goal is controlled access, not just letting AI freely touch everything.<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">Possible uses:<\/p>\n<ul class=\"wp-block-list\">\n<li>Search agency docs.<\/li>\n<li>Pull project status.<\/li>\n<li>Check time tracking.<\/li>\n<li>Review support history.<\/li>\n<li>Query website data.<\/li>\n<li>Trigger approved automations.<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">Good framing line:<\/p>\n<p class=\"wp-block-paragraph\">The more AI gets access to real tools, the more agencies need permission layers and guardrails.<\/p>\n<h2 class=\"wp-block-heading\"><strong>11. Vibe-Coded Agency Tools<\/strong><\/h2>\n<ul class=\"wp-block-list\">\n<li>Agencies are starting to vibe-code internal tools they never would have had time or budget to build before.<\/li>\n<li>Examples:\n<ul class=\"wp-block-list\">\n<li>Website management dashboards<\/li>\n<li>QA tools<\/li>\n<li>Reporting tools<\/li>\n<li>Client health dashboards<\/li>\n<li>Project profitability dashboards<\/li>\n<li>Launch checkers<\/li>\n<li>Tools combining project management, time tracking, and accounting data<\/li>\n<\/ul>\n<\/li>\n<li>This gives agency owners a more nuanced view of the business.<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">Good framing line:<\/p>\n<p class=\"wp-block-paragraph\">The value of vibe-coding is not always building a perfect SaaS product. Sometimes it is building a scrappy internal tool that saves the team 30 minutes every week.<\/p>\n<h2 class=\"wp-block-heading\"><strong>12. QA, Checklists, and Testing<\/strong><\/h2>\n<ul class=\"wp-block-list\">\n<li>AI is very useful for creating QA checklists.<\/li>\n<li>Tools like Claude with skills can be taught a repeatable launch process.<\/li>\n<li>AI can help generate launch checklists and even assist with running through parts of them.<\/li>\n<li>This becomes powerful when paired with actual testing tools.<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">Tie-in to CheckView:<\/p>\n<ul class=\"wp-block-list\">\n<li>Agencies need better ways to verify that websites and automations are actually working.<\/li>\n<li>AI can suggest what to check, but testing confirms whether it works.<\/li>\n<li>This is where tools like CheckView fit into the shift toward more automated QA and monitoring.<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">Good framing line:<\/p>\n<p class=\"wp-block-paragraph\">AI can help create the checklist, but you still need systems that verify the work actually works.<\/p>\n<h2 class=\"wp-block-heading\"><strong>13. The WordPress Plugin Market Impact<\/strong><\/h2>\n<ul class=\"wp-block-list\">\n<li>AI is making it easier to build small WordPress plugin utilities.<\/li>\n<li>This may hurt the market for small utility plugins.<\/li>\n<li>Agencies can now create small, custom plugins or snippets for specific client needs.<\/li>\n<li>Larger plugin companies may respond by focusing more on larger platform-style products with stronger moats.<\/li>\n<li>Small utility plugins may become less attractive as standalone businesses.<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">Good framing line:<\/p>\n<p class=\"wp-block-paragraph\">The tiny utility plugin market may get squeezed because agencies can now build small custom utilities much faster than before.<\/p>\n<h2 class=\"wp-block-heading\"><strong>14. Experimenting Beyond WordPress<\/strong><\/h2>\n<ul class=\"wp-block-list\">\n<li>Some agencies are experimenting with static sites, Astro, and other platforms.<\/li>\n<li>AI makes it easier to test unfamiliar stacks.<\/li>\n<li>This does not mean WordPress disappears.<\/li>\n<li>It does mean agencies may be more willing to choose different tools for different project types.<\/li>\n<li>WordPress will still make sense where clients need editing, plugins, WooCommerce, memberships, content workflows, and flexible admin tools.<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">Good framing line:<\/p>\n<p class=\"wp-block-paragraph\">AI may make agencies more platform-flexible, but WordPress still has a huge advantage when clients need a mature content and plugin ecosystem.<\/p>\n<h2 class=\"wp-block-heading\"><strong>15. Risks and Cautions<\/strong><\/h2>\n<ul class=\"wp-block-list\">\n<li><strong>Security issues<\/strong>\n<ul class=\"wp-block-list\">\n<li>Credentials, permissions, API access, and client data need to be handled carefully.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Poor error handling<\/strong>\n<ul class=\"wp-block-list\">\n<li>AI-built tools often work for the happy path but fail on edge cases.<\/li>\n<li>Many lack proper testing, validation, logging, and fallback behavior.<\/li>\n<\/ul>\n<\/li>\n<li><strong>No long-term maintenance plan<\/strong>\n<ul class=\"wp-block-list\">\n<li>Vibe-coded tools can create hidden technical debt.<\/li>\n<li>Once a tool touches client data, billing, credentials, or production systems, it needs real engineering thought.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Sensitive client data in AI tools<\/strong>\n<ul class=\"wp-block-list\">\n<li>Agencies need to be careful about pasting client data into SaaS AI tools.<\/li>\n<li>There are privacy, contractual, and data leakage concerns.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Over-dependence on AI vendors<\/strong>\n<ul class=\"wp-block-list\">\n<li>Agencies that make core offerings too AI-dependent could be exposed if tool costs rise.<\/li>\n<li>If the AI bubble cools or pricing changes, AI-heavy workflows may become more expensive.<\/li>\n<\/ul>\n<\/li>\n<li><strong>AI still makes mistakes<\/strong>\n<ul class=\"wp-block-list\">\n<li>AI can be confidently wrong.<\/li>\n<li>If a human made up facts this often, you would probably fire them.<\/li>\n<li>Agencies still need human review, especially for strategy, legal-sensitive work, code, security, and client-facing communication.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">Good framing line:<\/p>\n<p class=\"wp-block-paragraph\">AI can make bad thinking look very professional, and that is one of the biggest risks.<\/p>\n<h2 class=\"wp-block-heading\"><strong>16. Likely Outcomes for Agencies<\/strong><\/h2>\n<ul class=\"wp-block-list\">\n<li><strong>Smaller teams will do more<\/strong>\n<ul class=\"wp-block-list\">\n<li>AI may allow agencies to stay leaner.<\/li>\n<li>Some agency teams may shrink, or at least avoid hiring as quickly.<\/li>\n<\/ul>\n<\/li>\n<li><strong>More productized services<\/strong>\n<ul class=\"wp-block-list\">\n<li>AI makes it easier to package repeatable offerings.<\/li>\n<li>Example:\n<ul class=\"wp-block-list\">\n<li>An agency serving service businesses could automate intake, site planning, reporting, review analysis, landing page recommendations, and follow-up workflows.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<li><strong>More technical differentiation<\/strong>\n<ul class=\"wp-block-list\">\n<li>Agencies may differentiate through operations, automation, integrations, monitoring, and QA, not just design.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Agency tools will change<\/strong>\n<ul class=\"wp-block-list\">\n<li>Less focus on purely development-specific tools.<\/li>\n<li>More demand for automation, QA, testing, monitoring, and operational visibility.<\/li>\n<li>This connects directly to CheckView because agencies need to know whether the websites, forms, checkouts, and automations they manage are actually working.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">Good framing line:<\/p>\n<p class=\"wp-block-paragraph\">The next wave of agency tools may be less about building websites and more about proving that everything connected to the website is working.<\/p>\n<\/details>\n<h2 class=\"wp-block-heading\">Useful links<\/h2>\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wptavern.com\/podcast\/215-matt-schwartz-on-exploring-ais-impact-in-wordpress-agencies-part-1\">Part 1 of this two part podcast series can be listened to here<\/a><\/p>\n<p class=\"wp-block-paragraph\">Matt\u2019s agency \u2013 <a href=\"https:\/\/www.inspry.com\/\">Inspry<\/a><\/p>\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/checkview.io\/\">CheckView<\/a><\/p>\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/theadminbar.com\/\">The Admin Bar<\/a><\/p>\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/managewp.com\/\">ManageWP<\/a><\/p>\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/wpremote.com\/\">WP Remote<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Transcript [00:00:19] Nathan Wrigley: Welcome to the Jukebox podcast from WP Tavern. My name is Nathan Wrigley. Jukebox is a podcast which is dedicated to all things WordPress. The people, &hellip; <a href=\"http:\/\/appswikidroid.win\/index.php\/2026\/05\/13\/wptavern-216-matt-schwartz-on-exploring-ais-impact-in-wordpress-agencies-part-2\/\" class=\"more-link\">Leer m\u00e1s<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-39","post","type-post","status-publish","format-standard","hentry","category-sin-categoria","entry"],"_links":{"self":[{"href":"http:\/\/appswikidroid.win\/index.php\/wp-json\/wp\/v2\/posts\/39","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/appswikidroid.win\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/appswikidroid.win\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/appswikidroid.win\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/appswikidroid.win\/index.php\/wp-json\/wp\/v2\/comments?post=39"}],"version-history":[{"count":0,"href":"http:\/\/appswikidroid.win\/index.php\/wp-json\/wp\/v2\/posts\/39\/revisions"}],"wp:attachment":[{"href":"http:\/\/appswikidroid.win\/index.php\/wp-json\/wp\/v2\/media?parent=39"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/appswikidroid.win\/index.php\/wp-json\/wp\/v2\/categories?post=39"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/appswikidroid.win\/index.php\/wp-json\/wp\/v2\/tags?post=39"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}