Na této stránce používáme soubory cookie. Prohlédněte si naše Zásady ochrany osobních údajů pro více informací.

Penetration test of an app built with AI: when and why to do it

A penetration test is a controlled attempt to break into an app. When to consider one for an AI-built app, how broad it should be and how to choose a supplier.

Penetration test of an app built with AI: when and why to do it

Short answer: A penetration test is a controlled attempt to deliberately break into an app from an attacker's position. Whether you need one and how broad it should be depends on the type of app, its architecture, the sensitivity of the data and the impact of a possible incident. It makes sense when there is something to lose: payments, personal data, access to company systems, and after major changes. It does not replace automated checks or code review, but it verifies whether the flaws that remain can actually be exploited.

Why it matters

AI can write secure code today if it gets the right assignment and rules, but like a developer it makes mistakes. In October 2026, Veracode tested 18 language models, and on average they wrote secure code in 62% of cases, the best models in 73%. That is a noticeable improvement over 2025, but even the best models failed in roughly one task out of four. Veracode therefore recommends supplementing AI code generation with continuous independent verification.

A penetration test is one way to find out which of the possible flaws can actually be exploited, before someone else finds out. For a director it is a decision about risk, not about technology: how much is at stake, when to check it and how much control is proportionate. An overview of typical flaws in apps written with AI is in the article on the security of apps written with AI.

Not sure whether your app needs a penetration test? Book 30 minutes online with us. We will tell you what we would check first and whether it is time for a pentest. Book an online call

What a penetration test is and what it is not

A penetration test is a simulated attack led by a person, aiming to find exploitable paths into a running app. It is often confused with two other kinds of checks.

CheckWhat it doesWhat it findsWhat it does not find
Automated check and AI code reviewTools and AI go through the code, libraries and settingsOutdated libraries, keys in code, typical injections, wrong configuration, flaws across the whole codebaseFlaws in business logic, wrongly set permissions, combinations of small things
Review by a personAn experienced developer focuses on substantial risks and the stability of the foundationMissing access control, insecure design, weak architectureWhat shows up only at runtime under attack
Penetration testA person attacks the running appExploitable paths, account takeover, data leaks, chains of flawsFlaws outside the agreed scope

The automated check is the cheapest and should run on every change. There is so much code written with AI that a person cannot efficiently read all of it by hand, so automation goes first and then a person on the substantial risks. A penetration test is the most demanding of the three and is done when there is something to lose. When it is preceded by an automated check and a human one, it finds fewer things, but what it finds is substantial.

A penetration test is also not a certificate. The output is a report with findings as of the date of the test and within the scope you specified. In half a year, with new features, the situation may be different.

When to consider a penetration test

The impact of a flaw decides: whom it hits, how large it is and whether it can be reversed. A wrong write to a database can sometimes be restored from a backup, while money sent or personal data leaked usually cannot. For a starting app for dozens of users without sensitive data, it is reasonable to prioritize function and validating the market. For a financial app or a large system, much more thorough checking is needed. You would typically consider a penetration test in these situations:

  • The app handles payments, financial data or personal data of many people. This is where the cost of a flaw is highest and hardest to recover.
  • The app has access to company systems. Integrations with internal systems, an admin panel or an AI agent with the right to change data. Often a bigger risk than the code itself is the architecture and permissions.
  • After a major change. A new integration, a change of architecture, a move to another framework or adding an AI agent can open a path that was not there before.
  • When a customer, a contract or a regulation asks for it. Larger companies often require security tests from suppliers. Companies subject to the Cybersecurity Act (effective from 1 November 2025) must manage risks and put security measures in place. Whether that also means penetration tests for you, verify with a lawyer or a security specialist.

The scope follows the same factors. For a typical web app, a test usually focuses on the backend, the API, login and access rights. For a prototype that changes its foundations every week, a pentest usually makes no sense, because the report will be out of date in a month. First go through the checklist on whether your AI-built app is ready for real users.

What to expect from a penetration test

A good pentest has four parts. If an offer leaves one out, ask why.

  • Scope. Together you decide what is tested: which parts of the app, which user roles, whether also the infrastructure and the AI layer (chatbot, agent). The scope sets both the price and the value. It is ideally tested on an environment with test data, not on live customer data.
  • Report. A report with findings ordered by severity, with a description of the exploit and a recommended fix, ideally with a summary for leadership. If you do not understand what a finding means for the business, have it explained.
  • Remediation. Findings are fixed by your team or by the partner who maintains the app. First what allows access to someone else's data or account takeover, then the rest.
  • Retest. After the fixes the tester verifies that the findings are gone and that the fix did not open something new.

The result is a list and a status: what was found, what is fixed and what is a knowingly accepted risk. You can show that to a customer, an auditor or an investor.

How to choose a penetration test supplier

Choose by methodology, scope and clarity of the output, not just by price. We do not recommend specific firms, but you can tell whether a supplier knows what they are doing.

  • They ask about the business and the architecture. They want to know what the app does, what data it processes and what it has access to, so that they attack what is truly valuable.
  • They work to a public methodology. Most often to OWASP, meaning public lists and testing procedures for web applications, and for apps with AI also the OWASP lists for language models and agents.
  • They separate an automated scan from a manual test. If the offer is only the output of a tool, it is not a pentest.
  • They can test the AI layer. For an app with a chatbot or an agent, ask whether they test prompt injection and the agent's permissions.
  • They count on a retest and deliver a report for leadership and for developers.

If the app is maintained by an external partner, it is sensible for the pentest to be done by someone else. An independent view is the point of the test.

When a penetration test yes and when no

You usually do not need a pentest yet when:

  • the app is a prototype and its foundations change every week,
  • it stores no sensitive data and has no access to company systems,
  • a possible flaw is easy to reverse and harms nobody seriously,
  • you do not have the basics yet: access control, keys outside the code, up-to-date libraries, automated checking.

Consider a pentest when:

  • the app handles payments, financial data or personal data of many people,
  • it has access to internal systems or contains an AI agent that can change data,
  • a flaw would be hard to reverse or would have a large impact on customers or the company,
  • a customer, a contract or a regulation requires it,
  • a major change has happened since the last test.
If you found yourself in the second list, send us a few sentences about your app and what it processes. We will get back to you with how we would proceed. Send an inquiry

How MeguMethod approaches it

In App as a Service, architecture, security and code review are the technical guardrails we set before every launch. We use AI already from specification, and each project has its own rules for AI, so the code stays consistent. Every change is first checked by automated AI review and then by our engineers for security, architecture and function. A penetration test then verifies, according to the app's risks, what those guardrails hold.

An example from practice: the Android app Photo Creation Guide for documenting crash tests in automotive has encrypted offline storage, had to meet the OWASP Top 10 and passed a security review and a penetration test with no findings, on the first attempt. That was possible because security was handled from the design, not just before the test.

Next step

Do you want to get your app to a state where a pentest verifies something, rather than just listing the basics? See how App as a Service works, or book 30 minutes online right away and we will go through where your app stands.

FAQ

What is a penetration test in simple terms?

A controlled attempt by a person to break into a running app the way an attacker would. The result is a report with findings, their severity and a recommendation for fixes. It differs from an automated check, which looks for known patterns, and from a code review, which reads the code but does not attack it.

Does an app written with AI need a penetration test more than another app?

The same rules apply to it. The type of app, the sensitivity of the data and the impact of a flaw decide, not whether the code was written by a person or by AI. With AI code, though, it matters that an automated and a human check happen before the test, because a lot of code is produced, and quickly.

When is the right time for the first penetration test?

When the foundation is done (access control, keys outside the code, up-to-date libraries, automated checking) and the app starts working with data or systems where a flaw would have a large and hard to reverse impact.

What should a penetration test focus on?

On where the biggest risk is. For a typical web app, usually the backend, the API, login and access rights. For an app with an AI agent, also prompt injection and the agent's permissions.

Is an automated check enough instead of a pentest?

An automated check is the necessary minimum and should run on every change. But it will not find missing access control, flaws in logic or chains of small flaws. For apps where a flaw has a high impact, it is therefore supplemented by a check by a person and, according to the risks, by a pentest.

AppCast

České appky, které dobývají svět

Poslouchejte recepty na úspěšné aplikace. CEO Megumethod Vráťa Zima si povídá s tvůrci českých aplikací o jejich businessu.

Zrnitý barevný tvarCEO MeguMethod Vratislav ZimaČmáranice MeguMethod