# AI Governance for Mid-Sized Companies: Lean, Effective, Auditable > A mid-sized company does not need a corporate rulebook. It needs five parts: named roles, a short policy, a risk-based approval path, an inventory of every AI application, and a few checkable controls. That minimum can be set up in weeks and grows with the number of applications. Canonical URL: https://www.ruebstudio.de/en/guides/ai-strategy/ai-governance-framework Author: Dr.-Ing. Marcus Rüb, Founder Published: 2026-10-13 Once more than a few people in a company use AI tools, the same questions come up. What is allowed? Who decides? Which data may go in? And does anyone know what is actually running? AI governance is the organised answer to those questions. This article describes a minimum system that fits a mid-sized business, and places the law and the main standards in context. Status: October 2026. It is not legal advice. ## What is AI governance, and why does a mid-sized company need it? AI governance is the set of rules, roles and controls a company uses to decide which AI applications it runs, under what conditions and with what oversight. In a mid-sized firm it is less about committees than about clarity: every application has an owner, a risk tier and a documented route to approval. The benefit is practical. Without governance you get shadow AI: staff paste quotes, contracts or customer data into personal accounts of public chat services because nobody offered a sanctioned alternative. With governance there is an approved route that is faster than the private one. That is the real lever. Governance that only prohibits gets bypassed; governance that offers an easy path to approval gets used. Three reasons to start now: - **Liability and data protection:** if personal data or trade secrets go into AI services, you must know where they end up. You can only answer that if you know which services are in use. - **Regulation:** since 2 February 2025 the EU AI Act has required organisations that use AI to take steps on AI literacy. Further duties for operators of high-risk systems follow (see below). - **Customer questionnaires:** buyers can ask in supplier questionnaires how AI is managed. An inventory and a policy turn that into a short answer. ## What are the building blocks of a lean governance system? A lean system has five building blocks: roles, policy, approval process, inventory and controls. Each fits on a page or in a table. Together they answer who decides, what applies, how an application goes live, what is running, and how that is checked. ### Roles At least three roles make sense. In small firms the same people can fill them: - **AI lead:** decides on approvals, keeps the inventory, and is the contact for business units and auditors. This person needs real decision authority, not just a coordination remit. - **Business owner per application:** the person in the business unit who owns the process and answers for what the agent produces. They decide what the agent may do. - **Advisory functions:** data protection, IT security and, where relevant, the works council are involved in higher-risk approvals, not in every small decision. The point is simple: one person carries the responsibility, and everyone knows who it is. ### Policy The policy states what staff may and may not do. It works when it fits on two pages and contains concrete examples: which data classes may go into which services, which tools are approved, that outputs are checked before they reach customers, and whom to ask when unsure. ### Risk-based approval Not every application needs the same scrutiny. A three-tier process is usually enough; details follow in the next section. ### Inventory The inventory is a table of all AI applications in the company: name, purpose, vendor or model, data types, owner, risk tier, approval date and next review. Without this list, any statement about AI in the company is guesswork. ### Controls Controls are a few checkable measures: spot checks of agent outputs, a review of access rights, an evaluation of logs, and an annual reconciliation of the inventory with the real IT landscape. ## What does an approval process look like that does not stall projects? A risk-based approval process sorts applications into three tiers: approved by default, quick review, and full review. The tier depends on the data involved, who sees the output and what the output triggers. Most applications get an answer within days, and effort goes only where it reduces risk. A workable grid uses three questions: 1. **What data goes in?** Public or uncritical content, internal documents, personal data, or trade secrets. 2. **Who sees the result?** Only the person who ran it, other colleagues, or customers and third parties. 3. **What happens with the result?** It is a draft a person reviews, or it automatically triggers an action such as an order, an email or a booking. The answers give the tier: - **Tier 1 (standard approval):** uncritical data, internal output only, a human always in between. Example: summarising public standards. The AI lead approves by entering it in the inventory, within a few days. - **Tier 2 (quick review):** internal documents or a small amount of personal data, or output that goes to customers after review. Data protection and IT security check against a short checklist. - **Tier 3 (full review):** decisions about people, automatically triggered actions with financial or legal effect, and uses that may fall under the EU AI Act's high-risk rules. This tier includes a risk analysis, test evidence, an oversight concept and legal review. One thing matters more than the grid: a deadline. If people wait two weeks for an answer, they route around the process. Set response times the AI lead must meet, and measure them. For agents that operate tools on their own, one more rule applies: the more rights an agent has, the higher its tier. An agent that only reads and drafts is a different case from one that sends emails or changes records in an ERP system. The [glossary entry on AI agents](https://www.ruebstudio.de/en/glossary/ai-agent) explains what the term covers. ## What does the EU AI Act require of companies that use AI? The EU AI Act distinguishes providers and deployers. Mid-sized companies are mostly deployers: organisations that use AI systems under their own authority. Since 2 February 2025 they have had to promote AI literacy, and certain practices are banned. Further duties for deployers of high-risk systems apply later. According to the [European Commission's timeline](https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai) (as of 3 August 2026), the key dates are: - **2 February 2025:** prohibited practices and AI literacy obligations apply. - **2 August 2026:** the Act becomes generally applicable, with exceptions. Transparency rules take effect in August 2026. - **2 December 2027:** high-risk rules for the Annex III use cases, such as employment and education. This date was postponed from the original one. - **2 August 2028:** high-risk rules for AI built into regulated products under Annex I, also postponed. Two duties matter most for governance. **AI literacy (Article 4).** Under [Article 4](https://artificialintelligenceact.eu/article/4/), providers and deployers must take steps to build AI literacy among their staff and others who operate or use AI systems on their behalf, adapted to prior knowledge, context of use and the people affected. The text states that it does not require guaranteeing any specific level of AI literacy. In practice: training and guidance you can document. **Deployer duties for high-risk systems (Article 26).** According to [Article 26](https://artificialintelligenceact.eu/article/26/), a deployer of a high-risk system must, among other things, assign human oversight to people with the necessary competence, training and authority, monitor operation according to the provider's instructions, and inform the provider and the market surveillance authority of risks and suspend use. Automatically generated logs, where the deployer controls them, must be kept for at least six months unless other law provides otherwise. Employers must also inform workers' representatives and affected workers before putting such a system into use at the workplace. Whether a use counts as high-risk depends on the case, for example AI that screens job applicants. That classification belongs in tier 3 of the approval process and should be reviewed by a lawyer. This overview does not replace legal advice. ## Which standards and frameworks help you structure the work? Two frameworks come up most often: ISO/IEC 42001 and the NIST AI Risk Management Framework. Both are voluntary. They supply structure and vocabulary, but they replace neither your own risk assessment nor your legal duties. For a mid-sized firm they are guidance, not a prerequisite. **ISO/IEC 42001:2023.** The standard, titled "Information technology – Artificial intelligence – Management system", was published as edition 1.0 on 18 December 2023. According to the [IEC webstore](https://webstore.iec.ch/publication/90574), it sets out requirements and guidance for establishing, implementing, maintaining and continually improving an AI management system, and applies to organisations of any size or type. If you already run a management system under other ISO standards, the building blocks in this article can be attached to it. **NIST AI Risk Management Framework.** The [NIST framework](https://www.nist.gov/itl/ai-risk-management-framework), version 1.0 of 26 January 2023, is described by NIST as intended for voluntary use. Its four core functions are Govern, Map, Measure and Manage. NIST notes that the framework is being revised as part of the White House AI Action Plan, so check the current status. The five building blocks here line up with those functions: Govern (roles, policy), Map (inventory, context), and Measure and Manage (controls, approval). That is one reading, not an official mapping. Certification under ISO/IEC 42001 can make sense if buyers require it or if you sell AI products yourself. For purely internal use, a lean system without a certificate is usually enough. ## What does the system need to work in daily practice? Governance works when it is built into existing routines: approval is part of procurement, the inventory is part of IT documentation, training is part of onboarding. That takes three preconditions: backing from management, visibility of the IT landscape, and a culture where asking questions is welcome. **Backing.** The AI lead must be able to stop an application, and management must stand behind that decision. Without it, governance turns into a recommendation. **Visibility.** The inventory starts with a survey. Ask each business unit which AI services they use, check software licences and expense records for AI subscriptions, and review access logs from your proxy or single sign-on. The survey may show more in use than expected. **Culture.** Reporting an application must not be punished. An amnesty period for existing, unreported use helps fill the inventory quickly. **Training.** Short, role-based formats work better than a single mandatory session: half an hour on data classes for everyone, a deeper session for agent owners and another for approvers. Record attendance and content; it documents the measures under Article 4. **Technical guardrails.** Governance on paper is not enough if the technology does not support it. Approved tools with central access, least-privilege rights for agents, logging of inputs and actions, and the ability to switch an agent off centrally turn rules into enforceable controls. Where data should not leave the company, the hosting model is part of the decision too; the page on [sovereign AI](https://www.ruebstudio.de/en/sovereign-ai) gives an overview. ## In which steps do you build the minimum system? The minimum system comes together in five steps: survey what exists, assign responsibility, write the policy and risk tiers, approve existing applications retroactively, and schedule controls. In a firm of manageable size this takes six to eight weeks alongside normal work. A fixed timeframe stops it from fizzling out. 1. **Survey (weeks 1–2).** List every AI service, plugin and in-house script in use. Result: a first version of the inventory. 2. **Assign responsibility (week 2).** Name the AI lead and a deputy, and settle how data protection, IT security and the works council take part. 3. **Write policy and tiers (weeks 3–4).** Two pages of policy, plus the three tiers with examples from your own operations. Management signs off. 4. **Approve existing use (weeks 4–6).** Assign every listed application to a tier. Assess high-risk candidates and anything with personal data first. Retire applications that cannot be approved by a set date, and offer an approved alternative. 5. **Schedule controls (weeks 6–8).** Put spot checks, rights reviews, log evaluation and inventory reconciliation in the calendar, each with an owner. After that, operation takes over: new applications go through the approval path, the inventory is maintained, and once a year you review the system itself. Ask: how many applications are in the inventory, how many are running without approval, how long does an approval take, and were there incidents? If you are not yet sure which applications would make sense in your company, the overview of [AI use cases by department](https://www.ruebstudio.de/en/guides/finding-ai-use-cases/ai-use-cases-by-department) is a good starting point. For a concrete agent and the controls that apply to it, see the article on the [AI quoting agent](https://www.ruebstudio.de/en/guides/ai-agents-by-department/ai-quoting-agent-rfq). ## Where are the limits, and what typically goes wrong? Lean governance has limits. It does not protect against model errors, it does not replace legal review, and it works only if people follow it. The most common failures are an overloaded rulebook, an AI lead without authority, a stale inventory, and controls nobody carries out. - **Too much at once.** Nobody reads a 40-page rulebook. Start with two pages and add rules when a concrete case calls for them. - **Bans without alternatives.** Blocking a tool without offering an approved one creates shadow AI. - **Owners without time.** Business owners must actually be able to run spot checks. Plan time for it. - **A static inventory.** Models and services change. When a vendor swaps a model or adds features, the risk tier may change. Require that changes are reported. - **Paper instead of technology.** A policy that forbids entering confidential data but offers no technical block or alternative will be broken sooner or later. - **False comfort from certificates.** A certificate says nothing about whether one application works correctly in your process. Test results on your own material. - **A moving legal target.** The deadlines for high-risk rules were recently postponed. Do not rely on individual dates; check the current text of the regulation when a decision depends on it. The system also does not answer whether an application is worth building in the first place. That is a question for use-case evaluation, not for governance. ## How do you get started? Start with the inventory. A table of every AI application in the company takes a week and produces insights immediately. Then name the AI lead, write the two-page policy and define the three approval tiers. Everything else follows from practice. A useful rule of thumb for the first session: take the three most-used AI services and answer seven questions for each. Who owns it? What data goes in? Where is it processed? Who sees the output? Is there a human before it takes effect? What logs exist? When is it reviewed next? Any missing answer is your first gap. If building this needs support at leadership level, the page on [fractional AI leadership](https://www.ruebstudio.de/en/fractional-head-of-ai) outlines how roles, strategy and governance can be brought together in one engagement. ## Frequently asked questions ### Is AI governance legally required for a mid-sized company? There is no single obligation called AI governance. Specific duties do follow from the EU AI Act, such as promoting AI literacy (Article 4) and deployer duties for high-risk systems (Article 26). A lean governance system is the practical way to show you meet them. ### Do you need ISO/IEC 42001 certification? No. The standard is voluntary and describes an AI management system. It is useful as a way to structure your work; certification usually makes sense only when customers explicitly ask for it. ### Who should own AI governance? One named person with authority to approve and to stop applications, usually from management or IT leadership. A committee without that authority only slows things down. ### How much effort does a minimum system take? A first version needs a handful of working sessions: build the inventory, define risk tiers, write a one- to two-page policy and define the approval path. Ongoing effort is mostly keeping the inventory current and running spot checks. ## Sources - [European Commission: Regulatory framework on AI (application timeline, as of 3 August 2026)](https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai) - [EU AI Act, Article 4: AI literacy](https://artificialintelligenceact.eu/article/4/) - [EU AI Act, Article 26: Obligations of deployers of high-risk AI systems](https://artificialintelligenceact.eu/article/26/) - [NIST: AI Risk Management Framework](https://www.nist.gov/itl/ai-risk-management-framework) - [IEC: ISO/IEC 42001:2023, Artificial intelligence – Management system](https://webstore.iec.ch/publication/90574)