Building an Engineering Competency Framework: From Skills to Organisational Capability
As engineering organisations grow, one of the biggest challenges is understanding what capabilities they actually have and what capabilities they will need in the future.
It is easy to say that an organisation has “great engineers”. It is much harder to answer more practical questions:
Do we have the right skills for the technology we are adopting?
Where are our biggest capability gaps?
Are critical skills concentrated in only a few people?
What skills should we develop internally?
Where do we need to recruit?
Are our engineering disciplines working together effectively?
How do we create meaningful career paths for engineers?
This is where an engineering competency framework becomes valuable.
A competency framework provides a structured way of describing the knowledge, skills, behaviours and experience expected across different engineering disciplines. Done well, it is not simply an HR document or a matrix used during performance reviews. It becomes a mechanism for understanding engineering capability, guiding development, supporting career progression and shaping the future engineering organisation.
The key is to build the framework around capability rather than job titles.
What Is an Engineering Competency Framework?
A competency framework defines what good looks like across different engineering disciplines and levels of experience. It can describe capabilities such as:
Technical knowledge
Practical engineering skills
Architecture and design
Problem solving
Quality practices
Security
Automation
Operational responsibility
Collaboration
Technical leadership
Communication
Strategic thinking
Importantly, competency does not mean simply knowing about something. There is a significant difference between “I have worked with Kubernetes” and, “I can design, deploy, secure, troubleshoot, and operate Kubernetes-based workloads in production.”
A good framework therefore needs to distinguish between awareness, knowledge, application and mastery.
It should also recognise that different disciplines require different competencies. A Cloud Engineer, Quality Engineer, Data Engineer and Software Engineer may all need knowledge of security, automation and architecture, but they will apply those capabilities in very different ways.
1. Software Engineering
Software Engineering will typically form one of the largest competency areas within an engineering organisation. However, modern software engineering goes well beyond the ability to write code.
A Software Engineering competency framework might include:
Programming languages and development practices
Object-oriented and functional programming
API design
Distributed systems
Microservices
Event-driven architecture
Database design
Design patterns
Testing and testability
Version control
CI/CD
Observability
Performance engineering
Secure coding
Cloud-native development
Dependency management
AI-assisted development
At more advanced levels, the emphasis should increasingly move from implementation towards engineering decisions. An experienced engineer should be able to explain why a particular architecture, technology, or approach is appropriate, and what trade-offs it introduces. Competency frameworks should not accidentally create a culture where career progression simply means learning more technologies. Engineering maturity is increasingly about judgment, not just technical breadth.
2. Quality Engineering
Quality Engineering should be treated as an engineering discipline in its own right rather than simply a collection of testing skills. The competency framework could cover:
Test strategy
Test design
Automation
Unit and integration testing
API and contract testing
UI automation
Performance testing
Security testing
Test data management
Quality engineering practices
Continuous testing
CI/CD quality gates
Observability
Production validation
Risk-based testing
Quality metrics
AI/LLM evaluation
At more advanced levels, Quality Engineers should increasingly influence how systems are designed to be testable and reliable, rather than simply testing completed software.
This also creates a natural progression from traditional testing towards modern Quality Engineering. It also moves the dialogue in quality engineering from “Can we test this?” to “How do we engineer this system so that quality can be continuously demonstrated?”
3. DevSecOps
Security and operations increasingly overlap with engineering, making DevSecOps a critical competency rather than a specialist activity performed at the end of delivery. Competencies could include:
Secure development practices
SAST
SCA
Secret detection
Container security
Infrastructure-as-Code security
Dependency management
CI/CD security
Identity and access management
Security automation
Policy as Code
Vulnerability management
Threat modelling
Supply-chain security
Runtime security
Incident response
Security observability
Engineers should progressively move from understanding security requirements to being able to build security into engineering processes and platforms. At advanced levels, competency should include the ability to design secure delivery pipelines, establish automated controls, and balance security requirements against engineering and business needs.
The goal is not to turn every engineer into a security specialist. It is to make security part of engineering competence.
4. Cloud Engineering
Cloud has become fundamental to modern software delivery, but cloud competency is much broader than knowing how to use a particular provider. A Cloud competency framework could cover:
Cloud architecture
Compute
Networking
Storage
Containers
Kubernetes
Serverless
Infrastructure as Code
Cloud security
Identity and access management
Observability
Reliability engineering
Cost optimisation
Disaster recovery
Scalability
High availability
Cloud-native design
Platform engineering
An engineer at an introductory level might understand basic cloud concepts and services. A proficient engineer might be capable of deploying and operating production workloads. An advanced engineer should be able to design secure, resilient, scalable, and cost-effective cloud platforms. This distinction helps prevent cloud competency from becoming a checklist of certifications and instead focuses on practical engineering capability.
5. Data Engineering
As organisations become increasingly data-driven and increasingly dependent on AI, data engineering becomes a foundational engineering capability. Competencies might include:
Data modelling
SQL
Data pipelines
ETL/ELT
Streaming
Event-driven architectures
Data lakes and lakehouses
Data quality
Data governance
Data lineage
Data security
Metadata management
Data observability
Data privacy
Schema management
Distributed data processing
At more advanced levels, engineers should understand not only how to move data but how to build reliable and trustworthy data platforms. This is particularly important for AI systems. If an organisation wants to build reliable AI applications, it cannot treat data quality as someone else's problem. Data becomes part of the engineering architecture itself.
6. AI Engineering
AI Engineering deserves its own competency area because it introduces capabilities that do not fit neatly into traditional software engineering. Competencies could include:
AI and machine learning fundamentals
LLM concepts
Prompt engineering
RAG
Embeddings and vector databases
Model selection
AI application architecture
Agentic systems
Tool calling
AI evaluation
Prompt and model testing
AI observability
Guardrails
AI security
Data preparation
Model and prompt lifecycle management
Human-in-the-loop design
AI governance
Responsible AI
Importantly, AI competency should not simply mean knowing how to use ChatGPT or an AI coding assistant. The more important capability is understanding how AI changes engineering decisions. For example, an AI Engineer should understand that an LLM application may produce probabilistic rather than deterministic behaviour. This affects architecture, testing, observability, security, and quality engineering.
AI therefore introduces a new dimension to engineering competency: The ability to engineer systems that incorporate probabilistic intelligence while remaining reliable, secure, observable, and governable. This will likely become one of the fastest-evolving areas of any engineering competency framework.
7. Architecture
Architecture is sometimes treated as something reserved for architects. That can create a significant capability gap. Senior engineers across disciplines should understand architectural principles and be capable of making appropriate architectural decisions within their domain.
Architecture competencies might include:
Architectural principles
System design
Distributed systems
Integration patterns
API architecture
Event-driven architecture
Security architecture
Cloud architecture
Data architecture
AI architecture
Scalability
Resilience
Performance
Architecture decision records
Technology selection
Non-functional requirements
Architecture governance
At higher levels, the focus should increasingly shift towards trade-offs and organisational impact.
An architect needs to consider not just whether something can be built, but whether it should be built, how it will evolve, what risks it introduces, and how it fits within the broader technology landscape. This is also where engineering judgment becomes particularly important.
8. Engineering Leadership
Leadership should not be reserved for people with “Manager” or “Head of” in their job title. Technical leadership is an important engineering competency in its own right.
Leadership competencies could include:
Technical decision-making
Communication
Mentoring
Coaching
Stakeholder management
Conflict resolution
Facilitation
Strategic thinking
Influencing without authority
Decision ownership
Change leadership
Engineering culture
Developing other engineers
Managing technical risk
This creates multiple possible career paths. An engineer does not necessarily have to become a people manager to progress. They could develop towards:
Technical Specialist → Senior Engineer → Staff Engineer → Principal Engineer → Distinguished Engineer
while another person might pursue:
Senior Engineer → Team Lead → Engineering Manager → Senior Engineering Manager → Engineering Executive
A competency framework can make these paths visible without suggesting that one is more valuable than the other.
Competencies Should Overlap
One of the most important design principles is that these disciplines should not exist as isolated silos.
Consider security.
Software Engineering needs secure coding.
Quality Engineering needs security testing.
DevSecOps needs automated security controls.
Cloud Engineering needs cloud security.
Data Engineering needs data protection.
AI Engineering needs AI security and responsible AI.
Architecture needs security architecture.
Leadership needs to understand security risk.
The same applies to areas such as automation, observability, resilience, and engineering practices. This means the framework should contain both Discipline-specific competencies. The capabilities that make someone effective within their specialist field.
Cross-engineering competencies
The capabilities expected across the broader engineering organisation. Examples include:
Agile engineering
Quality
Security
Automation
Cloud
Observability
Architecture
Data
AI
Operational ownership
Collaboration
Engineering judgment
This creates a T-shaped engineering capability model: deep expertise in one discipline combined with sufficient breadth across the engineering ecosystem.
Competency Levels Matter
The framework also needs a consistent way to describe progression. A simple model might look like:
Awareness: Understands the concepts and terminology
Working Knowledge: Can apply the capability with guidance
Proficient: Independently applies the capability in real situations
Advanced: Handles complex problems and guides others
Expert: Defines strategy, establishes standards and influences the wider organisation
The exact terminology is less important than having clear behavioural descriptions. For example, saying someone is “Level 4 in Cloud” is not particularly useful.
Instead it should say: “Can independently design and implement highly available cloud workloads, identify architectural risks, optimise cost and reliability, and guide other engineers through cloud design decisions.”
Now the competency becomes observable.
From Competency Framework to Career Framework
A competency framework becomes particularly powerful when connected to career development. Engineers should be able to look at the framework and understand:
Where am I today?
What am I good at?
Where are my gaps?
What does the next level look like?
What experiences do I need?
What skills should I develop?
What opportunities exist within the organisation?
This changes development conversations to focus less on learning specific technical skills and rather on improving across proficiencies as measured by the output across many different areas. That is a much more actionable conversation.
Measuring Organisational Capability
The framework can also provide a view beyond individual development. Leadership can aggregate competency information to understand organisational capability. For example:
The purpose is not to rank people.
The purpose is to identify organisational questions such as:
Where are we dependent on a small number of specialists?
Which capabilities are becoming strategically important?
Where should we invest in training?
Where should we recruit?
Which teams need additional support?
Which capabilities should become communities of practice?
Where should engineering standards be strengthened?
This turns the competency framework into a strategic engineering tool.
Don't Build a Framework That Becomes a Spreadsheet
There is a significant danger here. An organisation can spend months creating a beautifully structured competency matrix containing hundreds of skills, dozens of levels and thousands of cells - only for nobody to use it.
A useful competency framework should be:
Simple enough to understand.
Specific enough to be meaningful.
Flexible enough to evolve.
Practical enough to influence development.
And most importantly, it should be connected to real engineering work. Competencies should be demonstrated through:
Projects
Architecture decisions
Production experience
Code
Automation
Incident response
Technical leadership
Mentoring
Problem solving
Delivery outcomes
A competency should not be awarded simply because someone completed a training course. Training demonstrates learning. Experience demonstrates competency.
The Framework Must Evolve
Technology changes too quickly for a competency framework to remain static. Five years ago, organisations may have had relatively mature competency models for software development, testing, infrastructure and architecture.
Today, they need to consider capabilities such as:
AI-assisted engineering
LLM application development
AI evaluation
Agentic systems
Platform engineering
Cloud-native security
Data engineering
AI governance
Tomorrow, there will be capabilities we have not yet identified. The framework should therefore be treated as a living engineering capability model, reviewed periodically against technology strategy, organisational needs and the changing engineering landscape.
From Skills Inventory to Engineering Strategy
The real value of an engineering competency framework is not the matrix itself. It is what the organisation can do with the information. Once capability is understood, engineering leaders can make more informed decisions about:
Recruitment: What skills genuinely need to be acquired externally?
Learning: Where can capability be developed internally?
Career development: What experiences and opportunities can engineers pursue?
Team composition: Do teams have the right combination of skills?
Succession planning: Where is critical knowledge concentrated?
Technology strategy: Do we have the skills required to adopt our planned technologies?
Engineering governance: Where do standards, practices or controls need strengthening?
Investment: Where should engineering capability receive additional funding and attention?
This is where the competency framework connects directly to the broader engineering strategy.
Conclusion: Engineering Capability Is a System
Ultimately, an engineering organisation is not simply a collection of individuals with different technical skills. It is a system of capabilities.
Software Engineering depends on Quality Engineering. Quality depends on automation and architecture. Architecture depends on cloud, data, and security knowledge. AI depends on software, data, cloud, security and quality. Leadership connects these capabilities and creates the environment in which they can operate effectively.
A competency framework makes those relationships visible. It should not become another HR spreadsheet or an annual assessment exercise. It should become a living map of engineering capability - helping engineers understand how they can grow, helping teams understand where they need support, and helping engineering leaders understand where the organisation needs to invest.
And as engineering continues to evolve through cloud, automation, data and AI, understanding capability may become just as important as measuring delivery.
Comments