DevOps on IBM i
IBM i Production Deployment: Protection, Rollback, and Release Safety
A practical look at protecting IBM i production deployments, handling emergency fixes, planning rollback, managing database changes, and making releases safer.
Production deployment on IBM i should be deliberate, controlled, traceable, and boring. Once a team has a Git and build flow in place, the next step is protecting production and planning for failure before failure happens.
A Git and TOBi build flow gives IBM i teams a stronger delivery foundation.
But production needs extra care.
Production is where users, jobs, batch windows, integrations, database objects, reports, and business operations all come together. A deployment process that works casually in development may not be safe enough for production.
This article continues the Git and TOBi deployment discussion with production protection, emergency fixes, rollback planning, database changes, and release safety.
Key idea
Production deployment should not depend on memory, heroics, or luck. It should follow a known path with clear controls and a realistic rollback plan.
Protecting production
Production should not be treated like development.
The main branch should represent an intentional release, not a casual update.
A direct push to production should usually be restricted because production changes carry different risk. A production deployment can affect users, scheduled jobs, integrations, batch processes, reports, database objects, downstream systems, and business operations.
A practical Git flow should make production changes deliberate.
dev
↓
uat
↓
main
↓
production
The goal is not to slow the team down unnecessarily.
The goal is to make production deployment predictable.
Production deployment should be boring.
The more predictable the production flow is, the less risk the team carries during release.
Practical production protections
A production branch should have stronger controls than development.
Useful protections may include:
- Restricting who can update
main - Requiring merges instead of direct commits
- Blocking force pushes
- Blocking accidental branch deletion
- Requiring a successful build before promotion
- Logging every production deployment
- Tagging or tracking successful releases
- Keeping a last known stable production reference
The exact controls depend on the team and tooling.
But the principle is simple:
Production should only change through a known and reviewable path.
If a team cannot answer what changed, who changed it, when it changed, and how it reached production, the deployment process needs improvement.
Dev to UAT to Main
A simple promotion process may work like this:
Developer completes work
↓
Feature branch merges into dev
↓
dev deploys to development
↓
developer testing happens
↓
dev merges into uat
↓
uat deploys to test
↓
business or QA validates
↓
uat merges into main
↓
main deploys to production
This gives the team a clear path.
It also makes the environment state easier to understand.
If something is in uat, it should be in the UAT environment.
If something is in main, it should represent production-ready code.
The branch structure should help the team understand where a change is in the delivery lifecycle.
Why IBM i promotion needs discipline
IBM i applications often have long-running business processes, scheduled jobs, integrations, batch windows, and production dependencies.
A small source change can have a wide impact.
For example:
- A display file change may require dependent programs to be recompiled.
- A service program change may affect multiple bound programs.
- A table change may affect programs, queries, logical files, indexes, constraints, or external integrations.
- A CL change may affect job setup, library lists, overrides, or batch processing.
- A command change may affect both interactive and batch users.
- A trigger or stored procedure change may affect applications that do not directly call the changed program.
This is why promotion discipline matters.
Git helps track source movement, but IBM i knowledge is still required to understand impact.
A good deployment process combines both:
Git discipline + IBM i platform knowledge
That combination is where safer delivery begins.
Emergency fixes
Emergency fixes happen.
A production issue may require a fast change. In many IBM i shops, the traditional response is to compile directly into production and move on.
That may solve the immediate issue, but it can create long-term risk if the change is not brought back into the Git flow.
A safer emergency fix process should still have a path back to source control:
Emergency production fix
↓
Capture source change
↓
Commit to Git
↓
Merge back into dev and uat
↓
Confirm environments are aligned
The goal is not to pretend emergency fixes will never happen.
The goal is to make sure emergency fixes do not become hidden differences between production and source control.
Rollback thinking
Rollback should not be an afterthought.
It is easy to say:
We can just go back to the previous version.
But on IBM i, rollback may not always be that simple.
Source rollback and object rollback are not always the same thing.
A rollback may involve:
- Rebuilding previous source
- Restoring previous objects
- Reversing database changes
- Handling data conversions
- Restarting jobs
- Reprocessing failed messages
- Revalidating dependent objects
- Reconnecting integrations
That does not mean rollback is impossible.
It means rollback needs planning.
Tracking the last stable production version
One useful approach is to track the last successful production release.
For example:
PROD-STABLE
Update the stable reference only after deployment, health checks, and agreed business checks pass. A successful compile alone does not establish that a release works in production. Keep an immutable release tag and a deployment record containing the commit, object package, configuration version, and database migration level.
Conceptually:
main deployment starts
↓
build succeeds
↓
deployment and release checks pass
↓
PROD-STABLE is updated
If a later production deployment fails, the process has a known point to return to.
Conceptually:
main deployment starts
↓
build, deployment, or release checks fail
↓
stop promotion and assess partial changes
↓
restore the verified previous release and compatible database state
↓
repeat release checks and log the rollback result
This does not solve every rollback problem.
Checking out an earlier commit and running an incremental build is not a complete rollback: newer object timestamps can cause targets to be skipped, and objects introduced by the failed release may remain. Use a tested restore or clean-build procedure that accounts for added objects, bound dependencies, active jobs, authorities, and database changes.
But it is much better than having no stable reference at all.
Rollback is not only a Git problem
A common mistake is thinking rollback means only checking out an older commit.
That may be enough for a simple program change.
It may not be enough for a database change, data conversion, or object dependency issue.
For example, rollback becomes more complicated when a deployment includes:
- Table changes
- Key changes
- Data conversions
- New constraints
- Removed columns
- Renamed objects
- Service program signature changes
- Long-running batch jobs
- Messages already sent to external systems
A mature rollback plan should separate different types of changes.
Program rollback may be one process.
Database rollback may be another.
Integration rollback may require coordination with external systems.
Database changes need extra care
Database changes are often the hardest part of rollback.
Changing a program is one thing.
Changing a table, key field, constraint, index, or data structure is another.
A table change may involve:
- Data conversion
- Dependent objects
- Long-running rebuilds
- Locks
- Constraints
- Foreign keys
- Logical files
- Application downtime
- Backout scripts
A good deployment process should treat database changes differently from ordinary program compiles.
A practical approach is to separate database migration scripts and review them carefully before production.
For example:
src/qsqlsrc/migrations/
20260706_add_invoice_status.sql
20260708_alter_customer_key.sql
Each migration should be reviewed with rollback in mind.
Before running a production migration, the team should understand:
- How long it may take
- What objects depend on it
- Whether it requires downtime
- How data will be backed up
- What the backout plan is
- Whether the change can be safely repeated
- Whether the application can run during the change
Database modernization is valuable, but it must be handled with discipline.
A realistic first production approach
A first version of production deployment does not need to be perfect.
A practical first version may include:
1. Only main deploys to production.
2. Only approved users can update main.
3. Every production deployment logs the branch, commit, environment, and result.
4. Verified production deployments update a stable reference.
5. Failed production builds stop clearly.
6. Rollback steps are documented, even if not fully automated yet.
That is already a major improvement over manual production compiles with no clear release history.
Start with control and visibility.
Automation can mature from there.
What makes IBM i DevOps different
IBM i DevOps has to account for platform-specific realities.
For example:
- Source members compile into objects.
- Compile order can matter.
- Library lists can affect compile and runtime behavior.
- Service programs and binding directories require planning.
- Database objects may have dependencies.
- Display files and printer files may affect dependent programs.
- Commands may depend on CPP programs.
- Job descriptions may control runtime behavior.
- Scheduled jobs and batch windows matter.
- Authority and adopted authority can affect execution.
- Production may have long-running jobs that cannot simply be interrupted.
This does not make DevOps impossible on IBM i.
It means DevOps has to be designed with IBM i in mind.
The mistake is not using Git, automation, or modern tools.
The mistake is using them without respecting how IBM i applications are actually built and operated.
A practical first version
A good first version does not need to solve everything.
Start with something simple and reliable:
1. Developers work in feature branches.
2. Changes merge into dev.
3. dev deploys to the development environment.
4. dev is promoted to uat.
5. uat deploys to the test environment.
6. uat is promoted to main.
7. main deploys to production.
8. Verified production deployments update a stable release reference.
That is already a big improvement over manual source movement, unclear promotion steps, direct production compiles, and environment-specific tribal knowledge.
Once the first version is stable, improve it gradually.
Later improvements
After the basic flow is working, teams can add more controls and visibility.
Useful next steps may include:
- Pull request checks
- Build status reporting
- Deployment dashboards
- Automated test cases
- Static analysis
- Impact analysis
- Object dependency checks
- Change approval workflows
- Deployment notifications
- Rollback automation
- Environment drift checks
- Security checks before promotion
- Audit history for deployments
These improvements should come after the basic flow is reliable.
Do not overbuild before the team trusts the process.
Start small, but design in the right direction.
The first IBM i DevOps flow does not need every feature. It needs to be clear, repeatable, and safe enough that the team can build on it.
Common mistakes to avoid
Some mistakes are easy to make when introducing Git and deployment automation on IBM i.
Treating Git as only a backup
Git should not be used only as a place to store a copy of source.
The real value comes from history, branching, review, promotion, traceability, and controlled deployment.
Ignoring environment differences
Development, UAT, and production are not the same.
Library lists, database libraries, object libraries, authorities, job descriptions, data volumes, and scheduled jobs can all differ.
Deployment automation should account for those differences.
Allowing uncontrolled production updates
If anyone can push directly to production, the team has not really improved control.
Production should have stronger rules than development.
Automating before understanding the manual process
Automation does not fix a broken process.
It can make a broken process fail faster.
Before automating, understand the current build and deployment steps clearly.
Forgetting rollback
Rollback planning should happen before the first major failed deployment.
Waiting until production is down is too late.
The human side of IBM i DevOps
Tools matter, but people matter too.
A team moving from manual source handling to Git-based delivery may need time to adjust.
Developers may need to learn:
- Branching
- Merging
- Pull requests
- Build logs
- Promotion rules
- How deployment is triggered
- What to do when a build fails
Operations teams may need to understand:
- Which branch maps to which environment
- How production releases are approved
- Where deployment logs are stored
- How rollback works
- Who owns failed deployments
A successful IBM i DevOps process should be understandable to both developers and operations teams.
If only one person understands the deployment flow, the team has not really reduced risk.
The goal
The goal of Git and TOBi on IBM i is not to make IBM i look like every other platform.
The goal is to make IBM i delivery safer, more traceable, more repeatable, and easier to understand.
Git gives the team source control, history, branching, and promotion structure.
TOBi helps organize repeatable IBM i builds.
Deployment automation connects branches, environments, and build steps.
Together, they can help IBM i teams move from manual change handling toward a more modern engineering process.
Modern IBM i DevOps should respect the platform while improving how teams deliver change.
That is the balance.
Respect the platform.
Preserve the business logic.
Modernize the engineering around it.
References
Documentation and supporting sources for this article.

Comments
Share your thoughts, questions, or real-world IBM i experiences related to this article.