DevOps and Continuous Delivery for Oracle SOA and BPM
June 21, 2015
The goal of Continuous Delivery and DevOps is to help software development teams drive waste out of their process by simultaneously automating the process of software delivery and reducing the batch size of their work. This allows organizations to rapidly, reliably, and repeatedly deliver software enhancements faster, with less risk and less cost.
Continuous Integration (CI) is the practice of automatically building and testing a piece of software; either each time code is committed by a developer or in environments with a large number of small commits, or a long-running build on a regular scheduled basis.
Continuous Delivery (CD) goes a step further to automate the build, packaging, deployment, and regression testing, so that it can be released at any time into production. Continuous deployment takes this another step further, in that code is automatically deployed into production, rather than when the business decides to release the code.
DevOps (development and operations) builds on Continuous Delivery and is used to mean a type of agile and collaborative relationship between Development and IT Operations. The goal of DevOps is to change and improve the relationship between Development and Operations by advocating better communication and collaboration to enable the business to deploy features into production quickly, with minimum risk and to detect and quickly correct problems when they do occur, without disrupting other services.
Work in Small Batches
The batch size is the unit at which code under development is promoted between stages, such as SIT, UAT, and Pre-Prod, in the development process. Under a traditional development process, the code from multiple developers working for weeks or months is batched up and integrated together. During the integration process, numerous defects will be surfaced. Some will be the result of a lack of unit testing, but many will be down to invalid assumptions about the various pieces of code developed in isolation and how they will work together as part of the overall solution.
This is especially the case for Oracle SOA and BPM projects, which involve integrating multiple systems together. It is a common mistake for all parties to agree on the interfaces between the systems and then go off and code independently, with each party making invalid assumptions about how the other systems will behave. The bigger the batch, the longer these assumptions remain undiscovered, and the greater the number of defects in the batch. A significant amount of the time taken to fix a defect is actually spent trying to isolate the problem and determine the root cause, rather than fixing the problem.
The issue with a big batch is that many of the defects are interwoven, and that the volume of code that needs be analyzed to troubleshoot a defect is greater. In addition, code based on invalid assumptions can often require significant re-work once these invalid assumptions are discovered; the longer these remain undiscovered, the greater the amount of invalid code written and the greater the amount of re-work required. As a result, the amount of time taken to identify and fix defects increases exponentially with the batch size.
Continuous delivery promotes the use of small batches, where new features are developed incrementally and promoted into the various test environments on a regular and frequent basis. Small batches mean problems are caught immediately and instantly localized, making it far simpler to identify the root cause and fix. In the case of invalid assumptions, these are discovered far earlier in the process, when they are cheap to fix, and results in higher-quality software.
Software components that are implemented in isolation are full of assumptions about the other components with which they will be integrated. The sooner we can identify these assumptions, the smaller the impact and the associated waste will be. Small batches enable us to integrate these components earlier in their respective development lifecycles, and thus reduce the risk and overall impact on the project.
Process for Releasing / Deploying Software MUST be Repeatable and Reliable
To enable the development team to work in small batches, we need to remove the waste in the current build and deployment process. This requires that the process for releasing/deploying software MUST be efficient, repeatable, and reliable.
This is achieved by automating each step in the software delivery process, as manual steps will quickly get in the way, become a bottleneck, or risk introducing unintended variation. This means automating the build and deployment of code, the provisioning of middleware environments, plus the testing of code.
Minimise Differences Between Environments
A common anti-pattern is deploying to a production-like environment only after development is complete. It is unfortunately all too common for solutions to fail on first deployment to any environment. Small inconsistencies between environments, such as disparities in the configuration of deployed SOA/BPM composites and OSB services, adapter configurations, WebLogic resources, or applied patches can cause issues with deployed code that are difficult to diagnose and rectify. This means that there can be almost no confidence that a particular software release will work successfully if it has never been tested in a production-like environments. To avoid this, deployment should always be to production-like environments. Each time we make a deployment to any environment, we are making changes to that environment, which means that it is no longer in-alignment with production. If the release passes, and the code gets promoted through to the next stage and into production then that is not an issue. But if the release fails, we need to restore the environment back to its pre-deployment state, prior to deploying the next release.
Build Quality In
W. Edwards Deming, in his famous management guideline, stated:
"Cease dependence on mass inspection to achieve quality and improve the process and build quality into the product in the first place”.
This means ensuring improvement and quality assurance at every step of the value stream instead of testing just the final product for compliance to requirement.
For software, it translates to writing automated tests at multiple levels (unit, component, and acceptance) and automating their execution as part of the build – test – deployment pipeline.
This way, whenever a commit happens (which is a change being made to either of the application, its configuration, or the environment and software stack that it runs on), an instance of the pipeline runs and so do the automated tests that verify / validate business expectations in form of test cases.
Applying Continuous Delivery in the development of Oracle Middleware projects can deliver significant reductions in development time and costs.
Download White Paper
In subsequent posts I will go into further details on how we can apply DevOps and Continuous Delivery to Oracle BPM and SOA projects. Click here download a white paper on Best Practice for Implementing Continuous Delivery for Oracle Middleware.