11/09/2021

How a small task force revamped and modernized a gigantic CI pipeline - DevOps Transformation

One of the typical cases for DevOps transformation is when a team has the ownership of a component they shouldn't own in the first place. For example, when you have super complex CI/CD, and the developers don't manage the CI/CD jobs directly but with help from the operations team.

Let's imagine this scenario ... Team A is developing great software, but due to the complexity of that software, they always relied on the Team B to operate their CI/CD jobs because they have more experience in that area. Now let's see the workflow of any change in the CI/CD:

  • Team A want's to add a new step in their CI/CD jobs, so they open a ticket for the Team B with all details.
  • Team B find will need to find a time slot for the new task because in the end of the day they also have their own tasks.
  • Team B found the time slot and started to work on the new change.
  • Team B does a high-level test/check to make sure the change works as expected, but also since they are not so involved in Team A project, they cannot really guarantee that the change is done as expected 100%.
  • Team B inform Team A with the changes and ask them to do more deeper checks to make sure the change is as expected.
  • Team A either confirms everything is as expected or asks for some fixes, then redo the process all over again.

As you see, this is simply because the 2 teams are coupled and it's super hard to hand over as it is with the current system complexity. And what exacerbates the problem when the use case is technically complex!

And as a rule of thumb when we are talking about DevOps, DevOps doesn't mean that all engineers would have the "same" knowledge and skills, but have the "common" knowledge and skills! (so I don't expect that every developer should be a system or Kubernetes guru!).

Hence comes the "developer enablement" part in the DevOps transformation, where the DevOps engineers should build systems and workflows that are convenient and handy to use, and the most important thing ever, to share knowledge. And by doing that, teams become more autonomous and move faster.

This becomes super important as the company growing otherwise operations teams would need to grow linearly as other development teams grow. In contrast, in the DevOps model, you could have a small team that is able to support many teams at the same time by providing a unified platform and tools to be used by software engineers.

So what's all that about?

Camunda Platform Runtime is an open-source BPM engine, and it has an advanced CI setup (9 databases, 6 application servers, 7 Java versions, multiple types of tests like unit, integration, E2E), we talk about 400 jobs per version (the CI setup needs to support 6 previous versions of the core engine). However, it used old complex Jenkins DSL freestyle jobs, which had many downsides and slowed everything down!

So we decided to change that! 3 engineers from 2 teams collaborated and tamed that gigantic CI and use the modern declarative pipelines. And that involved architecting and design the new CI flow, DevOps transformation, knowledge share, and many more. TBH, this project is one of my best projects so far!

It's hard to imagine this project to be done without Jenkins! It's simply battle-tested software! Also Jenkins' extensibility helped us a lot, using Jenkins plugins and shared libraries; Jenkins is more like a framework to develop CI/CD that can fit any use case!

However, sometimes it's so complex for simple use cases. Also, because of its flexibility and its huge ecosystem, things could go wild easily! So if you asked me, I'd say that Jenkins is a double-edged sword!

Sounds interesting? Read more details about our revamp success story on JenkinsIsTheWay.io

Enjoy :-)

Powered by Blogger.

Hello, my name is Ahmed AbouZaid and this is my "lite" technical blog!

I'm a passionate DevOps engineer, Cloud/Kubernetes specialist, Free/Open source geek, and an author.

I believe in self CI/CD (Continuous Improvements/Development), also that "the whole is greater than the sum of its parts".

DevOps transformation, automation, data, and metrics are my preferred areas. And I like to help both businesses and people to grow.

Popular Posts