Isn't it frustrating when the CI/CD pipelines triggers all the workflows when small, trivial change has been made to the code and pushed to the Version Control System? While frequent commits and automated pipelines are crucial for maintaining code quality and rapid deployment, triggering all CI/CD processes for every minor change can be counterproductive.
I also faced the same issue many time, and I went on to search the best possible practices to avoid unnecessary triggering of workflows. There wasn't any standard solution available for this challenge, perhaps I was able to secure different ways (cheat codes) which were widespread among different resources. This blog summarizes all my finding regarding how to avoid unnecessary triggers for a small, trivial changes in the code and streamline the workflow.
One of the most effective ways to prevent all pipelines from running on every commit is to implement selective triggers. This approach allows you to specify which changes should initiate the CI/CD process.
1) Use Path Filters
We can configure your pipeline to run only when meaningful changes are made by filtering commit messages - The pipeline won't run if only .md or .txt files are changed.
push:
branches:
- main
paths-ignore:
'**.md' # Ignore markdown files (e.g., documentation updates)
'**/*.txt' # Ignore text files
pull_request:
types: [opened, synchronize, reopened]```This configuration would not trigger the pipeline when changes are made to the .md or .txt files, ignoring modifications to documentation or the README file.
2) Leverage Commit Messages
Some CI/CD systems allow you to trigger or skip builds based on commit message patterns. You can add [skip ci] to the commit message, so that pipeline will recognize this and skip execution. For instance,
git commit -m "Updated README [skip ci]"
You can use specific keywords in your commit messages to indicate whether a full pipeline run is necessary.
3) Use Conditional Stages
Structure your pipeline with conditional stages that only execute when relevant changes are detected. You can add a step in your CI pipeline to check whether meaningful changes have been made before proceeding with the build. This approach allows you to run specific parts of your pipeline based on the nature of the code changes:
> check_changes:
> runs-on: ubuntu-latest
> steps:
> - uses: actions/checkout@v3
> - name: Check code changes
> run: |
> if git diff --name-only HEAD^ HEAD | grep -qvE 'README.md|docs/'; then
> echo "Relevant changes detected. Proceeding..."
> else
> echo "No relevant changes. Skipping pipeline."
> exit 0
> fi```The script checks if changes were made outside ignored files like README.md and docs directory and eventually exits the pipeline if no significant changes are found.
Some better programming practices:
Use Feature Flags
Implement feature flags in your codebase to decouple code deployment from feature release. This allows you to merge small changes into the main branch without necessarily triggering a full deployment process.
Commit Early and Often
It is highly encouraged to commit frequently with small, atomic changes. This practice not only aligns with CI/CD best practices but also makes it easier to identify which changes require full pipeline execution.
To cap it all, by implementing these strategies, you can significantly reduce unnecessary CI/CD pipeline triggers for small code changes. This optimization not only saves computational resources but also speeds up the feedback loop for developers, allowing them to work more efficiently. Remember, the goal is to strike a balance between maintaining code quality through comprehensive testing and achieving rapid, efficient development cycles.
