Top CI/CD providers for Flutter
CI/CD is a method to frequently deliver apps to customers by introducing automation into the stages of app development. The main concepts attributed to CI/CD are continuous integration, continuous delivery, and continuous deployment. CI/CD is a solution to the problems integrating new code can cause for development and operations teams (AKA “integration hell”). (https://www.redhat.com/en/topics/devops/what-is-ci-cd)
Developers enjoy continuous integration, and most mainstream smartphone developers have found it incredibly easy to integrate with CI frameworks. This wasn't always so…
A few years ago CI/CD was considered a practice for most web developers. But in recent years it’s become best practice and a core task/process in setting up a project, we must pick a service that can offer the best experience both for teams and the product in general.
Top 5 CI/CD providers for Flutter:
There are a lot of other CI/CD providers but e.g. Jenkins/ TravisCI but we’d stick with these five because they are more focused on proving this service to mobile applications/ mobile development teams.
Here’s a list of the top CI/CD providers currently for mobile applications developed with Flutter.
- Circle CI
- Fast Lane
- Github Actions
- Bitrise
- Codemagic.io
My Method:
So I’d be rating these 5 platforms by the following:
- User Experience: How easy is it to set up? Are there any configurations?
- Features: What awesome features make this CI/CD provider and unique?
- Speed: How fast was the total overall experience?
Setting up our Test CI project
For test purposes, we would be creating a default counter Flutter project by running flutter create test_ci which will be pushed to GitHub. This would be used as a base project to test each one of the listed CI/CD providers.
Circle CI
If you’ve ever been on youtube there’s a slim chance you might have seen an ad talking about Circle CI, if you're a dev there’s a higher chance you have seen it. Circle CI has been there for a while now.
Top features:
- Up to 16-way parallelization
- Ability to test a lot of code pushes concurrently
- Docker Integrations
- Comprehensive cache dependencies
- Builds can be automated with Fastlane
Integrations: Bitbucket, GitHub, Sauce Labs, Coveralls, Heroku, Docker, Firebase, Slack, Jira, etc.
Clients: Udacity, Nextdoor, Kickstarter, Teespring, Coinbase, Avant, etc.
Pricing: Pricing is flexible consisting of 3 plans: Free, Performance, and Scale
Let’s Go
Okay now, let’s dive in and try to run our demo test_ci project here.
Testing Circle CI
Create a new Circle CI account, on checking their list of supported languages/frameworks. Flutter is not on the list but we have Android and iOS flows. There’s an official article on how to set up the build process for both Android and iOS, it contains a lot of manual setup and knowledge of their yml structure.
Create a folder called .circleci in your project’s root directory and next create a configuration file such that the file at /YOUR_PROJECT/.circleci/config.yml.
The first thing to note is like many CI/CD platforms, Circle CI supports using docker images in your build workflow.
Our Experience
The above workflow DID NOT WORK, probably because we didn't have a better understanding of how CI/CD on Circle CI works. Finding proper documentation too on how to set it up from scratch was futile as well. After searching for a long time for suitable simple workflow, basically trial by error as the workflow on their page was from 2019 hence no longer testable.
Tried another workflow and it was really not as fun as expected it to be. This is because we are missing tools/commands to run/directly integrate with Flutter. E.g. Fastlane. We weren’t able to get it running with Circle CI.
Circle CI is very powerful too in professional hands, but to the eyes of a junior or even midlevel developer/ QA tester, it demands a lot of technical know-how and experience to understand, manage and run workflows for mobile applications.
— Features: Circle CI has a lot of integrations, lots of analytics that could prove useful to developers and testers alike.
— Speed: Average build time with errors 10 seconds.
Fast Lane
Fastlane is one of the most popular tools for Flutter CI/CD. It's not just a standalone CI/CD tool as it can be integrated into other CI tools. It uses Fastile scripts and a CLI tool to run these scripts and perform both pre-build, post-build, and build actions.
Top Features
- Fastlane integrates effortlessly into existing CI services including Travis CI, Circle CI, Jenkins CI, and Bitrise CI.
- Automatically capture localized screenshots for each language and device your app supports
- Easily store configurations for team-wide use.
- Easily distribute beta builds to testers
- Publish artifacts to respect stores
- Automatic code signing
Testing Fastlane
We would still be using our test_ci with the default flutter app boilerplate code to text Fastlane.
- Install the Fastlane CLI by running:
$ sudo gem install fastlane. - By running
cd ios && fastlane ioswe can run the CLI and easily follow the setup process.
Let’s try and create our own flow manually:
Scripts
- Create a new file in your android project at
/fastlane/Fastfile. - Then Do the same for the ios project.
These scripts will carry out the defined task such as flutter run && flutter build.
Our Experience
Fastlane has a really smooth experience because of it’s documentation and tool. It’s fast and easy to install and run but might be a bit complex if you haven't dealt with a ruby-like syntax for writing workflows and commands.
By running cd ios && fastlane beta we can automate our build process
Similarly by running cd android && fastlane beta , we can see our automated build process.
Note
One thing to note is that using Fastlane in the cloud you would need to use a cloud CI tool like TravisCI, Codemagic CI, or CircleCI. Where:
• A Flutter environment would have to be created,
• Your project pulled from your choice VCS.
• Fastlane bundle installed and ran.
Github Actions
Released on November 13th of 2019, Github Actions has proved a very useful tool for developers globally. Github Actions is great because it brings CI/CD directly to your favorite VCS.
Top Features
- Run workflows from GitHub event (push, merge, create PRs).
- Run Linux, macOS, Windows, ARM, and containers, Matrix builds, multiple languages.
- Community-powered workflows.
Users: Bepro Company. HENNGE K.K, Craftbase, UpstageAI, GitBook, Kong, General, Evooq.
Integrations: Jira, Trello, Asana, HubSpot, Zendesk, Basecamp, Wrike, e.t.c.
Pricing
Github Actions uses a pay-as-you-go pricing plan for private repositories but for public repositories, it's completely free.
Testing Github Actions
By clicking the “Setup workflow yourself” button on the Actions page, we can get started on editing our configuration file as follows:
Next, we commit the updated configurations to git.
Note
To create a GitHub release we would need to create a token from our GitHub account and then save it to the repo as a secret and then create a tag for the artifact to be released to. Unless our workflow would break at:
1. Configure Token
Navigate to https://github.com/settings/tokens/new and select repo and workflow permission then generate. Making sure to copy the generated token.
Lastly, we add the token to our project as a secret.
2. Create a Release Tag
According to https://github.com/ncipollo/release-action#notes
You must provide a tag either via the action input or the git ref (i.e push / create a tag). If you do not the action will fail.
If the tag of the release you are creating does not exist yet, you should set both the
tagandcommitaction inputs.commitcan point to a commit hash or a branch name (ex -main).
So we modify our workflow to this:
# now release the just created build
- name: Create a Release APK
uses: ncipollo/release-action@v1
with:
artifacts: "build/app/outputs/apk/release/*.apk"
token: ${{ secrets.TOKEN }}
commit: master
tag: "latest"We pull our latest changes from the repo, next we run the 4 highlighted commands below to create a “beta-1.0” tag.
Result
Navigating to the workflow details page we can watch our build go in in realtime:
Once it’s done go to your repository’s releases page and voila:
Our Experience
Github Actions is great, but a lot of work has to be done manually to get this flow up and running. Lots of manual tuning and there’s no GUI if you want to get anything done you need to modify the configuration file. Also releasing binaries from builds would need you to configure tokens and git tags. In all a bit complex for beginners.
Bitrise
Bitrise — Mobile Continuous Integration and Delivery platform offers a tool to Save time, money, and developer frustration with fast, flexible, and scalable mobile CI / CD that just works.
Top Features
- Flexible configuration and intuitive setup
- Helpful setup wizard
- Intuitive UI and easy to use
- Security with no compromise
- Firebase Test Lab integration
Integrations from Bitrise include: iOS, Android, React Native, Android SDK, Apache Cordova, Xamarin, GitHub, Bitbucket, GitLab, Bower, etc.
- Users: Virgin Mobile, TNT, Zeplin, Duolingo, Starva, Wunder, and more
Pricing
Bitrise gives a lot of plans and also a 14-day free trial for cloud plans but you can use it for free for personal projects.
Testing Bitrise
Jumping in right away:
- Signup with Github.
- Selected our repo. Our project framework was detected and it started automatically setting up a default workflow:
A few minutes more and it’s done and we can move directly to the created workflow that is already running:
Our Experience
Overall Bitrise is nice to use, easy to startup, and run with. We are offered a bitrise.yml configuration file plus a GUI Workflow Editor. That was easy.
Codemagic CI/CD
Codemagic CI/CD is dedicated to building tools that aid Flutter/Mobile Applications gives you simple powerful ways to automate tests and workflows with simple clicks/config files and an easy-to-use User Interface.
Top features:
- Automatic configuration and setup
- Integrates with Bitbucket, GitHub, GitLab, and any other source repository
- Concurrent builds
- Custom unit and UI tests using emulators or actual hardware
- Test automation with Firebase / Amazon Device Farm and test parallelization
- Automatic publishing to Google Play, iTunes Connect, HockeyApp, Crashlytics, TestFairy
- Scheduled builds
- Data encryption for all sensitive data
- First-class customer support
- Easy-to-understand documentation
Integrations: Appium, BitBucket, GitLab, Fastlane, Slack, Email, AWS, Azure, Browser Stack, Crashlytics, iTunes Connect, Gradle Build Tool, Cordova, CocoaPods, Codecov, Carthage, etc.
Clients: Invoice Ninja, Toyota, Schneider Electric, Criton and more.
Pricing: Codemagic has both free, a flexible pay as you go, and business plans. These rates are cheaper than most if not all other CI/CD providing similar services and quaility.
Testing Codemagic.io
Let’s take it for a spin.
• Go to https://codemagic.io
• Login with your preferred option
You will be presented with a list of your Flutter projects from your VCS (Github, Bitbucket, Gitlab). You can select from there or manually click on “Add repository”.
After click “Add application”, we can run a build right away with the default configurations already set.
Next, we click “Start new Build” to begin a new build process.
A few minutes later and we are notified that our build is done and our app bundle is ready to be downloaded:
Everything in less than 10 mins.
Summary
We have gone through a lot of process procedures and tested a couple of CI/CD tools. Here’s a summary
- Fastlane :
— User Experience: Fastlane is more of an automaton than a full-on CI/CD tool as it has to be integrated with other cloud-based CI/CD platforms like Bitrise and Travis CI. While it’s a scripting language (ruby) Fastfiles are easy to read and understand, it might take a lot of time to learn and understand them, save for the CLI that makes it easy to setup.
— Features: Fastlane is a powerful tool, with the ability to integrate into multiple tools and mainstream CI/CD tools like Bitrise, Circle CI, and the likes.
— Speed: Average build time 52 seconds, Fastlane lives up to the “fast” in its name, as it is blazing fast stand-alone but there’s no telling how much it will slow down when integrated into other CI workflows.
- Github Actions :
— User Experience: Github Actions is superb because right from your repository you can set up a workflow modify it and run it, but it lacks a simple easy to use workflow editor, which means you have to solely rely on writing scripts in yml or yaml . While having this as is nice having a GUI Where users can easily click their way to a build can save time and a lot of stress.
— Features: Github Actions has a very large collection of actions that can be used, and extended in building a CI workflow.
— Speed: Total build time was 6 mins 21 sec Relatively slower than traditional CI workflows because there's no direct caching of dependencies.
- Bitrise :
— User Experience: Easy to set up, easy to use ad integrate, but with lots of items displayed at the same time it's easy to get lost in your workflow, config files seem a bit complex to modify and also you need to modify the workflow manually before building.
— Features: Bitrise offers over 300+ integrations into services and platforms like slack, Test/Code coverage services e.t.c.
— Speed: With a base build time of 8 mins 45 sec, Bitrise is fast when it comes to its execution, but a little slower than Github Actions.
- Codemagic CI/CD
— User Experience: Set up a workflow and run from scratch in less than 10 mins, easy to use with proper documentation, lots of integrations too, experience is supersmooth and the default workflow can be run right away without any configurations.
— Features: Codemagic offers seamless integrations into services and platforms.
- — Speed: With a base build time of 6 mins 37 sec, Codemgic CI/CD is blazing fast.
In the end, the choice of your CI/CD tool should be based on your needs and how easy you feel it would be to integrate into your workflow or your automated build process.
Other Links
