As you’re probably aware by now, we are carrying on with our work on Aegees, it’s not just a new messaging application but we intend to make it the safest messenger available in the Galaxy. We have pledged to give our users a completely hack-proof app, something no one has managed to accomplish before now — and on top of that, we’re committed to doing it in the shortest time possible.
Toiling away, day and night, to live up to our promises, we were practically “drowning” in the enormous quantity of code we had to generate. Our code management process, that had worked so well during earlier stages, began to crack under the pressure due to the sheer volume of development. Well, we figured it was time to roll up our sleeves and find a new solution that would keep us on track to the glorious Aegees future that we all dream of every night — at least we do on those rare nights when we aren’t working, of course.
We stick rigidly to the principle of ‘separation of concerns’, because nothing works better. All our engineering work is stored in two major repositories; namely the aegees-client repository and the aegees-server repository. The work in both places is performed in these three major branches:
· dev, for all new development;
· stable, where we work on performance stability for each new chunk of functionality we write;
· release is where we keep a full and exact copy of the current release at any given time.
The client repository has three additional threads where we work on the client-side of the app for iOS and Android devices, and of course the desktop version. They are called respectively:
This structure allows us to clearly separate back-end and front-end development, the raw code and the ‘almost ready’ code that just needs to be finalised. We chose to work with Github, because, as it justifiably claims, it is the world’s leading software development platform, and we can vouch for that.
Our Basic Workflow Overview
Usually, dev is the target branch for any code we write for new functionality. We use it for all back-end (server) development and all front-end (client) cross-platform development. For platform-specific tasks, we use the dev-ios, dev-android and dev-desktop threads.
Let’s briefly go over the workflow for a new task. A developer starts work by creating a sub-thread in the target branch; its title consists of target branch name and a task number, for example, AEG-1234. Usually, each task in completed in a series of logically connected changes and revisions, which in Github are called “commits”. When that’s done, the responsible developer submits a request to merge the sub-thread with the target branch.
After that, the new code is reviewed and validated by other developers on the team, and the original developer is given the green light to merge his code into the main branch.
One of our vital rules is that all developers reviewing any code to be merged into the main branch have verify that it is free of any detected bugs that might need more than two days to fix. We introduced this sign off requirement to keep up with our airtight project schedule.
Similarly, developers, who write code for one of the three client platforms, work within their respective sub-threads and request permission to merge with the main branch, i.e. dev. They in turn have their work reviewed by peers. We also regularly update dev with the newest verified code for all platforms to ensure that the main branch always has the complete and latest version of all the development.
We believe that this approach to structure and workflow brings several advantages:
· Developers get to distribute tasks themselves and are able to complete them without testers getting involved;
· The logic of branches and sub-threads, along with the mandatory peer review, minimizes the risk of any bad code getting through to the next stage;
· It keeps the continuous integration process going efficiently, because everyone works in a sub-thread and all merge requests are moderated;
· There are no rigid rules for baseline updates so we can regulate the workload flexibly while keeping the main branch up to date.
Further Development Workflow
At the next stage, we work with the code either in the stable thread — when our goal is to introduce fixes before releasing the product, or in the release thread — that’s when we need to make changes to the released version.
When errors are discovered in the released version, fixes can be performed in either of the two threads depending on how serious an error is and how much of a change it needs to be fixed. That decision rests with the lead developer in charge of the release.
Once that is clear, the workflow is similar — a new sub-thread is created and the work completed in one “commit”. After that, the developer in charge requests a merger.
Before permission to merge can be given, the code needs to undergo review, pass autotests, be approved by testers (who can run any tests they see fit), and get the ok from the relevant team leader responsible for either back or front-end, or platform development, or a relevant combination of those.
To merge the code into the release branch, the request must additionally be approved by the head of development, project manager and release manager.
Once the release baseline is updated, the next step is to request a reverse merger to update the stable thread, and after that’s done, a further request is made to reverse-update dev, the development master branch.
That’s how we ensure that all work-streams are always up-to-date with the most current version of the current release.
All merger requests are initiated and managed by one person, the one who makes changes to the code.
We even perform reverse mergers when the error fixes are very minor and not expected to affect the main branches at all. These mergers help us ensure that there are no unwanted changes across all baselines, i.e. that they are intact and identical.
Once a developer assigns a ‘commit’ destined for merger with stable or release, this triggers an ‘autotest’ that runs looking for a reverse merger with dev or stable respectively. If the autotest fails to find an expected change by the end of the day, it sends off notifications to the developer, release manager and project manager reminding them that the workflow is not complete.
The key advantages we see in this workflow are:
· All changes are thoroughly tested in close cooperation with the testing team;
· The “one merger for one change” principle allows us to track all changes and prevents changes that might trigger negative effects at a later stage;
· Strict merger rules make sure there can be no merger conflicts because of unaccounted changes.
Testing and Production Workflow
Testing happens in two stages. Feature testing, as the name suggests, is focused on testing a particular feature that we have developed and wish to add to the product, and complex testing puts the entire application through a tryout with the new feature embedded in it. Feature testing runs on the dev branch, (or any of its sub-branches, i.e. dev-ios, dev-android, dev-desktop depending on where the feature is to go to) while complex testing is run on the stable branch.
When development is complete for a new Aegees release, the release manager initiates a request to merge the dev branch with the stable branch. This requires approval from our four key managers:
· head of development;
· project manager;
· head of testing;
· head of production.
The production team has to make sure that the stable branch has been updated with all changes made in the release branch, and that the dev branch has been updated with all the changes in the stable branch. They run the checks and either produce recommendations for some outstanding mergers or, if everything is ok, issue an approval.
Once the new code is stable, the testing manager initiates a request to merge the stable branch into the final product, i.e. the release branch. This also requires a sign-off from the same four key members of our team, i.e.:
· head of development;
· project manager;
· head of testing;
· head of production.
The production team must ensure that the stable branch is updated with all changes made in the release branch, and, just as before, they either produce recommendations for any outstanding mergers or issue an approval.
When the stable code finally gets to be committed to the release branch, this effectively means that we’re about to issue a new product release, and our production department will follow through by rolling out a new public release of the application. This job is supervised by the release manager. To be honest, that vacancy on our team is still open, and for the moment the job is filled by various heads of other teams on a rotating basis.
This workflow definitely gives us some great advantages, such as:
· a smooth and seamless merger process that ensures the conflict-free and successful transfer of all newly developed code into the relevant new product release.
· Strict code review and merger rules mean we stay on schedule and all code is optimally stable.
· The production department makes sure that all branches are updated to reflect the current version of the product at all times.
It’s important to note that fast-forward merge is not allowed in this workflow on Github.
With the planned global roll-out date for Aegees getting closer every day, we had to act really fast — and we did! It took us just two months to reorganize our code management system and just a couple of days to complete all the technicalities. We’re learning fast and super-fast, if we have to, and now everything runs like clockwork, so we can focus on our main goal — making our dream come true and delivering a fantastic app to the global audience right on time.