As a project evolves, so does its project management system — that’s especially true in software development, and even more so when you’re trying to develop the most protected messenger in the Galaxy (and possibly beyond). It’s impossible to stick to just one system and apply it across every project stage and workflow. So, on top of all the development and testing, the Aegees team is constantly streamlining, adjusting and optimizing the project management system to keep everything running efficiently and at full speed. We spoke to Aegees managing partner to delve into his experiences of managing a large, versatile team that’s working on a large-scale and very challenging project.
So, regarding project management, what methods and systems did you choose for Aegees? Did you change any in the process, and why?
Today, most companies have embraced an agile methodology in their software development, and there are three main agile software development frameworks to choose from. Actually, strike that, there are in fact two because Extreme Programming (XP) is getting old now, so the best two are Scrum and Kanban. We’ve used Scrum for a long while on our project. It’s an iteration-focused methodology (well, all agile frameworks are to a greater or lesser degree, but Scrum more than others). A “sprint”, or iteration, is the basic unit of development in Scrum. It works like this: first, we define a sprint’s duration; usually, that’s two weeks, and the main idea is that each sprint must deliver a product version with some ready functionality. The thing with Scrum is that it requires a working product at the end of the sprint that really is done, completed. This is called a “potentially shippable product increment” in Scum terms, and it means that at the end of each sprint, we deliver a new portion of software that is of value for the product from the end user’s point of view.
After a while, though, we ran into some limitations of the Scrum methodology that started to set up a few problems for our project. Our team kept growing, and you see, Scrum requires daily review meetings, called “daily scrums”, or stand-up meetings, where each and every team member stands up and says, “Yesterday, I did this and that; today, I’m going to do this and that; I expect input from X, and I’ll deliver my output to Y”. So when our team had grown sufficiently, that review became a nightmare, wasting dozens of highly skilled professionals’ time who were just waiting for their turn while others sorted out their issues.
Also, at some point, Scrum’s main “virtue,” i.e. the requirement to deliver potentially shippable product increments at the end of each sprint, turned into a bottleneck. An approach like that dictates that we break all the work into small intervals, to ensure a tangible delivery at the end of each one. For precisely the same reason though, long-term planning and design begin to suffer, because those “ready bits” don’t fit like Lego blocks into a ready product, they affect each other in ways that need to be identified, tested and fixed.
So we found ourselves in a situation where the number of errors and bugs caused by integration began to snowball. Yes, the Scrum approach makes you work fast, and the results are measurable, which is great, but the further we worked, the less stable and predictable the outcome of the sum-total of our development became.
So, we had two problems on our hands. One was that the stand-up meetings had become too long and inefficient. That could have been addressed by breaking the large team into several smaller Scrum teams. We figured though that a move like that would make overall team management too challenging and decided to look for another solution.
The other problem was all the accumulating errors. Each new function triggered responses across the entire version, resulting in new bugs. Fixing them would often cause more new bugs, and so on, so overall output quality and efficiency began to deteriorate.
We decided to focus on quality first because that’s our highest priority. We made changes to the process, introduced more layers, then revised and improved them. In the end, most of our development scope ended up being managed via a Kanban process, and we increased our design and research time for each function. Now they take as much time as development, unlike with Scrum, where the focus was on development. Previously, we just started writing code for a function right away. Now, we begin by researching all existing solutions for that function, defining its scope clearly, defining the scope of development for the current release and for subsequent releases, working out the function’s architecture, defining the API for the task and testing workflow. And all of that has to be approved by the relevant managers before development can even begin.
So, in the end, yes, you might say that our pace has slowed considerably. Today, we complete about 4 times less development than we used to in a given amount of time. However, on the other hand, in the long run, thanks to more thorough preparation, we have fewer integration issues and spend a lot less time on fixes. As a result, performance in terms of time and quality has gone up considerably. If you do one job slowly, but thoroughly, you finish it sooner than if you have to re-do it over and over again.
Haste makes waste?
Yes, though not always. It’s important to recognize when the time has come to slow down. By no means do we want to say anything against Scrum; it’s a great methodology, but in our case, it began to make more sense to switch methods than to keep adjusting Scrum to our expanding needs.
So how long have you been using Kanban, and what are its main advantages?
We’ve been using it for two months now. Some of our workflows still run on Scrum, but tasks that are ready for integration testing, are mostly done via the Kanban process.
When development is complete, say, for a particular function for the app, it gets tested separately first (the assumption being that it could still cause problems after integration). If the initial tests show it’s functional and worth our time, we go on and fix all the bugs identified at this stage.
After that, the function is integrated into the current stable version of the app, and integration tests are applied. We need to ensure that each new function, when added, doesn’t set us back in terms of the app’s overall performance and stability. This is where we use the Kanban approach now, and in my opinion, it works much better. Now, we get what we expect, and that’s what we needed.
Another improvement was ditching the stand-up meetings. With a Kanban board, we can call review meetings by column. Each column has its owners; designers, testers, developers, and so on, and each column is reviewable. We review, identify the needs and add resources if needed. If we find a conflict, we fix it. There’s no need any more to gather the entire team or assign different smaller teams per task every time, it’s all taken care of by the framework.
Thanks to this, our review process has become faster and more efficient. Each log gives us a very clear idea of the current status and outstanding tasks. That’s such a relief after stands-up that could last for up two full hours instead of the intended 20 minutes! When meetings like that ended, the only thing you knew for sure was that you were tired. So I’d say, both methods are good, it just depends on the project’s needs. As our project evolved, we made changes.
Speaking of improvements, could you give us an estimate of how much faster or more efficient the process has become because of the changes?
I can’t really give you a number until January when we’ll have collected enough data for tasks that have completed the full cycle, from start to release.
So far, we have had one product owner on our project, i.e. the person who maintains the backlog, which means that the backlog contains uniformly logged data with the same level of detail across all entries. This allows for some reliable forecasting, based on backlog analysis. Right now, we are working on 10 to 20 tasks, once they are completed, we’ll have full stats for all of them. That will give us the numbers that say how much faster we’ve become. As for efficiency, I can answer that question right away. Before the changes, the only thing we could rely on, in terms of integration, was the team’s professional forecast of what we could expect. That forecast was never good. Now, we don’t need a forecast. Things just work.
So human error is not a factor anymore?
Well, you can’t rule it out completely, but it has been minimized.
I see that Code Review rules in your process say that the task being moved to the Feature Testing column shows that the pull request for the task has been approved, the code was successfully integrated into the Dev branch and all autotests have been successful as well. How often does a task fail to migrate to that column?
Actually, they all fail on the first attempt, all of them! There is always something, the autotests, the same tests that we wrote ourselves, always find some errors. And that’s their primary purpose, to find something that could have slipped our attention. Code review means peer review. We have developers check each other’s work. The process is not perfect, there’s still room for improvement. I’m thinking of maybe introducing some target numbers, like a number of bugs per review, to make sure there are no hack jobs, excuse the pun. Sometimes the guys just ok a piece because it looks good, and they don’t go into detail. That’s where the autotests are a big help, and they are so vicious that hardly anything gets through them on the first attempt.
The Released Column is for all tasks that have been approved as fully completed. How many tasks do you have there?
The Released column is for front-end tasks; there are 107 of them in total now. Some examples are application authorization via a choice of three identifiers; a quick reboot of the authorization session; user invites to the app; blacklist management. Each of these tasks is split into a number of sub-tasks, such as, for example, adding contacts, deleting contacts, editing a contact’s profile.
You plan for your messenger application to include a cryptocurrency wallet. How much workload does that add to the development?
For each cryptocurrency, we set up a relevant Blockchain node. Bitcoin has its own Blockchain, Ethereum has its Blockchain, so does Litecoin, and so on. Each cryptocurrency has a network of nodes, and you have to have a full node to be able to work with them. A full node is one that can fully verify all of the rules of that Blockchain.
So, to fit our app with cryptocurrency services, we set up full nodes of the selected cryptocurrencies in our infrastructure and “wrap” them into our standard API for cryptocurrencies. The API runs all the communications between our app’s servers and each currency’s Blockchain network.
The end user will get a fully functioning cryptocurrency wallet, including a desktop version, the option to set up a new wallet, and the possibility to execute personal transactions using their private key. The wallet will provide the interface, while the app itself will simply serve as a conduit for all that communication, channelled to and from the user’s wallet identified by user login data.
The wallet’s interface will replicate the standard set of Blockchain functions. There aren’t many: request a balance report, request balance updates, generate a transaction. The wallet will be identifiable by the user’s login data, which will enable any other user on the same network to select the user by their login info and, for instance, send them some “cryptocash”.
One improvement that we introduced here is that we made it possible for our app users to exchange same-currency transactions using user login data only, whereas standard wallet solutions used in other systems usually require a very long alphanumeric ID generated to identify each user on the network.
So, overall, we keep it simple and easy, which doesn’t call for much extra work. We don’t expect any trouble developing the cryptocurrency part of our app, and the only reason we’re not on it right now is that we are working on calling and messaging, and those functions are a lot trickier to get right. And it was our decision not to start any new workflows until we’d finished those two and are completely satisfied with them, although, if I remember correctly, we have already “wrapped” at least five cryptocurrencies into our API.
Which ones, may I ask?
Let me see, these are Monero, Bitcoin, Etherium, Dash…
Yes, NEM and possibly some others. Etherium gave us a bit of trouble because its nodes are not very stable. They sometimes just go off and need resetting, completely out of the blue. It needs to be continuously watched. That adds a bit of a challenge.
Overall, with Etherium, the codebase quality isn’t particularly high, but it’s fully functional nonetheless.
Is there anything you would like to add, perhaps?
Well, maybe a few words about the product owner’s role in our project. There are three areas on our Kanban board where he has full control of things. First, he maintains the backlog, which means that he decides which tasks we work on, and he makes sure we stick to our priorities.
He also owns the Ready for Breakdown column. Here, we have fully researched tasks with drafted design solutions and ready test cases. The product owner reviews them to make sure the proposed designs are right for the intended output. He can request changes, or accept the job as it is if it’s fine.
He also owns the Done column. That’s where all completed tasks go. He can ask testers or developers who worked on them to walk him through the ready version of the software in question. If it works well, the product owner sends the task to the Released column. These three areas of control provide a very efficient management solution.
The Kanban board is also organized in such a way that it is easy to see all the bottlenecks. If you see tasks piling up in a column, it means that area is lacking resources, and you can quickly take care of the need.
Thank you so much for the interview.
Of course, there can be no perfect recipes in the area of project management. It all depends on the project stage and its priorities, and as the project moves on, some changes could be in order. Finding the best project management solution for every project stage is a craft, or perhaps, an art, but definitely a full-time job on its own too, as important as all the production work streams, like design, development and testing.