Kali Linux is a well-loved Debian-based Linux distribution aimed at advanced Penetration Testing and Security Auditing. We sat down with Ben Wilson (@g0tmi1k), senior developer at Kali, to hear more about why Kali Linux moved to GitLab and see if they've noticed any changes to their project since adopting GitLab as their DevOps solution.
Why did you decide to move to GitLab?
We decided to move from Gitolite to GitLab around April 2019 to make it possible for our community to contribute to Kali. Our previous setup didn't allow anyone to sign up, so the community couldn't help out. Another complication was using a mixture of services such as Google Docs and Phabricator, and we wanted to condense our tool stack. We love that GitLab is a single platform for the whole software development lifecycle.
One thing that was important for us is that we didn't want to reinvent the wheel. We tried to choose something open-source with advanced functionality, an active community, and a company behind it. GitLab ticked every box.
Another factor for our decision was that GitLab's API is significantly more feature-rich than competitor APIs, which allowed us to automate and integrate into anything that we wanted. For example, we can fully automate the process of remotely forking a repository then apply our configurations.
That way, we don't have to download a git repository only to push it up again. This is a big time-saver for us and significantly simplifies the workflow. Some of the configuration that we can now automatically apply are:
- Being able to drop the relationship between forks
- Configure the default branch
- Disable unused features for a repository (e.g., not everything requires their own wiki)
- Populate a description for the repository
- Set up CI paths
- Set up email notification on any activity to our private mailing list
We take advantage of various open source tools that leverage GitLab's API, such as Debian Salsa. We can use these tools to automate things like updates to email distribution lists and our configuration of GitLab admin settings and repository structure. We contribute any changes we make to these tools back upstream so that other communities can leverage GitLab's API's power the way we do.
An additional perk to GitLab is its usability. The way you can organize projects makes it a more intuitive experience for people who want to contribute. For example, having sub-groups and projects allows us to keep a clean layout in a folder-like structure. For those interested, you can see how we've organized the Kali project in GitLab.
How are you using GitLab at Kali Linux?
We're using GitLab's top-tier SaaS version, which is hosted on GitLab.com, thanks to the GitLab for Open Source program. Using this version and hosting it on GitLab is easier for us because it's less infrastructure to maintain. We have many unique pieces of infrastructure so it's nice to reduce the load when we can. We're using a wide range of features to manage the entire Kali Linux project, consisting of 564 active repositories.
Some of the most essential GitLab features for us are:
- Source Code Management: We're using GitLab to host the source code to all our packages and build scripts and custom tools.
- Wiki: We use the wiki functionality for internal documentation. Markdown makes it easy for everyone to contribute.
- Project management: We track tasks and short/long term goals with GitLab as well as the timelines for our project. We use issue tracking, threads, labels, milestones, weights, and everything else designed for project management.
- User Permissions: We like the functionality of GitLab's user permissions, which allows us to have "one-off" users on specific projects as well as automatic expiration after a particular time.
- Security: As a cybersecurity-focused Linux distro, security is paramount to us. We like that GitLab has 2FA and project access tokens.
- Analytics: We are still discovering the functionality here, but we like seeing user statistics around code review and contribution.
- Performance: We're able to use GitLab's Content Delivery Network (CDN) for great performance across the globe.
We're hoping to leverage GitLab's CI/CD features and the container management capabilities more regularly in the near future.
We're also looking to use GitLab Pages for hosting our website instead of our self-hosted WordPress instance. By using Hugo, we can write the content with a mixture of HTML and Markdown. Hugo makes it very simple, easy to update, and has straightforward change tracking. GitLab Pages then can serve up the static output.
There were several problems we were facing with WordPress that made us consider moving away, such as plugins that weren't properly maintained, security issues that made us require VPN access to admin pages. The other benefits to the move are that static pages will load faster, and our community can help fix typos on our website through merge requests. Once we make the move to GitLab Pages, we'll start to make greater use of GitLab's CI/CD functions to statically generate the websites.
Another thing we're becoming more familiar with is all of GitLab's project management features. One of the reasons we chose GitLab instead of other DevOps tools is that it's a single platform for the whole software development lifecycle, and we're looking to use more of its features. Since we're on the top-tier SaaS plan, we have every functionality available to us and we're eager to make use of it.
What are some of the changes you've noticed in your open-source community since starting to use GitLab?
The most significant change is that we only allowed contributions from employees before moving to GitLab. Since the switch to GitLab, we've adopted a new mindset and now allow anyone to help out.
GitLab's user-friendly design has made it easy for our community to get started, and we've started to receive merge requests from the public as well as bug reports and bug fixes.
It's been exhilarating to see these contributions land! We are working on increasing these contributions in 2021 with a "Kali Summer of Code" and are considering doing a giveaway for people who have made a significant contribution.
We've also experienced changes to our development practices. For example, we can now have more effective discussions about commit differences and can link to individual commits to pinpoint problems. It's easier to update items from the internal wiki, edit web pages, and merge requests. I also like that GitLab has a built-in automatic save feature to help when you're drafting something and either multitasking or on-the-go.
Finally, GitLab's to-dos and long-term planning features allow us to plan ahead for the future of Kali development. For example, we've replaced ad-hoc solutions done by individuals via emails and to-do list text files on each person's computer since moving to GitLab.
What are some challenges you've had with implementing GitLab for your community? How did you overcome those challenges?
During the switchover from the old system to GitLab, we discovered various things that were hardcoded.
To help with this, we automated a find and replace, and followed up with various manual searches to ensure that all links and references were located. This ended up taking about two hours. We also left the old web server up for a year, which pointed to the new URL structure to ensure that there weren't any missing links and references. We redid the layout of the site, so it took a while to recreate all the redirects.
Another challenge was the sheer size of Kali. We had to import roughly 1,000 repositories when we set up GitLab. We managed to migrate most of them in a day and completed the migration within a week once we managed to get the group structure in place. We set up separate groups for different access levels to repositories for build scripts, internal non-public files, Android, phone, build scripts, store, packages, recipes, tools, and websites.
Importing other items (code packages, build scripts, and custom tools from our self-hosted git) took longer because they were in many different formats. When we did the import we cleaned up to determine which items were no longer in use and archived them. The next step was making sure our custom tools were hosted on GitLab and then configuring the tools and packages appropriately. Next, we imported several repositories. We also needed to create files that were not previously tracked in our repository. Finally, we converted our WordPress-based content to Markdown using an open source project, then manually verified and cleaned it up.
We chose not to carry over existing issues because we wanted to have a clean start. In general, we only imported what was important. Everything we ended up with is what we cared about and what we wanted to track.
What do you think GitLab is doing well in supporting open source communities, and what should GitLab do to improve in this area?
We really like that GitLab has an outreach program for open source projects with dedicated people for the job role. They actively contacted us to become a GitLab Open Source Partner and we're glad to have joined as one!
One of the things that we appreciate about GitLab is that the company is open source. The transparency that comes with that allows us, and anyone else, to see the company's progress. GitLab is setting an example for how open source companies can work alongside their communities, and it's something we are learning from too.
What advice would you have for other open source communities that are looking to implement GitLab?
The sooner you make the switch, the easier and better! Once you move, you'll see that it's less work to maintain and there are more features to use.
When beginning your migration, make sure to set up a test project first to help plan the structure ahead of doing the main project switch. Look up and explore features ahead of time so you know what GitLab can do rather than discover the functionality when using it. GitLab has a GitLab Learn portal, which we hear is going to continue to be improved to help with user education.
What are some of the new things on the horizon for Kali Linux?
- KaBoxer: A framework to manage applications in containers on Kali
- New kali.org website using GitLab Pages
- Programs to increase community contributions to Kali
Is there anything else you'd like to share with us that we haven't asked you?
We have only scratched the surface of what GitLab has offered - and they keep putting in more features. We are planning on taking their upcoming training to make sure we are fully up-to-date on their offerings.
Last but certainly not least, we have heard a rumor that the founders of Kali are so dedicated to the project that they have Kali logo tattoos. Is this true?
Very true! The original founders both have Kali tattoos, as do various current members.
We also have some pretty cute baby onesies that are a hit.
Kali Linux has some cute baby onesies. CC BY-SA 4.0 Ben Wilson
About Kali Linux
Kali Linux (formerly known as BackTrack-Linux) is a Debian-based Linux distribution aimed at advanced Penetration Testing and Security Auditing. Kali Linux contains several hundred tools targeted toward various information security tasks, such as Penetration Testing, Forensics, and Reverse Engineering. Kali Linux is a multi platform solution, accessible and freely available to information security professionals and hobbyists.
