DevOps Engineer

Overview

During my fruitful time as a DevSecOps (DSO) Engineer, I assisted in maintaining and updating pipelines while also supporting queries from the Application team. During this time, I helped with three crucial CI/CD pipelines, where I worked closely with my DSO team and the Application team as they underwent a major data migration in Azure Data Factory.

Pipeline delivery and migration

I supported the updating and testing of the pipelines to ensure that their code was successfully deployed to the various environments: DEV, SIT, UAT, ORT, and PROD. In addition, I also helped to promote one of the pipelines from GCC1 (Government on Commercial Cloud 1) to GCC2 (Government on Commercial Cloud 2).

Besides supporting the major pipeline projects, I helped to fine-tune the pipelines' performance by utilising Kusto Query to analyse build agent health details from the AKS logs, as well as Azure Command-Line Interface (CLI) and kubectl to monitor pods in real time and troubleshoot errors. I even went as far as creating my own script to automatically collect pod health logs.

In addition, I conducted research on third-party tools such as Lighthouse to determine their applicability to the project.

Pipeline and build-agent troubleshooting

Riding on the work I did with the pod health logs, I also helped my team monitor the overall health of our build agents and AKS clusters. Sometimes, the Application team would send a query through the helpdesk asking why their pipeline had failed.

To understand the issue, I would first identify which stage of the pipeline had failed and check the console logs for more information. Normally, if the failure occurred during the NPM test, it meant that their application did not pass the build test.

If the failure occurred at SonarQube, Fortify, or Mend, it meant that their code did not meet IRAS standards or had identified code smells. If this was the case, I would screenshot the reason for the failure and send it back to the developers.

Otherwise, we would dive deeper into the issue by obtaining the build agent name, examining the console logs from Azure Kubernetes Service (AKS), and checking whether the pod still existed. If it did, I would use kubectl to access the pod and gather more detailed information for troubleshooting. Sometimes, I would also consult my team when fine-tuning the build agent configuration in the Helm file.

Network infrastructure

Outside of the pipeline and AKS work, I assisted my supervisor in documenting our network infrastructure, particularly the connections between the different DSO infrastructure components.

Conclusion

Overall, I really enjoyed working there. It was probably the best internship experience I could have asked for. To this day, I still miss the company of my co-workers and our discussions about how to tackle different tasks and challenges.

Being a DSO Engineer exposed me to many different Azure services and gave me a much better understanding of how they are used in a real working environment. More importantly, the experience deepened my interest in cloud technology and automation.