This article was published as a part of the Data Science Blogathon
Introduction
In this article, we will discuss DevOps, the two phases of DevOps, its advantages, and why we need DevOps along with CI and CD Pipelines.
Before DevOps, software development teams, quality assurance (QA) teams, security, and operations would test the code for several months. As a result, software updates sometimes go through months or even years.
Dev is the short form of development, and Ops stands for operations.
The development of software with high quality by increasing the robotic and merging of IT Operations. is known as DevOps.
In other words, by automating and combining the work of development and IT operations teams, DevOps, as defined, specifies a software development process and an organizational culture shift that accelerates the delivery of higher-quality products.
DevOps is the combination of two terms Development and Operations. We will discuss in detail the two phases of DevOps, the Development process, the first phase, and the operation process, the second phase.
Two Phases of the DevOps
A. Improvement (Code Development)
Writing project code, testing, correcting issues, creating new features, performing updates, and patching code are all part of the development phase. Plan, Code, Building, and Testing the code are the four processes of the development phase. Developers make use of a variety of technologies in this phase to reduce the complexity of the development phase.
1. The Plan stage covers everything that happens before the developers start writing code, and it is where a Product Manager plays a role. By gathering or collecting feedback project roadmap is structured.
The product roadmap can be broken down into Epics, Features, and User Stories, creating a backlog of tasks, then be used to plan sprints and allocate tasks to the team to begin development.
2. Code
The team uses the static code analyzer plugin to ensure the standards of the code are met and secure.
3. Build
The Build phase starts when the developer commits their code to a shared code repository.
If the build or any tests fails and the pull request from Git fails, the developer gets notified to resolve the issue. By continuously checking code changes in a shared repository and running builds and tests, we can minimize integration issues that arise when working on a shared codebase and highlight breaking bugs early in the development lifecycle.
4. Test
The test phase leads to a series of manual and automated tests. Manual testing can be traditional User Acceptance Testing (UAT), which uses the application as the customer to highlight any issues or refinements before deploying into production.
Automates test will simultaneously check for security by scanning the application and checking the changes made to the infrastructure, i.e., the type of networking attached to the application, to test the performance.
B. Operations (Deployment Operations)
Deployment refers to the changes updated from one environment to another. For example, the live website should remain live without getting affected then the local environment or development environment should be there to make changes or updates.
Release, Operate, Display, and Monitor are the four steps in the Operations phase.
1. Release
When the changes are ready to be implemented from the development environment, the release phase is one of the main steps in DevOps Pipelines. Each code change has passed a series of manual and automated tests, and the operations team is sure of no breaking issues or regressions.
Any build that reaches this point may be automatically deployed depending on an organization’s level of DevOps maturity.
At the release stage, you can add a manual approval process that only lets certain people in an organization approve a release into production.
Deploy A build is finally released into production when it is ready for the big time. Several tools and procedures can automate the release process.
2. Operate
The company has developed a method for customers to provide feedback on their service and tools to collect and prioritize feedback. This feedback loop is important because the customer is the best testing team in the world and knows what they want better, as the customer spends a lot more time testing the application than the DevOps pipeline ever could. This information is worth its weight in gold, so you must capture it.
3. Display and Monitor
Monitoring the environment is the “final” step in the DevOps cycle. Analytics on customer behavior, performance, errors, and more builds on the customer feedback during the Operate phase.
To close the loop on the procedure, the Product Manager and the development team receive all of this information back. It would be straightforward to assert that this is the new beginning of the loop, but the actual course of action is never-ending. There is no beginning or end; all that matters is a product’s ongoing development throughout its lifespan or when people move on or no longer require it.
CI and CD Pipelines
DevOps and continuous integration/continuous delivery (CI/CD) are now so crucial that no tech-first company can afford to ignore them. These companies strive for maximum operational efficiency and exceptional product quality. Development teams use CI and CD worldwide to deliver code changes quickly and reliably. DevOps principles, on the other hand, encourage collaboration between development and operations teams to streamline all product development processes.
Continuous integration developers merge their changes into the main branch as frequently as possible by creating a build and running automated tests. The developer’s changes are validated. By doing this, you avoid integration issues that can arise when merging changes into the release branch on release day.
Continuous Delivery: Since it automatically deploys all code changes to a testing or production environment after the build stage, continuous delivery is an extension of continuous integration.
Theoretically, with continuous delivery, you can decide to release on a daily, weekly, or biweekly or as often as your company requires. However, you should deploy to production as soon as possible to ensure that you release small batches that are simple to troubleshoot in the event of a problem if you truly want to reap the benefits of continuous delivery.
Advantages of CI and CD Pipelines
1. Automating the deployment process to speed up deployment: The length of the software delivery process reduces by the more effective use of automated testing in the development phase.
2. In addition, updates made by a developer to a cloud application can go live just minutes after written to continuous deployment and automatic provisioning.
3. Lowers the costs involved in developing traditional software.
4. Error detection sooner in the development process.
Key Difference Between DevOps and CI/CD
DevOps and CI/CD are related concepts that work together to speed up and streamline software development. They share some differences.
A DevOps team’s workflow best practice is to follow the implementation of CI/CD practices, also known as “establishing a CI/CD pipeline.”
Establishing a CI and CD pipeline term refers to coding, building, and testing automatically to deliver software. In combination with the agile DevOps Team, it enables the delivery of high-quality software. By agile DevOps, we mean agile techniques are used for project management by DevOps Team.
Agile is just a way of managing a project by delivering high-quality software quickly with few customer errors, enabling changes in the code frequently.
There are a few advantages of agile management, as discussed below.
-
It focussed on the continuous delivery of software, enabling no complaints by customers with frequent changes.
-
Enabling the small group to work in a Team, with each member having a daily meeting.
Product updates can be distributed quickly, effectively, and automatically through CI/CD, which is its primary goal. Additionally, this procedure reduces product flaws, increasing average levels of customer satisfaction. CI/CD pipeline boosts “quality at speed,” adds value to the operational and product development teams, and increases the company’s business value.
The integration of production cycle processes and the merging of development and operations functions are the goals of DevOps.
Microsoft Azure
Azure is a cloud-based platform that offers to design and build your applications in a simple, reliable way. It provides support on tools Visual Studio Code and SDK Library.
Microsoft Azure is a collection of all the tools needed for development and the Integration process.
-
Azure Repos – Azure Repos is a set of tools that manage your changes in code and save it for later use.
-
Azure Pipelines – Azure pipelines refer to the CI and CD combination and build and test the code to deliver it to the target.
-
Azure Test Plans – Azure Test plans to provide tools that enable various tests, such as manual, browser-based, and user testing.
-
Azure Artifacts – Azure artefacts help to manage the code and packages, which can be shared within the team publicly in one place.
-
Azure Boards – Azure Boards provide tools to help track your code, any issues, and bugs, and support agile, scrum, and Kanban processes.
Source: Microsoft Azure
Advantages of Microsoft Azure Pipelines
- High availability.
The uptime guarantee you receive is 99.9%, as you can see when you read the legal agreement that governs Azure’s services. It indicates approximately 4.5 hours of downtime in real-time hours. For the same price as Azure, comparable services from rival brands offer uptime percentages of 98%, 95%, and occasionally as low as 90%. - It supplies you with a solid security profile.
Multi-factor authentication and application passwords provide an additional layer of security to prevent information theft. - Azure provides good options for scaling.
With Azure, you don’t have to buy data packets or other upgrades to get the computer power you need for that day or two every month. Select the necessary upgrades by clicking on them and when you can remove them. It’s easier to pay only for what you use with this structure. - It is a cost-effective IT budget solution.
You don’t have to spend on IT infrastructure as other businesses do because you use a cloud provider. It means that SMBs can use their technology to become globally competitive immediately. When you need something, you only buy it when you need it. In return, Azure’s cloud environment lets you launch internal or customer applications. - For data access, multiple redundancies exist.
Access to your data. They can guarantee continuous availability at levels that are higher than their rivals. You can still access another data center whenever you need to insert or remove data for your business.
It supports many different languages like Java, Python, etc. Azure gives you access to tools like Visual Studio after you build apps in the language of your choice, such as.NET, Java, or Node.js.
- App connectors are made available to you.
With about 150 connectors with your Azure subscription, you can connect devices, data, and applications. One of the best connectors, Office 365 lets you combine coding and administration tasks on a single platform. Dropbox, Google Services, Twitter, Salesforce, and other services are also compatible with Azure. - Increase the delivery speed of the app
Can manage large, complex applications in Azure, including REST APIs, Power Shell, and BASH. Because of built-in support for analytics, patching, monitoring, backups, and site recovery. - A wide range of artificial intelligence services is available to you.
The Azure-provided tools make it possible to create several AI-powered experiences. AI services enable you to serve more people and build the customer experience with your brand.
We will use DevOps Azure to create CI and CD pipelines and set upstream analytics projects. Before that, if you don’t have an account, you can create your account at https://dev.azure.com.
Upstream Analytics Projects
1. Click on the New project tab on the right side and click on create.
2. When you create a new project, choose the name for your project and then click the advanced tab to choose from basic to agile.
It is how it will look project once formed. Now we have to create the pipelines; we will choose the option pipelines.
3. Now, create a new pipeline. We will form our first pipeline by clicking on the create pipeline button.
The next page will take you to connect options to a different repository like Azure, GitHub, or others.
4. We choose a classic editor without YAML.
We will have to take permission from GitHub because our code is in GitHub. After that, we will browse the repository and continue.
5. We can choose the Development template as ASP.NET or Select a template as an Empty Job.
6. From the task tab near the agent Job, select plus sign and on search, add npm and install.
7. After the npm install, click on the install and change the display name according to you, and from the command option, chooses the option custom and add the script.
install -g azure-streamanalytics-cicd
8. From the Task tab, click on Agent, again click on the plus sign, and type in the search command line to add.
We have to add a command line to enable the script to run.
9. Change the display name of the downloaded command line.
10. Now we have to build it. In the variable, write new variables
deployPath values Deploy output path values Output project root path with value root path.
11. From the task, go to the Agent option and choose the plus sign to add the command line. Change the display name and write in the script.
RootPath should be according to your project.
12. Add test path in a variable with the value Test.
13. Now, add the command line by going to task and choosing the plus sign from the Agent tab. Change the display name and add it to the script.
Change the root Path according to the project.
14. Now go to the task, and from the Agent tab, choose plus sign and search for copy files.
Source: Microsoft Azure
15. Inside the copy files, add the following scripts, where to copy files, source, and the target folder.
$(build.artifactstagingdirectory) $(system.defaultworkingdirectory)/$(outputPath)/
16. From the control option, choose even if the previous task and build failed.
Source: Microsoft Azure
17. To enable continuous build and testing, we will check to enable it in the Trigger tab.
18. Enabled the option, and from the task tab from the Agent option, choose plus sign to add and publish build artifacts.
19. Inside it, choose the option even if the build and task get failed.
20. Click on the tab save and queue; type run and test or anything you wish to write in the comment.
Now we will click on the tab save and queue our settings.
21. Run the Pipelines
After saving and running the pipelines, we will get the result mentioned published.
Operation Process
Continuous Integration and Continuous deployment complete the automation cycle while reducing or eliminating human involvement in the deployment process. When source code updates pass the testing stage, automated DevOps systems instantly deploy them into the production environment, saving time and enhancing user happiness.
Automation in continuous deployment speeds up user feedback loops. Additionally, its purpose is to divide the deployment of a release, either by making it inaccessible to consumers (black releases) or by making it active for a select group of users to test out new features and get feedback (feature toggles or switches).
Due to automation, releasing the code in tiny batches reduces the risk of making significant code modifications.
After the Integration step, we will continue to the operation phase, which consists of the Release, Operate, Display, and Monitor.
Release
We have done our part of the Integration process and will go for the operation process using the Releases tab.
Add Deploy
We have to add a display name and choose our subscription.
Create Release
We have to choose to create a release tab on the right side of the page.
Source: Microsoft Azure
Conclusion
Our Objective was to run Continuous Development and Continuous Integration pipelines on Microsoft Azure. To answer a few questions about why do we need Microsoft Azure? What are Continuous Development and Continuous Integration? What do we understand by DevOps? We have achieved our purpose.
Key Takeaways
1. In this article, firstly, we have mentioned the meaning of DevOps and its two phases. Second, using Continuous Development and Continuous Integration and why we need them. Now you should be able to define Continuous Development and Continuous Integration.
Continuous development is nothing but software development that surrounds or encircles many DevOps processes, i.e., Continuous Integration is nothing but building and testing the code automatically whenever the changes occur, followed by testing, delivery, and deployment.
Continuous Integration and Continuous Development have gained popularity recently due to their various features.
It results in quick implementation and feedback. This method is more reliable and provides immediate release and removal of errors with minimal code change.
2. We mentioned why we need Microsoft Azure. You should now know why we need it and what its purpose is.
Azure is a platform that helps you build your applications on the cloud and edges by providing you with various products, services, and tools to solve recent challenges and problems.
3. We also mentioned how to create and run CD and CI Pipelines in DevOps Microsoft Azure, and now you should be able to run on your own.
The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion.
I have done my Master Of Science in Biotechnology and Master of Science in Bioinformatics from reputed Universities. I have written a few research papers, reviewed them, and am currently an Advisory Editorial Board Member at IJPBS.
I Look forward to the opportunities in IT to utilize my skills gained during work and Internship.
https://aster28.github.io/SoniaSinglaBio/site/
