Cloud & DevOps Essentials!
What is DevOps?
- DevOps refers to a set of cultural philosophies, practices, and tools
- Reduces gaps between the teams
- Faster Delivery, Evolving & Improving Product, Less Resources
- DevOps Lifecycle:
- Plan
- Code
- Build
- Test
- Release
- Deploy
- Operate
- Monitor
source : by Izzy Azeri on Mabl
Resources
- gitlab.com
- DevOps is a combination of software development (dev) and operations (ops). It is defined as a software engineering methodology which aims to integrate the work of development teams and operations teams by facilitating a culture of collaboration and shared responsibility.
- Core DevOps principles
- Automation of the software development lifecycle
- Collaboration and communication
- Continuous improvement and minimization of waste
- Hyperfocus on user needs with short feedback loops
- Azure Tech Blog
- DevOps is the union of people, process, and technology to continually provide value to customers. By adopting a DevOps culture along with DevOps practices and tools, teams gain the ability to better respond to customer needs, increase confidence in the applications they build, and achieve business goals faster.
- AWS Tech Blog
- DevOps is the combination of cultural philosophies, practices, and tools that increases an organization’s ability to deliver applications and services at high velocity: evolving and improving products at a faster pace than organizations using traditional software development and infrastructure management processes. This speed enables organizations to better serve their customers and compete more effectively in the market.
Difference between DevOps & Agile
Agile
- Development of small modules and testing in quick iterations
- Development & testing going on parallely
- Interatively merging and releasing
- Dev + QA teams work together but Ops team still work in isolation
DevOps
- Dev + QA + Ops teams work in collaboration
- Automating the workflow
- Collaboration between the teams
What is CI/CD?
- CI/CD falls under DevOps (the joining of development and operations teams) and combines the practices of continuous integration and continuous delivery. CI/CD automates much or all of the manual human intervention traditionally needed to get new code from a commit into production, encompassing the build, test (including integration tests, unit tests, and regression tests), and deploy phases, as well as infrastructure provisioning. With a CI/CD pipeline, development teams can make changes to code that are then automatically tested and pushed out for delivery and deployment.
Resources
Key DevOps Tools & Cloud Services
| DevOps Tool / Cloud Service | Open Source | AWS | Azure | GCP | |
|---|---|---|---|---|---|
| 1. | Version Control Systems | * Git | * AWS CodeCommit | * Azure Repos * GitHub Enterprise |
* GCP Cloud Source Repositories |
| 2. | Build Tools | * Apache Maven * Gradle * SBT * Poetry |
* AWS CodeBuild | * Azure DevOps | * GCP Cloud Build |
| 3. | CI/CD | * Jenkins | * AWS CodeDeploy * AWS CodePipeline |
* Azure DevOps | * GCP Cloud Deploy |
| 4. | Container Orchestration | * Docker Swarm * Kubernetes |
* AWS Elastic Kubernetes Service (EKS) | * Azure Kubernetes Service (AKS) | * Google Kubernetes Engine (GKE) |
| 5. | Configuration Management Tools | * Puppet * Chef * Ansible |
* AWS Config | * Azure Policy | * GCP Asset Inventory |
| 6. | Continuous Monitoring Tools | * Grafana * ELK * Nagios |
* AWS CloudWatch | * Azure Monitor | * GCP Cloud Monitoring |
| 7. | Workflow Orchestration | * Apache Airflow * Oozie |
* AWS Step Functions | * Azure Logic Apps | * Workflows |
| 8. | * Storage * Infrequently accessed storage |
* Hadoop File System (HDFS) * Ceph |
* AWS Simple Storage Service (S3) * Amazon S3 Glacier |
* Azure Blob Storage * Azure Archive Storage |
* Cloud Storage * Cloud Storage Archive |
| 9. | Compute | * Amazon Elastic Compute Cloud (EC2) | * Azure Virtual Machines | * GCP Compute Engine | |
| 10. | Data Warehouse | * Spark SQL * Apache Hive * Presto |
* Amazon Redshift * Amazon Athena |
* Azure Synapse Analytics | * BigQuery |
| 11. | Serverless (functions as a service) |
* Knative * OpenFaaS * Fn |
* AWS Lambda | * Azure Functions Serverless Compute | * Cloud Functions |
| 12. | ML/AI Training Compute | * AWS Elastic Compute Cloud (EC2) * AWS UltraClusters |
* Azure Virtual Machines * GPU Optimized VMs |
* Cloud GPUs * Cloud TPU |
|
| 13. | Container Registry | * Docker Registry * Quay |
* Amazon Elastic Container Registry (ECR) | * Azure Container Registry | * Artifact Registry |
| 14. | Job Scheduling | * Apache ActiveMQ | * Amazon EventBridge | * Azure Scheduler | * Cloud Scheduler |
| 15. | Event Handling | * Apache Kafka * Trigger Mesh |
* AWS EventBridge | * Azure Event Grid | * Eventarc |
| 16. | Service Mesh | * OpenStack * OpenVPN |
* Amazon VPC | * Azure VPN Gateway | * Cloud Router |
| 17. | Identity & Access Management | * Open Identity Platform * Apache Syncope * Keycloak |
* Amazon Identity and Access Management | * Azure Identity Management | * Identity and Access Management |
| 18. | Logging | * Grafana Loki * OpenObserve |
* Amazon CloudWatch Logs | * Azure Monitor Logs | * Cloud Logging |
| 19. | Data Streaming | * Apache Kafka | * AWS Kinesis | * Azure Service Bus Messaging | * Pub/Sub |
| 20. | Audit Logging | * Graylog | * AWS CloudTrail | * Azure Audit Logs | * Cloud Audit Logs |
| 21. | In-Memory Cache | * Memcached * Redis |
* AWS ElastiCache | * Azure Cache | * GCP Memorystore |
| 22. | Data Processing | * Apache Spark | * AWS Elastic MapReduce (EMR) | * Azure Data Lake Analytics | * GCP Dataproc |
| 23. | Document Database | * MongoDB | * Azure Cosmos DB | * Amazon DocumentDB * Amazon DynamoDB |
* Firestore |
| 24. | Persistent Disk | * OpenEBS * Portworx |
* Amazon Elastic Block Store (EBS) | * Azure Disk Storage | * Block storage |
| 25. | Serverless Data Integration | * Airbyte * Talend * Airflow |
* AWS Glue | * Azure Data Factory | * GCP Dataflow |
| 26. | Load balancing | * Traefik * OpenStack |
* AWS Elastic Load Balancing | * Azure Load Balancer | * Cloud Load Balancing |
| 27. | NoSQL Database | * Apache Cassandra | * Amazon DynamoDB | * Azure Cosmos DB | * Cloud Bigtable |
| 28. | Machine Learning | * H2O.ai * MLflow * Cortex * Ploomber * Zilliz’s Towhee |
* Amazon Sagemaker | * Azure AI Platform * Azure Machine Learning |
* Vertex AI |
| 29. | Graph DBMS | * Neo4j | * AWS Neptune | * Azure Cosmos DB | |
| 30. | * Command-line interface (CLI) * Client libraries |
* AWS SDKs * AWS CLI |
* Azure SDKs * Azure CLI |
* Cloud SDK |
Leave a Comment