Deployment and Operations Management

Software Deployments

Types of Deployment:

  • Big Bang
  • Phased Rollout
  • Parallel Adoption

Deployment Strategies:

  • Rolling Deployment
    • Changing launch configuration to specific version will rollout the changes
  • A/B Testing
    • Using Route53 we can specific the ALB which gets the traffic
  • Canary Release
    • Deploy new version on production - if no errors are detected, deploy the rest
  • Blue-Green Deployment
    • Create new ALB and ASG with new version
    • Switch to the new version using Route53
    • Switch back to the old version using Route53
    • The goal of blue/green deployment is to achieve immutable infrastructure - you don’t make changes to your application after it’s deployed but redeploy altogether
    • How to achieve Blue-Green Deployment on AWS:
      • Update DNS with Route53 to point to new ELB or instance
      • Swap ASG of new instances behind the ELB
      • Change ASG LC to use new AMI and terminate old instances
      • Swap environment URL on Elastic Beanstalk
      • Clone stack using AWS Opswork and update DNS

Blue-Green Deployments are not recommended:

  • If the code is very tightly coupled with database schema
    • Schema should be forward and backward compatible in best case
  • The upgrade requires special upgrade routines to be run during deployment
  • Off-the-shelf products might not be blue-green friendly

CI, CD, CD

  • Continuous Integration
    • Merge code changes to main branch as frequently as possible with automated testing as you go
  • Continous Delivery
    • You have automated your release process to the point you can deploy at the click of a button
  • Continuous Deployment
    • Each code change that passes all stages of the release process is released to production with no human intervention required

Continuous Integration Pipeline:

  • Get latest from Repo
  • Make changes
  • Unit Testing
  • Commit to Repo
  • Integration Testing
  • Acceptance Testing
  • Deploy to Production
  • Smoke Testing

CI/CD Consideration:

  • Objective is to create smaller, incremental compartmentalized improvements and features
  • Lowers deployment risk and tries to limit negative impact
  • Test Automation must be strong
  • Feature toggle patterns useful for dealing with in-progress features not read for release (versus more traditional branching strategies)
  • Microservice architectures lend themselves well to CI/CD practices

AWS Development Lifecycle Tools

  • AWS CodeCommit
  • AWS CodeBuild
  • AWS CodeDeploy
  • AWS CodePipeline
  • AWS X-Ray
  • AWS CodeStar

Elastic Beanstalk

  • Orchestration service to make it easy to deploy scalable web packages
  • Wide range of supported platforms - Docker, PHP, Java, NodeJS
  • Multiple Environments within Application (DEV, QA, PRD…)
  • Great for ease of deployment, but not great if you need a lot of control and flexibility

Deployment Options

  • All At Once
    • minimal deployment time, downtime, manual rollback process
  • Rolling
    • one by one - terminates old version instances and replaces the new instances
    • no downtime expected, manual rollback process
  • Rolling with Additional Batch
    • launch new version instances, and then take old instances out of service
    • no downtime, manual rollback process
  • Immutable
    • launch a full set of instances in a separate ASG and cut over when health checks are passed
    • no downtime, terminate new instances to rollback
  • Blue/Green
    • CNAME DNS entry is changes when new version is up, old version is preserved
    • no downtime, rollback is achieved through URL SWAP

CloudFormation

  • Infrastructure as Code
  • JSON/YAML to model and provision entire landscapes
  • Repeatable, automatic deployments and rollbacks
  • Nest components for reusability
  • Supports over 300 Resource Types
  • Supports custom resources via SNS / Lambda

Concepts

  • Templates
  • Stacks
  • Change Sets

Stack Policies

  • Protect certain resources from being unintentionally deleted or updated
  • Once created, stack policy cannot be deleted, but can be modified via the CLI

CloudFormation Best Practices

  • AWS Provides Python “helper scripts” which can help you install software and start services on EC2
  • CloudFormation should be used to make changes - don’t apply changes directly to the resources
  • Make use of Change Sets to identity potential trouble spots
  • Use Stack Policies to protect against accidential changes for sensitive resources
  • Use version control systems like CodeCommit or Github to track changes to templates

Elastic Container Service

There are 2 main services:

  • ECS: Elastic Container Service
    • Managed, highly available and highly scalable platform
    • AWS-specific platform that supports Docker containers
    • Leverages AWS services: Route53, ALB, CloudWatch, etc.
    • Collection of containers are called tasks, tasks provide a service
    • Limited extensibility
  • EKS: Elastic Kubernetes Service
    • Managed, highly available and highly scalable platform
    • Comptaible with upstream K8s so its easy to lift and shift from other K8s
    • A hosted K8s platform that handles many things internally
    • Collection of containers is called “Pods”. They can share resources and access to each other
    • Extensible via a wide variety of third-party community ad-ons

Launch Types:

  • Amazon EC2 Launch Type
    • You provision EC2 instances
    • You are responsible for upgarding, patching and taking care of EC2 pool
    • You must handle cluster optimization
    • Allows more granular control over infrastructure
  • Amazon Fargate Launch Type
    • Fargate automatically provisions underlying resources
    • Fargate provisions compute as needed
    • Fargate handles cluster optimization
    • Limited control, infra is automated

API Gateway

  • Managed, high availability service to front-end REST APIs
  • Backed with custom code via Lambda, as a proxy for another AWS Service or any other HTTP API on AWS or elsewhere
  • Regionally based, private or edge optimized (deployed via CloudFront)
  • Supports API Keys, Usage Plans for user identitifcation, throttling and quota management
  • Can be published as product and monetized via AWS marketplace
  • API Gateway can cache responses

Management Tools

AWS Config

  • Allows to assess, audit and evaluate configuration of your AWS resources
  • Very useful for Configuration Management as part of ITIL Program
  • Creates baseline for various configuration settings and files - tracks variations against tha baseline
  • AWS Config rules can check resources for certain desired conditions and if violations are found, the resources is flagged as “noncompliant”

AWS OpsWorks

  • Managed instance of Chef or Puppet
  • Provide configuration management to deploy code, automate tasks, configure instances, perform upgrades, etc.
  • Has three offerings
    • OpsWorks for Chef Automate
    • OpsWorks for Puppet
    • OpsWorks Stacks
      • AWS creation compatible with Chef recipe
      • Uses Chef solo client installed on EC2 instances to run Chef recipes
      • Supports on-prem servers with agent installed

AWS OpsWorks Stacks

  • Stacks
    • Collection of resourced needed to support a service or application
  • Layers
    • Different components of application delivery hierarchy
    • Stacks can be cloned - but only within the same region
    • When you create stack you must specify a region, this stack can control only resources in that region

AWS System Manager

  • Centralized console for a wide variety of system management tasks
  • Designed for managing a large fleet of systems - tens or hundreds
  • SSM Agent enables System Manager features and support all OSs supported by OS as well as back to Windows Server 2003
  • SSM Agent is installed by default on recent AWS-provided base AMIs for Linux and Windows
  • Manages AWS-based and on-prem based systems via the agent

Services:

  • Inventory - collects information from managed instances about applications, versions, meta data
  • State Manager - create states that represent a certain configuration is applied to instances
  • Logging - CloudWatch Log agent and stream logs directly to CloudWatch from instances
  • Parameter Store - storing configuration data, connecton strings, passwords
  • Insights Dashboard - account-level view of Cloudtrail, Config, Trust Advisor
  • Resource Groups - group resource through tagging for organization
  • Mantenance Window - defined schedules for instances to patch, update apps, run scripts and more
  • Automation - automating routine maintenance tasks and scripts
  • Run Command - Run commands without SSH or RDP
  • Patch Manager - automates the application of certain patches
    • uses baselines to defined which patches are auto-approved for update (e.g. pre-defined baseline for Windows Server 2008 - 2016)
    • supports creation of custom basedline to define applying critical, optional, and important updates
  • AWS System Manager Documents
    • defines the actions that Systems Manager performs on your managed instances

AWS System Manager Documents

  • Command Document
    • Run Command and State Manager
    • Used to execute commands. State Manager uses command documents to apply a configuration. These actions can be run on one or more targets at any point during the lifecycle of an instance.
  • Policy Document
    • Used wiht State Manager
    • Enforce a policy on your targets
  • Automation Document
    • Used with Automation
    • Used when performing common maintenance and deployment tasks

Enterprise Applications

  • Amazon App-Stream
    • Enables user to run apps in the browser
  • AWS Client VPN
  • Amazon Chime
    • Online meeting and video conferencing service
  • Alexa for Business
    • Deploy Alexa functionality and skills internally
    • Management functionality more appropriate for an enterprise organization than buying and provisioning individual Alexa device
  • Amazon WorkLink
    • Provides secure access to internal web applications for mobile devices
    • When mobile user requests an app it’s rendered on a secure machine then image is sent to the mobile client
  • Amazon WorkSpaces
    • Workspace as a service
  • AWS Connect
    • Fully managed contact center with configurable call handling, inbound / outbound telephony, interactive voice response, chatbot and analytics
  • Amazon WorkDocs
    • Document storage and collaboration platform: supports version management, sharing and collaborative edits
  • Amazon WorkMail
    • Fully managed email and calendar as a service
    • Comptaible with Microsoft Exchange (Outlook), IMAP, Android and iOS clients

AWS Machine Learning Landscape

AI Services

App Developers, no ML experience required

  • Amazon Comprehend
  • Amazon Lex
  • Amazon Polly
  • Amazon Rekognition
  • Amazon Translate
  • Amazon Transribe
  • Amazon Personalize

ML Services

ML Developers and Data Scientists

  • Amazon Sage Maker
    • Ground Truth
    • Notebooks
    • Training
    • Hosting
    • Algorithms
    • Marketplace

ML Frameworks & Infrastructure

ML Researchers and Academics

  • Frameworks
    • MX Net
    • TensorFlow
  • Interfaces
    • Gluon
    • Keras
  • Amazon GreenGrass
  • Amazon EC2
  • AWS Deep Learning AMIs