Posts

Showing posts from 2017

AWS - Storage Gateway

Storage Gateway It connects on-premise with cloud based storage. Storage gateway's software appliance is available for download as VM image. Once installed on on-premise datacenter and associated with AWS by activation process. AWS management console is used now(after 3) to configure storage options. iSCSI is an acronym for Internet Small Computer Systems Interface, an Internet Protocol-based storage networking standard for linking data storage facilities. Connection flow - Customer on-premise host - AWS Gateway -->Data connect or AWS VPC --> S3 Direct connect is direct line between customer's on premise data centre. Through VPC directly connecting to EC2 instance. Four types of storage gateways File gateway (NFS-S3) Volume gateway (iSCSI-block based storage) Stored volume Cached volume Tape gateway (VTL-Virtual tapes for backup) File gateway - Files are stored as objects in your S3 buckets, accessed through a NFS mount point. All me...

AWS-Whitepaper-OverviewOfAmazonWebServices

                         Overview of Amazon Web Services Introduction The AWS Cloud provides a broad set of infrastructure services. What are infrastructure services?such as computing power, storage options, networking and databases. How infrastructure services are delivered? on-demand, available in seconds, with pay-as-you-go pricing. In 2006, Amazon Web Services (AWS) began offering IT infrastructure services to businesses in the form of web services — now commonly known as cloud computing.   Why we need cloud computing service?replace up-front capital infrastructure expenses. A cloud services platform such as Amazon Web Services owns and maintains the network-connected hardware required for these application services, while you provision and use what you need via a web application . As cloud computing has grown in popularity, several different models and deployment strategies have emerged to help ...

AWS -EC2

                                            Amazon EC2 The Amazon EC2 simple web service interface allows you to obtain and configure capacity with minimal friction. Amazon EC2 works in conjunction with Amazon VPC to provide security and robust networking functionality. Connect your existing IT infrastructure to resources in your VPC using industry-standard encrypted IPsec virtual private network (VPN) connections. Dedicated Instances are Amazon EC2 instances that run on hardware dedicated to a single customer for additional isolation. Dedicated Hosts , which are physical servers with EC2 instance capacity fully dedicated to your use. Dedicated Hosts can help you address compliance requirements and reduce costs by allowing you to use your existing server-bound software licenses. You pay for the compute capacity you actually consume .  EC2 Instance Purch...

AWS - EC2-FrmCloudGuru

Image
EC2 instance pricing options Virtual machines on cloud provision in minutes through web portal. EC2 standard instances charges by hour in windows/Linux and in seconds also for linux. Reserved instances for known need that will arise for short term. Spot instances are available for bidding from a pool of available instances.. Dedicated hosts are physical server dedicated for your use. EC2 standard (OnDemand) For application implementation with low cost and leveraging cloud flexibility. Pay as you go.  Application that should not be interrupted. Application being developed or tested on EC2 for the first time. No upfront payment. Reserved instance For application with steady usage or predictable usage. That need reserved capacity at certain points of time. Upfront payment to reduce total cost of computing. Standard RI's (up to 75% of on demand) Convertible RI's (up to 54% of on demand) Scheduled RI's Spot instances For applications that ...

AWS - QnA

AWS - Security Groups

Security Group All inbound traffic is blocked by default. All outbound traffic is allowed. Changes to security group takes effect immediately. You can have any number of EC2 instances within a security group. Multiple security group attached to an EC2 instance. Security groups are STATEFUL (A service that is actively processing state data). You can not block specific IP address using security group instead use NACL. You can specify allow rules but not deny rules.

AWS - EBS

EBS -Elastic block storage To create block storage volumes. Attach to EC2 instance and create file system, run database etc. Automatic backup or storage replicated. Cannot mount 1EBS volume to multiple EC2 instances, instead use EFS. EBS volume types General purpose SSD (GP2) Balances both price and performance. Ratio 3IOPS per GB up to 10K IOPS. Burst up to 3000 IOPS Provisioned IOPS SSD (IO1) For I/O intensive application like RDBMS Use if application needs above 10K IOPS. Can support up to 20K IOPS per volume. Magnetic storage (Old school) Throughput optimised HDD (ST1) Frequently accessed workloads. For Big data, Data warehouse, Log processing. Can not be boot volume. Cold HDD (SC1) Lowest of all for infrequently accessed workloads.  File server Can not be boot volume. Magnetic storage (Standard) Lowest cost per GB of all that is bootable. Magnetic volumes ideal for workloads.

AWS - Storage - Rebuild-InProgress

S3 S3 is object based storage on AWS cloud. Objects are files like videos, pictures etc. 0byte to 5TB size files can be uploaded. There is no limit on upload. Files are stored in buckets. S3 is universal namespace (two same name bucket can't exist globally), so each bucket name must be unique. S3 url syntax https://S3-<region name>.amazonaws.com/<bucketname>. Putting new object on S3 gives read after write consistency (no delay in read). For any update or deletion of object propagation of change will take time. S3 fundamentals are Key(name), Value(data), VersionID, Metadata, ACLs Write/Upload to S3 is notified by HTTP-200 message. Faster upload of larger files can be done using multipart upload tool. S3 storage classes/Tier S3 (durable, immediately available, frequently accessed). S3-IA (durable, immediately available, infrequently accessed). S3-Reduced redundancy object (data that can be quickly reproducible from backup, like thumbnail etc). Gl...

AWS - Transfer acceleration

Transfer acceleration It uses cloudfront edge network to accelerate upload of data to S3 (we can directly upload also). We can also directly upload to edge location using distinct edge network URL.

AWS - Snowball

Snowball Before snowball, there was import/export disk accelerates moving large amount of data into and out of AWS through portable devices (like external HD, physically transporting them to AWS). This method created overhead of management of different portable devices. Snowball in AWS console comes under migration. We need to create a job/request/order Snowball. Once hardware delivered to connect to Snowball we need to download client. Once hardware setup is done, download credentials from Snowball dashboard. We need credential to unlock and connect to Snowball Snowball can import and export data to S3. Introduced Snowball (Amazon created physical box) with below types Snowball (standard) : is peta-byte scale data transport solution. Transfer your data to snowball then transport that to Amazon and Amazon will upload data to AWS. Amazon uses multiple layer of security in data transport. 80TB available on all region. Snowball edge : 100TB storage capacity comes with on bo...

AWS - Network and Content Delivery

Network and Content Delivery Virtual Private Cloud(VPC) are virtual systems that are used to host services and application. Amazon DNS service is Route53(53 is DNS port). CloudFront is used for content delivery. Direct connect is used to physically connect AWS data centre through telephone lines. CloudFront Content delivery network(CDN) is a system of distributed servers (network) that deliver webpages and web contents to requestor based on geographic locations of user. Edge location are the locations where the content will be cached, separate entity from AZ or region. Origin of content that is distributed by CDN are EC2, S3, Elastic load balancer or route53. Distribution is name given to the CDN which consists of the edge locations. Two type distribution Web for websites and RTMP(real time messg protocol) for adobe flash. User hits URL, a request is first sent to edge location, if the object found request is served else request sent to S3 bucket that contains requeste...

AWS- Global Infrastructure

AWS Global Infrastructure Is the physical infrastructure that supports all software that is available to end user as services. Spread across different countries, established as data centre. In AWS terminology they are categorised as Regions, Availability zone and Edge locations. Regions are marked geographical area. Each region has Availability zone(Data Centre), minimum two to support failover can have more. Edge locations are physical systems distributed among regions to cache frequent queries and request for faster access.

AWS - Compute

Compute EC2 are VM on cloud run as an instance. EC2 container services are used to support cluster infrastructure. Elastic Beanstalk used to deploy code on amazon web services. Lambda is used as a service where we just deploy our code and set some events or triggering point for code to run and respond. LightSail to deploy code for those who has no experience of AWS deployment. 

AWS - Databases

Databases Relational databases (Oracle, SQL server, Aurora). DynamoDB is non-relational databases. RedShift is data warehouse solution. Elasticache is a cache service for database.

AWS - Migration

Migration Snowball service to migrate all disk data to AWS cloud. Database migration services(DMS) to migrate on premise database to AWS cloud or to migrate between AWS clouds. Server migration services to migrate on premise VM to AWS.

AWS - Analytics

Analytics Athena service allows to run SQL query on S3. Elastic map reduce(EMR) service to process large amount of data. Cloud search Elastic search Kinesis to analyse real time data. Data pipeline service to move data. Quick sight is business analytics tool.

AWS - Management Tools

Management Tools Cloud watch monitor AWS environment like EC2 instance etc. Cloud formation is a way of turning infrastructure into code of AWS like template. Cloud trail is used for auditing. Opsworks is used for automating deployment of server using chef. Config manager to monitor environment for any break, auditing with alert. Service catalog allows to authorise services. Trusted advisor is automating advisor for environment.

AWS - Application services

Application services Step function is used to visualise whats going on with application and its services it is using. Simple workflow service (SWF). API gateway allows to access backend services. AppStream is used to streaming desktop application. Elastic transcoder is used to format video for different devices.

AWS - Developer tools

Developer tools CodeCommit CodeBuild CodeDeploy CodePipeline

AWS - Mobile services

Mobile services MobileHub Cognito DeviceFarm MobileAnalytics PinPoint

AWS - Business Productivity

Business Productivity WorkDocs WorkMail

AWS - Internet of Things(IOT)

Internet of Things(IOT) iOT to monitor many devices

AWS - Desktop and App streaming

Desktop and App streaming Workspaces is like VDI. App Streaming 2.0

AWS - Artificial Intelligence

Artificial Intelligence Alexa Lex Polly Machine Learning Rekognition

AWS - Messaging

Messaging SNS SQS SES

AWS - Critical Terms

Critical Terms Users - End users. Groups - A collection people under one set of permissions. Roles - Create and assign them to AWS resources. Policies - It is a document that defines one or more permissions.

AWS - Security and Identity

AWS - Security and Identity Identity Access Management (IAM) used to manage user, groups, permission, etc. Inspector is an agent on VM to monitor security and reporting. Certificate Manager managing SSL certificate. Directory service for Active directory service and other directory service. Web application firewall for application level protection. Artifacts list of access-controlled documents 1 - IAM It is used to manage users, permission and level of access to AWS console. IAM console to manage user account access. Identity federation: Linking to users multiple accounts like Facebook, Twitter, AD service etc. Multifactor authentication[MFA]. Supports temporary access for users and devices. Custom password policy. Integrates other AWS services. Support PCI DSS[The Payment Card Industry Data Security Standard (PCI DSS) is an information security standard for organisations that handle branded credit cards from the major card schemes]. IAM is not region specific ...

AWS - Solution Architect Associate

AWS Solution Architect Associate Services to learn for exam AWS Global infrastructure - Physical infrastructure for all services. Network and Content Delivery Compute Storage Database Security and Identity Management Tools Desktop and App Streaming - High level Messaging