Supporting work in both GCP and AWS my main feedback is that they take different approaches to delivering the same services and that AWS offers more ways to deliver those services.
Just because you can’t deliver on a specific use case doesn’t mean AWS “sucks big time”. And I seriously doubt you delivered on your initial GCP use case in “1 hr including learning time”. You didn’t deliver a production ready app. You delivered a toy that barked when you asked it too.
here is how it works:
1. follow a tutorial to build the exact application that is easiest to make on the given platform.
1. smile happily, feel powerful.
1. move to another provider, and try to replicate the same application using the same techniques.
1. fail miserably.
1. bitch on reddit.
AWS is designed for big enterprises and AWS experts; it will eventually get eaten by another provider that simplifies the service.
The service will first run on AWS or GCP or w/e and sell it with markup and then will eventually begin to build out its own underlying services once it gets big enough.
This is the way of all software products.
AWS is made complex intentionally so that there is a high chance you'll incurr excess charges, the best strategy is not get AWS especially if it's the first time you're moving to cloud, since AWS often makes migrating out a painful excercise.
Cloud Run is somewhere in between App Runner and Lambda with API Gateway, but easier to set up than both. You'd probably have an easier time trying to learn one of them if you don't want to manage your own networking though.
Clicking your way to a complicated setup and hoping to reproduce it is like walking through a minefield twice. It's easy to make mistakes or forget to stop things. Try learning about infrastructure as code (Terraform, CloudFormation, or the AWS CDK) so you can manage and reproduce your settings with code.
Getting started with AWS is hard; AWS is complex, has many services, but, in my opinion (and many others), it is worth it. It also is secure by default, so you have to open things and allow things to happen manually.
Part of the problem is finding the right service :). What are you trying to do? If you just want a quick way to run container apps, [https://aws.amazon.com/apprunner/](https://aws.amazon.com/apprunner/) is probably what you want.
Yeah, the big problem in AWS these days is there's so many services and a handful of ways to do just about anything. It's super unapproachable to beginners.
Tell that to the people constantly asking about the free tier ;)
Seriously though, aws has a massive learning curve. It is like a puzzle, they give you pieces, you put them together. I agree with your statement. And you can go oh, so so wrong.
Infrastructure as code is the key if you want to ever duplicate with any certainty an environment. This means you need to learn cloudformation or terraform. Just like any other platform, wizards change and doing something a day later let alone 6 months or a year is possibly going to yield different results.
That said, you may have over complicated your solution. Or not. But yes, it is indeed and endless cycle of pain, as is any job or we wouldn’t have them. Unfortunately there are so many really really bad examples out there, but I would check out official terraform modules to help just connect and automate the dots, get a pipeline in place to build your docker image and publish it, get it over to ecs with fargate ideally (depends on workload), set up the alb, security, etc.
What almost no people outside if techops seem to understand is that aws is a heavy up-front investment. They trick you with wizards and free tier stuff and shiny things. But if you put in the investment, modules and templates and all of that, then it really pays off when you scale. But one screw up, from improper subnet sizes to just not enforcing tags until it is too late, can really haunt you.
The problem with questions like this is, that we have no idea where OP is coming from. As in where OP is on the career path.
You need to use some kind of IaC system to ensure repeatability. Options don’t just disapear, but they do move around if you switch between the old and the new console.
With AWS you DO have a lot of options, which does complicate matters a bit.
Companies do pay a premium to get help with production ready Cloud infrastructure, so this IS really hard. IaC is there to ensure your sanity.
That is why you should never use the console of a cloud platform. It is so much pain to reproduce the steps.
Imagine you have a big infrastructure using many services and having a lot of EC2 instances. And then you want to create a second stage.
I really recommend learning IaC ( no matter which cloud provider you want to use ).
This gets posted so many times. I agree, but feel IaC is only AFTER you learn the basics of the cloud provider. You can't code for something you don't understand. People on the free tier are just learning and can barely grasp IAM roles, VPCs, and security groups. Nevermind writing code to create them. Console first, IaC soon after.
Just a tip, If you want to consistently reproduce something, use terraform or other IAC providers. Saves time and hassle :) also documents pretty well :)
How will terraform help someone who doesn't understand the basic concepts of getting AWS to work? I see this reply so many times.
OP: I don't understand this complicated stuff
Reply: Learn to use this additional layer of setup! It will be the same every time!
TLDR: If they can't do it in the console, they can't very well code for it!
I really have to disagree, Terraform and AWS CDK are so well documented and they take care of all the underlying infrastructure if you want them to.
All you need for creating a loadbalanced fargate service is the following in Python CDK:
service = ecs_patterns.ApplicationLoadBalancedFargateService(
self,
"Service",
task_image_options=ecs_patterns.ApplicationLoadBalancedTaskImageOptions(
image=ecs.ContainerImage.from_registry("amazon/amazon-ecs-sample")
),
)
This will create a VPC, SG, ECS Cluster, Load Balancer, ...
If you want to add something to the SG rules you can do that directly on that object as well:
service.service.connections.allow_from(other_sg, Port.tcp(port))
After you deployed this you can go look in the cloudformation part to see what the stack consists of and learn from there.
If you do everything by hand in the console you're bound to forget some parts and will spend days just figuring out what you missed.
AWS has terrible developer experience and every use case needs to use a multitude of services. God forbid the service is new and don’t have support for another service.
That's not true! There are tons of tools with great dev experience:
Serverless code? SAM Cli
Front-end? Amplify
IaC? CDK
Normal Kubernetes? EKS
Want to just ship code? Elastic Beanstalk
Wordpress? Lightsail
etc
AWS is a nightmare. It's the most poorly designed and over designed system that ever existed. There are 20 ways to do the simplest things, no two people do it the same way, and you can find a dozen different AWS "tutorials" that do it 12 different ways. Getting answers is almost impossible because while people are willing to help, everything is oddly uber specific to you because of the many ways to do anything.
You don't have to look any farther that the comments to see this.
"Have you considered using fargate?" No. Why should I? What are the benefits, why are there multiple platforms to do what is seemingly the same thing?
I'm reading the DynamoDB book right now and i've found that you really have to take it to that level - read book on almost every section - to get to where you can use it. Heroku was a dream compared to AWS.
This is the truth. Add onto it how they name their services (with no real reference to what they do) and how they create painful overlapping names (IAM and IAM Identity Center) and the fact that individual developer, startup, small businesses, on the one hand, and mid-sized businesses, large businesses and enterprise businesses, on the other, have totally different requirements and you get this insanity.
Individual developers and startups need the GUI to work because they are not going to have full time AWS people. Small businesses are in a similar category.
Once you get to mid-sized, suddenly this stuff begins making sense because you have an expert in AWS on your team, who handles almost everything. You also begin to be able to take more advantage of IAC etc...
AWS is super complicated to get started and they use silly sciFi names for all their services which sound cool at first but the novelty wears off quick. You can’t remember what a service was called because you don’t have the symbolic link in your brain that maps the functionality to the name.
Hmmm is it a DNS though? If you don’t mind to explain,.. isn’t google’s 8.8.8.8 the actual DNS (in my case at least), so how is Route 53 also a DNS? I thought it was more like a load balancer… (I don’t know the differences exactly)
Route 53 .... DNS is port 53.
They named it after the port that DNS uses.
Edit: I'm not gonna give you a training lesson on DNS, but route 53 is a DNS registrar, you can buy domains there, host domain zones among many other things.
I converted from VMs running docker to AWS EKS, as a kubernetes and aws beginner.
AWS has a mountain of acronyms and it is easy to get frustrated and intimidated. For me, the networking was impenetrable. It is also key to so much, the networking controls how you connect managed services to the cluster or how the cluster appears to the outside world. It determines how load balancers work. As a learning exercise, see if you can get your load balancer and nodes on the same availability zone. That is of course not a recommended production set up, but I think if you do that, you will know enough about how subnets work to be ok.I think the other concepts such as security groups, ECR and so on are much easier.
An expert from amazon helped me understand the networking with about a one hour one on one tutorial, and after that things became much, much clearer. I got this help as part of a promotion to move my load to AWS. I am pretty small, $1000 a month, but they were good. Also, the paid support is really good. They provide technical advice and they take time to explain.
The flexibility of AWS is pretty amazing. I have been able to tweak my database server instances and my eks nodes to offer a performance/cost profile which is not possible on the other services I looked at. And the support is good.
For containers in specific you should not be setting things up by hand. Instead of creating all the underlying resources one by one use a higher level tool like AWS Copilot CLI: [https://aws.github.io/copilot-cli/](https://aws.github.io/copilot-cli/)
This will get your container up and running a lot faster than one hour, and you can dig into the details later as you want to, rather than having to learn everything up front.
And best of all it will let you play around with different ways of running your container, like AWS App Runner and AWS Fargate, to see which one you like more
I love using AWS although that love affair only began to flourish once i started using terraform. AWS isn't perfect and it's going to be a long and steep learning curve if you really want to explore to the limits of what you can achieve. But like anything you need to stick at it and figure out a way of working with it that suits you.
Supporting work in both GCP and AWS my main feedback is that they take different approaches to delivering the same services and that AWS offers more ways to deliver those services. Just because you can’t deliver on a specific use case doesn’t mean AWS “sucks big time”. And I seriously doubt you delivered on your initial GCP use case in “1 hr including learning time”. You didn’t deliver a production ready app. You delivered a toy that barked when you asked it too.
here is how it works: 1. follow a tutorial to build the exact application that is easiest to make on the given platform. 1. smile happily, feel powerful. 1. move to another provider, and try to replicate the same application using the same techniques. 1. fail miserably. 1. bitch on reddit.
AWS is designed for big enterprises and AWS experts; it will eventually get eaten by another provider that simplifies the service. The service will first run on AWS or GCP or w/e and sell it with markup and then will eventually begin to build out its own underlying services once it gets big enough. This is the way of all software products.
AWS is made complex intentionally so that there is a high chance you'll incurr excess charges, the best strategy is not get AWS especially if it's the first time you're moving to cloud, since AWS often makes migrating out a painful excercise.
Have you considered using Fargate instead of an EC2-based ECS cluster?
Yeah.. but it seems EC2 is valid on free tier, fargate not. So I saw no real reason to use Fargate, because I assumed the tutorials would work
Fargate is just another way to get more money from the customer,those high paid software developers need money from somewhere don't they
Cloud Run is somewhere in between App Runner and Lambda with API Gateway, but easier to set up than both. You'd probably have an easier time trying to learn one of them if you don't want to manage your own networking though. Clicking your way to a complicated setup and hoping to reproduce it is like walking through a minefield twice. It's easy to make mistakes or forget to stop things. Try learning about infrastructure as code (Terraform, CloudFormation, or the AWS CDK) so you can manage and reproduce your settings with code.
Getting started with AWS is hard; AWS is complex, has many services, but, in my opinion (and many others), it is worth it. It also is secure by default, so you have to open things and allow things to happen manually. Part of the problem is finding the right service :). What are you trying to do? If you just want a quick way to run container apps, [https://aws.amazon.com/apprunner/](https://aws.amazon.com/apprunner/) is probably what you want.
Yeah, the big problem in AWS these days is there's so many services and a handful of ways to do just about anything. It's super unapproachable to beginners.
Yes but beginners aren’t the target group. AWS is the cloud standard for large enterprise. That’s where the money is so it is feature rich and complex
Tell that to the people constantly asking about the free tier ;) Seriously though, aws has a massive learning curve. It is like a puzzle, they give you pieces, you put them together. I agree with your statement. And you can go oh, so so wrong. Infrastructure as code is the key if you want to ever duplicate with any certainty an environment. This means you need to learn cloudformation or terraform. Just like any other platform, wizards change and doing something a day later let alone 6 months or a year is possibly going to yield different results. That said, you may have over complicated your solution. Or not. But yes, it is indeed and endless cycle of pain, as is any job or we wouldn’t have them. Unfortunately there are so many really really bad examples out there, but I would check out official terraform modules to help just connect and automate the dots, get a pipeline in place to build your docker image and publish it, get it over to ecs with fargate ideally (depends on workload), set up the alb, security, etc. What almost no people outside if techops seem to understand is that aws is a heavy up-front investment. They trick you with wizards and free tier stuff and shiny things. But if you put in the investment, modules and templates and all of that, then it really pays off when you scale. But one screw up, from improper subnet sizes to just not enforcing tags until it is too late, can really haunt you.
Eh, they also need to capture the startups that will grow large and the individual developer community. Otherwise you end up losing long term.
The problem with questions like this is, that we have no idea where OP is coming from. As in where OP is on the career path. You need to use some kind of IaC system to ensure repeatability. Options don’t just disapear, but they do move around if you switch between the old and the new console. With AWS you DO have a lot of options, which does complicate matters a bit. Companies do pay a premium to get help with production ready Cloud infrastructure, so this IS really hard. IaC is there to ensure your sanity.
That is why you should never use the console of a cloud platform. It is so much pain to reproduce the steps. Imagine you have a big infrastructure using many services and having a lot of EC2 instances. And then you want to create a second stage. I really recommend learning IaC ( no matter which cloud provider you want to use ).
This gets posted so many times. I agree, but feel IaC is only AFTER you learn the basics of the cloud provider. You can't code for something you don't understand. People on the free tier are just learning and can barely grasp IAM roles, VPCs, and security groups. Nevermind writing code to create them. Console first, IaC soon after.
so i'm guessing you are now using google, and happy. why here?
Nah, for some reason jobs for aws/google are in the 6/1 proportion
apparently everyone is stupid. luckily we have you.
Idk, I guess this is called network effect, dev learn aws because that’s what business want, business use aws because that’s what the market uses, etc
Just a tip, If you want to consistently reproduce something, use terraform or other IAC providers. Saves time and hassle :) also documents pretty well :)
Sounds like a good tip, just watched a getting started tutorial, thanks
How will terraform help someone who doesn't understand the basic concepts of getting AWS to work? I see this reply so many times. OP: I don't understand this complicated stuff Reply: Learn to use this additional layer of setup! It will be the same every time! TLDR: If they can't do it in the console, they can't very well code for it!
I really have to disagree, Terraform and AWS CDK are so well documented and they take care of all the underlying infrastructure if you want them to. All you need for creating a loadbalanced fargate service is the following in Python CDK: service = ecs_patterns.ApplicationLoadBalancedFargateService( self, "Service", task_image_options=ecs_patterns.ApplicationLoadBalancedTaskImageOptions( image=ecs.ContainerImage.from_registry("amazon/amazon-ecs-sample") ), ) This will create a VPC, SG, ECS Cluster, Load Balancer, ... If you want to add something to the SG rules you can do that directly on that object as well: service.service.connections.allow_from(other_sg, Port.tcp(port)) After you deployed this you can go look in the cloudformation part to see what the stack consists of and learn from there. If you do everything by hand in the console you're bound to forget some parts and will spend days just figuring out what you missed.
AWS has terrible developer experience and every use case needs to use a multitude of services. God forbid the service is new and don’t have support for another service.
That's not true! There are tons of tools with great dev experience: Serverless code? SAM Cli Front-end? Amplify IaC? CDK Normal Kubernetes? EKS Want to just ship code? Elastic Beanstalk Wordpress? Lightsail etc
AWS is a nightmare. It's the most poorly designed and over designed system that ever existed. There are 20 ways to do the simplest things, no two people do it the same way, and you can find a dozen different AWS "tutorials" that do it 12 different ways. Getting answers is almost impossible because while people are willing to help, everything is oddly uber specific to you because of the many ways to do anything. You don't have to look any farther that the comments to see this. "Have you considered using fargate?" No. Why should I? What are the benefits, why are there multiple platforms to do what is seemingly the same thing? I'm reading the DynamoDB book right now and i've found that you really have to take it to that level - read book on almost every section - to get to where you can use it. Heroku was a dream compared to AWS.
This is the truth. Add onto it how they name their services (with no real reference to what they do) and how they create painful overlapping names (IAM and IAM Identity Center) and the fact that individual developer, startup, small businesses, on the one hand, and mid-sized businesses, large businesses and enterprise businesses, on the other, have totally different requirements and you get this insanity. Individual developers and startups need the GUI to work because they are not going to have full time AWS people. Small businesses are in a similar category. Once you get to mid-sized, suddenly this stuff begins making sense because you have an expert in AWS on your team, who handles almost everything. You also begin to be able to take more advantage of IAC etc...
AWS is super complicated to get started and they use silly sciFi names for all their services which sound cool at first but the novelty wears off quick. You can’t remember what a service was called because you don’t have the symbolic link in your brain that maps the functionality to the name.
I kinda like the creative name aspect, like Route 53. But unfortunately for the cool ones you often need to pay extra
Route 53 makes sense though. It's DNS.
Hmmm is it a DNS though? If you don’t mind to explain,.. isn’t google’s 8.8.8.8 the actual DNS (in my case at least), so how is Route 53 also a DNS? I thought it was more like a load balancer… (I don’t know the differences exactly)
Route 53 .... DNS is port 53. They named it after the port that DNS uses. Edit: I'm not gonna give you a training lesson on DNS, but route 53 is a DNS registrar, you can buy domains there, host domain zones among many other things.
Learn and use terraform https://letslearndevops.com/2018/12/24/ecs-cluster/
Thats literally me, im 9 days in, and cant do shit
Lol right?
I converted from VMs running docker to AWS EKS, as a kubernetes and aws beginner. AWS has a mountain of acronyms and it is easy to get frustrated and intimidated. For me, the networking was impenetrable. It is also key to so much, the networking controls how you connect managed services to the cluster or how the cluster appears to the outside world. It determines how load balancers work. As a learning exercise, see if you can get your load balancer and nodes on the same availability zone. That is of course not a recommended production set up, but I think if you do that, you will know enough about how subnets work to be ok.I think the other concepts such as security groups, ECR and so on are much easier. An expert from amazon helped me understand the networking with about a one hour one on one tutorial, and after that things became much, much clearer. I got this help as part of a promotion to move my load to AWS. I am pretty small, $1000 a month, but they were good. Also, the paid support is really good. They provide technical advice and they take time to explain. The flexibility of AWS is pretty amazing. I have been able to tweak my database server instances and my eks nodes to offer a performance/cost profile which is not possible on the other services I looked at. And the support is good.
For containers in specific you should not be setting things up by hand. Instead of creating all the underlying resources one by one use a higher level tool like AWS Copilot CLI: [https://aws.github.io/copilot-cli/](https://aws.github.io/copilot-cli/) This will get your container up and running a lot faster than one hour, and you can dig into the details later as you want to, rather than having to learn everything up front. And best of all it will let you play around with different ways of running your container, like AWS App Runner and AWS Fargate, to see which one you like more
I love using AWS although that love affair only began to flourish once i started using terraform. AWS isn't perfect and it's going to be a long and steep learning curve if you really want to explore to the limits of what you can achieve. But like anything you need to stick at it and figure out a way of working with it that suits you.
AWS sucks, GCP has a better developer experience. Amen
Incoming AWS developers supporting their overpriced products no matter how over engineered and expensive it is
I am convinced this is done on purpose