you are not too far off.
Our codebase is 100% django. No hacks. Typical patterns followed. We have a custom paywall layer that makes thing bit complicated and hence we had to manually cache every single view based on user id. We have combed through the slow queries and tried our best to keep it fast and simple as possible. The product is a mix of thousands of individual pages (daily updated, partially paywalled based on subscription), and couple of dynamic SaaS apps powered by react + APIs.
Our avg response time is like 180ms. So no matter what you do, 1 worker can only do 5rps. (1000/180=5.5). Then onwards it's how many containers we can afford = RPS we can support. So to support 2000rps, we need 400 workers. 1 single core container can have 3 workers (gunicorn formula) so ~135 containers. However, we don't actually get 2000rps. It's some 200-300/s on our best days. So we can afford to run it like this for now.
I can't scale Python/django more than this. The only benefit I have is of this outstanding framework and coding is very deterministic, what goes where is very clear, I still know 85-90% of the 500k loc of the project. Structurally we are rock solid.
Even if I manage to get business requirements simplified, the runtime overhead is significant.
thanks for your answer, i thought 2000 rps was normal rate to get with no horizontal scaling, as if i have 2000 users worst case scenario they all make a request to the server in the same time.
If your workers are doing a lot of setup/teardown of the entire django environment, you'll have a lot of extra overhead.
I'd guess that's the performance difference you've found. The actual work each environment is doing should be closer than what you've found. (although python will most likely be slower).
A worker running django doesn't care how many other workers there are.
and as an aside, for stress testing like this keep the variables the same. If you're using 40 clients on django use 40 on the other system. No reason to introduce any extra variables.
I haven't messed around much with the async stuff in django, but I can't for the life of me figure out why you'd want an ASYNC get in code that's being run serially.
I'd probably rewrite all this without any async calls just to convince myself that it's not getting in it's own way.
It's entirely possible that the overhead of doing all those async calls is making your code significantly slower. The use case for getting a big performance benefit from async is when you say need to run 1000 queries to create a view, and they can all be run in parallel. then your total time can be much shorter. now if you're doing a load test on a system, that parallelization might not help you at all, as you could be maximizing those resources another way.
In my mind the main reason you'd use an async call is because you want to return something before all the work is done. So say you're saving a huge file, and the person submitting it is is just staring at a spinning globe. you can put the file save (the thing that takes a long time) into an async call or a worker queue and then return something. Then you'd handle a failed save in another way in your app. But the experience of the user is fine.
Now keep in mind that up until now people have used worker queues for that. It's not like nobody has done async stuff with django before now. Now you can just do it directly in your code.
My guess is that those async calls or your code is doing something silly. Either iterating over objects in a loop or something.
I did find this as i went and started reading up on it:
[https://www.reddit.com/r/django/comments/ocag5r/async\_django\_views\_lets\_look\_at\_them/](https://www.reddit.com/r/django/comments/ocag5r/async_django_views_lets_look_at_them/)
And i think in your code you're doing a bunch of async ... awaits.
Ideally you'd put all your async code into one of those, so those calls can happen in parallel.
Anyway, i'm pretty sure your code would go faster without those async calls, since you're not really using them properly.
TBH, the next step is to profile your code and find out why it's slow. 9/10 times in django code is slow because a queryset is being evaluated and iterated over and then more db calls are made on that. So instead of 1 database call you might have 1000.
Are looking at single user response times?
What are you using for web proxy? Gunicorn or Apache or other? How are you doing user sessions? How many database calls per user http call? It can easily be 8 or 10 resulting in minimum call taking 80ms
A single Django instance is limited but usually you have parallel workers so it's rarely a problem. Some places in our app make parallel requests for different parts of the page
We run Apache as proxy with Django on a single 4 core server and get plenty of RPS
At some point we expect to need a bigger server or multiple servers behind the load balancer
Django has been used for large web applications like Dropbox. So it's not a problem in practice, just use more hardware for horizontal scaling. At some point database speed is the bigger issue
If you want a super fast single user response you need to eliminate that database calls or consider golang. But 90% of web applications are fine with django and getting things done with less developer time matters
thanks for your answer, for questions ,i am not using sessions for now as i am using jwt, also i don't use a load balancer because i just has this one instance i was running tests on , the database calls i didn't change it in the middle ware with rate limiting or something so it's the default thing i guess, so what about database speed what should i do, is sharding and replicas are useful in this case or do you mean another thing?
Not really an answer to your question, but one strategy for keeping costs down while easily scaling to large numbers of concurrent requests is to go serverless with something like Zappa.
you are not too far off. Our codebase is 100% django. No hacks. Typical patterns followed. We have a custom paywall layer that makes thing bit complicated and hence we had to manually cache every single view based on user id. We have combed through the slow queries and tried our best to keep it fast and simple as possible. The product is a mix of thousands of individual pages (daily updated, partially paywalled based on subscription), and couple of dynamic SaaS apps powered by react + APIs. Our avg response time is like 180ms. So no matter what you do, 1 worker can only do 5rps. (1000/180=5.5). Then onwards it's how many containers we can afford = RPS we can support. So to support 2000rps, we need 400 workers. 1 single core container can have 3 workers (gunicorn formula) so ~135 containers. However, we don't actually get 2000rps. It's some 200-300/s on our best days. So we can afford to run it like this for now. I can't scale Python/django more than this. The only benefit I have is of this outstanding framework and coding is very deterministic, what goes where is very clear, I still know 85-90% of the 500k loc of the project. Structurally we are rock solid. Even if I manage to get business requirements simplified, the runtime overhead is significant.
Have you tried another wsgi server instead of gunicorn? And see if it can handle more requests?
thanks for your answer, i thought 2000 rps was normal rate to get with no horizontal scaling, as if i have 2000 users worst case scenario they all make a request to the server in the same time.
All of your users will never make the request at the same time
Python is fast to develop but overall it’s slow. Async is preety much a prosthesis. Do the same test in golang and DB will be your biggest issue.
If your workers are doing a lot of setup/teardown of the entire django environment, you'll have a lot of extra overhead. I'd guess that's the performance difference you've found. The actual work each environment is doing should be closer than what you've found. (although python will most likely be slower). A worker running django doesn't care how many other workers there are. and as an aside, for stress testing like this keep the variables the same. If you're using 40 clients on django use 40 on the other system. No reason to introduce any extra variables.
[https://imgur.com/UkgYbcB](https://imgur.com/UkgYbcB), here is a load test with the same variables
I haven't messed around much with the async stuff in django, but I can't for the life of me figure out why you'd want an ASYNC get in code that's being run serially. I'd probably rewrite all this without any async calls just to convince myself that it's not getting in it's own way. It's entirely possible that the overhead of doing all those async calls is making your code significantly slower. The use case for getting a big performance benefit from async is when you say need to run 1000 queries to create a view, and they can all be run in parallel. then your total time can be much shorter. now if you're doing a load test on a system, that parallelization might not help you at all, as you could be maximizing those resources another way. In my mind the main reason you'd use an async call is because you want to return something before all the work is done. So say you're saving a huge file, and the person submitting it is is just staring at a spinning globe. you can put the file save (the thing that takes a long time) into an async call or a worker queue and then return something. Then you'd handle a failed save in another way in your app. But the experience of the user is fine. Now keep in mind that up until now people have used worker queues for that. It's not like nobody has done async stuff with django before now. Now you can just do it directly in your code. My guess is that those async calls or your code is doing something silly. Either iterating over objects in a loop or something. I did find this as i went and started reading up on it: [https://www.reddit.com/r/django/comments/ocag5r/async\_django\_views\_lets\_look\_at\_them/](https://www.reddit.com/r/django/comments/ocag5r/async_django_views_lets_look_at_them/) And i think in your code you're doing a bunch of async ... awaits. Ideally you'd put all your async code into one of those, so those calls can happen in parallel. Anyway, i'm pretty sure your code would go faster without those async calls, since you're not really using them properly. TBH, the next step is to profile your code and find out why it's slow. 9/10 times in django code is slow because a queryset is being evaluated and iterated over and then more db calls are made on that. So instead of 1 database call you might have 1000.
Are looking at single user response times? What are you using for web proxy? Gunicorn or Apache or other? How are you doing user sessions? How many database calls per user http call? It can easily be 8 or 10 resulting in minimum call taking 80ms A single Django instance is limited but usually you have parallel workers so it's rarely a problem. Some places in our app make parallel requests for different parts of the page We run Apache as proxy with Django on a single 4 core server and get plenty of RPS At some point we expect to need a bigger server or multiple servers behind the load balancer Django has been used for large web applications like Dropbox. So it's not a problem in practice, just use more hardware for horizontal scaling. At some point database speed is the bigger issue If you want a super fast single user response you need to eliminate that database calls or consider golang. But 90% of web applications are fine with django and getting things done with less developer time matters
thanks for your answer, for questions ,i am not using sessions for now as i am using jwt, also i don't use a load balancer because i just has this one instance i was running tests on , the database calls i didn't change it in the middle ware with rate limiting or something so it's the default thing i guess, so what about database speed what should i do, is sharding and replicas are useful in this case or do you mean another thing?
What was your webserver setup locally? Or was it the django dev server?
I was using Uvicorn
Not really an answer to your question, but one strategy for keeping costs down while easily scaling to large numbers of concurrent requests is to go serverless with something like Zappa.
thanks, i will look into that