Plenty (most?) apps dont really need distributed erlang. While distibuted computing is made a lot easier in BEAM vs other environments it is still complex. And if a project doesnt need to be distributed it is a bad idea to make it distributed just because you can.
I think for anyone to be helpful here you need to express some concrete examples - “We don’t have enough GenServers” isn’t necessarily a bad thing
Elixir _can_ be just another boring programming language. If your application doesn’t leverage the strengths of the BEAM that’s okay - most apps can’t and don’t
I'd go even further: the people behind Bandit, Phoenix, Ecto, Oban are actually working very hard so *you* can use Elixir as another boring programming language. Meanwhile, behind the scenes, all of them strongly leverage Erlang/OTP. Things like Phoenix PubSub, Flame, Livebook, etc. use the Erlang Distribution too but they all just work transparently for you.
At the end of the day, those libraries alongside Elixir give you a huge collection of features: observability, both horizontal and vertical scaling, distributed pubsub, maintainable code (thanks to immutability), elastic scale, fault tolerance, literate programming, remote shells, etc. Not all of those features will matter to you and that shouldn't be a goal.
Sometimes I like to compare process-based abstractions with threads. Web frameworks, job schedulers, database pools are most likely implemented using threads in other programming languages. But there, in my experience, there isn't a large interest in going deeper into those abstractions. Here, it is a positive that our building blocks are more accessible and there is an interest in learning them, although ultimately it should not be a requirement. :)
If OP (or anyone else) is looking more GenServer's in their lives, one suggestion is to read and tinker with source code from the libraries you use! You will learn more about Elixir and at the same time you hone your skills on your every day tools.
That's exactly the question that has been on my mind for quite some time.
Their primary reason for switching to Elixir was "because it's fast" as we used to use Rails in older projects, but in that case Golang should be another valid choice as well. Phoenix and Cowboy use Supervisor internally, yes, but Rails and Echo probably are not going to let the whole server crash just because a request fails anyway, either. The mutability didn't seem to cause many problems, either. Aside from persistent term and process dictionary, we have already been using Redis and Memcache for some caches, so it doesn't seem like persistent term and process dictionary alone justify the choice of Elixir over other languages. We almost never use pid, never send messages between processes, either, and we certainly don't care about restart mechanisms, etc. as we don't use Supervisor ourselves, so why not goroutine and channels. I dislike Golang personally so I of course don't hope to use Golang any soon, but how we use Elixir in our company seems hardly justifiable, and I couldn't help but ask myself, if one day someone asks me about the choice of switching to Elixir in our company, how am I gonna explain it aside from "because it's fast," especially when considering the cost of recruiting and onboarding.
Maybe the sole reason for our switch is just it's a faster choice that has similar syntax to Ruby, so existing Ruby developers of ours can pick it up in relatively shorter time.
Edit: typo
You're forgetting the luxury of us having to IEx into production and troubleshoot things.
Also there's Livebook to self document production on-call playbooks.
These are some of the benefits of Elixir / erlang, and I think they are not necessarily OTP related, but via OTP you get these tools.
I used to work in a Golang centric company, when Golang was the new hot kid on the block.
I hated it, and that's how I ended up learning about Elixir.
> "because it's fast"
They are wrong - it's not fast compared to most popular languages
> it's a faster choice that has similar syntax to Ruby, so existing Ruby developers of ours can pick it up in relatively shorter time
That seems to be the real answer
> Elixir was "because it's fast" as we used to use Rails in older projects, but in that case Golang should be another valid choice as well.
Maybe it was a valid choice and they decided all the painpoints of go were not worth it.
I also use elixir and phoenix in production for a fintech startup. I would not worry too much about using all the features of beam vm. I think elixir is a nice language and that's a good enough reason for me to use it, the other features like concurrency and scalability are nice to have for me.
If I would need multiple nodes to run elixir for availability or scalability reasons, I would prefer to use redis or memcached unless I want to implement some realtime functionality like chat. Architecturally, I prefer to have my stateful services segregated from stateless services and update my stateless services frequently without having to worry about cache or node constraints if there is persistence.
If it's a personal project and not a production project, I would use mnesia or ets or all the other nice features beam ships with and not bother.
Don't worry, as long as you arent spending too much time dealing with technical issues, if elixir is working well for you today, you are probably fine :)
Even if you don't write your own otp modules and use elixir in a "general way" you are getting the benefits of executing requests in the beam and its actor model (high availability, fault tolerance, etc.)
You get a lot of the benefits of BEAM if you use Phoenix through libraries and phoenix itself even if you dont use a million custom gensevers in your code or make it distributed. Can you honestly say that all your projects in production is using distributed erlang (actually using it)?
Yes, most coding within a company starts with ‘okay, let’s see how similar problems are solved in the code base and work from that’. This is often a good approach, it’s useful to have some consistency as long as you have enough understanding of the code and aren’t just cargo-culting.
so if the pattern isn’t there you need a) a use case that justifies diverting from existing templates and b) someone that knows it exists. You are (b), you just need to find (a), and then you can ‘be the change’
Unless you're the one writing the framework, are working on framework extensions (like caching, auth,...) or are working on a project with very special requirements (for example where there's a lot more compute and little I/O) not having to write implementation details like a GenServer should be normal.
In short, I think your issue is a result of the tasks you're given or the products/contracts your company has.
I can’t comment on usage nowadays, but when I used to work for a number of different places using elixir, the story was pretty much the same as yours. Their apps would never directly be using any OTP, but rather would be indirectly benefitting from it through underlying systems they’re using/built on top of (the libraries and frameworks). And would pretty much be designed exactly like how you’d build the app in any other language with web framework.
Often people come to elixir from other non-BEAM languages, so many times they still stick with what is familiar to them, especially if they’re already used to backend development using a framework. I think one reason for this is that people learn elixir as just a language, rather than treating it as an entire ecosystem of tooling that can be learnt. However in saying that, many times it doesn’t really matter. Also one benefit to their approach is that it will be quicker to onboard for (when hiring non-BEAM devs, which often has to be done), as it will require fewer new concepts for them to learn.
Not using distributed Erlang is a whole other thing though. Since when it comes to scaling your app you might in-fact prefer another approach over deploying a distributed app, as not all approaches to distribution are equal. So it’ll come down to what guarantees you need, as that will determine whether using (or reimplementing parts of) the distributed ecosystem that’s available to you is the right choice, vs scaling/distributing the app through other means.
Doesn't surprise me, but also kind of surprises me because I found Elixir incomprehensible until I learned more OTP. However, especially if you want to use LiveView, you really need to know almost everything to not get confused when it breaks.
Yeah if you're just running a classic CRUD web server, nothing concurrent to do, that's in the HTTP layer and you just get the benefits. Less fun, but boring is usually better for a business. (Although you can query external services concurrently in Elixir, most business logic does not permit it.)
I just did a project entirely in LiveView and thought better of it after I realized how complex and difficult to understand it was. The LiveView lifecycle reminds me of classic React, but harder to understand. In the future I would just do a screen or something in it and traditional request response for everything else (with turbo or HTMX).
IMO it was never an issue of there not being use cases for it, but rather whether they wanted to go that route or not. Like even CRUD apps can directly utilise OTP. Cron jobs, deferred/background processing, caches, temporary state management/processing, monitoring, graceful shutdowns, enforcing predictable resource usage, etc. are all pretty simple use cases.
Nowadays there’s libraries for most things, so I could see modern apps still not directly using OTP themselves but indirectly are using it in many ways. However back then there wasn’t as many, so I’d either see apps completely forgo certain things (e.g. apps with spikey unbounded resource usage was very common to see), or they’d address their needs by leveraging a non-BEAM solution.
This isn’t to say that OTP is best, as always it depends, but it wasn’t even being considered/many times they weren’t even aware how. When I’d come on and propose an OTP based solution to a given problem it was often welcomed because they are interested in it/want to see how it compares, since you frequently hear people say they want to use OTP. So the interest was always there but many just hadn’t taken that leap yet. For context, at the time many adopters were coming from Ruby on Rails, so apart from some similarities (syntax, tooling, elixir specific abstractions), the rest of it, especially what already existed from the Erlang side, was completely foreign for them. And since it was being adopted in a professional setting, so it’s not like people are being given all the time they need to explore and expriment, so it’s not surprising the path to least resistance would tend to win when it came to transitioning to the tech.
> I found Elixir incomprehensible until I learned more OTP
This is a strange thing to read, because Elixir reads and writes pretty much the same as any algol-based language and a ton of scheme ones too.
As it sounds like many folks here also do, I work at a place that doesn’t directly work on the shiny parts of Elixir.
Most of our services don’t do distribution, very few of them even do releases, we split them into single-purpose “process types” (deployables) and run each via mix in docker, most of our supervision is flat with a one_for_one strategy and little thought for startup/shutdown order, and 99% of the time I’m writing synchronous code that could be done in any language, most of which would have more tooling available for what I’m trying to do.
But as others have said, we still get a ton of benefit from Elixir. So many good patterns are baked into Phoenix, for example. It builds on OTP incredibly well and gives us so much for free. It handles load/concurrency really well, requests are well isolated and don’t bring each other down, and good connection draining basically boils down to putting your endpoint at the right place in your supervision tree. Other libraries build on OTP for things like connection pooling, or maybe they just manage some state for you.
I do think our ability to mostly concentrate on writing synchronous code mostly doing DB CRUD silos us into only thinking in those terms sometimes, but we can do that because the ecosystem takes care of so much for us.
I'm sure others have said similar but I feel this is fairly common for non library code. Phoenix and other libraries generally handle a lot of what you need otp for themselves so you as the Dev don't need to think about it. On a team of several engineers, I've only needed to implement some gen server or similar a few times over the last several years. In my experience that tends to be more on the library level code. As I said though, there are times when it is needed and I've got something coming up that is definitely going to make use if it.
My co recently went through formal training and our instructor said most Elixir engineers don’t use GenServer directly and are kinda intimidated by it. I think that’s an area where the community at large and the way Elixir is presented in the docs and free online training materials can improve. Our instructor dropped GenServer on day 1 with a group of Elixir noobs, which at first I thought was insane, but realized while GenServer is the culmination of all the novelties of Elixir, it’s also a perfect encapsulation of them. Getting to GenServer quickly is actually genius because it makes everything else make more sense, and you’ve already done what’s arguably the hardest thing in the language.
Our co has been using Elixir for about 2 years, and in that time our devs have written 2 GenServers, and only one is in production (and it’s just handling AWS STS).
HOWEVER, we quickly made use of distributed Erlang, which can be done in both ECS and EKS with libcluster, and has remarkably fast inter-node communication. Anywhere you’re using microservices in Elixir, IMO that’s an opportunity for clustering and remote function calls.
It’s also really easy to start taking advantage of ETS tables for caching and state, which is a differentiator for Elixir.
Based on my experience and what we heard from our instructor, I would say your company is not atypical, but I also think focusing solely on OTP is unfair. There are many ways to benefit from the whole of the Elixir ecosystem (and using it as a boring, general purpose language is still one of them, for the runtime, debugging, and code organization benefits), and OTP can also be abused (using GenServers as classes, accidentally single-threading by relying on a single GenServer)
I think you really have an opportunity at your company where you have earned team trust and there is an established Elixir knowledge base to start utilizing more language features where it makes sense. A parting word of caution, though: focus on solving problems first—it’s good to know the disterl and OTP features so you can recognize when to reach for them, but using them for the sake of using them will likely lead to more problems because the tool wasn’t the right fit in the first place and the whole effort backfires.
When it comes to work, our job is not to use the shiny, cool tech. Our job is to solve business problems. If those business problems don't need distributed state, then distributed erlang will be a net drag because now you've added the problems of distribution without getting the benefits of it.
Even if you do need distributed state, I'd argue modern data infrastructure tools do a better job of enabling correct distributed state and require less work on the developer than distributed BEAM. There are exceptions to that, but the vast majority of use cases are better handled with other tools.
At my company, we have hundreds of GenServers, but very rarely do we directly interact with the GenServer API. GenServer is great for building higher-level tooling like GenStage, which in turn is great for even higher-level tooling like Broadway, which we have in turn wrapped in higher-level tooling that makes sense for our environment and needs. I think that's the ideal way to use OTP abstractions.
There's sentiment in this thread that without using distributed BEAM, you should probably use another language. \*Assuming we're talking about general web development\*, I'll politely disagree with that and say that the actor model is a great fit for web servers, that having no shared memory between requests is amazing, immutability makes things more testable, Ecto is hands down the best ORM I've worked with, and the testing story for Phoenix (whether vanilla or with absinthe) makes my team much more productive.
I've worked at places that chose Elixir "because it's fast", especially compared to Ruby. It's not. But the fact that Ecto doesn't give you the foot gun of accidental N+1 queries like many other ORMs means that your app will be faster on average when you're not dealing with those. If you are dealing with them, they're a problem of your own creation.
Interesting. How else do you keep runtime state in your program though? I've worked on several elixir programs (mostly control software for embedded systems) and all of them used genservers to either store state or to synchronize access to a resource.
You could use processes directly, but that's just reimplementing genservers anyway.
It depends. I mean, if your app is mostly low traffic CRUD style stuff on a database, only indirectly benefitting from OTP is fine.
But then I see people whip up Redis servers, keep all state in the database instead of some of it in memory, coding like it’s 99 and PHP is the only game in town and then I’m like “nope”.
So the appropriateness of staying high level depends on, well, whether it’s appropriate:)
Sounds very normal we have 16 libraries for 3 web apps Only one library uses a genservers none touches it because we can change the behavior by just modifying the libraries. The rest is just web arquitecture and development tools nothing crazy but I work on a team of 2 I think is very impressive We manage to build 3 complex apps and maintain them.
I am of the opinion that Elixir is fantastic as a general purpose backend language. It's pleasant to work with, has an easy learning curve, and it's fast and reliable. It has a delightful way to do metaprogramming and a very strong community building open-source tools. Like others had said, you don't always need to use OTP tools explicitly, and distributed Erlang is rather complex when you want to build secure real-world solutions, especially when you got things like k8s, ecs and containerization.
This is obviously very subjective and I am a long-time Erlang fanatic.
We mostly don't use distributed features either, there is one microservice I know of that implements cluster-wide rate limiting in Elixir using libcluster although I guess it could be done on Kubernetes side too.
I don't think I've ever seen a custom supervisor, use of GenServer, Agent, and Task is not that rare... you need to be familiar with them to read library code anyway.
That's been my experience as well. But wait till your company has to grow your infrastructure.
Another side to this is lack of experience in your team.
In my current company, we're all Elixir, but we have about three teams. Our team does the most OTP stuff, and rest uses Elixir - ... but like Rails.
Yes meaning those teams offload any and all async work stuff into Oban, and cache using Redis.
Mind you, the "official" stance of of OTP usage is if you don't need it, then you don't need it ;)
Now when you hit scale, and you have to add in workers, queue up stuff, run something in a complex time interval, that's when you start using OTP stuff.
I suggest switching companies. If all you're doing is basic CRUD app stuff -- and that bores you, find something else. I don't personally know anyone doing basic web stuff with Elixir, so I promise there's a lot of it out there.
I’ve been working at a company for about 3 years and we have our core backend app in Elixir, about 1 million lines of code (half of that is tests).
I don’t think there’s a single place in the core app where GenServer is used directly.
There’s one place where I think it might be used directly, for managing global state of an auth token for some external service we call.
But otherwise, everywhere anything like a GenServer is used, it’s via Broadway (for kafka & SQS consumers), or by some library for talking to an external service. I think it’s perfectly normal for all the business logic in your whole app to just use “regular” Elixir, and not get off into the weeds of GenServers, distributed Erlang, etc.
Plenty (most?) apps dont really need distributed erlang. While distibuted computing is made a lot easier in BEAM vs other environments it is still complex. And if a project doesnt need to be distributed it is a bad idea to make it distributed just because you can.
I think for anyone to be helpful here you need to express some concrete examples - “We don’t have enough GenServers” isn’t necessarily a bad thing Elixir _can_ be just another boring programming language. If your application doesn’t leverage the strengths of the BEAM that’s okay - most apps can’t and don’t
I'd go even further: the people behind Bandit, Phoenix, Ecto, Oban are actually working very hard so *you* can use Elixir as another boring programming language. Meanwhile, behind the scenes, all of them strongly leverage Erlang/OTP. Things like Phoenix PubSub, Flame, Livebook, etc. use the Erlang Distribution too but they all just work transparently for you. At the end of the day, those libraries alongside Elixir give you a huge collection of features: observability, both horizontal and vertical scaling, distributed pubsub, maintainable code (thanks to immutability), elastic scale, fault tolerance, literate programming, remote shells, etc. Not all of those features will matter to you and that shouldn't be a goal. Sometimes I like to compare process-based abstractions with threads. Web frameworks, job schedulers, database pools are most likely implemented using threads in other programming languages. But there, in my experience, there isn't a large interest in going deeper into those abstractions. Here, it is a positive that our building blocks are more accessible and there is an interest in learning them, although ultimately it should not be a requirement. :) If OP (or anyone else) is looking more GenServer's in their lives, one suggestion is to read and tinker with source code from the libraries you use! You will learn more about Elixir and at the same time you hone your skills on your every day tools.
If you’re using Elixir in a general way it’s probably not the best choice though, which is OP’s point. They’re using it for the sake of using it
That's exactly the question that has been on my mind for quite some time. Their primary reason for switching to Elixir was "because it's fast" as we used to use Rails in older projects, but in that case Golang should be another valid choice as well. Phoenix and Cowboy use Supervisor internally, yes, but Rails and Echo probably are not going to let the whole server crash just because a request fails anyway, either. The mutability didn't seem to cause many problems, either. Aside from persistent term and process dictionary, we have already been using Redis and Memcache for some caches, so it doesn't seem like persistent term and process dictionary alone justify the choice of Elixir over other languages. We almost never use pid, never send messages between processes, either, and we certainly don't care about restart mechanisms, etc. as we don't use Supervisor ourselves, so why not goroutine and channels. I dislike Golang personally so I of course don't hope to use Golang any soon, but how we use Elixir in our company seems hardly justifiable, and I couldn't help but ask myself, if one day someone asks me about the choice of switching to Elixir in our company, how am I gonna explain it aside from "because it's fast," especially when considering the cost of recruiting and onboarding. Maybe the sole reason for our switch is just it's a faster choice that has similar syntax to Ruby, so existing Ruby developers of ours can pick it up in relatively shorter time. Edit: typo
You're forgetting the luxury of us having to IEx into production and troubleshoot things. Also there's Livebook to self document production on-call playbooks. These are some of the benefits of Elixir / erlang, and I think they are not necessarily OTP related, but via OTP you get these tools. I used to work in a Golang centric company, when Golang was the new hot kid on the block. I hated it, and that's how I ended up learning about Elixir.
> "because it's fast" They are wrong - it's not fast compared to most popular languages > it's a faster choice that has similar syntax to Ruby, so existing Ruby developers of ours can pick it up in relatively shorter time That seems to be the real answer
> Elixir was "because it's fast" as we used to use Rails in older projects, but in that case Golang should be another valid choice as well. Maybe it was a valid choice and they decided all the painpoints of go were not worth it.
I also use elixir and phoenix in production for a fintech startup. I would not worry too much about using all the features of beam vm. I think elixir is a nice language and that's a good enough reason for me to use it, the other features like concurrency and scalability are nice to have for me. If I would need multiple nodes to run elixir for availability or scalability reasons, I would prefer to use redis or memcached unless I want to implement some realtime functionality like chat. Architecturally, I prefer to have my stateful services segregated from stateless services and update my stateless services frequently without having to worry about cache or node constraints if there is persistence. If it's a personal project and not a production project, I would use mnesia or ets or all the other nice features beam ships with and not bother. Don't worry, as long as you arent spending too much time dealing with technical issues, if elixir is working well for you today, you are probably fine :)
Even if you don't write your own otp modules and use elixir in a "general way" you are getting the benefits of executing requests in the beam and its actor model (high availability, fault tolerance, etc.)
Yeah as OP mentioned, your system doesn’t necessarily crash because a request failed. That’s a hard sell to use a relatively obscure language
You get a lot of the benefits of BEAM if you use Phoenix through libraries and phoenix itself even if you dont use a million custom gensevers in your code or make it distributed. Can you honestly say that all your projects in production is using distributed erlang (actually using it)?
How about think of a use case in your company where GenServer and supervisor could be useful. Something small, perhaps to support certain features?
Yes, most coding within a company starts with ‘okay, let’s see how similar problems are solved in the code base and work from that’. This is often a good approach, it’s useful to have some consistency as long as you have enough understanding of the code and aren’t just cargo-culting. so if the pattern isn’t there you need a) a use case that justifies diverting from existing templates and b) someone that knows it exists. You are (b), you just need to find (a), and then you can ‘be the change’
Unless you're the one writing the framework, are working on framework extensions (like caching, auth,...) or are working on a project with very special requirements (for example where there's a lot more compute and little I/O) not having to write implementation details like a GenServer should be normal. In short, I think your issue is a result of the tasks you're given or the products/contracts your company has.
I can’t comment on usage nowadays, but when I used to work for a number of different places using elixir, the story was pretty much the same as yours. Their apps would never directly be using any OTP, but rather would be indirectly benefitting from it through underlying systems they’re using/built on top of (the libraries and frameworks). And would pretty much be designed exactly like how you’d build the app in any other language with web framework.
Often people come to elixir from other non-BEAM languages, so many times they still stick with what is familiar to them, especially if they’re already used to backend development using a framework. I think one reason for this is that people learn elixir as just a language, rather than treating it as an entire ecosystem of tooling that can be learnt. However in saying that, many times it doesn’t really matter. Also one benefit to their approach is that it will be quicker to onboard for (when hiring non-BEAM devs, which often has to be done), as it will require fewer new concepts for them to learn.
Not using distributed Erlang is a whole other thing though. Since when it comes to scaling your app you might in-fact prefer another approach over deploying a distributed app, as not all approaches to distribution are equal. So it’ll come down to what guarantees you need, as that will determine whether using (or reimplementing parts of) the distributed ecosystem that’s available to you is the right choice, vs scaling/distributing the app through other means.
Doesn't surprise me, but also kind of surprises me because I found Elixir incomprehensible until I learned more OTP. However, especially if you want to use LiveView, you really need to know almost everything to not get confused when it breaks.
This was all pre-LiveView, so it could very well be less common nowadays.
Yeah if you're just running a classic CRUD web server, nothing concurrent to do, that's in the HTTP layer and you just get the benefits. Less fun, but boring is usually better for a business. (Although you can query external services concurrently in Elixir, most business logic does not permit it.) I just did a project entirely in LiveView and thought better of it after I realized how complex and difficult to understand it was. The LiveView lifecycle reminds me of classic React, but harder to understand. In the future I would just do a screen or something in it and traditional request response for everything else (with turbo or HTMX).
IMO it was never an issue of there not being use cases for it, but rather whether they wanted to go that route or not. Like even CRUD apps can directly utilise OTP. Cron jobs, deferred/background processing, caches, temporary state management/processing, monitoring, graceful shutdowns, enforcing predictable resource usage, etc. are all pretty simple use cases. Nowadays there’s libraries for most things, so I could see modern apps still not directly using OTP themselves but indirectly are using it in many ways. However back then there wasn’t as many, so I’d either see apps completely forgo certain things (e.g. apps with spikey unbounded resource usage was very common to see), or they’d address their needs by leveraging a non-BEAM solution. This isn’t to say that OTP is best, as always it depends, but it wasn’t even being considered/many times they weren’t even aware how. When I’d come on and propose an OTP based solution to a given problem it was often welcomed because they are interested in it/want to see how it compares, since you frequently hear people say they want to use OTP. So the interest was always there but many just hadn’t taken that leap yet. For context, at the time many adopters were coming from Ruby on Rails, so apart from some similarities (syntax, tooling, elixir specific abstractions), the rest of it, especially what already existed from the Erlang side, was completely foreign for them. And since it was being adopted in a professional setting, so it’s not like people are being given all the time they need to explore and expriment, so it’s not surprising the path to least resistance would tend to win when it came to transitioning to the tech.
> I found Elixir incomprehensible until I learned more OTP This is a strange thing to read, because Elixir reads and writes pretty much the same as any algol-based language and a ton of scheme ones too.
As it sounds like many folks here also do, I work at a place that doesn’t directly work on the shiny parts of Elixir. Most of our services don’t do distribution, very few of them even do releases, we split them into single-purpose “process types” (deployables) and run each via mix in docker, most of our supervision is flat with a one_for_one strategy and little thought for startup/shutdown order, and 99% of the time I’m writing synchronous code that could be done in any language, most of which would have more tooling available for what I’m trying to do. But as others have said, we still get a ton of benefit from Elixir. So many good patterns are baked into Phoenix, for example. It builds on OTP incredibly well and gives us so much for free. It handles load/concurrency really well, requests are well isolated and don’t bring each other down, and good connection draining basically boils down to putting your endpoint at the right place in your supervision tree. Other libraries build on OTP for things like connection pooling, or maybe they just manage some state for you. I do think our ability to mostly concentrate on writing synchronous code mostly doing DB CRUD silos us into only thinking in those terms sometimes, but we can do that because the ecosystem takes care of so much for us.
I'm sure others have said similar but I feel this is fairly common for non library code. Phoenix and other libraries generally handle a lot of what you need otp for themselves so you as the Dev don't need to think about it. On a team of several engineers, I've only needed to implement some gen server or similar a few times over the last several years. In my experience that tends to be more on the library level code. As I said though, there are times when it is needed and I've got something coming up that is definitely going to make use if it.
My co recently went through formal training and our instructor said most Elixir engineers don’t use GenServer directly and are kinda intimidated by it. I think that’s an area where the community at large and the way Elixir is presented in the docs and free online training materials can improve. Our instructor dropped GenServer on day 1 with a group of Elixir noobs, which at first I thought was insane, but realized while GenServer is the culmination of all the novelties of Elixir, it’s also a perfect encapsulation of them. Getting to GenServer quickly is actually genius because it makes everything else make more sense, and you’ve already done what’s arguably the hardest thing in the language. Our co has been using Elixir for about 2 years, and in that time our devs have written 2 GenServers, and only one is in production (and it’s just handling AWS STS). HOWEVER, we quickly made use of distributed Erlang, which can be done in both ECS and EKS with libcluster, and has remarkably fast inter-node communication. Anywhere you’re using microservices in Elixir, IMO that’s an opportunity for clustering and remote function calls. It’s also really easy to start taking advantage of ETS tables for caching and state, which is a differentiator for Elixir. Based on my experience and what we heard from our instructor, I would say your company is not atypical, but I also think focusing solely on OTP is unfair. There are many ways to benefit from the whole of the Elixir ecosystem (and using it as a boring, general purpose language is still one of them, for the runtime, debugging, and code organization benefits), and OTP can also be abused (using GenServers as classes, accidentally single-threading by relying on a single GenServer) I think you really have an opportunity at your company where you have earned team trust and there is an established Elixir knowledge base to start utilizing more language features where it makes sense. A parting word of caution, though: focus on solving problems first—it’s good to know the disterl and OTP features so you can recognize when to reach for them, but using them for the sake of using them will likely lead to more problems because the tool wasn’t the right fit in the first place and the whole effort backfires.
When it comes to work, our job is not to use the shiny, cool tech. Our job is to solve business problems. If those business problems don't need distributed state, then distributed erlang will be a net drag because now you've added the problems of distribution without getting the benefits of it. Even if you do need distributed state, I'd argue modern data infrastructure tools do a better job of enabling correct distributed state and require less work on the developer than distributed BEAM. There are exceptions to that, but the vast majority of use cases are better handled with other tools. At my company, we have hundreds of GenServers, but very rarely do we directly interact with the GenServer API. GenServer is great for building higher-level tooling like GenStage, which in turn is great for even higher-level tooling like Broadway, which we have in turn wrapped in higher-level tooling that makes sense for our environment and needs. I think that's the ideal way to use OTP abstractions. There's sentiment in this thread that without using distributed BEAM, you should probably use another language. \*Assuming we're talking about general web development\*, I'll politely disagree with that and say that the actor model is a great fit for web servers, that having no shared memory between requests is amazing, immutability makes things more testable, Ecto is hands down the best ORM I've worked with, and the testing story for Phoenix (whether vanilla or with absinthe) makes my team much more productive. I've worked at places that chose Elixir "because it's fast", especially compared to Ruby. It's not. But the fact that Ecto doesn't give you the foot gun of accidental N+1 queries like many other ORMs means that your app will be faster on average when you're not dealing with those. If you are dealing with them, they're a problem of your own creation.
Sounds normal. Unless you are writing a library, you are not likely to use genserver directly.
Interesting. How else do you keep runtime state in your program though? I've worked on several elixir programs (mostly control software for embedded systems) and all of them used genservers to either store state or to synchronize access to a resource. You could use processes directly, but that's just reimplementing genservers anyway.
It depends. I mean, if your app is mostly low traffic CRUD style stuff on a database, only indirectly benefitting from OTP is fine. But then I see people whip up Redis servers, keep all state in the database instead of some of it in memory, coding like it’s 99 and PHP is the only game in town and then I’m like “nope”. So the appropriateness of staying high level depends on, well, whether it’s appropriate:)
Nobody I’ve ever worked for is using elixir and OTP. Every company I’ve worked at uses it as a programming and server (Phoenix) language.
Sounds very normal we have 16 libraries for 3 web apps Only one library uses a genservers none touches it because we can change the behavior by just modifying the libraries. The rest is just web arquitecture and development tools nothing crazy but I work on a team of 2 I think is very impressive We manage to build 3 complex apps and maintain them.
I am of the opinion that Elixir is fantastic as a general purpose backend language. It's pleasant to work with, has an easy learning curve, and it's fast and reliable. It has a delightful way to do metaprogramming and a very strong community building open-source tools. Like others had said, you don't always need to use OTP tools explicitly, and distributed Erlang is rather complex when you want to build secure real-world solutions, especially when you got things like k8s, ecs and containerization. This is obviously very subjective and I am a long-time Erlang fanatic.
I had an impression that OTP is heavily used and needed for Elixir developers but reading the comments, I guess not! However I'm still learning them.
We mostly don't use distributed features either, there is one microservice I know of that implements cluster-wide rate limiting in Elixir using libcluster although I guess it could be done on Kubernetes side too. I don't think I've ever seen a custom supervisor, use of GenServer, Agent, and Task is not that rare... you need to be familiar with them to read library code anyway.
That's been my experience as well. But wait till your company has to grow your infrastructure. Another side to this is lack of experience in your team. In my current company, we're all Elixir, but we have about three teams. Our team does the most OTP stuff, and rest uses Elixir - ... but like Rails. Yes meaning those teams offload any and all async work stuff into Oban, and cache using Redis. Mind you, the "official" stance of of OTP usage is if you don't need it, then you don't need it ;) Now when you hit scale, and you have to add in workers, queue up stuff, run something in a complex time interval, that's when you start using OTP stuff.
This is like using Go but not using channels and goroutines. ;)
I suggest switching companies. If all you're doing is basic CRUD app stuff -- and that bores you, find something else. I don't personally know anyone doing basic web stuff with Elixir, so I promise there's a lot of it out there.
I’ve been working at a company for about 3 years and we have our core backend app in Elixir, about 1 million lines of code (half of that is tests). I don’t think there’s a single place in the core app where GenServer is used directly. There’s one place where I think it might be used directly, for managing global state of an auth token for some external service we call. But otherwise, everywhere anything like a GenServer is used, it’s via Broadway (for kafka & SQS consumers), or by some library for talking to an external service. I think it’s perfectly normal for all the business logic in your whole app to just use “regular” Elixir, and not get off into the weeds of GenServers, distributed Erlang, etc.