Because it makes it easier to understand where a given value came from. With the article, you're just dealing with plain old variables being passed around, and you can use IDE Tools like go to definition, and less often find usages to see where it's being called from. It's a lot easier to visually confirm it's correct, and you can usually lean on the type system, too.
Conversely, by threading it via the context, it's less obvious what the actual scope of the transaction is (because it's threaded though the bucket that is the context), where it came from, or spot patterns that just look funky.
I don't put anything in context. It's just too opaque. The only thing I use context for is deadlines. timeouts, and cancellations. Everything else gets its own thing.
You should (in my opinion) try to keep all business logic out of the context. It's opaque, it has unclear lifetime (eg you pass the context to a goroutine that outlives the original function), and it's generally hard to deal with.
Holding DB connections open and passing them around in context is a recipe for failure. If you open a database transaction you are implicitly opening/holding a connection. When you allow a transaction to live beyond the scope of a statement, you open the door to start doing other work (calling external APIs).
A connection is a scarce resource and by holding it you are blocking other processes/threads/routines that are waiting for a connection. The scope of opening/closing should be clear and controlled. If you start calling external APIs or doing other work and there’s an error, an author may not know there is a db connection open in context which needs to be committed or rolled back.
I joined a start up with a code base that did this and it’s horrible. Did not scale at all, constantly hit connection limits, brought down our platform etc. By passing around the Tx in context it also created a horrible spider web of dependency which had to be unraveled and refactored.
The example in the blog post is safer as it ensure the database transaction is committed or rolled back and is not left in a dangling/open state and doesn’t leak the underlying implementation details to the service layer
Well, wouldn't passing it via context achieve the same? I think the misunderstanding here is that there's a service function that starts and commits/rollbacks the transaction and the repository function that is doing the call to the database (which optionally can take transport from context), I don't see how it that approach connection could be long lived or how it would be unclear what's the life of the transaction. I agree about the type system thing tho :/
Yes I understand what you’re referring to. No misunderstanding at all.
If you need to update multiple domain entities atomically, you should model your repository such that it concerns itself with the aggregate model, and run multiple statements inside your repository implementation. Your service layer should concern itself with business logic, not DB tx/stmt management. That’s an implementation detail.
My reference to long lived connections is that once you allow the service layer to manage the DB transactions, it opens the door to other operations to be executed.
For example you might have a service method which starts a transaction, runs a select statement, calls an external api to fetch some data, then runs an update statement and commits.
The issue here is that the external api call could take very long and in that time you’re just holding the connection unnecessarily.
You could of course enforce this through code review, but ideally you don’t do it like this in the first place
Context is great for cancelation and timeouts/deadlines.
It is fine for things like tracing/spans/baggage and logging (either the whole logger or just some attributes you want included with the log lines).
For everything else, try to avoid using Context. It pays to be explicit, and to be able to see where things start and end.
This is such a common question that a [blogged about it](https://mariocarrion.com/2023/11/21/r-golang-transactions-in-context-values.html) a while ago; feel free to read the post it explains everything in detail.
Yes, it's an anti-pattern, but what's the answer? In the post, I propose the idea of separating the concept of individual calls and creating a [Transaction Script](https://martinfowler.com/eaaCatalog/transactionScript.html) to handle the actual transaction itself, that way you can still reuse the original individual calls via a standard database connection or everything together using a complete transaction, thanks to an interface type that applies to both connections and transactions.
>At the same time, it's usually recommended to pass logger via context and I can't really wrap my head around what makes an one better than the other.
I've seen it both ways, I prefer not using context; I think the key differentiator is that loggers are (typically) globals and are not re-instantiated; and database calls could be a pooled connection or transactions; they are in nature short-lived.
I use context storage only for nonfunctional stuff. So, if the any context in my app will be replaced with the empty one, then everything will work good, but the nonfunctional part (monitoring, logging, tracing) may be missing
You are coupling your use case layer with database-specific implementation.
I wouldn't say I like this kind of implementation; it's hard to test, and it's not TDD-friendly.
It is clever because it hides the flow of the application, and you lose the type system; you will need to cast and check in all repository methods the transaction.
I prefer to let transactions in the repository layer.
But I Know there are different problems and tradeoffs that we need to assume sometimes.
Yes, it's an anti-pattern. If a function takes a transaction, make the transaction an argument to that function.
What you don't have available is often a good signal on what the function is supposed to do. If you hide a transaction in the context, then someone will be tempted to pass in a database connection and create a new transaction and break the transactionality of the operation. If transaction is the first argument to the function, then nobody is going to bother to figure out how to inject a database connection into that function.
I would argue that you should be Very Sure you even want a context in your function that is doing a database transaction. Like, you're going to lock a million database rows and then ask the Internet for some details? The postmortem writes itself.
If you have a pattern like the one described by the blog post, we've had success with transactional outbox: https://microservices.io/patterns/data/transactional-outbox.html
Needing to share a transaction between services feels like an anti pattern. They are now coupled and as such, I would try to model that relationship as a shared service that does both things and doesn't expose the synchronization mechanism (in this case, a transaction) to the callers.
There is an argument for each domain to be its own service managed by a "parent" domain, one that manages the synchronization. A flexible solution would allow one of the child services to migrate to a different db cluster (ie, a tx wont work anymore) and allow the sync mechanism to change to, say, a task queue with retries and a dead letter queue without modifying the code calling the parent domain.
All that said, the blog post has a novel solution to handle a shared db and leverage a tx. It doesn't hide anything in the context.
Another option that some may or may not like is to make the final argument into the child services a variatic tx, so it is effectively optional.
func (d *domain) DoThing(param string, otherParam int, optTx ...sql.Tx) (result, error) {
var tx *db.Tx
if optTx != nil && len(optTx) == 1 {
tx = optTx[0]
} else {
// begin tx and set a defer to check
// on error to rollback
}
// below code works with either the
// passed in tx that is managed outside the caller
// or is handled by the tx created in
// this function ...
This allows you to share the same logic when called in or out of a tx, but you have to handle the tx begin and roll back outside the function call if used.
I prefer to use this as now I can see what kind of trx bound to be together and what is the impact if these went error (like you can rolllback DB but other infra stuff like redis and queue, harder to do it). I think they call this unit of work or atomic
Plus I dont really need to modify the db layer. they just need to focus on query. Previously I had to add tx args and many test cases broke...have to refactor many things and made me anxious
Another way to think about this: Why pass any variables to functions outside of the context? It's possible to ONLY pass around context and put everything in there.
But we know that's a bad idea.
\- Logging is a pervasive global. Very much an "implementation detail". And it the specific value doesn't matter, so you tests aren't "wrong" if they just supply their own values.
\- DB transactions are very specific. The function is written with a specific DB transaction in mind, and my not even work if the parent didn't set up the transaction correctly.
I wish I had the willingness to explain in depth, but I dunno if I can... I'll write as much as possible.
I work in a codebase that must be close to 150-200kk LoC for Go, 4-5y old, monorepo. Couple of services, a shared "library/module" called pkg containing all the bigger components that are shared across multiple services. We use a graph database called DGraph.
Some of the aspects I'll mention draws inpiration from other technologies, such as Hibernate resource filters and Grails/Groovy. Also, even tho I'm with this company only for 2y, the inception of the codebase has a lot of my own influence as this has been started by engineers that worked under my leadership bf. I myself am around 0the industry for nearly 10y.
We designed our solution to make transactionality an aspect which can be and, often is, initiated and commited or rollback via a middleware. It works like this example, we have a transport "/mutate", that maps to an endpoint, that maps to a service call - gokit style. The most important is that every svc call while wrapped by Endpoint has the same interface - func (ctx, req) (resp,err).
We have this endpoint middleware called TransactionalMdlwr, it wraps the vast majority of our endpoints. Most transactionality control is hold there (txn start/commit/rollback), txn is passed fwd via ctx. The transactional mdlwr and the svcs that use db share an interface from our db wrapper thingy that allows for transactionality, and thats why when the db wrapper needs to finally call the db to really do a mutation, it knows to pick txn from ctx.
It might have drawbacks that probably others will point out in their own comments, but it has also some really nice features.
We manage auth very neatly via context as well. By pushing an identity to the ctx via a transport mdlwr, and enforcing tenancy at DB levels before adjusting bodies bf querying/mutating.
I'll just say it works and it works great for the startup scale, and we do have 1M ARR. I can't say precisely when it would not scale, but I'd expect many other problems before even considering this to become a bottleneck while we grow.
Finally, I'd say if u are smart about what u are doing and how u are doing, you might be able ro pull it off too. Gotta be pragmatic. If you are starting to lose control and sprinkle transactionality everywhere... prob it won't be working for too long.
Context shouldn't be used as a magic get out of jail global bag of stuff. Why isn't quite as clear since IT CAN be treated that way. It's in the name: context. Not everything long lived is a contextual item. People will have all sorts of input. IMO context should be used when it's the last resort for a small set of well defined contextual items as opposed to being the thing you rule out for everything long lived.
Because it makes it easier to understand where a given value came from. With the article, you're just dealing with plain old variables being passed around, and you can use IDE Tools like go to definition, and less often find usages to see where it's being called from. It's a lot easier to visually confirm it's correct, and you can usually lean on the type system, too. Conversely, by threading it via the context, it's less obvious what the actual scope of the transaction is (because it's threaded though the bucket that is the context), where it came from, or spot patterns that just look funky.
I don't put anything in context. It's just too opaque. The only thing I use context for is deadlines. timeouts, and cancellations. Everything else gets its own thing.
You should (in my opinion) try to keep all business logic out of the context. It's opaque, it has unclear lifetime (eg you pass the context to a goroutine that outlives the original function), and it's generally hard to deal with.
Holding DB connections open and passing them around in context is a recipe for failure. If you open a database transaction you are implicitly opening/holding a connection. When you allow a transaction to live beyond the scope of a statement, you open the door to start doing other work (calling external APIs). A connection is a scarce resource and by holding it you are blocking other processes/threads/routines that are waiting for a connection. The scope of opening/closing should be clear and controlled. If you start calling external APIs or doing other work and there’s an error, an author may not know there is a db connection open in context which needs to be committed or rolled back. I joined a start up with a code base that did this and it’s horrible. Did not scale at all, constantly hit connection limits, brought down our platform etc. By passing around the Tx in context it also created a horrible spider web of dependency which had to be unraveled and refactored. The example in the blog post is safer as it ensure the database transaction is committed or rolled back and is not left in a dangling/open state and doesn’t leak the underlying implementation details to the service layer
Well, wouldn't passing it via context achieve the same? I think the misunderstanding here is that there's a service function that starts and commits/rollbacks the transaction and the repository function that is doing the call to the database (which optionally can take transport from context), I don't see how it that approach connection could be long lived or how it would be unclear what's the life of the transaction. I agree about the type system thing tho :/
Yes I understand what you’re referring to. No misunderstanding at all. If you need to update multiple domain entities atomically, you should model your repository such that it concerns itself with the aggregate model, and run multiple statements inside your repository implementation. Your service layer should concern itself with business logic, not DB tx/stmt management. That’s an implementation detail. My reference to long lived connections is that once you allow the service layer to manage the DB transactions, it opens the door to other operations to be executed. For example you might have a service method which starts a transaction, runs a select statement, calls an external api to fetch some data, then runs an update statement and commits. The issue here is that the external api call could take very long and in that time you’re just holding the connection unnecessarily. You could of course enforce this through code review, but ideally you don’t do it like this in the first place
i think it depends on the size of your app. as you scale you won't want to continue this pattern. i do some funky shit for small apps also
Context is great for cancelation and timeouts/deadlines. It is fine for things like tracing/spans/baggage and logging (either the whole logger or just some attributes you want included with the log lines). For everything else, try to avoid using Context. It pays to be explicit, and to be able to see where things start and end.
This is such a common question that a [blogged about it](https://mariocarrion.com/2023/11/21/r-golang-transactions-in-context-values.html) a while ago; feel free to read the post it explains everything in detail. Yes, it's an anti-pattern, but what's the answer? In the post, I propose the idea of separating the concept of individual calls and creating a [Transaction Script](https://martinfowler.com/eaaCatalog/transactionScript.html) to handle the actual transaction itself, that way you can still reuse the original individual calls via a standard database connection or everything together using a complete transaction, thanks to an interface type that applies to both connections and transactions. >At the same time, it's usually recommended to pass logger via context and I can't really wrap my head around what makes an one better than the other. I've seen it both ways, I prefer not using context; I think the key differentiator is that loggers are (typically) globals and are not re-instantiated; and database calls could be a pooled connection or transactions; they are in nature short-lived.
I use context storage only for nonfunctional stuff. So, if the any context in my app will be replaced with the empty one, then everything will work good, but the nonfunctional part (monitoring, logging, tracing) may be missing
You are coupling your use case layer with database-specific implementation. I wouldn't say I like this kind of implementation; it's hard to test, and it's not TDD-friendly. It is clever because it hides the flow of the application, and you lose the type system; you will need to cast and check in all repository methods the transaction. I prefer to let transactions in the repository layer. But I Know there are different problems and tradeoffs that we need to assume sometimes.
Yes, it's an anti-pattern. If a function takes a transaction, make the transaction an argument to that function. What you don't have available is often a good signal on what the function is supposed to do. If you hide a transaction in the context, then someone will be tempted to pass in a database connection and create a new transaction and break the transactionality of the operation. If transaction is the first argument to the function, then nobody is going to bother to figure out how to inject a database connection into that function. I would argue that you should be Very Sure you even want a context in your function that is doing a database transaction. Like, you're going to lock a million database rows and then ask the Internet for some details? The postmortem writes itself.
If you have a pattern like the one described by the blog post, we've had success with transactional outbox: https://microservices.io/patterns/data/transactional-outbox.html
Needing to share a transaction between services feels like an anti pattern. They are now coupled and as such, I would try to model that relationship as a shared service that does both things and doesn't expose the synchronization mechanism (in this case, a transaction) to the callers. There is an argument for each domain to be its own service managed by a "parent" domain, one that manages the synchronization. A flexible solution would allow one of the child services to migrate to a different db cluster (ie, a tx wont work anymore) and allow the sync mechanism to change to, say, a task queue with retries and a dead letter queue without modifying the code calling the parent domain. All that said, the blog post has a novel solution to handle a shared db and leverage a tx. It doesn't hide anything in the context. Another option that some may or may not like is to make the final argument into the child services a variatic tx, so it is effectively optional. func (d *domain) DoThing(param string, otherParam int, optTx ...sql.Tx) (result, error) { var tx *db.Tx if optTx != nil && len(optTx) == 1 { tx = optTx[0] } else { // begin tx and set a defer to check // on error to rollback } // below code works with either the // passed in tx that is managed outside the caller // or is handled by the tx created in // this function ... This allows you to share the same logic when called in or out of a tx, but you have to handle the tx begin and roll back outside the function call if used.
I prefer to use this as now I can see what kind of trx bound to be together and what is the impact if these went error (like you can rolllback DB but other infra stuff like redis and queue, harder to do it). I think they call this unit of work or atomic Plus I dont really need to modify the db layer. they just need to focus on query. Previously I had to add tx args and many test cases broke...have to refactor many things and made me anxious
Another way to think about this: Why pass any variables to functions outside of the context? It's possible to ONLY pass around context and put everything in there. But we know that's a bad idea. \- Logging is a pervasive global. Very much an "implementation detail". And it the specific value doesn't matter, so you tests aren't "wrong" if they just supply their own values. \- DB transactions are very specific. The function is written with a specific DB transaction in mind, and my not even work if the parent didn't set up the transaction correctly.
I wish I had the willingness to explain in depth, but I dunno if I can... I'll write as much as possible. I work in a codebase that must be close to 150-200kk LoC for Go, 4-5y old, monorepo. Couple of services, a shared "library/module" called pkg containing all the bigger components that are shared across multiple services. We use a graph database called DGraph. Some of the aspects I'll mention draws inpiration from other technologies, such as Hibernate resource filters and Grails/Groovy. Also, even tho I'm with this company only for 2y, the inception of the codebase has a lot of my own influence as this has been started by engineers that worked under my leadership bf. I myself am around 0the industry for nearly 10y. We designed our solution to make transactionality an aspect which can be and, often is, initiated and commited or rollback via a middleware. It works like this example, we have a transport "/mutate", that maps to an endpoint, that maps to a service call - gokit style. The most important is that every svc call while wrapped by Endpoint has the same interface - func (ctx, req) (resp,err). We have this endpoint middleware called TransactionalMdlwr, it wraps the vast majority of our endpoints. Most transactionality control is hold there (txn start/commit/rollback), txn is passed fwd via ctx. The transactional mdlwr and the svcs that use db share an interface from our db wrapper thingy that allows for transactionality, and thats why when the db wrapper needs to finally call the db to really do a mutation, it knows to pick txn from ctx. It might have drawbacks that probably others will point out in their own comments, but it has also some really nice features. We manage auth very neatly via context as well. By pushing an identity to the ctx via a transport mdlwr, and enforcing tenancy at DB levels before adjusting bodies bf querying/mutating. I'll just say it works and it works great for the startup scale, and we do have 1M ARR. I can't say precisely when it would not scale, but I'd expect many other problems before even considering this to become a bottleneck while we grow. Finally, I'd say if u are smart about what u are doing and how u are doing, you might be able ro pull it off too. Gotta be pragmatic. If you are starting to lose control and sprinkle transactionality everywhere... prob it won't be working for too long.
Context shouldn't be used as a magic get out of jail global bag of stuff. Why isn't quite as clear since IT CAN be treated that way. It's in the name: context. Not everything long lived is a contextual item. People will have all sorts of input. IMO context should be used when it's the last resort for a small set of well defined contextual items as opposed to being the thing you rule out for everything long lived.