What is proper workflow for insuring "transactional procedures" in case of exceptions
<p>In programming web applications, Django in particular, sometimes we have a set of actions that must all succeed or all fail (in order to insure a predictable state of some sort). Now obviously, when we are working with the database, we can use transactions.</p>

<p>But in some circumstances, these (all or nothing) constraints are needed outside of a database context</p>

<p>(e.g. If payment is a success, we must send the product activation code or else risk customer complaints, etc)</p>

<p>But lets say on some fateful day, the <code>send_code()</code> function just failed time and again due to some temporary network error (that lasted for 1+ hours)</p>

<p>Should I log the error, and manually fix the problem, e.g. send the mail manually  </p>

<p>Should I set up some kind of work queue, where when things fail, they just go back onto the end of the queue for future retry?</p>

<p>What if the logging/queueing systems also fail? (am I worrying too much at this point?)</p>

<p>We use microservices in our company and at least once a month, we have one of our microservices down for a while. We have <code>Transaction</code> model for the payment process and statuses for every step that go before we send a product to the user. If something goes wrong or one of the connected microservices is down, we mark it like <code>status=error</code> and save to the database. Then we use cron job to find and finish those processes. You need to try something for the beginning and if does not fit your needs, try something else.</p>

python django exception transactions