Transactions – yesterday & today – Example

View Github Repository Open presentation in a new window

Scooletz

See all presentation from Scooletz

Transactions – yesterday & today – Example

0 0

transactions-presentation

The transactions presentations

On Github Scooletz / transactions-presentation

Transactions

yesterday & today

Szymon Kulec

@Scooletz

http://blog.scooletz.com

Transactions

?

Example

extracted from MSDN magazine

Publish example

1. public void Publish(object o)

2. {

3. var factory = new ConnectionFactory();

4. var conn = factory.CreateConnection();

5. using ( var channel = conn.CreateModel())

6. {

7. // code to define the RabbitMQ channel

8. var json = SerializeToJSON(o);

9. var messageBodyBytes = Encoding.UTF8.GetBytes(json);

10. channel.BasicPublish("CustomerUpdate", "",

11. props, messageBodyBytes);

12. }

13. }

Save example

1. public bool Save(Customer c)

2. {

3. using ( var ctx = new MyDbCtx())

4. {

5. ctx.Customers.Add(c);

6. int response = ctx.SaveChanges();

7. if (response > 0) {

8. Publish(c);

9. return true;

10. }

11. return false;

12. }

13. }

Is this Save better?

1. public bool Save(Customer c)

2. {

3. using (var ctx = new MyDbCtx())

4. {

5. ctx.Customers.Add(c);

6. Publish(c);

7. return ctx.SaveChanges() > 0;

8. }

9. }

Simplified

1. public bool Save(Customer c)

2. {

3. DoDatabase(c);

4. DoQueue(c);

5. }

Simplified with some tweaks

1. public bool Save(Customer c)

2. {

3. PreDoDatabase(c);

4. PreDoQueue(c);

5. PostDoDatabase(c);

6. }

Example outcome

  • The example shows not only queue-db integration but service-db & others as well
  • Even with different ordering, we cannot ensure the transaction between queue & database
  • It's possible to get into an inconsistent state because of exceptions, processes being killed, etc.

Distributed transactions

Distributed transactions

  • A transaction across more than one system, database, queue, resource
  • Simulates good old-fashioned ACID transaction
  • Requires a transaction manager
  • May be used across the network

Distributed transactions - 2PC

  • 2PC - two phase commit
  • 1st prepares all the transactions across systems being used in the transaction
  • 2nd goes through the systems and ACKing the transaction

TransactionScope

  • .NET high-level abstraction over transaction
  • When used only with a single resource, it's just a wrap around transaction
  • Can propagate to a distributed transaction when another transactional resource used
  • The process of registering a transaction in a scope is called enlistement

TransactionScope applied

1. public bool Save(Customer c)

2. {

3. using( var ts = new TransactionScope())

4. {

5. PreDoDatabase(c);

6. PreDoQueue(c);

7. PostDoDatabase(c);

9. ts.Complete();

10. }

11. }

Distributed transactions are the solution !

NOT!

Disadvantages of distributed transactions

  • Not always supported (try RabbitMQ or REST services or Cassandra or MongoDb)
  • Latency - you need additional network hops
  • Longer calls - longer locks - worse throughput
  • Orphans - state is unknown (no, they won't become 007)

If not distributed

then what? :(

Going local

Simplified example once again

but with a ctx passed to Publish...

1. public bool Save(Customer c)

2. {

3. using (var ctx = new MyDbCtx())

4. {

5. ctx.Customers.Add(c);

6. Publish(c, ctx);

7. return ctx.SaveChanges() > 0;

8. }

9. }

With the Publish defined now as...

1. public void Publish(Customer c, MyDbCtx ctx)

2. {

3. var publishIntent = PublishIntent.CreateFrom (c);

4. ctx.PublishIntents.Add (publishIntent);

5. }

Going local

  • Transaction is local
  • A different modelling saved us from going distributed
  • A different modelling saved us from dropping system in a unknown state
  • ... but left with unpublished intents
  • Now all we should do is just published intents via queue and delete them in a tx
  • ... but this requires distributed transaction again :/
  • Is happiness really possible???

Delivery guarantees

Delivery guarantees

  • at-most-once
  • exactly-once
  • at-least-once

This method provides at-most-once delivery without getting into distributed problems

1. public void RealPublish(MyDbCtx ctx, RabbitMqSender sender)

2. {

3. var intent = ctx.PublishIntents.FirstOrDefault();

4. ctx.PublishIntents.Remove(intent);

5. ctx.SaveChanges();

6. sender.Send(intent);

7. }

This method provides at-least-once delivery without getting into distributed problems

1. public void RealPublish(MyDbCtx ctx, RabbitMqSender sender)

2. {

3. var intent = ctx.PublishIntents.FirstOrDefault();

4. sender.Send(intent);

5. ctx.PublishIntents.Remove(intent);

6. ctx.SaveChanges();

7. }

Delivery guarantees

  • It's easy to at-least/at-most once deliver the message
  • Can we make it exactly-once then?
  • YES!

Exactly-once delivery

  • We need to be ensured that receiver gets the message
  • ... hence at-least-once is required.
  • Under some conditions duplicates can appear on the receiver side
  • What can be done?

Exactly-once delivery with idempotent receiver

  • Mark every message with a unique id
  • On the receiver side use local transaction to process the message and store some state
  • Additonally, in the very same transaction store the message id
  • Process only messages previously not marked as processed

This method handles the message being sent and assumes that the message has Id property with a unique message id

1. public void ReceiveAtLeastOnce(OtherDbCtx ctx, Message msg)

2. {

3. using(var tx = ctx.Database.BeginTransaction())

4. {

5. var done = ctx.ProcessedIds.Find(msg.Id);

6. if (done != null)

7. return;

9. ProcessAndSaveState(ctx);

10. ctx.ProcessedIds.Add (new ProcessedId {Id = msg.Id});

12. tx.Commit();

13. }

14. }

Delivery guarantees summary

exactly-once = at-least-once + idempotent-receiver

Transactions in new databases

Transactions in new databases

  • So many trends, approaches - this will not cover everything
  • Many of them does not imply any transactions
  • It's common to have transactions only for the given key
  • There's no notion of a transaction locking rows for reads

Transactions in new databases - examples

Name Features Azure Table Storage partition key + row key, tx only across partition RavenDB transactions spans across documents, but indeces are not transactional! Riak key-value, values are replace atomically but can use multi-write as well EventStore transaction only for a given stream, idempotent receiver in the db MongoDb sometimes saves the data, if you're lucky you can query it later on, or not

The error that drained many bitcoin wallets (improper use of MongoDb)

1. mybalance = database.read("account-number")

2. newbalance = mybalance - amount

3. database.write("account-number", newbalance)

4. dispense_cash(amount) // or send bitcoins to customer

Modelling question - money transfer

  • The database can hold transaction only on one key
  • We are allowed to have a debit on an account
  • How would you model a transfer across accounts?
  • We can use EventStore for example

Modelling question - money transfer

  • There is no account but only its number
  • The atomic operation of a transfer is the thing that we save
  • An account is a sum of all transfers from-to

Summary

  • Be aware of the possible failures
  • Try to model towards local transactions
  • exactly-once = at-least-once + idempotent-receiver
  • Model is only a model: choose wisely its first class citizens
  • Use error-injection if you cannot see the whole picture yet. This will drive you.
  • Don't drop the ball. There's a big chance that you're dealing with money :P

TO DO

Questions?

Szymon Kulec

@Scooletz

http://blog.scooletz.com

Transactions yesterday & today Szymon Kulec @Scooletz http://blog.scooletz.com