Optimizing Heavy Web Service

Mateusz Kubuszok > Scalac

Agenda

when optimize
optimizing queries
caching requests
keeping things simple

Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.

-- Donald Knuth

How can we tell if optimization is needed and where?

Measure!

  trait Service[Request, Response] {

    def apply(request: Request): Future[Response]
  }

  object Service {

    def apply[Request, Response](body: Request => Future[Response]) =
      new Service[Request, Response] {

        override def apply(request: Request) = body(request)
      }
  }

def monitored[Request, Response](name: String)
                                (service: Service[Request, Response]) =
  new Service[Request, Response] {

    override def apply(request: Request) = {
      NewRelic.incrementCounter(name)
      val start = System.currentTimeMillis
      val response = service(request)
      response onComplete { case _ =>
        val time = System.currentTimeMillis - start
        NewRelic.recordResponseTimeMetric(name, time)
      }
      response onError { case ex => NewRelic.noticeError(ex) }
      response
    }
  }

val getUsersBillings: Service[UsersBillingsReq, UsersBillingsRes] =
  monitored("UserServices.getUserBillings") {
    Service { request =>
      Future {
        val formattedBillings = for {
          user     <- userRepository.fetchUsers(request.userIds)
          contract <- user.contracts
          billings <- contract.billings
        } yield formatBilling(user, contract, billing)

        UsersBillingsRes(formattedBillings)
      }
    }
  }

Queries to the database

n+1 query problem
recalculating things all over again

n+1 query

often consequence of using the Active Record pattern
usually not that much of an issue when the amount of requests is low, operations are simple and DB latency is low (e.g. small blogs, very simple CRUDs)
kills performance when service is heavily used, requires data from a long chain of relations and latency is non-negligible

// avg. user has 3 contacts
// avg. contract has 2 billings

val formattedBillings = for {
  user     <- userRepository.fetchUsers(request.userIds)
              // 1 DB query
  contract <- user.contracts
              // next 3 DB queries on avg.
  billing  <- contracts.billings
              // next 3*2=6 DB queries on avg.
} yield formatBilling(user, contract, billing)
        // total 10 queries on avg.

// avg. user has 3 contacts
// avg. contract has 2 billings

val users     = userRepository.fetchUsers(request.userIds)
                  .map(user => (user.id, user)).toMap
val contracts = contractRepository.findForUserIds(users.keys)
                  .map(contract => (contract.id, contract)).toMap
val billings  = billingRepository.findByContractIds(contracts.keys)
                // 3 queries in total

val formattedBillings = for {
  billing  <- billings
  contract <- contract.get(billing.contractId).toSeq
  user     <- users.get(contract.userId).toSeq
} yield formatBilling(user, contract, billing)

Caching DB requests

Result we'd like to cache

Future {
  val users     = userRepository.fetchUsers(request.userIds)
                    .map(user => (user.id, user)).toMap
  val contracts = contractRepository.findForUserIds(users.keys)
                    .map(contract => (contract.id, contract)).toMap
  val billings  = billingRepository.findByContractIds(contracts.keys)

  val formattedBillings = for {
    billing  <- billings
    contract <- contract.get(billing.contractId).toSeq
    user     <- users.get(contract.userId).toSeq
  } yield formatBilling(user, contract, billing)

  UsersBillingsRes(formattedBillings)
}

How could be acheive that with Redis?

{
  "EntityKey(user, 1)" : {
    "user-billings-1,2,5": "",
    ...
  },
  "EntityKey(user, 2)" : {
    "user-billings-1,2,5": "",
    ...
  },
  ...
  "user-billings-1,2,5": [serialized value],
  ...
}

case class EntityKey(type: String, id: String)

trait CacheContext[T] {

  def entityKeys(value: T): Seq[EntityKey] // entities result depends on
  def serializer: Serializer[T]            // T -> String
  def deserializer: Deserializer[T]        // String -> T
}

trait CacheHandler {
  def getOrPut[T](valueKey: String, ttl: Duration)
                 (valueF => Future[T]))
                 (implicit cacheContext: CacheContext[T],
                           executionContext: ExecutionContext): Future[T]
  def invalidate(entityKeys: EntityKey*)
}

Obtaining values from Redis

def get[T](key: String)
          (implicit cacheContext: CacheContext[T],
                    executionContext: ExecutionContext): Future[T] =
  redisClient.mget(key) map { gets =>
    gets.headOption map (_.toArray) map cacheContext.deserializer
  }

Storing values into Redis

def put[T](valueKey: String, value: T, ttl: Duration)
          (implicit cacheContext: CacheContext[T],
                    executionContext: ExecutionContext): Future[Unit] = {
  val entityKeys = cacheContext.entityKeys(value)
  val bytes = cacheContext.serializer(value).bytes
  val transaction = redisClient.multi()
  transaction.setex(valueKey, ttl.toSeconds, bytes)
  entityKeys map (_.toString) map { entityKey =>
    transaction.hmset(entityKey, Map(valueKey -> Array[Bytes]()))
    transaction.expire(entityKey, maxTtl.toSeconds)
  }
  transaction.exec() map (())
}

def getOrPut[T](valueKey: String, ttl: Duration)
               (valueF: => Future[T]))
               (implicit cacheContext: CacheContext[T],
                         executionContext: ExecutionContext): Future[T] =
  get[T](valueKey) flatMap { optionValue =>
    optionValue map Future.successful getOrElse {
      for {
        value <- valueF
        _     <- put[T](valueKey, value, ttl)
      } yield value
    }
  }

Invalidating values

val transaction = redisClient.multi()
val keys = entityKeys map (_.toString) map { key =>
  key -> transaction.hgetall(key) }
for {
  _ <- transaction.exec()
  invTransaction = redisClient.multi()
  allInvalidatedKeys <- Future.sequence(keys map { case (key, keyMapF) =>
    keyMapF map { keyMap =>
      val invKeys = keyMap.keys
      invTransaction.hdel(key, invKeys:_*)
      invKeys
    }
  }) map (_.flatten)
  _ = invTransaction.del(allInvalidatedKeys:_*)
  _ <- invTransaction.exec()
} yield ()

implicit val userBillingsContext = new CacheContext[UsersBillingsRes] {

  def entityKeys(value: UsersBillingsRes): Seq[EntityKey] = ...
  def serializer: Serializer[UsersBillingsRes]            = ...
  def deserializer: Deserializer[UsersBillingsRes]        = ...
}

cacheHandler.getOrPut[UsersBillingsRes](
    s"user-billings-${request.userIds.mkString}", 10 minutes) {
  Future {
    // ...

    UsersBillingsRes(formattedBillings)
  }
}

Caching API requests

trait UsersControllerImpl extends UsersController {

  def getBillingsForCurrentUserAndContractor(userId: Long) =
      authenticatedRequest { request =>
    val observedEntityId   = userId
    val observedEntityType = "user"
    val currentUserId = currentUser.id

    for {
      result <- userServices.getContractBillingsForPair(
          ContractBillingsForPairReuqest(userId, currentUserId))
    } yield {
      // create JSON from user
    }
  }
}

Lets improve CacheHandler a little

def getOrPut[T](valueKey: String, ttl: Duration, request: Request)
               (valueF => Future[T]))
               (implicit cacheContext: RequestCacheContext[T],
                         executionContext: ExecutionContext): Future[T] = {
  val uri    = request.uri
  val header = request.headers.get(Headers.Authentication)
                              .getOrElse("")
  val params = request.params.map { case (name, value) =>
    name + ":" + value.sorted.toString
  }.toSeq.sorted
  val requestSuffix = s"$uri-$header-${params.mkString}"

  getOrPut[T](s"$value-$requestSuffix", ttl)(valueF)
}

def getBillingsForCurrentUserAndContractor(userId: Long) =
    authenticatedRequest { request =>
  val observedEntityId = userId
  val currentUserId    = currentUser.id

  cacheHandler.getOrPut[BillingJsonResult](
      "api-billings-for-$userId", 10 minutes, request) {
    for {
      result <- userServices.getContractBillingsForPair(
          ContractBillingsForPairReuqest(userId, currentUserId))
    } yield {
      // create JSON from user
    }
  }
}

Removal of redundant queries

val users     = userRepository.fetchUsers(request.userIds)
                  .map(user => (user.id, user)).toMap
val contracts = contractRepository.findForUserIds(users.keys)
                  .map(contract => (contract.id, contract)).toMap
val billings  = billingRepository.findByContractIds(contracts.keys)

val formattedBillings = for {
  billing  <- billings
  contract <- contract.get(billing.contractId).toSeq
  user     <- users.get(contract.userId).toSeq
} yield formatBilling(user, contract, billing)

UsersBillingsRes(formattedBillings)

[
  {
    "id": 5543,
    "date": "2015-10-23",
    ...
    "_links": ...,
    "_embedded": {
      "user": {
        "id": 1,
        "name": "John",
        "surname": "Smith",
        ...
      },
      "contract": {
        "id": 646,
        ...
      },
      "billings": {
        "id": 766,
        ...
      }
    }
  },
  ...
]

Was it all really needed?

No!

We actually needed:

Billing
User ID
Contract ID

// UserId -> Billings
val billingsByUsers: Map[Long, Seq[Billing]] =
    billingRepository.findByUserIds(request.userIds)

val formattedBillings = for {
  (userId, billings) <- billingsByUsers
  billing            <- billings
  contractId        = billing.contractId
} yield formatBilling(userId, contractId, billing)

UsersBillingsResponse(formattedBillings)

[
  {
    "id": 5543,
    "date": "2015-10-23",
    "userId": 1,
    "contractId": 646,
    ...
  },
  ...
]

Optimizing Heavy Web Service – Mateusz Kubuszok > Scalac – Caching API requests

MateuszKubuszok

Optimizing Heavy Web Service – Mateusz Kubuszok > Scalac – Caching API requests

0 1

OptimizingHeavyWebServicePresentation

Optimizing Heavy Web Service

Mateusz Kubuszok > Scalac

Agenda

How can we tell if optimization is needed and where?

Measure!

Queries to the database

n+1 query

Caching DB requests

Result we'd like to cache

How could be acheive that with Redis?

Obtaining values from Redis

Storing values into Redis

Invalidating values

Caching API requests

Lets improve CacheHandler a little

Removal of redundant queries

Was it all really needed?

Summary

Questions?

Thank you!

Optimizing Heavy Web Service – Mateusz Kubuszok > Scalac – Caching API requests

MateuszKubuszok

Optimizing Heavy Web Service – Mateusz Kubuszok > Scalac – Caching API requests

0 1 (function() { var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true; po.src = 'https://apis.google.com/js/platform.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s); })();

OptimizingHeavyWebServicePresentation

Optimizing Heavy Web Service

Mateusz Kubuszok > Scalac

Agenda

How can we tell if optimization is needed and where?

Measure!

Queries to the database

n+1 query

Caching DB requests

Result we'd like to cache

How could be acheive that with Redis?

Obtaining values from Redis

Storing values into Redis

Invalidating values

Caching API requests

Lets improve CacheHandler a little

Removal of redundant queries

Was it all really needed?

Summary

Questions?

Thank you!

0 1