Massively Parallel Serverless Computing in AWS

By Yogesh Chinta

Why Cloud?

  1. Do our workloads vary in storage requirements? No, our workload does not differ in storage requirements but is driven by desired performance and reliability for transferring, reading, and writing the data.
  2. Do our workloads vary in compute requirements? Yes, Our workloads do vary in compute requirements. Some event footprints take a few seconds, and others take a few hours, hence the need to choose an appropriate memory-to-compute ratio for the containers. You could optimize and find the sweet spot ratio (2GB RAM:1 vCPU) for the entire workload or customize the ratio ([1GB RAM:1 vCPU], [2GB RAM:1 vCPU]) for every workload.
  3. Do our workloads require high network bandwidth/latency? No, Because our workloads do not typically interact with each other, the workloads' feasibility or performance is not sensitive to the bandwidth and latency capabilities of the network between containers. Therefore, clustered placement groups are not necessary for our case because they weaken the resiliency without providing a performance gain.