Tools
Tools
Kumbhare, A.G., Simmhan, Y., Frincu, M. 
ORCID: 0000-0003-1034-8409 and Prasanna, V.K.,
2015.
Reactive resource provisioning heuristics for dynamic dataflows on cloud infrastructure.
IEEE Transactions on Cloud Computing, 3 (2), pp. 105-118.
ISSN 2168-7161
|
Text
1393281_Frincu.pdf - Post-print Download (2MB) | Preview |
Abstract
The need for low latency analysis over high-velocity data streams motivates the need for distributed continuous dataflow systems. Contemporary stream processing systems use simple techniques to scale on elastic cloud resources to handle variable data rates. However, application QoS is also impacted by variability in resource performance exhibited by clouds and hence necessitates autonomic methods of provisioning elastic resources to support such applications on cloud infrastructure. We develop the concept of “dynamic dataflows” which utilize alternate tasks as additional control over the dataflow's cost and QoS. Further, we formalize an optimization problem to represent deployment and runtime resource provisioning that allows us to balance the application's QoS, value, and the resource cost. We propose two greedy heuristics, centralized and sharded, based on the variable-sized bin packing algorithm and compare against a Genetic Algorithm (GA) based heuristic that gives a near-optimal solution. A large-scale simulation study, using the linear road benchmark and VM performance traces from the AWS public cloud, shows that while GA-based heuristic provides a better quality schedule, the greedy heuristics are more practical, and can intelligently utilize cloud elasticity to mitigate the effect of variability, both in input data rates and cloud resource performance, to meet the QoS of fast data applications.
| Item Type: | Journal article | ||||||
|---|---|---|---|---|---|---|---|
| Publication Title: | IEEE Transactions on Cloud Computing | ||||||
| Creators: | Kumbhare, A.G., Simmhan, Y., Frincu, M. and Prasanna, V.K. | ||||||
| Publisher: | Institute of Electrical and Electronics Engineers | ||||||
| Date: | 1 April 2015 | ||||||
| Volume: | 3 | ||||||
| Number: | 2 | ||||||
| ISSN: | 2168-7161 | ||||||
| Identifiers: |
|
||||||
| Divisions: | Schools > School of Science and Technology | ||||||
| Record created by: | Jonathan Gallacher | ||||||
| Date Added: | 09 Dec 2020 10:00 | ||||||
| Last Modified: | 31 May 2021 15:10 | ||||||
| URI: | https://irep.ntu.ac.uk/id/eprint/41814 |
Actions (login required)
 |
Edit View |
Views
Views per month over past year
Downloads
Downloads per month over past year
