WP 6 -- Work Division and Scheduling

• Task1: Develop scheduler (ICL, XLab, Sean)
  • Milestone M2.5: Month 12 -- First, simple, version of the full scheduler integrated into portal (replaces "dummy" version inside prototype portal)
  • Deliverable D2.5: Month 12 -- code and related artifacts of the working scheduler
  • Milestone M2.6: Month 24 -- Subset of advanced scheduler features functioning and integrated into portal.
  • Deliverable D2.6: Month 24 -- code and related artifacts of the working second-stage scheduler
  • Milestone M2.7: Month 36 -- All advanced scheduler features working within the integrated system.
  • Deliverable D2.7: Month 36 -- code and related artifacts of the working final, advanced scheduler
• Task1: Develop scheduler (Leader ICL with contributions to integration by XLab and Sean)

Milestone M2.3: (24 Months after the start of the project) A working scheduler that includes some advanced features. Measured performance improvements over the simple prototype scheduler. Milestone M2.4: (36 Months after the start of the project) All advanced features of scheduler added and demonstrated working within the system, with measured performance improvements delivered to end-user applications, which are due to the dynamic monitoring and re-provisioning.

The scheduler will exploit a self-aware dynamic analysis approach which in an on-line manner and in real-time, will receive as input the state of all system resources, and calculate resource availability and expected execution times for tasks currently active within the portal and expected to be launched based on predictions derived from usage patterns. Scheduling decisions will combine detailed status information regarding expected response times, internal network delays, possible security risks, and possible reliability problems. The self-aware dynamic analysis will return a "short list" of the best instantaneous provisioning decisions, ranked according to performance, security, reliability, energy consumption and other relevant metrics. Based on the task to be provisioned, the scheduler will be able to decide on provisioning rapidly. The portal and collection of DSM runtime systems include monitoring of whether the performance objectives of the task are being met, which will re-trigger the scheduler to make a decision again if the observations indicate an unsatisfactory outcome, which will be based on the new system state as well as the overhead related to any changes in provisioning.

Task 2: Develop Model of system that speeds up the search for best work division and best scheduling choices. The model should be very light weight so that it can be used inside the runtime to make choices as a program executes. Employ the model inside a scheduler, which is plugged in to the CloudDSM system, and measure the improvement in work division and scheduling quality achievable with its use.

Task 3: Develop Theory of scheduling that establishes well founded mathematical basis for discussions of scheduling. Use theory to prove conclusions about particular scheduler approaches when applied to applications that have particular features. Apply the theory to drive the development of a scheduler that takes advantage of the additional information provided by the CloudDSM annotations. Plug the scheduler into the CloudDSM system and measure the improvement in scheduling choices achieved with its use.