CloudDSM.WP3 History

Hide minor edits - Show changes to markup - Cancel

April 01, 2014, at 08:31 PM by 178.11.216.240 -
Changed line 11 from:
  • Task1: Architecture of hierarchical DSM runtime system. Delivered runtime will be a federation. On each virtual machine, or in some cases physical machine, an individual runtime system will be in operation. Each of these individual runtime systems will interact with the others to form a collective, cooperative, overall runtime system, which presents a single address space abstraction. This task will define the architecture, interfaces, and protocols for this overall runtime system. Then for each class of hardware an individual task will implement the interfaces and protocols for that particular kind of hardware, as described below.
to:
  • Task2: Architecture of hierarchical DSM runtime system. Delivered runtime will be a federation. On each virtual machine, or in some cases physical machine, an individual runtime system will be in operation. Each of these individual runtime systems will interact with the others to form a collective, cooperative, overall runtime system, which presents a single address space abstraction. This task will define the architecture, interfaces, and protocols for this overall runtime system. Then for each class of hardware an individual task will implement the interfaces and protocols for that particular kind of hardware, as described below.
Changed line 22 from:
  • Task2: DSM runtime system on a shared memory x86 based Cloud server instance -- Sean with involvement by XLAB and Gigas
to:
  • Task3: DSM runtime system on a shared memory x86 based Cloud server instance -- Sean with involvement by XLAB and Gigas
Changed line 30 from:
  • Task3: DSM runtime system on a shared memory Power based Cloud server instance -- IBM plus Sean with involvement by XLAB and Gigas
to:
  • Task4: DSM runtime system on a shared memory Power based Cloud server instance -- IBM plus Sean with involvement by XLAB and Gigas
Changed line 39 from:
  • Task4: Integration testing. Deploy the various individual runtime systems on specific machines. Write test cases whose purpose is to expose performance issues and bugs. Execute the test cases, record the performance, report to the partners who developed the individual runtimes in tasks 2 through 6. Work with the partners to determine the reasons for particular performance anomalies, perhaps writing and running specific tests interactively. The partners for 2 through 6 will use the results to modify their individual runtime system implementations. -- Gigas leads -- partners for 2 through 6, plus XLAB are involved.
to:
  • Task5: Integration testing. Deploy the various individual runtime systems on specific machines. Write test cases whose purpose is to expose performance issues and bugs. Execute the test cases, record the performance, report to the partners who developed the individual runtimes in tasks 2 through 6. Work with the partners to determine the reasons for particular performance anomalies, perhaps writing and running specific tests interactively. The partners for 2 through 6 will use the results to modify their individual runtime system implementations. -- Gigas leads -- partners for 2 through 6, plus XLAB are involved.
April 01, 2014, at 09:15 AM by 178.11.216.240 -
Added lines 8-9:
  • Task1: participate in rapid prototyping process, together with members of WP 2 -- portal, WP 4 -- low level annotations, WP 5 -- transform tools, and WP 7 -- end application, to together get a minimum viable system up and running. This will create a high degree of communication between partners early in the project, to uncover hidden assumptions, develop shared ideas about approach, define clean interfaces and interactions between partners, workpackages, and deliverable pieces, and increase the smoothness and pace of progress for the rest of the project.
April 01, 2014, at 09:10 AM by 178.11.216.240 -
Added lines 4-7:

WP 2, the CloudDSM portal, will interact with the DSM runtime. The portal sends commands to the runtime to start work on behalf of applications. The runtime sends back updates to the portal regarding machine usage consumed by the commands. The portal also tells the Cloud stack when to suspend particular VMs, due to the DSM runtime inside the VM being in a busy-wait loop. This happens when a particular VM is waiting for results to be completed inside a different VM, and so is only in a busy-wait loop constantly polling for completion of the other VM.

WP 5, the transform tools, insert calls to the DSM API, as part of the transforms that they perform. In this way, the end developer, as in WP 7, uses a highly productive set of annotations/DSL, which are then transformed by WP 5 tools, into calls to WP 3's DSM runtime system.

April 01, 2014, at 09:02 AM by 178.11.216.240 -
Changed line 3 from:

The major goal of this workpackage is to develop the CloudDSM runtime system, which is responsible for implementing the low level behavior of the shared memory abstraction. It manages the shared data, and controls access to it. To accomplish this, it tracks the access rights granted to work units that reside on particular machines. A gi moving updates to shared memory between machines, and tracking which machine has ownership of which pieces of data

to:

The major goal of this workpackage is to develop the CloudDSM runtime system, which is responsible for implementing the low level behavior of the shared memory abstraction. It manages the shared data, and controls access to it. To accomplish this, it tracks the access rights granted to units of work (IE tasks or threads) that reside on particular machines. When work requests rights to a given region of shared addresses, the runtime controls the granting of those rights. It may suspend the work, wait for a different work unit to give up ownership, then move the updates to the machine where the requesting work unit resides. Thus, the DSM runtime manages granting and tracking of ownership of regions of addresses, moving updates to the address regions between machines, and manages suspend of work until movement completes.

April 01, 2014, at 08:53 AM by 178.11.216.240 -
Added lines 1-4:

WP 3 DSM Runtime System

The major goal of this workpackage is to develop the CloudDSM runtime system, which is responsible for implementing the low level behavior of the shared memory abstraction. It manages the shared data, and controls access to it. To accomplish this, it tracks the access rights granted to work units that reside on particular machines. A gi moving updates to shared memory between machines, and tracking which machine has ownership of which pieces of data

March 23, 2014, at 04:51 AM by 88.71.191.47 -
March 23, 2014, at 04:51 AM by 88.71.191.47 -
Deleted line 0:
Changed lines 12-13 from:
  • Task4: DSM runtime system on a shared memory x86 based Cloud server instance -- Sean with involvement by XLAB and Gigas
    • Milestone M2.: Month 12 --
to:
  • Task2: DSM runtime system on a shared memory x86 based Cloud server instance -- Sean with involvement by XLAB and Gigas
    • Milesttone M2.: Month 12 --
Changed line 20 from:
  • Task5: DSM runtime system on a shared memory Power based Cloud server instance -- IBM plus Sean with involvement by XLAB and Gigas
to:
  • Task3: DSM runtime system on a shared memory Power based Cloud server instance -- IBM plus Sean with involvement by XLAB and Gigas
Changed lines 28-36 from:
  • Task6: binary runtime specializer -- IBM (Q: any chance to get open source release? Or to apply to other ISA?). Use fat binary runtime optimization techniques to adjust the granularity and layout during execution. This is specific to IBM Power architecture, but it provides the blue print for implementing for other ISAs as well.
    • Milestone M2.: Month 12 --
    • Deliverable D2.: Month 12 --
    • Milestone M2.: Month 24 --
    • Deliverable D2.: Month 24 --
    • Milestone M2.: Month 36 --
    • Deliverable D2.: Month 36 --
  • Task7: Integration testing. Deploy the various individual runtime systems on specific machines. Write test cases whose purpose is to expose performance issues and bugs. Execute the test cases, record the performance, report to the partners who developed the individual runtimes in tasks 2 through 6. Work with the partners to determine the reasons for particular performance anomalies, perhaps writing and running specific tests interactively. The partners for 2 through 6 will use the results to modify their individual runtime system implementations. -- Gigas leads -- partners for 2 through 6, plus XLAB are involved.
to:
  • Task4: Integration testing. Deploy the various individual runtime systems on specific machines. Write test cases whose purpose is to expose performance issues and bugs. Execute the test cases, record the performance, report to the partners who developed the individual runtimes in tasks 2 through 6. Work with the partners to determine the reasons for particular performance anomalies, perhaps writing and running specific tests interactively. The partners for 2 through 6 will use the results to modify their individual runtime system implementations. -- Gigas leads -- partners for 2 through 6, plus XLAB are involved.
March 23, 2014, at 04:44 AM by 88.71.191.47 -
Changed lines 1-3 from:
  • Task1: Architecture of hierarchical DSM runtime system.
    • Milestone M3.: Month 12 --
    • Deliverable D3.: Month 12 --
to:
  • Task1: Architecture of hierarchical DSM runtime system. Delivered runtime will be a federation. On each virtual machine, or in some cases physical machine, an individual runtime system will be in operation. Each of these individual runtime systems will interact with the others to form a collective, cooperative, overall runtime system, which presents a single address space abstraction. This task will define the architecture, interfaces, and protocols for this overall runtime system. Then for each class of hardware an individual task will implement the interfaces and protocols for that particular kind of hardware, as described below.

-- Sean leads task, partners for 2 through 6, plus Imperial, plus INRIA, plus XLAB participate on aspects that involve their individual piece. INRIA will advise on how the architecture choices impact the toolchain design and implementation. Imperial will advise on how the choices impact the algorithms for runtime work division and deployment. XLAB will advise on how the architecture impacts the portal. Sean and IBM will provide input on how the architecture impacts the individual runtime system they are implementing.

  * Milestone   M3.: Month 9 -- architecture and API of DSM runtime delivered, based on experience gained in the rapid prototype development done in WP 2 and WP 6
  * Deliverable D3.: Month 9 -- deliver document detailing architecture and API of DSM runtime  
Changed lines 13-16 from:
  • Task1: Architecture of hierarchical DSM runtime system. Delivered runtime will be a federation. On each virtual machine, or in some cases physical machine, an individual runtime system will be in operation. Each of these individual runtime systems will interact with the others to form a collective, cooperative, overall runtime system, which presents a single address space abstraction. This task will define the architecture, interfaces, and protocols for this overall runtime system. Then for each class of hardware an individual task will implement the interfaces and protocols for that particular kind of hardware, as described below.

-- Sean leads task, partners for 2 through 6, plus Imperial, plus INRIA, plus XLAB participate on aspects that involve their individual piece. INRIA will advise on how the architecture choices impact the toolchain design and implementation. Imperial will advise on how the choices impact the algorithms for runtime work division and deployment. XLAB will advise on how the architecture impacts the deployment tool. Partners for 2 through 6 will provide input on how the architecture impacts the individual runtime system they are implementing.

  • Task4: DSM runtime system on a shared memory x86 based Cloud server instance -- Sean with involvement by XLAB
to:
  • Task4: DSM runtime system on a shared memory x86 based Cloud server instance -- Sean with involvement by XLAB and Gigas
Changed line 15 from:
  * Deliverable D2.: Month 12 --  
to:
  * Deliverable D2.: Month 12 --  simple, first prototype working inside rapid prototype created in WP 2.
Changed line 21 from:
  • Task5: DSM runtime system on a shared memory Power based Cloud server instance -- IBM plus Sean with involvement by XLAB
to:
  • Task5: DSM runtime system on a shared memory Power based Cloud server instance -- IBM plus Sean with involvement by XLAB and Gigas
Changed line 45 from:
  • Question: "Is the runtime going to be virtualized itself? Why not use standard bytecodes?" A: There won't be any bytecodes. CloudDSM is focused on high performance, and so the DSM runtime for many of the machines will be based on proto-runtime, due to its high performance and productivity benefits. Proto-runtime creates the equivalent of user-level "threads", which are called virtual processors because they capture a CPU context, and a CPU context virtualizes the bare processor pipeline. One of these VPs consists of its own stack, plus a data structure that holds a saved stack pointer, frame pointer, and program-counter contents. That set of state equals a point within the CPU's computation, and it can be used to switch the CPU around among different computation timelines. Each of those timelines behaves like its own, virtual, CPU processor pipeline.. hence, the name "virtual processor". But it's the lowest level, most bare bones "virtualization" possible.. there is no bytecode, no translation between ISAs, nothing but multiple CPU contexts.
to:
  • Question: "Is the runtime going to be virtualized itself? Why not use standard bytecodes?" A: There won't be any bytecodes. CloudDSM is focused on high performance, and so the DSM runtime for many of the machines will be based on proto-runtime, due to its high performance and productivity benefits. Proto-runtime creates the equivalent of user-level "threads", which are called virtual processors because they capture a CPU context, and a CPU context virtualizes the bare processor pipeline. One of these VPs consists of its own stack, plus a data structure that holds a saved stack pointer, frame pointer, and program-counter contents. That set of state equals a point within the CPU's computation, and it can be used to switch the CPU around among different computation timelines. Each of those timelines behaves like its own, virtual, CPU processor pipeline.. hence, the name "virtual processor". But it's the lowest level, most bare bones "virtualization" possible.. there is no bytecode, no translation between ISAs, nothing but multiple CPU contexts.
March 22, 2014, at 08:18 PM by 178.11.223.226 -
Deleted line 0:
Deleted lines 11-26:
  • Task2: DSM runtime system on Kalray HW that presents an API that is manipulatable by WP2 Cloud deployment tool -- Kalray (with involvement by Sean and XLAB)
    • Milestone M2.: Month 12 --
    • Deliverable D2.: Month 12 --
    • Milestone M2.: Month 24 --
    • Deliverable D2.: Month 24 --
    • Milestone M2.: Month 36 --
    • Deliverable D2.: Month 36 --
  • Task3: DSM runtime system on FORTH Cube HW that presents an API that is manipulatable by WP2 Cloud deployment tool -- FORTH (with involvement by Sean and XLAB)
    • Milestone M2.: Month 12 --
    • Deliverable D2.: Month 12 --
    • Milestone M2.: Month 24 --
    • Deliverable D2.: Month 24 --
    • Milestone M2.: Month 36 --
    • Deliverable D2.: Month 36 --
March 19, 2014, at 05:10 AM by 80.114.135.137 -
Added lines 1-62:
  • Task1: Architecture of hierarchical DSM runtime system.
    • Milestone M3.: Month 12 --
    • Deliverable D3.: Month 12 --
    • Milestone M3.: Month 24 --
    • Deliverable D3.: Month 24 --
    • Milestone M3.: Month 36 --
    • Deliverable D3.: Month 36 --
  • Task1: Architecture of hierarchical DSM runtime system. Delivered runtime will be a federation. On each virtual machine, or in some cases physical machine, an individual runtime system will be in operation. Each of these individual runtime systems will interact with the others to form a collective, cooperative, overall runtime system, which presents a single address space abstraction. This task will define the architecture, interfaces, and protocols for this overall runtime system. Then for each class of hardware an individual task will implement the interfaces and protocols for that particular kind of hardware, as described below.

-- Sean leads task, partners for 2 through 6, plus Imperial, plus INRIA, plus XLAB participate on aspects that involve their individual piece. INRIA will advise on how the architecture choices impact the toolchain design and implementation. Imperial will advise on how the choices impact the algorithms for runtime work division and deployment. XLAB will advise on how the architecture impacts the deployment tool. Partners for 2 through 6 will provide input on how the architecture impacts the individual runtime system they are implementing.

  • Task2: DSM runtime system on Kalray HW that presents an API that is manipulatable by WP2 Cloud deployment tool -- Kalray (with involvement by Sean and XLAB)
    • Milestone M2.: Month 12 --
    • Deliverable D2.: Month 12 --
    • Milestone M2.: Month 24 --
    • Deliverable D2.: Month 24 --
    • Milestone M2.: Month 36 --
    • Deliverable D2.: Month 36 --
  • Task3: DSM runtime system on FORTH Cube HW that presents an API that is manipulatable by WP2 Cloud deployment tool -- FORTH (with involvement by Sean and XLAB)
    • Milestone M2.: Month 12 --
    • Deliverable D2.: Month 12 --
    • Milestone M2.: Month 24 --
    • Deliverable D2.: Month 24 --
    • Milestone M2.: Month 36 --
    • Deliverable D2.: Month 36 --
  • Task4: DSM runtime system on a shared memory x86 based Cloud server instance -- Sean with involvement by XLAB
    • Milestone M2.: Month 12 --
    • Deliverable D2.: Month 12 --
    • Milestone M2.: Month 24 --
    • Deliverable D2.: Month 24 --
    • Milestone M2.: Month 36 --
    • Deliverable D2.: Month 36 --
  • Task5: DSM runtime system on a shared memory Power based Cloud server instance -- IBM plus Sean with involvement by XLAB
    • Milestone M2.: Month 12 --
    • Deliverable D2.: Month 12 --
    • Milestone M2.: Month 24 --
    • Deliverable D2.: Month 24 --
    • Milestone M2.: Month 36 --
    • Deliverable D2.: Month 36 --
  • Task6: binary runtime specializer -- IBM (Q: any chance to get open source release? Or to apply to other ISA?). Use fat binary runtime optimization techniques to adjust the granularity and layout during execution. This is specific to IBM Power architecture, but it provides the blue print for implementing for other ISAs as well.
    • Milestone M2.: Month 12 --
    • Deliverable D2.: Month 12 --
    • Milestone M2.: Month 24 --
    • Deliverable D2.: Month 24 --
    • Milestone M2.: Month 36 --
    • Deliverable D2.: Month 36 --
  • Task7: Integration testing. Deploy the various individual runtime systems on specific machines. Write test cases whose purpose is to expose performance issues and bugs. Execute the test cases, record the performance, report to the partners who developed the individual runtimes in tasks 2 through 6. Work with the partners to determine the reasons for particular performance anomalies, perhaps writing and running specific tests interactively. The partners for 2 through 6 will use the results to modify their individual runtime system implementations. -- Gigas leads -- partners for 2 through 6, plus XLAB are involved.
    • Milestone M2.: Month 12 --
    • Deliverable D2.: Month 12 --
    • Milestone M2.: Month 24 --
    • Deliverable D2.: Month 24 --
    • Milestone M2.: Month 36 --
    • Deliverable D2.: Month 36 --
  • Question: "Is the runtime going to be virtualized itself? Why not use standard bytecodes?" A: There won't be any bytecodes. CloudDSM is focused on high performance, and so the DSM runtime for many of the machines will be based on proto-runtime, due to its high performance and productivity benefits. Proto-runtime creates the equivalent of user-level "threads", which are called virtual processors because they capture a CPU context, and a CPU context virtualizes the bare processor pipeline. One of these VPs consists of its own stack, plus a data structure that holds a saved stack pointer, frame pointer, and program-counter contents. That set of state equals a point within the CPU's computation, and it can be used to switch the CPU around among different computation timelines. Each of those timelines behaves like its own, virtual, CPU processor pipeline.. hence, the name "virtual processor". But it's the lowest level, most bare bones "virtualization" possible.. there is no bytecode, no translation between ISAs, nothing but multiple CPU contexts.