Mapping and Dataflow Optimization Considering Thermal Effects in 3D/2.5D Multi-chilet System

Problem of Mapping and Thermal

Thermal problem

In 3D/2.5D stacked system, thermal is always a problem no matter in digital computing or analog computing, no matter in NVMs or VMs. This problem has several influeneces on difference memory devices and systems.

• Retention time – Mainly influence power concumption
  • Rentention time can decrease when temperature increases. This can be observed in all types of memory devices, like DRAM, SRAM, PCM, ReRAM.
  • The reduce of retention time may not influence the accuracy of memory cells since refreshing happens. However, this will largely increase the power consumption, since refreshing is power consuming operation.
  • Retention time also influences the life time of PCM/ReRAM, since reprogramming of NVM will lead to acclerating of device damage. The endurance of most NVM devices are finite.
• Thermal noise –Mainly Accuracy
  • DRAM and SRAM are robust to this noise.
  • Most memorisitive based devices will be influenced by temperature increaase. For example, heating changes the OFF conductance of RRAM change the crystal status of PCM.

    Mapping challenge

    Mapping techniques considering nowdays workload is non-trivial.

• Trade-off between data transfering overhead and heating dissipation.
  • Devices with heavy workload prefer not to be mapped closely considering hearing.
  • However, chiplets with heavy workload typlically has large data transfer amount which perfer not to be far away from contiguous chiplets.
• Workload unbalance.
  • Mapping CNN or dense tranformers is much easier compare to complicated workload like RAG or MoE.
  • Workload like RAG MoE have a property that: Different parameters may have different workload, which means some parameters will process heavy input some will not. And this happens dynamically and simutaneously at runtime.
  • Conventional mapping techqniques that mapping layers one-by-one to single or group of chiplets will lead to severe workload unbalance in MoE or RAG. We need to spread one layer as wide as possible at the same time make them contiguous to limit activation transfering overhead.

Recent works

1. Edit circuit to adaptively adjust output at the circuit level.
   • TRIM
2. Mapping solution with optimziation algorithms
   • TEFLON
3. Mapping solution with ML methods
4. Heuristic solutions
   • WRAP

back to homes