dev-discuss.pytorch.org/t/min-cut-optimal-recomputation-i-e-activation-checkpointing-with-aotautograd/467/1

Preview meta tags from the dev-discuss.pytorch.org website.

Linked Hostnames

7

Thumbnail

Search Engine Appearance

Google

https://dev-discuss.pytorch.org/t/min-cut-optimal-recomputation-i-e-activation-checkpointing-with-aotautograd/467/1

Min-cut optimal(*) recomputation (i.e. activation checkpointing) with AOTAutograd

TL;DR: We’ve implemented a min-cut based recomputation pass with AOTAutograd + NVFuser that consistently improves both memory and runtime across a wide range of models (including the TorchBench suite) for GPU training. …



Bing

Min-cut optimal(*) recomputation (i.e. activation checkpointing) with AOTAutograd

https://dev-discuss.pytorch.org/t/min-cut-optimal-recomputation-i-e-activation-checkpointing-with-aotautograd/467/1

TL;DR: We’ve implemented a min-cut based recomputation pass with AOTAutograd + NVFuser that consistently improves both memory and runtime across a wide range of models (including the TorchBench suite) for GPU training. …



DuckDuckGo

https://dev-discuss.pytorch.org/t/min-cut-optimal-recomputation-i-e-activation-checkpointing-with-aotautograd/467/1

Min-cut optimal(*) recomputation (i.e. activation checkpointing) with AOTAutograd

TL;DR: We’ve implemented a min-cut based recomputation pass with AOTAutograd + NVFuser that consistently improves both memory and runtime across a wide range of models (including the TorchBench suite) for GPU training. …

  • General Meta Tags

    8
    • title
      Min-cut optimal(*) recomputation (i.e. activation checkpointing) with AOTAutograd - compiler - PyTorch Developer Mailing List
    • charset
      utf-8
    • description
      TL;DR: We’ve implemented a min-cut based recomputation pass with AOTAutograd + NVFuser that consistently improves both memory and runtime across a wide range of models (including the TorchBench suite) for GPU training. …
    • generator
      Discourse 3.5.0.beta9-dev - https://github.com/discourse/discourse version 37d1849de66e13c724deaa6b460abf871b737ac5
    • theme-color
      #111111

  • Open Graph Meta Tags

    9
    • og:site_name
      PyTorch Developer Mailing List
    • og:type
      website
    • og:image
      https://canada1.discourse-cdn.com/flex036/uploads/pytorch1/original/1X/29732cf008ca2934b9d9e9a1a7691d71445e388a.jpeg
    • og:url
      https://dev-discuss.pytorch.org/t/min-cut-optimal-recomputation-i-e-activation-checkpointing-with-aotautograd/467/1
    • og:title
      Min-cut optimal(*) recomputation (i.e. activation checkpointing) with AOTAutograd

  • Twitter Meta Tags

    9
    • twitter:card
      summary
    • twitter:image
      https://canada1.discourse-cdn.com/flex036/uploads/pytorch1/original/1X/29732cf008ca2934b9d9e9a1a7691d71445e388a.jpeg
    • twitter:url
      https://dev-discuss.pytorch.org/t/min-cut-optimal-recomputation-i-e-activation-checkpointing-with-aotautograd/467/1
    • twitter:title
      Min-cut optimal(*) recomputation (i.e. activation checkpointing) with AOTAutograd
    • twitter:description
      TL;DR: We’ve implemented a min-cut based recomputation pass with AOTAutograd + NVFuser that consistently improves both memory and runtime across a wide range of models (including the TorchBench suite) for GPU training. Intro Recomputation (often called activation checkpointing) is a technique in which, instead of saving some activations for use in backwards, we recompute them during the backwards pass. Thus, we trade off longer runtime for less memory, right? Actually, we can do better than...

  • Item Prop Meta Tags

    51
    • position
      1
    • headline
      Min-cut optimal(*) recomputation (i.e. activation checkpointing) with AOTAutograd
    • datePublished
      2022-01-10T20:43:45Z
    • articleSection
      compiler
    • keywords

  • Link Tags

    27
    • alternate nofollow
      https://dev-discuss.pytorch.org/t/min-cut-optimal-recomputation-i-e-activation-checkpointing-with-aotautograd/467.rss
    • apple-touch-icon
      https://canada1.discourse-cdn.com/flex036/uploads/pytorch1/optimized/1X/6cd7da56682d360e2c6006ff3e31eb250c5a8675_2_180x180.png
    • canonical
      https://dev-discuss.pytorch.org/t/min-cut-optimal-recomputation-i-e-activation-checkpointing-with-aotautograd/467
    • icon
      https://canada1.discourse-cdn.com/flex036/uploads/pytorch1/optimized/1X/6cd7da56682d360e2c6006ff3e31eb250c5a8675_2_32x32.png
    • search
      https://dev-discuss.pytorch.org/opensearch.xml

Links

39