forums.fast.ai/t/adamw-and-super-convergence-blog/18836/2

Preview meta tags from the forums.fast.ai website.

Linked Hostnames

2

Thumbnail

Search Engine Appearance

Google

https://forums.fast.ai/t/adamw-and-super-convergence-blog/18836/2

AdamW and Super-convergence blog

I had some questions about the recent fast.ai blog post, “AdamW and Super-convergence is now the fastest way to train neural nets”, and in the absence of a comments topic this Deep Learning one seemed most appropriate. M…



Bing

AdamW and Super-convergence blog

https://forums.fast.ai/t/adamw-and-super-convergence-blog/18836/2

I had some questions about the recent fast.ai blog post, “AdamW and Super-convergence is now the fastest way to train neural nets”, and in the absence of a comments topic this Deep Learning one seemed most appropriate. M…



DuckDuckGo

https://forums.fast.ai/t/adamw-and-super-convergence-blog/18836/2

AdamW and Super-convergence blog

I had some questions about the recent fast.ai blog post, “AdamW and Super-convergence is now the fastest way to train neural nets”, and in the absence of a comments topic this Deep Learning one seemed most appropriate. M…

  • General Meta Tags

    8
    • title
      AdamW and Super-convergence blog - #2 by sgugger - Deep Learning - fast.ai Course Forums
    • charset
      utf-8
    • description
      I had some questions about the recent fast.ai blog post, “AdamW and Super-convergence is now the fastest way to train neural nets”, and in the absence of a comments topic this Deep Learning one seemed most appropriate. M…
    • generator
      Discourse 3.5.0.beta5-dev - https://github.com/discourse/discourse version eff31e0d42c535ee115317f5b63ff70d90097911
    • theme-color
      #fff

  • Open Graph Meta Tags

    9
    • og:site_name
      fast.ai Course Forums
    • og:type
      website
    • og:image
      https://forums.fast.ai/uploads/default/original/3X/b/3/b395de7a2ba00b82865031c97a8cadf3d80e71e5.png
    • og:url
      https://forums.fast.ai/t/adamw-and-super-convergence-blog/18836/2
    • og:title
      AdamW and Super-convergence blog

  • Twitter Meta Tags

    5
    • twitter:card
      summary
    • twitter:image
      https://forums.fast.ai/uploads/default/original/3X/b/3/b395de7a2ba00b82865031c97a8cadf3d80e71e5.png
    • twitter:url
      https://forums.fast.ai/t/adamw-and-super-convergence-blog/18836/2
    • twitter:title
      AdamW and Super-convergence blog
    • twitter:description
      For the NLP tasks, the measure used is the perplexity, which is the exponential of the validation/test loss. So lower is better and this is where amsgrad actually hurts training the most (since there is a substantial spike in ppl). For the tests with SGD, I need 150 epochs to get to the same perplexities on wikitext-2 (I had posted a notebook about this earlier in this thread) but someone may find a best set of hyper-parameters to get as fast as AdamW. On CIFAR-10, SGD gets to the same accurac...

  • Item Prop Meta Tags

    12
    • position
      1
    • headline
      AdamW and Super-convergence blog
    • datePublished
      2018-07-03T15:53:19Z
    • articleSection
      Deep Learning
    • keywords

  • Link Tags

    29
    • alternate nofollow
      https://forums.fast.ai/t/adamw-and-super-convergence-blog/18836.rss
    • apple-touch-icon
      https://forums.fast.ai/uploads/default/optimized/3X/b/3/b395de7a2ba00b82865031c97a8cadf3d80e71e5_2_180x180.png
    • canonical
      https://forums.fast.ai/t/adamw-and-super-convergence-blog/18836
    • icon
      https://forums.fast.ai/uploads/default/optimized/3X/b/3/b395de7a2ba00b82865031c97a8cadf3d80e71e5_2_32x32.png
    • url
      https://forums.fast.ai/u/mcskinner

Links

11