forums.fast.ai/t/super-convergence-ish-on-wikitext-2/17091

Preview meta tags from the forums.fast.ai website.

Linked Hostnames

4

Thumbnail

Search Engine Appearance

Google

https://forums.fast.ai/t/super-convergence-ish-on-wikitext-2/17091

Super convergence(ish) on wikitext-2

After a month or so of experimenting on wikitext-2 and training a LM as fast as possible, I wanted to share a few findings. Here is the notebook with my current results: in 150 epochs (instead of 750) I get to a perplexi…



Bing

Super convergence(ish) on wikitext-2

https://forums.fast.ai/t/super-convergence-ish-on-wikitext-2/17091

After a month or so of experimenting on wikitext-2 and training a LM as fast as possible, I wanted to share a few findings. Here is the notebook with my current results: in 150 epochs (instead of 750) I get to a perplexi…



DuckDuckGo

https://forums.fast.ai/t/super-convergence-ish-on-wikitext-2/17091

Super convergence(ish) on wikitext-2

After a month or so of experimenting on wikitext-2 and training a LM as fast as possible, I wanted to share a few findings. Here is the notebook with my current results: in 150 epochs (instead of 750) I get to a perplexi…

  • General Meta Tags

    8
    • title
      Super convergence(ish) on wikitext-2 - Part 2 & Alumni (2018) - fast.ai Course Forums
    • charset
      utf-8
    • description
      After a month or so of experimenting on wikitext-2 and training a LM as fast as possible, I wanted to share a few findings. Here is the notebook with my current results: in 150 epochs (instead of 750) I get to a perplexi…
    • generator
      Discourse 3.5.0.beta5-dev - https://github.com/discourse/discourse version eff31e0d42c535ee115317f5b63ff70d90097911
    • theme-color
      #fff

  • Open Graph Meta Tags

    9
    • og:site_name
      fast.ai Course Forums
    • og:type
      website
    • og:image
      https://forums.fast.ai/uploads/default/original/3X/b/3/b395de7a2ba00b82865031c97a8cadf3d80e71e5.png
    • og:url
      https://forums.fast.ai/t/super-convergence-ish-on-wikitext-2/17091
    • og:title
      Super convergence(ish) on wikitext-2

  • Twitter Meta Tags

    9
    • twitter:card
      summary
    • twitter:image
      https://forums.fast.ai/uploads/default/original/3X/b/3/b395de7a2ba00b82865031c97a8cadf3d80e71e5.png
    • twitter:url
      https://forums.fast.ai/t/super-convergence-ish-on-wikitext-2/17091
    • twitter:title
      Super convergence(ish) on wikitext-2
    • twitter:description
      After a month or so of experimenting on wikitext-2 and training a LM as fast as possible, I wanted to share a few findings. Here is the notebook with my current results: in 150 epochs (instead of 750) I get to a perplexity of 70.73 (vs 68 for the benchmark given by Stephen Merity) then 53.1 with the cache pointer (vs 52 or the benchmark given by Stephen Merity). Here is a list of things that helped: using the dropouts from Stephen Merity and not Jeremy’s (I think this part comes from the diff...

  • Item Prop Meta Tags

    60
    • position
      1
    • headline
      Super convergence(ish) on wikitext-2
    • datePublished
      2018-05-27T13:03:46Z
    • articleSection
      Part 2 & Alumni (2018)
    • keywords

  • Link Tags

    30
    • alternate nofollow
      https://forums.fast.ai/t/super-convergence-ish-on-wikitext-2/17091.rss
    • apple-touch-icon
      https://forums.fast.ai/uploads/default/optimized/3X/b/3/b395de7a2ba00b82865031c97a8cadf3d80e71e5_2_180x180.png
    • canonical
      https://forums.fast.ai/t/super-convergence-ish-on-wikitext-2/17091
    • icon
      https://forums.fast.ai/uploads/default/optimized/3X/b/3/b395de7a2ba00b82865031c97a8cadf3d80e71e5_2_32x32.png
    • search
      https://forums.fast.ai/opensearch.xml

Links

21