Mitigating Memorization in LLMs: @dair_ai observed this paper offers a modification of the following-token prediction goal called goldfish loss to aid mitigate the verbatim era of memorized training data. Tweet from Robert Graham (@ErrataRob): nVidia is in exactly the same placement as Sunshine Microsystems was within the early days in the d