BEGIN:VCALENDAR
PRODID:-//AddEvent Inc//AddEvent.com v1.7//EN
VERSION:2.0
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
BEGIN:STANDARD
DTSTART:20261101T010000
RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=11
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:20260308T030000
RRULE:FREQ=YEARLY;BYDAY=2SU;BYMONTH=3
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
DESCRIPTION:Tony will discuss optimization methods that leverage the linear minimization oracle (LMO) over a norm-ball and their application to training huge neural networks. In recent work\, Tony has proposed with his coauthors a new stochastic family of algorithms that uses the LMO to adapt to the geometry of the problem. The resulting update rule unifies several existing optimization methods under a single framework\, including spectral methods like Muon\, which we are able to prove rigorous convergence results for. Furthermore\, we propose an explicit choice of norm for deep architectures\, which\, as a side benefit\, guarantees the transferability of hyperparameters like learning rate across model sizes. Experimentally\, we demonstrate significant speedups on nanoGPT training using our algorithm\, Scion\, without any reliance on Adam. The proposed method is memory-efficient\, requiring only one set of model weights and one set of gradients\, which can be stored in half-precision.\n\nAntonio (Tony) Silveti-Falls is an associate professor of artificial intelligence at CentraleSupélec in the department of mathematics and a member of the Inria team OPIS. He works at the intersection of {stochastic\, nonsmooth\, noneuclidean} optimization theory and machine learning\, with a focus on developing and analyzing algorithms for large-scale problems.\n\nWebsite: https://tonysf.github.io/\n\n------\n\nCreate your own Add to Calendar links with addevent.com/r/a \n
X-ALT-DESC;FMTTYPE=text/html:Tony will discuss optimization methods that leverage the linear minimization oracle (LMO) over a norm-ball and their application to training huge neural networks. In recent work, Tony has proposed with his coauthors a new stochastic family of algorithms that uses the LMO to adapt to the geometry of the problem. The resulting update rule unifies several existing optimization methods under a single framework, including spectral methods like Muon, which we are able to prove rigorous convergence results for. Furthermore, we propose an explicit choice of norm for deep architectures, which, as a side benefit, guarantees the transferability of hyperparameters like learning rate across model sizes. Experimentally, we demonstrate significant speedups on nanoGPT training using our algorithm, Scion, without any reliance on Adam. The proposed method is memory-efficient, requiring only one set of model weights and one set of gradients, which can be stored in half-precision.<br><br>Antonio (Tony) Silveti-Falls is an associate professor of artificial intelligence at CentraleSupélec in the department of mathematics and a member of the Inria team OPIS. He works at the intersection of {stochastic, nonsmooth, noneuclidean} optimization theory and machine learning, with a focus on developing and analyzing algorithms for large-scale problems.<br><br>Website: <u><a href="https://tonysf.github.io/" target="_blank">https://tonysf.github.io/</a></u><br /><br />------<br /><br />Create your own Add to Calendar links with addevent.com/r/a <br>
UID:2e20f25f85d34933a193d6dc2f08e2faaddeventcom
SUMMARY:Tony Silveti-Falls - Training neural networks at any scale (ML Theory)
DTSTART;TZID=America/Los_Angeles:20250703T090000
DTEND;TZID=America/Los_Angeles:20250703T100000
DTSTAMP:20261023T194123Z
TRANSP:OPAQUE
STATUS:CONFIRMED
SEQUENCE:0
LOCATION:https://meet.google.com/nvg-ptgt-ucf?hs=122&authuser=0
X-MICROSOFT-CDO-BUSYSTATUS:BUSY
BEGIN:VALARM
TRIGGER:-PT30M
ACTION:DISPLAY
DESCRIPTION:Reminder
END:VALARM
END:VEVENT
END:VCALENDAR