BEGIN:VCALENDAR
PRODID:-//AddEvent Inc//AddEvent.com v1.7//EN
VERSION:2.0
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
BEGIN:STANDARD
DTSTART:20261101T010000
RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=11
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:20260308T030000
RRULE:FREQ=YEARLY;BYDAY=2SU;BYMONTH=3
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
DESCRIPTION:The Lottery Ticket Hypothesis (LTH) suggests that there exists a sparse winning ticket mask and weights that achieves the same generalization performance as the dense model while using significantly fewer parameters. LTH achieves this by iteratively sparsifying and re-training within the pruned solution basin. However\, this procedure is computationally expensive\, and a winning ticket’s sparsity mask does not generalize to other weight initializations. Recent work has suggested that Deep Neural Networks (DNNs) trained from random initialization find solutions within the same basin modulo weight symmetry\, and proposed a method to align trained models within the same basins. We propose permuting the winning ticket mask to align with the new optimization basin when performing sparse training from a different random initialization. Using this permuted mask\, we show it is possible to significantly increase the generalization performance of sparse training from random initialization with the permuted mask as compared to sparse training naively using the non-permuted mask. We empirically demonstrate that our proposed method improves the generalization of LTH with the new random initialization on multiple datasets (CIFAR-10/100 and ImageNet) using VGG11 and ResNet-20/50 models of varying widths.\n\nBio: "My name is Rohan. I'm a 2nd year MSc student at the University of Calgary in Canada working under Dr. Yani Ioannou. My research interests lie in understanding the geometry of NN loss landscapes\, sparse neural network training dynamics\, mean-field theory\, and random matrix theory. I studied Mathematics & Computer Science during my undergrad at the University of Toronto in Canada\, with a focus in probability & combinatorics."\n\n------\n\nCreate your own Add to Calendar links with addevent.com/r/a \n
X-ALT-DESC;FMTTYPE=text/html:The Lottery Ticket Hypothesis (LTH) suggests that there exists a sparse winning ticket mask and weights that achieves the same generalization performance as the dense model while using significantly fewer parameters. LTH achieves this by iteratively sparsifying and re-training within the pruned solution basin. However, this procedure is computationally expensive, and a winning ticket’s sparsity mask does not generalize to other weight initializations. Recent work has suggested that Deep Neural Networks (DNNs) trained from random initialization find solutions within the same basin modulo weight symmetry, and proposed a method to align trained models within the same basins. We propose permuting the winning ticket mask to align with the new optimization basin when performing sparse training from a different random initialization. Using this permuted mask, we show it is possible to significantly increase the generalization performance of sparse training from random initialization with the permuted mask as compared to sparse training naively using the non-permuted mask. We empirically demonstrate that our proposed method improves the generalization of LTH with the new random initialization on multiple datasets (CIFAR-10/100 and ImageNet) using VGG11 and ResNet-20/50 models of varying widths.<br /><strong><strong><br /></strong></strong><strong><strong>Bio</strong></strong>: "My name is Rohan. I'm a 2nd year MSc student at the University of Calgary in Canada working under Dr. Yani Ioannou. My research interests lie in understanding the geometry of NN loss landscapes, sparse neural network training dynamics, mean-field theory, and random matrix theory. I studied Mathematics &amp; Computer Science during my undergrad at the University of Toronto in Canada, with a focus in probability &amp; combinatorics."<br /><br />------<br /><br />Create your own Add to Calendar links with addevent.com/r/a <br>
UID:1c02b2ab88a6473cbcdd8c8b8792aaeaaddeventcom
SUMMARY:C4AI - Rohan Jain - Winning Tickets from Random Initialization: Aligning Masks for Sparse Training
DTSTART;TZID=America/Los_Angeles:20250117T090000
DTEND;TZID=America/Los_Angeles:20250117T100000
DTSTAMP:20261115T162040Z
TRANSP:OPAQUE
STATUS:CONFIRMED
SEQUENCE:0
LOCATION:https://meet.google.com/wdk-yipf-zjd
X-MICROSOFT-CDO-BUSYSTATUS:BUSY
BEGIN:VALARM
TRIGGER:-PT30M
ACTION:DISPLAY
DESCRIPTION:Reminder
END:VALARM
END:VEVENT
END:VCALENDAR