check the belt for alignment and tension as suggested - and check that the drive pulley is tight and not floating around on the shaft -it's easy to check just take out three bolts that hold the cover on and check it for play on the shaft - some bikes do not seem to come with any where near the correct torque
there are some pics around here of one that wasn't tight, wore the spline teeth, and made the exact noise you describe when you let off on the throttle - had to replace the drive pulley at 20k miles -
here it is
http://suzukisavage.com/cgi-bin/YaBB.pl?board=RubberSideDown;action=display;n...