Filed Pursuant to Rule 424(b)(3) Registration No. 333-118966 Prospectus Supplement to Prospectus dated November 24, 2004 1,818,504 Shares Casual Male Retail Group, Inc. Common Stock This prospectus supplement updates the prospectus dated November 24, 2004 relating to the offer for sale of up to an aggregate of 1,818,504 shares of common stock of Casual Male Retail Group, Inc. by the selling stockholders identified in the prospectus and this prospectus supplement, and any of their pledgees, donees, transferees or other successors in interest. We are providing this prospectus supplement to update the table in the prospectus under the caption "Selling Securityholders" to reflect (i) a transfer of a warrant to purchase 164,000 shares of common stock from Harbour Holdings, Ltd. to Millennium Partners, L.P. and (ii) a transfer of a warrant to purchase 116,000 shares of common stock from Skylands Special Investment LLC (f/k/a Strong Special Investment, L.P.) to Millennium Partners, L.P.. The amounts set forth below are based upon information provided to us by the selling stockholder (or their representatives), or on our records, and are accurate to the best of our knowledge. Unless we indicate otherwise, the information in this prospectus supplement is as of September 18, 2006. Number of Shares Common Stock Beneficially of Common Stock Number of Shares Owned After the Sales Selling Beneficially Owned of Common Stock ----------------------- Securityholders Before Any Sale Subject to Sale Number ** Percent*** - -------------- ------------------ ---------------- ----------------------- Harbour Holdings, Ltd. N/A 0 N/A N/A Skylands Special Investment LLC (f/k/a Strong Special Investment, L.P.) N/A 0 N/A N/A Millennium Partners, L.P. (1) 296,237 (2) 280,000 16,237 (2) * _____________________ * Less than 1%. ** Assumes that the selling securityholders will sell all of their shares of common stock subject to sale pursuant to this prospectus and prospectus supplement. We cannot assure you that the selling securityholders will sell all or any of their shares of common stock. *** Percentage ownership is based on 34,417,684] shares of common stock outstanding as of September 18, 2006, plus securities deemed to be outstanding with respect to individual stockholders pursuant to Rule 13d-3(d)(1) under the Securities Exchange Act of 1934. (1) Millennium Management, L.L.C., a Delaware limited liability company, is the managing partner of Millennium Partners, L.P., a Cayman Islands exempted limited partnership, and consequently may be deemed to have voting control and investment discretion over securities owned by Millennium Partners, L.P. Israel A. Englander is the managing member of Millennium Management, L.L.C. As a result, Mr. Englander may be deemed to be the beneficial owner of any shares deemed to be beneficially owned by Millennium Management, L.L.C. The foregoing should not be construed in and of itself as an admission by either Millennium Management, L.L.C. or Mr. Englander as to beneficial ownership of the shares of our common stock owned by Millennium Partners, L.P. (2) Includes 16,237 shares of our common stock held by Millenco, L.P., an affiliate of Millenium Partners, L.P. as of the close of business on September 18, 2006. This prospectus supplement is not complete without the prospectus dated November 24, 2004, and we have not authorized anyone to deliver or use this prospectus supplement without the prospectus. The date of this prospectus supplement is September 29, 2006.