Filed Pursuant to Rule 424(b)(3)
                                                Registration No. 333-118966

           Prospectus Supplement to Prospectus dated November 24, 2004

                                 1,818,504 Shares

                         Casual Male Retail Group, Inc.

                                  Common Stock

This prospectus supplement updates the prospectus dated November 24, 2004
relating to the offer for sale of up to an aggregate of 1,818,504 shares of
common stock of Casual Male Retail Group, Inc. by the selling stockholders
identified in the prospectus and this prospectus supplement, and any of their
pledgees, donees, transferees or other successors in interest.

We are providing this prospectus supplement to update the table in the
prospectus under the caption "Selling Securityholders" to reflect (i) a
transfer of a warrant to purchase 10,000 shares of common stock from WEC
Partners, L.P. to Zeke, LP and (ii) a transfer of a warrant to purchase 120,000
shares of common stock from Hourglass Master Fund, Ltd. to OTA, LLC.  The
amounts set forth below are based upon information provided to us by the
selling stockholder (or their representatives), or on our records, and are
accurate to the best of our knowledge.  Unless we indicate otherwise, the
information in this prospectus supplement is as of March 8, 2006.

                  Number of Shares                    Common Stock Beneficially
                  of Common Stock     Number of Shares  Owned After the Sales
Selling           Beneficially Owned  of Common Stock  ------------------------
Securityholders   Before Any Sale     Subject to Sale   Number *    Percent**
- --------------    ------------------  ----------------  -----------------------

WEC Partners, L.P.      N/A                  0            N/A        N/A
Zeke, LP (1)         90,000             90,000              0          -
Hourglass Master
  Fund, Ltd.            N/A                  0            N/A        N/A
OTA, LLC (2)        120,000            120,000              0          -
_____________________

*	Assumes that the selling securityholders will sell all of their shares of
common stock subject to sale pursuant to this prospectus and prospectus
supplement. We cannot assure you that the selling securityholders will sell
all or any of their shares of common stock.

**	Percentage ownership is based on 34,721,610 shares of common stock
outstanding as of March 7, 2006, plus securities deemed to be outstanding with
respect to individual stockholders pursuant to Rule 13d-3(d)(1) under the
Securities Exchange Act of 1934.

(1)	Edward N. Antoian, the general partner of Zeke, LP, has voting and
dispositive power over securities owned by Zeke, LP.

(2)	Ira M. Leventhal, Senior Managing Director of OTA, LLC, has voting
and dispositive power over securities owned by OTA, LLC.

This prospectus supplement is not complete without the prospectus dated November
24, 2004, and we have not authorized anyone to deliver or use this prospectus
supplement without the prospectus.

           The date of this prospectus supplement is March 8, 2006.