As filed with the Securities and Exchange Commission on November 22, 2004

                                                     Registration No. 333-118966
- --------------------------------------------------------------------------------

                       SECURITIES AND EXCHANGE COMMISSION
                              Washington, DC 20549
                               ------------------
                           AMENDMENT NO. 1 TO FORM S-3
                             REGISTRATION STATEMENT
                                      UNDER
                           THE SECURITIES ACT OF 1933
                               ------------------
                         CASUAL MALE RETAIL GROUP, INC.
             (Exact name of registrant as specified in its charter)

                               Delaware 04-2623104
                (State or other jurisdiction of (I.R.S. Employer
              incorporation or organization) Identification Number)


                555 Turnpike Street, Canton, Massachusetts 02021
                                 (781) 828-9300
    (Address, including zip code, and telephone number, including area code,
                  of registrant's principal executive offices)

                               ------------------

                                Dennis Hernreich
              Executive Vice President, Chief Operating Officer and
                             Chief Financial Officer
                         Casual Male Retail Group, Inc.
                               555 Turnpike Street
                           Canton, Massachusetts 02021
                                 (781) 828-9300
 (Name, address, including zip code, and telephone number, including area code,
                  of registrant's principal executive offices)

                               ------------------

                                    Copy to:
                              Peter G. Smith, Esq.
                       Kramer Levin Naftalis & Frankel LLP
                                919 Third Avenue
                            New York, New York 10022
                                 (212) 715-9100

                               ------------------

Approximate Date of Commencement of Proposed Sale to the Public: From time to
time after the effective date of this Registration Statement.

                               ------------------

   If the only securities being registered on this form are being offered
pursuant to dividend or interest reinvestment plans, check the following box.
|_|

   If any of the securities being registered on this form are to be offered on a
delayed or continuous basis pursuant to Rule 415 under the Securities Act of
1933, other than securities offered only in connection with dividend or interest
reinvestment plans, check the following box. |X|

   If this form is filed to register additional securities for an offering
pursuant to Rule 462(b) under the Securities Act, please check the following box
and list the Securities Act registration statement number of the earlier
effective registration statement for the same offering. |_|

   If this form is a post-effective amendment filed pursuant to Rule 462(c)
under the Securities Act, check the following box and list the Securities Act
registration statement number of the earlier effective registration statement
for the same offering. |_|

   If delivery of the prospectus is expected to be made pursuant to Rule 434,
please check the following box. |_|

                               ------------------

The registrant hereby amends this registration statement on such date or dates as may be necessary to delay its effective date until the registrant shall file a further amendment which specifically states that this registration statement shall thereafter become effective in accordance with Section 8(a) of the Securities Act of 1933 or until the registration statement shall become effective on such date as the Commission, acting pursuant to said Section 8(a), may determine.

Subject to Completion, dated November 22, 2004 1,818,504 Shares of Common Stock This prospectus relates to the offer and sale by the selling stockholders listed on pages 20-21 of up to 1,818,504 shares of common stock, par value $0.01 per share, of Casual Male Retail Group, Inc., which shares consist of (i) 1,182,400 shares of common stock issued or issuable upon the exercise of warrants currently held by the selling securityholders, (ii) 430,000 shares of common stock issuable upon the exercise of options currently held by the selling securityholders and (iii) 206,104 additional shares of common stock currently held by the selling securityholders. We issued through private placements in 2003 $29.56 million principal amount of 12% senior subordinated notes due 2010. Together with these notes, which in most cases were issued net of any commission for an aggregate purchase price equal to 98.4% of the aggregate principal amount, we also issued, through such private placements, detachable warrants to purchase 1,182,400 million shares of our common stock at exercise prices ranging from $4.76 to $7.32 per share. The warrants are exercisable through July 2, 2010. Our common stock is quoted on the Nasdaq National Market under the symbol "CMRG." The last reported sale price of our common stock on November 19, 2004 was $4.77 per share. Investing in our securities involves risks that are described in the "Risk Factors" section beginning on page 4 of this prospectus. We will not receive any cash proceeds from the sale of the shares of common stock offered under this prospectus. We are responsible for the payment of certain expenses incident to the registration of the securities. Neither the Securities and Exchange Commission, any state securities commission nor any other regulatory body has approved or disapproved of these securities or determined if this prospectus is truthful or complete. Any representation to the contrary is a criminal offense. The date of this prospectus is November 22, 2004. The information in this prospectus is not complete and may be changed. The selling securityholders may not sell these securities until the registration statement filed with the Securities and Excchange Commission is effective. This prospectus is not an offer to sell these securities and it is not soliciting an offer to buy these securities in any state where the offer or sale is not permitted.

Important Notice about the Information Presented in this Prospectus You should rely only on the information contained or incorporated by reference in this prospectus. We have not authorized any other person to provide you with different information. If anyone provides you with different or inconsistent information, you should not rely on it. For further information, see the section of this prospectus entitled "Where You Can Find More Information." We are not making an offer to sell these securities in any jurisdiction where the offer or sale is not permitted. You should assume that the information appearing in this prospectus is accurate as of the date on the front cover of this prospectus only. Our business, financial condition, results of operations and prospects may have changed since that date.

TABLE OF CONTENTS PROSPECTUS SUMMARY...........................................................1 RISK FACTORS.................................................................4 FORWARD LOOKING INFORMATION.................................................11 USE OF PROCEEDS.............................................................12 DESCRIPTION OF CAPITAL STOCK................................................12 CERTAIN UNITED STATES FEDERAL TAX CONSIDERATIONS............................15 SELLING SECURITYHOLDERS.....................................................18 PLAN OF DISTRIBUTION........................................................24 LEGAL MATTERS...............................................................25 EXPERTS.....................................................................25 WHERE YOU CAN FIND MORE INFORMATION.........................................26

PROSPECTUS SUMMARY The following summary may not contain all the information that may be important to you and is qualified in its entirety by the more detailed information appearing elsewhere or incorporated by reference in this prospectus. You should read the entire prospectus, especially the risks set forth under the heading "Risk Factors," as well as the information incorporated by reference, before making an investment decision. When used in this prospectus, the terms "Casual Male," "the Company," "we," "our" and "us" refer to Casual Male Retail Group, Inc. (formerly known as Designs, Inc.) and our consolidated subsidiaries, unless otherwise specified. References in this prospectus to years are to our 52-week or 53-week fiscal year, which ends on the Saturday nearest to January 31. For example, references to "fiscal 2003" mean our fiscal year ended January 31, 2004. Our Business We are the largest specialty retailer of big and tall men's apparel in the United States. We operate 489 Casual Male Big & Tall stores, the Casual Male catalog business, the Casual Male e-commerce website and 48 Levi's(R)/Dockers(R) Outlet by Designs outlet stores, all of which are located throughout the United States and Puerto Rico. Background Prior to May 2002, our business primarily consisted of owning and operating Levi's(R)/Dockers(R) and Candies(R) branded apparel mall and outlet stores. With limited opportunity to expand our mature Levi's(R)/Dockers(R) business, we acquired substantially all of the assets of Casual Male Corp. and certain of its subsidiaries at a bankruptcy court-ordered auction in May 2002. At the time of the acquisition, Casual Male Corp. was the largest retailer in the United States of men's clothing in the "big and tall" market. In April 2002, we entered into a joint venture with Ecko Complex, LLC, a leading design-driven lifestyle brand targeting young men and women, to open and operate Ecko Unltd.(R) branded outlet stores. Following our acquisition of Casual Male, we re-evaluated our strategic initiatives. In light of the significant opportunity to grow the Casual Male business and the continued significant deterioration in our Levi's(R)/Dockers(R) business, we announced that we would downsize and eventually exit the Levi's(R)/Dockers(R) business. We also announced that we would exit the Candies(R) outlet business, which we did by the end of fiscal 2002. Then in the second quarter of fiscal 2004, we sold our 50.5% interest in the Ecko joint venture to Ecko Complex, LLC. These decisions enabled management to focus our resources and energies primarily on growing our Casual Male business. Completion of Acquisition of Rochester Big & Tall On October 29, 2004, the Company completed its acquisition of substantially all of the assets of Rochester Big & Tall Clothing, Inc., a privately held company headquartered in San Francisco, California ("Rochester"). The purchase price was $15 million in cash plus the assumption of bank and subordinated debt of approximately $5 million, in addition to the assumption of identified operating liabilities such as accounts payable and accrued liabilities. There is a potential payment of an additional $4 million, to be paid over a three-year period subject to an earn-out provision. The Company financed the transaction with a new $7.5 million term loan from its primary lender and additional borrowings on its existing $90 million credit facility. The acquisition did not have any impact on the Company's overall liquidity.

Since the Casual Male acquisition, we have operated in two segments: our "Casual Male business" and our "Other Branded Apparel businesses." Casual Male Business Our Casual Male business is a multi-channel retailer that offers our customers multiple ways to purchase men's big and tall apparel. The business consists of: o 420 Casual Male Big & Tall full-price retail stores, located primarily in strip centers, power centers and stand-alone locations; o 69 Casual Male Big & tall outlet stores, located in outlet shopping centers; o the "Casual Male Big & Tall" catalog, of which we issued 17 editions in fiscal 2003; and o our e-commerce business, which includes the www.casualmale.com website and a Casual Male Big & Tall apparel shop on the Amazon.com website. Since our acquisition of the Casual Male business in May 2002, in order to revitalize the Casual Male brand and increase our share of the $5.3 billion men's big and tall apparel market, we have implemented several merchandising strategies, including: o changing our store format to merchandise our stores by lifestyle, such as traditional, active and contemporary; o targeting the fast-growing "under 30" big and tall customer segment; o announcing an exclusive marketing agreement with George Foreman, as well as launching an exclusive line of clothing with the George Foreman brand which commenced in Spring 2004; o launching a custom fit program, by which customers can purchase certain styles of clothing that are custom made to specific fit requirements; o broadening our merchandise offerings by introducing selected branded products, including professional sports apparel; and o introducing new systems infrastructure to improve inventory management, maintain in-stock positions in critical sizes for all stores and tailor lifestyle merchandise assortments to the demographic characteristics of each store. Other Branded Apparel Businesses Ecko Unltd.(R) From March 2002 through July 30, 2004, we operated a joint venture with Ecko Complex, LLC ("Ecko"), under which we owned and managed retail outlet stores bearing the name Ecko Unltd.(R) and featuring Ecko(R) branded merchandise. On July 30, 2004, the Company sold to Ecko its 50.5% interest in the joint venture for a purchase price of $800,000 in cash and a secured promissory note in the principal amount of $6.2 million. In addition, the Company will also continue to receive fees based on a percentage of sales for providing transitional services to the joint venture related to its operating and accounting systems, as needed until June 30, 2005. The above transaction resulted in a gain of approximately $3.1 million, which was included in our results of operations for the three and six months ended July 31, 2004. 2

Furthermore, pursuant to a mark-down allowance agreement entered into on July 30, 2004, Ecko also executed and delivered an additional secured promissory note for $1.0 million as a markdown allowance with respect to purchases of certain goods made by the Company from Ecko. Levi's(R)/Dockers(R) Outlets We currently operate 48 Levi's(R)/Dockers(R) outlet stores, 17 of which we expect to close by the end of fiscal 2004. We expect that the remaining Levi's(R)/Dockers(R) stores will either be closed on or before the end of their respective lease terms. Corporate Information We were originally incorporated in Delaware as Designs, Inc. in 1976. We changed our name to Casual Male Retail Group, Inc. shortly following our acquisition of substantially all of the assets of Casual Male Corp. and certain of its subsidiaries in May 2002. The address of our principal corporate and executive office is 555 Turnpike Street, Canton, Massachusetts 02021. The telephone number at our headquarters is (781) 828-9300. Our corporate website is located at http://www.cmrginc.com. The information contained on our website is not part of this prospectus. 3

Risk Factors You should carefully consider the following risks, as well as the other information contained in this prospectus or incorporated by reference in this prospectus, before investing in shares of our common stock. If any of the following risks actually occurs, our business, financial condition, operating results or prospects could be harmed. In that case, the trading price of our common stock could decline, and you might lose all or part of your investment. Risks Related to Our Company and Our Industry Our ability to continue to expand our Casual Male stores may be limited. A large part of our growth has resulted from the addition of new Casual Male stores and the increased sales volume and profitability provided by these stores. We will continue to depend on adding new stores to increase our sales volume and profitability. We believe that our ability to increase the number of Casual Male stores in the United States substantially in excess of the number of our current stores will be limited due to capital constraints, market conditions and other factors. When we enter new markets, we must: o obtain suitable store locations in light of the local real estate market conditions; o hire and train personnel; o establish distribution methods; and o advertise our brand names and our distinguishing characteristics to consumers who may not be familiar with us. As a result of these and other factors, opening new stores is often costly and entails significant risk. We cannot assure you that we will be able to open and operate new stores on a timely and profitable basis. The costs associated with opening new stores may negatively affect our results of operations. We may be unable to successfully predict fashion trends and customer preferences. Customer tastes and fashion trends are volatile and tend to change rapidly. Our success depends in large part upon our ability to effectively predict and respond to changing fashion tastes and consumer demands and to translate market trends to appropriate saleable product offerings. If we are unable to successfully predict or respond to changing styles or trends and misjudge the market for products or any new product lines, our sales will be lower and we may be faced with a substantial amount of unsold inventory or missed opportunities. In response, we may be forced to rely on additional markdowns or promotional sales to dispose of excess, slow-moving inventory, which would decrease our revenues and margins. In addition, the failure to satisfy consumer demand could have serious longer-term consequences, such as an adverse impact on our brand value and the loss of market share to our competitors. Our business is highly competitive, and competitive factors may reduce our revenues and profit margins. The United States men's big and tall apparel market is highly competitive with many national and regional department stores, specialty apparel retailers and discount stores offering a broad range of apparel products similar to the products that we sell. Besides retail competitors, we consider any manufacturer of big and tall merchandise operating in outlet malls throughout the United States to be a competitor. It is also 4

possible that another competitor, either a mass merchant or a men's specialty store or specialty apparel catalog, could gain market share in men's big and tall apparel due to more favorable pricing, locations, brand and fashion assortment and size availability. The presence in the marketplace of various fashion trends and the limited availability of shelf space also can affect competition. We may not be able to compete successfully with our competitors in the future and could lose brand recognition and market share. A significant loss of market share would adversely affect our revenues and results of operations. Our sales will decline if we do not successfully advertise and market our products. Our business is directly affected by the success or failure of our advertising and promotional efforts and those of our vendors. Future advertising efforts by us, our vendors or our other licensors may be costly and may not result in increased sales. If we were to undertake a major advertising campaign without success, then our failure to realize any revenues from our advertising and promotional expenditures, together with the possible adverse impact on our brand value and loss of market share, would have a negative impact upon our revenues. In either case, increased costs and decreased margins, accompanied by static or decreased revenues, would cause a decline in our results of operations. Our success significantly depends on our key personnel and our ability to attract and retain additional personnel. Our future success is dependent on the personal efforts, performance and abilities of our key management. For example, the loss of the services of David Levin, our President and Chief Executive Officer, or Dennis Hernreich, our Chief Operating Officer and Chief Financial Officer, each of whom is an integral part of our daily operations and is a primary decision maker in all our important operating matters, could significantly impact our business until adequate replacements could be identified and put in place. The loss of any of our senior management may result in: o a loss of organizational focus; o poor operating execution; o an inability to identify and execute potential strategic initiatives such as joint venture and licensing opportunities; o an impairment in our ability to identify new store locations; and o an inability to consummate possible acquisitions. These adverse results could, among other things, reduce potential revenues, prevent us from diversifying our product lines and geographic concentrations, and expose us to downturns in our markets. The loss of members of our senior management as well as our chairman, Seymour Holtzman, who also has many years of experience in the capital markets, could negatively impact our ability to obtain additional debt or equity financing for our operations or to refinance existing indebtedness, or the terms that might be negotiated for such financing or refinancing. Those circumstances in turn could ultimately result in a significant decline in profitability and decline in our financial condition. The competition is intense for the type of highly skilled individuals with relevant industry experience that we require and we may not be able to attract and retain new employees of the caliber needed to achieve our objectives. 5

We need to timely complete the implementation of our information systems and control procedures. We depend heavily upon technology and information systems to control inventory, sales, markdowns, merchandise on hand and other critical information. Any significant deficiencies in our management information systems resulting in less than optimal systems performance could have a negative impact upon our business. For example, since the information systems provide vital information with respect to specific merchandise sales at the SKU level, replenishment requirements to maintain optimum inventory levels, and sell through data from which markdown requirements are identified to most productively sell through poor selling SKUs, if that information is not consistently provided on a timely and accurate basis our sales could be severely impacted, or our gross margins could be adversely affected. We periodically review, improve and, under certain circumstances, replace our technology and management information systems to provide enhanced support to all operating areas. If such upgrades and enhancements are not successfully implemented, then the current systems may not be able to continue to support adequately our management information requirements. Currently, we are undergoing a significant effort to replace our existing antiquated legacy systems, as part of the process of integrating the historical Designs, Inc. and Casual Male operations. It is critically important to the successful operation of our business that the implementation of our systems integration process, which entails the replacement, enhancement, or upgrade of all Casual Male's vital former information systems, be completed within budget and in a timely manner without disruption to our daily operations. To implement these initiatives, we spent approximately $4.1 million in fiscal 2002, $4.0 million in fiscal 2003 and $1.5 million to date during fiscal 2004. We anticipate that the implementation will require approximately 12 additional months to complete at a remaining cost of approximately $10.5 million which includes upgrading our existing point-of-sale and register systems and implementing a Customer Relationship Management software system. If we are unable to complete these projects within budget and on time, our operating results will suffer. The loss of, or disruption in, our centralized distribution center could negatively impact our business and operations. All merchandise for our Casual Male stores is received into our centralized distribution center in Canton, Massachusetts, where the inventory is then processed, sorted and shipped to our stores. We depend in large part on the orderly operation of this receiving and distribution process, which depends, in turn, on adherence to shipping schedules and effective management of the distribution center. Although we believe that our receiving and distribution process is efficient and well positioned to support our expansion plans, we cannot assure you that events beyond our control, such as disruptions in operations due to fire or other catastrophic events, employee matters or shipping problems, would not result in delays in the delivery of merchandise to our stores. Although we maintain business interruption and property insurance, we cannot assure you that our insurance will be sufficient, or that insurance proceeds will be timely paid to us, in the event our distribution center is shut down for any reason or if we incur higher costs and longer lead times in connection with a distribution at our distribution center. We are dependent on third parties for the manufacture of the products we sell. We do not own or operate any manufacturing facilities and are therefore entirely dependent on third parties for the manufacture of the products we sell. Without adequate supplies of merchandise to sell to our customers in the styles and fashions demanded by our particular customer base, sales would decrease materially and our business would suffer. Furthermore, approximately 70% of our merchandise is branded product made specifically for Casual Male and our customers. In the event that manufacturers are unable or unwilling to ship products to us in a timely manner or continue to manufacture products for us, we would 6

have to rely on other current manufacturing sources or identify and qualify new manufacturers. We might not be able to identify or qualify such manufacturers for existing or new products in a timely manner and such manufacturers might not allocate sufficient capacity to us in order to meet our requirements. Our inability to secure adequate and timely supplies of product would negatively impact inventory levels, sales and gross margin rates, and ultimately our results of operations. In addition, even if our current manufacturers continue to manufacture our products, they may not maintain adequate controls with respect to product specifications and quality and may not continue to produce products that are consistent with our standards. If we are forced to rely on products of inferior quality, then our brand recognition and customer satisfaction would be likely to suffer. These manufacturers may also increase the cost to us of the products we purchase from them. If our suppliers increase our costs, our margins may be adversely affected. Should we experience significant unanticipated demand, we will be required to significantly expand our access to manufacturing, both from current and new manufacturing sources. If such additional manufacturing capacity is not available on terms as favorable as those obtained from current sources, then our revenues or margins, or both, will suffer. In addition, a significant portion of our merchandise is directly imported from other countries, and U.S. domestic suppliers who source their goods from other countries supply most of our remaining merchandise. If imported goods become difficult or impossible to bring into the United States, due to tariffs, embargoes or other reasons and if we cannot obtain such merchandise from other sources at similar costs, then our sales, gross margins and profit margins would significantly decline. Furthermore, in the event that commercial transportation is curtailed or substantially delayed, we may not be able to maintain adequate inventory levels of important merchandise on a consistent basis, which would negatively impact our sales and potentially erode the confidence of our customer base, leading to further loss of sales and an adverse impact on our results of operations. In extreme circumstances, it may be necessary to close less productive stores so as to consolidate important merchandise categories into our most productive stores, which would severely impact our results of operations and cash flow. Exiting our Levi's(R)/Dockers(R) business may subject us to significant costs and divert resources. In light of the continued significant deterioration in our Levi's(R)/Dockers(R) operations, we announced that we would downsize and eventually exit this business. In connection with this restructuring, we have incurred and will need to continue to incur significant exit costs associated with the termination of leases, liquidation of inventory and various employee matters. In addition, the restructuring of this business may divert managerial and other resources from our core businesses and may subject us to litigation. We have recorded restructuring charges totaling $41.3 million to date in connection with the restructuring of our Levi's(R)/Dockers(R) business, and expect to record additional restructuring charges as we complete this initiative. These charges have reduced and will continue to reduce our net income, and if future charges exceed our expectations, our stock price may be adversely affected. Our results of operations will be adversely affected if our George Foreman line of apparel is unsuccessful. We have entered into an exclusive endorsement and licensing arrangement for a men's apparel line with George Foreman, the well-known boxing personality. Under the terms of this arrangement, we are obligated to make significant payments to Mr. Foreman regardless of the success of the product line, and we intend to incur significant marketing costs in connection with the promotion of this product line. As a result 7

of these expenditures, if sales from this product line do not meet our expectations our results of operations will be adversely affected. Furthermore, we are subject to risks associated with having our brand identified with a celebrity personality. If our customers do not care for Mr. Foreman, or if this product line is not successful, our brand value will suffer. The loss of any of our key trademarks or licenses could adversely affect demand for our products. We own and use a number of trademarks and operate under several trademark license agreements. We believe that these trademarks have significant value and are instrumental in our ability to create and sustain demand for and to market our products. We cannot assure you that these trademarks and licensing agreements will remain in effect and enforceable or that any license agreements, upon expiration, can be renewed on acceptable terms or at all. In addition, any future disputes concerning these trademarks and licenses may cause us to incur significant litigation costs or force us to suspend use of the disputed trademarks. Our business is seasonal and is affected by general economic conditions. Like most other retail businesses, our business is seasonal. Historically, over 30% of our net sales have been made and approximately 70% of our operating income has been generated during November, December and January. Like other retail businesses, our operations may be negatively affected by local, regional or national economic conditions, such as levels of disposable consumer income, consumer debt, interest rates and consumer confidence. Any economic downturn might cause consumers to reduce their spending, which could negatively affect our sales. A sustained economic downturn would likely have an adverse affect on our results of operations. Acts of terrorism could negatively impact our operating results and financial condition. The continued threat of terrorism and heightened security measures in response to an act of terrorism may disrupt commerce and undermine consumer confidence which could negatively impact our sales by causing consumer spending to decline. Furthermore, an act of terrorism or war, or the threat thereof, could negatively impact our business by interfering with our ability to obtain merchandise from vendors or substitute suppliers at similar costs in a timely manner. Our cost savings and expense reductions resulting from the acquisition of Casual Male may be less than anticipated. We anticipate significant, continued cost savings following our May 2002 acquisition of substantially all the assets of Casual Male, primarily through headcount reductions, renegotiations of contractual arrangements for supplies and services associated with the operation for more favorable pricing terms, elimination of inefficient and costly business processes and costs by streamlining our management information systems, and economies of scale in purchasing. It is possible that some of the contemplated headcount reductions could fail to take place on the scale proposed due to unforeseen or underestimated needs for the employees in question. It is also possible that the cost savings associated with achieving purchasing economies fail to materialize due to unsuccessful negotiations with key vendors. There is also a cost to realizing the potential savings and these costs could potentially be higher than originally contemplated in management's projections. In such an instance, the amount of the cost savings would be offset by the higher costs of realizing the savings, thereby reducing the overall benefit of the acquisition of Casual Male and reducing our expected profitability. If there are substantial failures to achieve these cost savings, cash flow and the servicing of debt related to that acquisition could also be reduced. 8

We face greater challenges in managing several brands in multiple channels of distribution. Several retailers have had problems executing a corporate strategy aimed at operating multiple brands in multiple channels. We have expertise in the outlet channel of distribution, but our acquisition of Casual Male caused us to conduct operations in the specialty store and internet channels of distribution. We are now also responsible for all aspects of brand management with respect to the Casual Male brand, including advertising and promotion, and the servicing and merchandising of private label and brand merchandise. If the managing of multiple brands within multiple channels is poorly executed, we will not achieve our expected level of profitability, and could ultimately be compelled to eliminate the multiple brand strategy so that the organization may focus on a single brand strategy. A reduction in the size of our target market and shifts in customer purchasing habits will adversely affect our sales. As more and more food retailers begin to compete on the basis of providing more healthy menus, and American popular culture becomes more health conscious, the size of our target demographic could decrease, resulting in lower sales. In addition, recent statistics have shown that the overall levels of men's apparel sales have been decreasing, in part due to a lesser percentage of men's apparel being bought by women. If this trend continues and we are unable to adjust our business model to reflect the trend, our results of operations and cash flow will be impacted. Risks Related to Our Corporate Structure and Stock Our stock price has been and may continue to be extremely volatile due to many factors. The market price of our common stock has fluctuated in the past and may increase or decrease rapidly in the future depending on news announcements and changes in general market conditions. Between January 31, 2003 and November 1, 2004, the closing price of our common stock ranged from a low of $2.28 per share to a high of $10.57 per share. The following factors, among others, may cause significant fluctuations in our stock price: o news announcements regarding quarterly or annual results of operations; o monthly comparable store sales; o acquisitions; o competitive developments; o litigation affecting us; or o market views as to the prospects of the retail industry generally. Rights of our stockholders may be negatively affected if we issue any of the shares of preferred stock which our Board of Directors has authorized for issuance. We have available for issuance 1,000,000 shares of preferred stock, par value $.01 per share. Our board of directors is authorized to issue any or all of this preferred stock, in one or more series, without any further action on the part of shareholders. The rights of our shareholders may be negatively affected if we issue a series of preferred stock in the future that has preference over our common stock with respect to the payment of dividends or distribution upon our liquidation, dissolution or winding up. 9

State laws and our certificate of incorporation may inhibit potential acquisition bids that could be beneficial to our stockholders. We are subject to certain provisions of Delaware law which could delay or make more difficult a merger, tender offer or proxy contest involving us. In particular, Section 203 of the Delaware General Corporation Law prohibits a Delaware corporation from engaging in certain business combinations with any interested stockholder for a period of three years unless specific conditions are met. In addition, certain provisions of Delaware law could have the effect of delaying, deferring or preventing a change in control of us, including, without limitation, discouraging a proxy contest or making more difficult the acquisition of a substantial block of our common stock. The provisions could also limit the price that investors might be willing to pay in the future for shares of our common stock. For additional information, see the section of this prospectus entitled "Description of Capital Stock -- Antitakeover Effects of Provisions of Our Certificate of Incorporation, Our By-Laws and Delaware Law." Conversion of our 5% Senior Subordinated Notes could result in dilution to holders of our common stock. In the fourth quarter of fiscal 2003, we sold in a private transaction $100 million principal amount of convertible senior subordinated notes due 2024 (the "Convertible Notes"). If the holders of the Convertible Notes convert such notes, we would be required to issue to such holders approximately 9.39 million additional shares of common stock, which would result in dilution to holders of our common stock. Additionally, the 9.39 million additional shares of common stock would not have an impact on historical earnings per share of the Company because the conversion of such shares would have been antidilutive. 10

FORWARD LOOKING INFORMATION Certain statements contained in this prospectus constitute "forward-looking statements" within the meaning of the United States Private Securities Litigation Reform Act of 1995. In some cases, forward-looking statements can be identified by the use of forward-looking terminology such as "may," "will," "estimate," "intend," "continue," "believe," "expect" or "anticipate" or the negatives thereof, variations thereon or similar terminology. The forward-looking statements contained in this prospectus are generally located in the material set forth under the headings "Prospectus Summary" and "Risk Factors," but may be found in other locations as well. These forward-looking statements generally relate to plans and objectives for future operations and are based upon management's reasonable estimates of future results or trends. Although we believe that the plans and objectives reflected in or suggested by such forward-looking statements are reasonable, such plans or objectives may not be achieved. Actual results may differ from projected results due, but not limited, to unforeseen developments, including developments relating to the following: o overall economic and business conditions; o competitive factors in the industries in which we conduct our business; o changes in governmental regulation; o the demand for our goods and services; o the fact that our customers may cancel orders they have placed with us, in whole or in part, without advance notice; o changes in tax requirements, including tax rate changes, new tax laws and revised tax law interpretations; o changes in generally accepted accounting principles or interpretations of those principles by governmental agencies and self-regulatory groups; o developments in and results of litigation; o interest rate fluctuations, foreign currency rate fluctuations and other capital market conditions; o economic and political conditions in international markets, including governmental changes and restrictions on the ability to transfer capital across borders; o changes in the cost of raw materials used in our business; o the timing, impact and other uncertainties of acquisitions that we may consider or consummate; o our ability to achieve anticipated synergies and other cost savings in connection with such acquisitions; and o the other factors discussed under "Risk Factors" or elsewhere in this prospectus. You should read this prospectus completely and with the understanding that actual future results may be materially different from what we expect. All forward-looking statements contained in this prospectus and all subsequent written and oral forward-looking statements attributable to us or persons acting on our behalf are expressly qualified in their entirety by the foregoing factors. These forward-looking statements speak only as of the date on which they are made. We disclaim any obligation or undertaking to provide any updates or revisions to any forward-looking statement to reflect any change in our expectations or any change in events, conditions or circumstances on which the forward-looking statement is based. 11

USE OF PROCEEDS The selling securityholders will receive all of the net proceeds from the sales of shares of common stock pursuant to this prospectus, and the Company will receive none of the proceeds. DESCRIPTION OF CAPITAL STOCK We are authorized to issue a total of 76,000,000 shares, consisting of 75,000,000 shares of common stock and 1,000,000 shares of preferred stock. The following is a summary of some of the rights and privileges pertaining to our common stock. For a full description of our common stock and our preferred stock, you should refer to our certificate of incorporation and by-laws. Our Common Stock As of July 31, 2004, there were 34,207,108 shares of our common stock outstanding. The holders of our common stock are entitled to one vote per share on all matters to be voted upon by the stockholders. None of our common stockholders will be entitled to cumulate votes at any election of directors. Subject to preferences that are applicable to any series of our preferred stock that may come into existence in the future, the holders of our common stock are entitled to receive such dividends, if any, as may be declared from time to time by our board of directors out of legally available funds. In the event of our liquidation, dissolution or winding up, the holders of our common stock are entitled to share ratably in all assets remaining after payment of liabilities, subject to prior rights of any series of our preferred stock that may come into existence in the future. Holders of our common stock have no preemptive or conversion rights or other subscription rights. There are no redemption or sinking fund provisions available to the holders of our common stock. Our Preferred Stock Our board of directors has the authority, without action by the stockholders, to designate and issue preferred stock in one or more series and to designate the rights, preferences and privileges of each series, which may be greater than the rights of the common stock. It is not possible to state the actual effect of the issuance of any shares of preferred stock upon the rights of holders of the common stock until the board of directors determines the specific rights of the holders of such preferred stock. However the effects might include, among other things: o restricted dividends on the common stock; o diluting the voting power of the common stock; o impairing the liquidation rights of the common stock; or o delaying or preventing a fundamental change in control of our company without further action by our stockholders. Warrants As of July 31, 2004, there were outstanding warrants to purchase an aggregate of up to 3,377,871 shares of our common stock at a weighted average exercise price of $5.74 per share, including the warrants exercisable to purchase the common stock offered by this prospectus. These warrants are currently exercisable in full and expire between April 2007 and July 2010. 12

Options As of July 31, 2004, there were an aggregate of 3,067,433 shares of our common stock subject to outstanding options at a weighted average exercise price of $5.56 per share under our 1992 Stock Incentive Plan, as amended. In addition, as of July 31, 2004, there were an aggregate of 1,140,000 shares of our common stock issued outside of our 1992 Stock Incentive Plan subject to outstanding options at a weighted average exercise price of $3.40. As of July 31, 2004, 2,700,075 shares were reserved for future issuance upon exercise of options that may be granted under the 1992 plan. Registration Rights of Certain Holders From time to time, we have issued and may continue to issue shares of capital stock, warrants, convertible notes or other securities entitled to registration rights. All previously issued securities entitled to such registration rights, other than the shares of stock offered under this prospectus, have been registered pursuant to registration statements on Form S-3 filed in September 2002 and February 2004. Accordingly, all such previously granted registration rights have been satisfied. Antitakeover Effects of Provisions of Our Certificate of Incorporation, Our By-Laws and Delaware Law Provisions of our certificate of incorporation and by-laws, as well as provisions of Delaware law, could make it more difficult for a third party to acquire us and to remove incumbent officers and directors. These provisions, summarized below, are expected to discourage certain types of coercive takeover practices and inadequate takeover bids and to encourage persons seeking to acquire control of us to first negotiate with us. These provisions could discourage potential acquisition proposals and could delay or prevent a change in control. These provisions are also intended to enhance the likelihood of continuity and stability in the composition of our board of directors and in the policies formulated by our board of directors and to discourage certain types of transactions that may involve an actual or threatened change of control. These provisions are designed to reduce our vulnerability to an unsolicited acquisition proposal and to discourage certain tactics that may be used in proxy fights. These provisions could, however, have the effect of discouraging others from making tender offers for our shares and, as a consequence, they also may inhibit fluctuations in the market price of our shares that could result from actual or rumored takeover attempts. These provisions also may have the effect of preventing changes in our management. We believe that the benefits of increased protection of our ability to negotiate with the proponent of an unfriendly or unsolicited proposal to acquire or restructure us outweigh the disadvantages of discouraging those proposals because, among other things, negotiation of those proposals could result in an improvement of their terms. Preferred Stock. Our board of directors, without stockholder approval, has the authority under our certificate of incorporation to issue up to 1,000,000 shares of convertible preferred stock with rights superior to the rights of the holders of our common stock. As a result, preferred stock could be issued quickly and easily, could hurt the rights of holders of common stock and could be issued with terms calculated to delay or prevent a change of control or make removal of management more difficult. Stockholder Meetings. Certain provisions of our by-laws may have the effect of delaying, deferring or preventing a change of control through limitations on the right of stockholders to call, or determine the agenda for, special stockholder meetings, including a requirement for advance notification of stockholder proposals. Delaware Antitakeover Law. We are subject to Section 203 of the Delaware General Corporation Law, which prohibits a publicly held Delaware corporation from engaging in a business combination with an interested stockholder for a period of three years after the date of the transaction in which the person 13

becomes an interested stockholder, unless, before that date: (i) the board of directors of the corporation approves either the business combination or the transaction that resulted in the stockholder becoming an interested stockholder; (ii) upon consummation of the transaction that resulted in the stockholder becoming an interested stockholder, the interested stockholder owns at least 85% of the outstanding voting stock, excluding shares held by directors, officers and employee stock plans; or (iii) on or after the consummation date, the business combination is approved by the board of directors and by the affirmative vote at an annual or special meeting of stockholders of at least 66?% of the outstanding voting stock that is not owned by the interested stockholder. For purposes of Section 203, a business combination includes a merger, asset sale or other transaction resulting in a financial benefit to the interested stockholder. An interested stockholder is generally a person who, together with affiliates and associates of that person, (a) owns 15% or more of the corporation's voting stock or (b) is an affiliate or associate of the corporation and was the owner of 15% or more of the outstanding voting stock of the corporation at any time within the prior three years. Transfer Agent and Registrar The transfer agent and registrar for the common stock is American Stock Transfer and Trust Company. Its address is 59 Maiden Lane, Plaza Level, New York, NY 10038. Listing Our common stock is quoted on the Nasdaq National Market under the symbol "CMRG." 14

CERTAIN UNITED STATES FEDERAL TAX CONSIDERATIONS The following discusses the material U.S. federal income tax consequences to holders, and U.S. estate tax consequences to non-U.S. holders (defined below), relating to the ownership and disposition of common stock. This discussion is for general information only and does not address all aspects of U.S. federal income taxation that may be relevant to you in light of your personal circumstances. This summary is based on the provisions of the Internal Revenue Code of 1986, as amended (the "Code"), applicable existing and proposed U.S. Treasury regulations, and judicial authority and current administrative rulings and practice, all of which are subject to change, possibly on a retroactive basis, or to differing interpretation. Except as otherwise noted, this summary applies only to holders that hold our common stock as a capital asset within the meaning of Section 1221 of the Code (generally, for investment). It does not address tax consequences applicable to those U.S. holders that may be subject to special tax rules, including financial institutions, regulated investment companies, tax-exempt organizations, expatriates, persons subject to the alternative minimum tax provisions of the Code, pension funds, insurance companies, dealers in securities or foreign currencies, persons that will hold common stock as a position in a hedging transaction, straddle, conversion transaction or other risk reduction transaction for tax purposes, persons deemed to sell common stock under the constructive sale provisions of the Code, persons who hold common stock through a partnership or other pass through entity, or persons whose functional currency is not the U.S. dollar (except as disclosed below under "Non-U.S. Holders"). We have not sought any ruling from the Internal Revenue Service (the "IRS") with respect to the statements made and the conclusions reached in the following summary, and there can be no assurance that the IRS or a court will agree with our statements and conclusions. Moreover, this discussion does not address the effect of the federal estate and gift tax laws on U.S. holders or the effect of any applicable state, local or foreign tax laws. THE FOLLOWING DISCUSSION OF U.S. FEDERAL TAX CONSIDERATIONS IS FOR GENERAL INFORMATION ONLY. IT IS NOT TAX ADVICE. HOLDERS SHOULD CONSULT THEIR OWN TAX ADVISORS WITH RESPECT TO THE APPLICATION OF THE U.S. FEDERAL INCOME TAX AND ESTATE TAX LAWS TO THEIR PARTICULAR SITUATIONS AS WELL AS ANY TAX CONSEQUENCES ARISING UNDER THE LAWS OF ANY STATE, LOCAL OR FOREIGN TAXING JURISDICTION OR UNDER ANY APPLICABLE TAX TREATY. For purposes of this discussion, the term U.S. holder means a beneficial owner of common stock that is for U.S. federal income tax purposes: o a citizen or resident of the U.S.; o a corporation created or organized under the laws of the U.S. or any state; o an estate, the income of which is subject to U.S. federal income taxation regardless of its source; or o a trust if (a) its administration is subject to the primary supervision of a court within the U.S. and one or more U.S. persons have authority to control all of its substantial decisions, or (b) it has a valid election in effect under applicable Treasury regulations to be treated as a U.S. person. A non-U.S. holder means a holder of common stock (other than a partnership or entity treated as such for U.S. federal income tax purposes) that is not a U.S. holder for U.S. federal income tax purposes. Income earned through a foreign or domestic partnership or entity treated as such is generally attributed to its owners. A beneficial owner of common stock that is a partnership for U.S. federal income tax purposes, 15

and the partners in such a partnership, should consult their tax advisors about the U.S. federal income tax consequences of holding and disposing of the common stock. U.S. Holders Dividends Distributions received on our common stock will be treated as a dividend, subject to tax as ordinary income, to the extent of our current and accumulated earnings and profits as of the end of the year of distribution. For taxable years beginning after December 31, 2002 and before January 1, 2009, subject to certain exceptions, dividends received by individual shareholders generally would be taxed at the same preferential rates that apply to long-term capital gains. Any excess will be treated as a tax-free return of capital to the extent of the U.S. holder's adjusted tax basis in the common stock and thereafter as gain from the sale or exchange of that stock. Subject to applicable rules, U.S. holders that are corporations may be eligible to claim a deduction equal to a portion of any distributions received that are treated as dividends. Special rules may apply to corporate U.S. holders upon the receipt of any "extraordinary dividends" with respect to the common stock. Sale Upon the sale, exchange or other taxable disposition of our common stock, a U.S. holder will recognize capital gain or loss equal to the difference between (i) the amount of cash and the fair market value of any property received upon the disposition and (ii) the U.S. holder's adjusted tax basis in the common stock. Such capital gain or loss will be long-term if the U.S. holder's holding period is more than one year. Non-U.S. Holders Dividends If we make distributions on our common stock, those distributions generally will be treated as a dividend to the extent of our current and accumulated earnings and profits as of the end of the year of distribution. Subject to the discussion below of backup withholding, any such distribution treated as dividends to a non-U.S. holder generally will be subject to a 30% U.S. federal withholding tax, unless (i) the dividend is effectively connected with the conduct of a U.S. trade or business of the non-U.S. holder or a lower treaty rate applies and (ii) the non-U.S. holder provides us with proper certification as to the non-U.S. holder's exemption from, or as to the reduced rate of, withholding on Form W-8ECI or W-8BEN (or appropriate substitute form), respectively. If the dividend is effectively connected with the conduct of a U.S. trade or business, it will be subject to the U.S. federal income tax on net income that applies to U.S. persons generally and, under certain circumstances with respect to corporate holders, to the branch profits tax, which is generally imposed at a 30% rate, subject in each case to income tax treaty exceptions. Sale A non-U.S. holder will not be subject to U.S. federal income tax or withholding tax on gain realized on the sale, exchange, redemption or other disposition of common stock, unless: o in the case of an individual non-U.S. holder, that holder is present in the U.S. for 183 days or more in the year of the disposition and certain other requirements are met; or 16

o the gain is effectively connected with the conduct of a U.S. trade or business of the non-U.S. holder. If the gain is effectively connected to the conduct of a U.S. trade or business, it will be subject to the U.S. federal income tax on net income that applies to U.S. persons generally and, under certain circumstances with respect to corporate holders, to the branch profits tax, which is generally imposed at a 30% rate, subject in each case to income tax treaty exceptions. Notwithstanding the above, if we are or become a U.S. real property holding corporation (a "USRPHC"), a non-U.S. holder could be subject to federal income tax with respect to gain realized on the disposition of shares of common stock. Amounts withheld, if any, with respect to such gain pursuant to the rules applicable to dispositions of U.S. real property interests would be creditable against that non-U.S. holder's U.S. federal income tax liability and could entitle that non-U.S. holder to a refund upon furnishing required information to the IRS. In general, we would be a USRPHC if interests in U.S. real estate comprised most of our assets. We do not believe that we are a USRPHC or will become a USRPHC in the future. United States Federal Estate Tax Common stock actually or beneficially held by an individual who is not a citizen or resident of the U.S., as specifically defined for U.S. federal estate tax purposes, at the time of death (or who previously transferred such stock subject to certain retained rights or powers) will be subject to U.S. federal estate tax unless otherwise provided by an applicable estate tax treaty. Backup Withholding and Information Reporting Non-exempt U.S. holders will be subject to information reporting with respect to payments of dividends on, and possibly proceeds from the disposition of, common stock. Non-exempt U.S. holders who are subject to information reporting and who do not provide appropriate information when requested may be subject to backup withholding. U.S. holders should consult their tax advisors. Payments of dividends on common stock to non-U.S. holders will be subject to information reporting on Form 1042-S. If the common stock is held by a non-U.S. holder through a non-U.S., and non-U.S. related, broker or financial institution, backup withholding generally would not be required. Backup withholding may apply if the common stock is held by a non-U.S. holder through a U.S., or U.S. related, broker or financial institution and the non-U.S. holder fails to provide appropriate information. Non-U.S. holders should consult their tax advisors. Any amounts withheld from a payment under the backup withholding rules will be allowed as a refund or credit against a holder's federal income tax liability, provided that the required information is furnished to the IRS. Some holders (including, among others, U.S. corporations) are generally not subject to information reporting and backup withholding. 17

SELLING SECURITYHOLDERS The selling securityholders, including their transferees, pledgees or donees or their successors, may from time to time offer and sell a total of 1,818,504 shares of common stock under this prospectus, which amount consists of (i) 1,182,400 shares of common stock issued or issuable upon exercise of detachable warrants issued by us in private placements in 2003, (ii) 430,000 shares of common stock issuable upon exercise of options and (iii) 206,104 additional shares of common stock currently held. The following table sets forth certain information, as of June 22, 2004, about the selling securityholders for which we are registering common stock for resale to the public. To the best of our knowledge, none of the selling securityholders has any plan, arrangement, understanding, agreement or commitment to sell its securities. Within the past three years, the following persons have held the following positions or offices within Casual Male, or have had the following material relationship with the Company during such time: (a) Stephen Duff, who was a director of the Company from May 14, 2002 to February 26, 2004, is the Treasurer of Ninth Floor Corporation, the general partner of Clark Partners I, L.P., and is also the Chief Investment Officer of The Clark Estates, Inc., the beneficial holder of approximately 7.1% of the outstanding common stock of the Company; (b) Seymour Holtzman, the Chairman of the Company's Board of Directors and the beneficial holder of approximately 14.3% of the Company's outstanding common stock (principally held by Jewelcor Management, Inc.), is also the President and Chief Executive Officer, and indirectly, with his wife, the primary shareholder of Jewelcor Management, Inc., which is also party to a consulting agreement with the Company; (c) Baron Small Cap Fund is a series of Baron Asset Fund, which is an affiliate of Baron Capital Group, Inc., the beneficial holder of approximately 6.3% of the outstanding common stock of the Company; (d) Marc Holtzman is the son of Seymour Holtzman, the Chairman of the Company's Board of Directors; (e) Robert L. Patron, who was a director of the Company from October 1999 to May 2002, is the president of Business Ventures International Inc.; (f) Rose Gerszberg, Efrem Gerszberg, Seth Gerszberg, Marc Ecko, Marci Tapper, Donniel Zinkin, Ephraim and Devora Zinkin and Alberto Verdi are members of the family that owns the equity interests in, and Seth Gerszberg is the Chief Executive Officer of, Ecko Complex, LLC, with which the Company operated a joint venture from March 2002 through July 31, 2004; (g) Frank J. Husic, a director of the Company since June 30, 2003, is the controlling person with respect to the Frank J. Husic Rollover IRA and is also the sole shareholder and general partner of Husic Capital Management, the beneficial holder of approximately 6.5% of the outstanding common stock of the Company; (h) Alan S. Bernikow has been a director of the Company since June 30, 2003; (i) each of Raymond Perlman, Wayne T. Green and Efrem Gerszberg is an employee of an affiliate of Jewelcor Management, Inc., the beneficial holder of approximately 12.1% of the outstanding common stock of the Company and a party to a consulting agreement with the Company; (j) George Foreman, who was a director of the Company from March to April 2004, is the Company's spokesperson; and (k) Diane and Hugh Unger, the trustees of the Diane E. Unger Inter Vivos Trust are the parents of Jeff Unger, the Company's Vice President of Investor Relations . The information regarding the selling securityholders' beneficial ownership after the sales made pursuant to this prospectus assumes that all of the shares of common stock subject to sale pursuant to this prospectus shall have been sold. Each of the selling securityholders has provided the information set forth below relating to the number of shares such securityholder currently beneficially owns. The shares subject to sale pursuant to this prospectus may be offered from time to time, in whole or in part, by the selling securityholders or their transferees. Information about the selling securityholders may change over time. Any changed information given to us by the selling securityholders will be set forth in prospectus supplements if and when necessary. 18

Number of Shares of Common Stock Beneficially Common Stock Number of Shares Owned After the Sales Beneficially Owned of Common Stock --------------------------- Selling Securityholders Before Any Sale (1) Subject to Sale Number** Percent*** - --------------------------------------------- ------------------- ---------------- --------- ----------- Clark Partners I, L.P. (2) 2,446,324 100,000 2,346,324 6.8% - ------------------------------------------------------------------------------------------------------------------------- Jewelcor Management, Inc. (3) 4,123,973 286,104 3,837,869 11.1% - ------------------------------------------------------------------------------------------------------------------------- Baron Small Cap Fund (4) 2,257,353 100,000 2,157,353 6.0% - ------------------------------------------------------------------------------------------------------------------------- Paul R. Mancia DDS P.C. Profit Sharing Plan (5) 84,066 10,000 74,066 * - ------------------------------------------------------------------------------------------------------------------------- Marc L. Holtzman (6) 306,300 20,000 286,300 * - ------------------------------------------------------------------------------------------------------------------------- Benchmark Partners, L.P. (7) 176,000 26,000 150,000 * - ------------------------------------------------------------------------------------------------------------------------- Family Trust created under the duly probated Last Will and Testament of Samuel L. Lane, Deceased (8) 4,000 4,000 0 0 - ------------------------------------------------------------------------------------------------------------------------- Prism Partners I, L.P. (9) 20,000 20,000 0 0 - ------------------------------------------------------------------------------------------------------------------------- Prism Partners II Offshore Fund (10) 20,000 20,000 0 0 - ------------------------------------------------------------------------------------------------------------------------- Allen Brill (11) 44,000 4,000 40,000 0 - ------------------------------------------------------------------------------------------------------------------------- Diane E. Unger Inter Vivos Trust (12) 2,000 2,000 0 0 - ------------------------------------------------------------------------------------------------------------------------- Business Ventures International Inc. (13) 110,000 10,000 100,000 * - ------------------------------------------------------------------------------------------------------------------------- Rose Gerszberg (14) 20,000 20,000 0 0 - ------------------------------------------------------------------------------------------------------------------------- Efrem Gerszberg (15) 10,000 10,000 0 0 - ------------------------------------------------------------------------------------------------------------------------- Seth Gerszberg (16) 20,000 20,000 0 0 - ------------------------------------------------------------------------------------------------------------------------- Marc Ecko (17) 20,000 20,000 0 0 - ------------------------------------------------------------------------------------------------------------------------- Marci Tapper (18) 20,000 20,000 0 0 - ------------------------------------------------------------------------------------------------------------------------- Lawrence Seidman Retirement Plan and Trust (19) 4,000 4,000 0 0 - ------------------------------------------------------------------------------------------------------------------------- Frank J. Husic Rollover IRA (20) 20,000 20,000 0 0 - ------------------------------------------------------------------------------------------------------------------------- Alan S. Bernikow (21) 26,802 4,000 22,802 * - ------------------------------------------------------------------------------------------------------------------------- WEC Partners, L.P. (22) 46,600 10,000 36,600 * - ------------------------------------------------------------------------------------------------------------------------- Far West Capital Partners, L.P. (23) 527,639 80,000 447,639 - ------------------------------------------------------------------------------------------------------------------------- Robert G. Schiro 2001 Trust (24) 97,000 16,000 81,000 * - ------------------------------------------------------------------------------------------------------------------------- Alberto Verdi (25) 6,000 6,000 0 0 - ------------------------------------------------------------------------------------------------------------------------- Harbour Holdings Ltd. (26) 164,000 164,000 0 0 - ------------------------------------------------------------------------------------------------------------------------- Strong Special Investment, L.P. (27) 116,000 116,000 0 0 - ------------------------------------------------------------------------------------------------------------------------- Ephraim Zinkin and Devora Zinkin (28) 2,000 2,000 0 0 - ------------------------------------------------------------------------------------------------------------------------- Donniel Zinkin (29) 2,000 2,000 0 0 - ------------------------------------------------------------------------------------------------------------------------- Raymond Perlman (30) 11,000 4,000 7,000 * - ------------------------------------------------------------------------------------------------------------------------- Wayne T. Green (31) 12,000 2,000 10,000 * - ------------------------------------------------------------------------------------------------------------------------- FPA Crescent Fund (32) 60,000 60,000 0 0 - ------------------------------------------------------------------------------------------------------------------------- Zeke LP (33) 230,000 80,000 150,000 * - ------------------------------------------------------------------------------------------------------------------------- Hourglass Master Fund, Ltd. (34) 120,000 120,000 0 0 - ------------------------------------------------------------------------------------------------------------------------- George Foreman (35) 215,000 100,000 115,000 * - ------------------------------------------------------------------------------------------------------------------------- Seymour Holtzman (36) 4,933,405 330,000 4,603,405 13.3% - ------------------------------------------------------------------------------------------------------------------------- - ------------------------------- *Less than 1%. 19

**Assumes that the selling securityholders will sell all of their shares of common stock subject to sale pursuant to this prospectus. We cannot assure you that the selling securityholders will sell all or any of their shares of common stock. ***Percentage ownership is based on 34,199,109 shares of common stock outstanding as of June 22, 2004, plus securities deemed to be outstanding with respect to individual stockholders pursuant to Rule 13d-3(d)(1) under the Exchange Act. (1) Beneficial ownership is determined in accordance with the rules of the Securities and Exchange Commission and includes generally voting and/or investment power with respect to securities. Shares of common stock subject to warrants, options or convertible stock currently exercisable or convertible, or exercisable or convertible within 60 days of June 22, 2004 are deemed outstanding for the purpose of computing the percentage beneficially owned by the person holding such warrants, options or convertible stock but are not deemed outstanding for the purpose of computing the percentage beneficially owned by any other person. (2) The amount shown as beneficially owned before any sale includes, and the amount shown as subject to sale consists of, 100,000 shares of common stock issuable upon exercise of a warrant. The Board of Directors of Ninth Floor Corporation, the general partner of Clark Partners I., L.P., exercises sole voting and dispositive power with respect to the shares subject to sale underlying the warrant owned by Clark Partners I., L.P., which warrant was acquired in the ordinary course of business. (3) The amount shown as beneficially owned before any sale includes, and the amount shown as subject to sale consists of, 80,000 shares of common stock issuable upon exercise of a warrant and 206,104 shares of common stock currently held by Jewelcor Management, Inc. as compensation under a consulting agreement with the Company. Seymour Holtzman is the Chairman, President and Chief Executive Officer of, and indirectly with his wife the primary shareholder of, Jewelcor Management, Inc., and, in such capacities, exercises sole voting and dispositive power with respect to the shares subject to sale currently held by Jewelcor Management, Inc. and underlying the warrant owned by Jewelcor Management, Inc., which warrant was acquired in the ordinary course of business. (4) The amount shown as beneficially owned before any sale includes 1,507,353 shares of common stock issuable upon exercise of warrants, and the amount shown as subject to sale consists of 100,000 shares of common stock issuable upon exercise of a warrant. Clifford Greenberg is the Portfolio Manager of Bamco Inc., the investment adviser to Baron Small Cap Fund, and, in such capacity, exercises sole voting and dispositive power with respect to the shares subject to sale underlying the warrant owned by Baron Small Cap Fund, which warrant was acquired in the ordinary course of business. (5) The amount shown as beneficially owned before any sale includes, and the amount shown as subject to sale consists of, 10,000 shares of common stock issuable upon exercise of a warrant. Paul R. Mancia, the President, Secretary and Treasurer of the Paul R. Mancia DDS P.C. Profit Sharing Plan, exercises sole voting and dispositive power with respect to the shares subject to sale underlying the warrant owned by the Paul R. Mancia DDS P.C. Profit Sharing Plan, which warrant was acquired in the ordinary course of business. (6) The amount shown as beneficially owned before any sale includes, and the amount shown as subject to sale consists of, 20,000 shares of common stock issuable upon exercise of a warrant. (7) The amount shown as beneficially owned before any sale includes, and the amount shown as subject to sale consists of, 26,000 shares of common stock issuable upon exercise of a warrant. Richard Whitman and Lorraine DiPaolo, the general partners of Benchmark Partners, L.P., exercise shared voting and dispositive power with respect to the shares subject to sale underlying the warrant owned by Benchmark Partners, L.P., which warrant was acquired in the ordinary course of business. (8) The amount shown as beneficially owned before any sale includes, and the amount shown as subject to sale consists of, 4,000 shares of common stock issuable upon exercise of a warrant. 20

(9) The amounts shown as beneficially owned before any sale and shown as subject to sale consist of 20,000 shares of common stock issuable upon exercise of a warrant. Jerald Weintraub is the manager of Weintraub Capital Management LLC, the general partner of Prism Partners I, L.P., and, in such capacity, exercises sole voting and dispositive power with respect to the shares subject to sale underlying the warrant owned by Prism Partners I, L.P., which warrant was acquired in the ordinary course of business. (10) The amounts shown as beneficially owned before any sale and shown as subject to sale consist of 20,000 shares of common stock issuable upon exercise of a warrant. Jerald Weintraub is the manager of Weintraub Capital Management LLC, the investment advisor of Prism Partners II Offshore Fund, and, in such capacity, exercises sole voting and dispositive power with respect to the shares subject to sale underlying the warrant owned by Prism Partners II Offshore Fund, which warrant was acquired in the ordinary course of business. (11) The amount shown as beneficially owned before any sale includes, and the amount shown as subject to sale consists of, 4,000 shares of common stock issuable upon exercise of a warrant. (12) The amount shown as beneficially owned before any sale includes, and the amount shown as subject to sale consists of, 2,000 shares of common stock issuable upon exercise of a warrant. Diane E. Unger and Hugh Unger are the trustees of the Diane E. Unger Inter Vivos Trust and, in such capacity, exercise shared voting and dispositive power with respect to the shares subject to sale underlying the warrant owned by the Diane E. Unger Inter Vivos Trust, which warrant was acquired in the ordinary course of business. (13) The amount shown as beneficially owned before any sale includes, and the amount shown as subject to sale consists of, 10,000 shares of common stock issuable upon exercise of a warrant. Robert L. Patron, the President of Business Ventures International Inc., exercises sole voting and dispositive power with respect to the shares subject to sale underlying the warrant owned by Business Ventures International Inc., which warrant was acquired in the ordinary course of business. (14) The amounts shown as beneficially owned before any sale and shown as subject to sale consist of 20,000 shares of common stock issuable upon exercise of a warrant. (15) The amounts shown as beneficially owned before any sale and shown as subject to sale consist of 10,000 shares of common stock issuable upon exercise of a warrant. (16) The amounts shown as beneficially owned before any sale and shown as subject to sale consist of 20,000 shares of common stock issuable upon exercise of a warrant. (17) The amounts shown as beneficially owned before any sale and shown as subject to sale consist of 20,000 shares of common stock issuable upon exercise of a warrant. (18) The amounts shown as beneficially owned before any sale and shown as subject to sale consist of 20,000 shares of common stock issuable upon exercise of a warrant. (19) The amount shown as beneficially owned before any sale includes, and the amount shown as subject to sale consists of, 4,000 shares of common stock issuable upon exercise of a warrant. Lawrence Seidman, the Trustee for Lawrence Seidman Retirement Plan and Trust, exercises sole voting and dispositive power with respect to the shares subject to sale underlying the warrant owned by Lawrence Seidman Retirement Plan and Trust, which warrant was acquired in the ordinary course of business. (20) The amount shown as beneficially owned before any sale includes, and the amount shown as subject to sale consists of, 20,000 shares of common stock issuable upon exercise of a warrant. Frank Husic, the managing partner of the Frank J. Husic Rollover IRA, exercises sole voting and dispositive power with respect to the shares subject to sale underlying the warrant owned by the Frank J. Husic Rollover IRA, which warrant was acquired in the ordinary course of business. (21) The amount shown as beneficially owned before any sale includes 4,000 shares of common stock issuable upon exercise of a warrant and 20,000 shares of common stock issuable upon exercise of options. The 21

amount shown as subject to sale consists of, 4,000 shares of common stock issuable upon exercise of a warrant. (22) The amount shown as beneficially owned before any sale includes, and the amount shown as subject to sale consists of, 10,000 shares of common stock issuable upon exercise of a warrant. Warren E. Clifford, the general partner of WEC Partners, LP, exercises sole voting and dispositive power with respect to the shares subject to sale underlying the warrant owned by WEC Partners, LP, which warrant was acquired in the ordinary course of business. (23) The amount shown as beneficially owned before any sale includes, and the amount shown as subject to sale consists of, 40,000 shares of common stock issuable upon exercise of a warrant. Robert G. Schiro is the General Partner of Far West Capital Management, LP, the General Partner of Far West Capital Partners, L.P., and, in such capacity, exercises sole voting and dispositive power with respect to the shares subject to sale underlying the warrant owned by Far West Capital Partners, L.P., which warrant was acquired in the ordinary course of business. (24) The amount shown as beneficially owned before any sale includes, and the amount shown as subject to sale consists of, 16,000 shares of common stock issuable upon exercise of a warrant. Robert G. Schiro, the Trustee for Robert G. Schiro 2001 Trust, exercises sole voting and dispositive power with respect to the shares subject to sale underlying the warrant owned by Robert G. Schiro 2001 Trust, which warrant was acquired in the ordinary course of business. (25) The amounts shown as beneficially owned before any sale and shown as subject to sale consist of 6,000 shares of common stock issuable upon exercise of a warrant. (26) The amounts shown as beneficially owned before any sale and shown as subject to sale consist of 164,000 shares of common stock issuable upon exercise of a warrant. Charles A. Parquelet is the Portfolio Manager of Skylands Capital, LLC, the Investment Advisor to Harbour Holdings Ltd., and, in such capacity, exercises sole voting and dispositive power with respect to the shares subject to sale underlying the warrant owned by Harbour Holdings Ltd., which warrant was acquired in the ordinary course of business. (27) The amounts shown as beneficially owned before any sale and shown as subject to sale consist of 116,000 shares of common stock issuable upon exercise of a warrant. Charles A. Parquelet is the Portfolio Manager of Skylands Capital, LLC, the Investment Advisor to Strong Special Investment, L.P., and, in such capacity, exercises sole voting and dispositive power with respect to the shares subject to sale underlying the warrant owned by Strong Special Investment, L.P., which warrant was acquired in the ordinary course of business. (28) The amounts shown as beneficially owned before any sale and shown as subject to sale consist of 2,000 shares of common stock issuable upon exercise of a warrant. Ephraim and Devora Zinkin exercise joint voting and dispositive power with respect to the shares subject to sale underlying the warrant jointly owned by them. (29) The amounts shown as beneficially owned before any sale includes, and the amount shown as subject to sale consists of, 2,000 shares of common stock issuable upon exercise of a warrant. (30) The amount shown as beneficially owned before any sale includes, and the amount shown as subject to sale consists of, 4,000 shares of common stock issuable upon exercise of a warrant. (31) The amount shown as beneficially owned before any sale includes, and the amount shown as subject to sale consists of, 2,000 shares of common stock issuable upon exercise of a warrant. (32) The amounts shown as beneficially owned before any sale and shown as subject to sale consist of 60,000 shares of common stock issuable upon exercise of a warrant. Steve Romick, the President of FPA Crescent Fund and the Senior Vice President of First Pacific Advisors, Inc., the investment adviser to FPA Crescent Fund, exercises sole voting and dispositive power with respect to the shares subject to sale underlying the warrant owned by FPA Crescent Fund, which warrant was acquired in the ordinary course of business. 22

(33) The amount shown as beneficially owned before any sale includes, and the amount shown as subject to sale consists of, 80,000 shares of common stock issuable upon exercise of a warrant. Ed Antoian, the general partner of Zeke LP, exercises sole voting and dispositive power with respect to the shares subject to sale underlying the warrant owned by Zeke LP, which warrant was acquired in the ordinary course of business. (34) The amounts shown as beneficially owned before any sale and shown as subject to sale consist of 120,000 shares of common stock issuable upon exercise of a warrant. John Barton is the Managing Member of Tablerock Fund Management, LLC, the Advisor to Hourglass Master Fund, Ltd., and, in such capacity, exercises sole voting and dispositive power with respect to the shares subject to sale underlying the warrant owned by Hourglass Master Fund, Ltd., which warrant was acquired in the ordinary course of business. (35) The amount shown as beneficially owned before any sale consists of 215,000 shares of common stock issuable upon exercise of stock options, and the amount shown as subject to sale consists of 100,000 shares of common stock issuable upon exercise of stock options. (36) The amount shown as beneficially owned before any sale includes 4,123,973 shares owned of record by Jewelcor Management, Inc., of which Mr. Holtzman is the Chairman, President and Chief Executive Officer and, indirectly with his wife, the primary shareholder. The amount shown as beneficially owned before any sale includes, and the amount shown as subject to sale consists of, 330,000 shares of common stock issuable upon exercise of stock options. The amount shown as beneficially owned before any sale does not include 66,667 shares currently exercisable pursuant to stock options granted to Mr. Holtzman on July 15, 2004. 23

PLAN OF DISTRIBUTION The selling securityholders and their successors, which include their pledgees, donees, partnership distributees and other transferees receiving the warrants or the common stock from the selling securityholders in non-sale transfers, may sell the common stock directly to purchasers or through underwriters, broker-dealers or agents. Underwriters, broker-dealers or agents may receive compensation in the form of discounts, concessions or commissions from the selling securityholders or the purchasers. These discounts, concessions or commissions may be in excess of those customary in the types of transactions involved. The common stock may be sold in one or more transactions at: o fixed prices that may be changed; o prevailing market prices at the time of sale; o prices related to the prevailing market prices; o varying prices determined at the time of sale; or o negotiated prices. These sales may be effected in transactions, which may involve cross or block transactions, in the following manner: o on any national securities exchange or quotation service on which the common stock may be listed or quoted at the time of sale; o in the over-the-counter-market; o in transactions otherwise than on these exchanges or services or in the over-the-counter market (privately negotiated transactions); o through the writing and exercise of options, whether these options are listed on an options exchange or otherwise; or o through any combination of the foregoing. Selling securityholders may enter into hedging transactions with broker-dealers or other financial institutions which may in turn engage in short sales of the common stock and deliver these securities to close out short positions. In addition, the selling securityholders may sell the common stock short and deliver the common stock to close out short positions or loan or pledge the common stock to broker-dealers that in turn may sell such securities. Selling securityholders may sell or transfer their shares of common stock other than by means of this prospectus. In particular, any securities covered by this prospectus that qualify for sale pursuant to Rule 144 under the Securities Act may be sold thereunder, rather than pursuant to this prospectus. The aggregate proceeds to the selling securityholders from the sale of the common stock will be the purchase price of the common stock less any discounts and commissions. A selling securityholder reserves the right to accept and, together with its agents, to reject any proposed purchase of common stock to be made directly or through agents. We will not receive any of the proceeds from this offering. In order to comply with the securities laws of some jurisdictions, if applicable, the holders of common stock may sell in some jurisdictions through registered or licensed broker-dealers. In addition, under certain circumstances in some jurisdictions, the holders of the common stock may be required to 24

register or qualify the securities for sale or comply with an available exemption from the registration and qualification requirements. Our common stock is quoted on the Nasdaq National Market. The selling securityholders and any underwriters, broker-dealers or agents that participate in the sale of the common stock may be deemed to be "underwriters" within the meaning of the Securities Act. In this case, any discounts, commissions, concessions or profit they earn on any resale of the shares of the common stock may be underwriting discounts and commissions under the Securities Act. In addition, selling securityholders who are deemed to be "underwriters" within the meaning of the Securities Act will be subject to the prospectus delivery requirements of the Securities Act and may be subject to statutory liabilities, including, but not limited to, liabilities under Sections 11, 12 and 17 of the Securities Act. The selling securityholders and any other persons participating in the distribution of the common stock will be subject to applicable provisions of the Exchange Act and the rules and regulations thereunder. Regulation M of the Exchange Act may limit the timing of purchases and sales of the common stock by the selling securityholders and any such other person. In addition, Regulation M may restrict the ability of any person participating in the distribution to engage in market-making activities with respect to the particular securities being distributed for a period of up to five business days prior to the commencement of the distribution. This may affect the marketability of the common stock. If required, the specific common stock to be sold, the names of the selling securityholders, the respective purchase prices and public offering prices, the names of any agent, dealer or underwriter and any applicable commissions or discounts with respect to a particular offer will be set forth in an accompanying prospectus supplement or, if appropriate, a post-effective amendment to the registration statement of which this prospectus is a part. LEGAL MATTERS The validity of the common stock issuable upon exercise of the warrants issued to the selling securityholders is being passed upon for us by Kramer Levin Naftalis & Frankel LLP, New York, New York. EXPERTS The consolidated financial statements of Casual Male Retail Group, Inc. appearing in Casual Male's Annual Report (Form 10-K) for the year ended January 31, 2004 (fiscal 2003), have been audited by Ernst & Young LLP, independent registered public accounting firm, as set forth in their report thereon included therein and incorporated herein by reference. Such consolidated financial statements have been incorporated herein by reference in reliance upon such report given on the authority of such firm as experts in accounting and auditing. 25

WHERE YOU CAN FIND MORE INFORMATION We file annual, quarterly and current reports, proxy statements and other information with the SEC. You may read and copy materials that we have filed with the SEC at the SEC's public reference room located at 450 Fifth Street, N.W., Room 1024, Washington, D.C. 20549. Please call the SEC at 1-800-SEC-0330 for further information on the public reference room. Our SEC filings also are available to the public on the SEC's web site at www.sec.gov, which contains reports, proxies and information statements and other information regarding issuers that file electronically. Access to this information as well as other information on the Company is also available on the Company's corporate website at http://www.cmrginc.com and clicking on "Investor Relations." This prospectus "incorporates by reference" information that we have filed with or furnished to the SEC under the Exchange Act, which means that we are disclosing important information to you by referring you to those documents. Any statement contained in this prospectus or in any document incorporated or deemed to be incorporated by reference into this prospectus will be deemed to be modified or superseded for purposes of this prospectus to the extent that a statement contained in this prospectus or any subsequently filed document which also is, or is deemed to be, incorporated by reference into this prospectus modifies or supersedes that statement. Any statement so modified or superseded will not be deemed, except as so modified or superseded, to constitute a part of this prospectus. We incorporate by reference into this prospectus the following documents that we have previously filed with the SEC and any future filings that we make with the SEC under Sections 13(a), 13(c), 14 or 15(d) of the Exchange Act after the date of this prospectus until all of the securities covered by this prospectus are sold by the selling securityholders: o Our Annual Report on Form 10-K for the fiscal year ended January 31, 2004; o Our Quarterly Reports on Form 10-Q for the fiscal quarters ended May 1, 2004 and July 31, 2004; o Our Current Reports on Form 8-K filed on March 25, 2004, April 9, 2004, April 14, 2004, May 11, 2004, May 20, 2004, June 4, 2004, July 8, 2004, August 4, 2004, August 11, 2004, August 19, 2004, September 2, 2004, October 7, 2004, November 4, 2004 and November 18, 2004 (excluding information deemed "furnished" as opposed to "filed" under the rules of the SEC); and o All other reports filed by us pursuant to Section 13(a) or 15(d) of the Exchange Act since the end of the fiscal year covered by the annual report referred to above. You may request a copy of these filings at no cost, by writing or telephoning us at the following address: Casual Male Retail Group, Inc. 555 Turnpike Street Canton, Massachusetts 02021, Attn: Dennis R. Hernreich (781) 828-9300 26

PART II INFORMATION NOT REQUIRED IN THE PROSPECTUS Item 14. Other Expenses Of Issuance And Distribution The registrant is paying certain of the expenses related to this offering. The following table sets forth the approximate amount of fees and expenses payable by the registrant in connection with this Registration Statement and the distribution of the shares of the securities being registered hereby. The selling securityholders will bear all underwriting discounts, commissions or fees attributable to the sale of the registrable securities. SEC registration fee $1,366.30 Legal fees and expenses $10,000.00 Accounting fees and expenses $10,000.00 Printing and engraving expenses $5,000.00 Miscellaneous $537.99 ---------- Total $27,500.00 Item 15. Indemnification of Directors and Officers The General Corporation Law of the State of Delaware, as currently in effect, permits charter provisions eliminating the liability of directors for breach of fiduciary duty, except that directors remain liable for (i) any breach of the directors' duty of loyalty to a company or its stockholders, (ii) acts or omissions not in good faith or which involve intentional misconduct or a knowing violation of law, (iii) any payment of a dividend or approval of a stock repurchase that is illegal under Section 174 of the Delaware General Corporation Law, or (iv) any transaction from which the directors derived an improper personal benefit. The registrant's Restated Certificate of Incorporation, as amended (the "Certificate"), provides that no director of the registrant shall be personally liable to the registrant or to any of its stockholders for monetary damages arising out of such director's breach of fiduciary duty, except to the extent that the elimination or limitation of liability is not permitted by the Delaware General Corporation Law. The effect of this provision of the Certificate is that directors cannot be held liable for monetary damages arising from breaches of their duty of care, unless the breach involves one of the four exceptions described in the first sentence of this Item 15. The provision does not prevent stockholders from obtaining injunctive or other equitable relief against directors, nor does it shield directors from liability under federal or state securities laws. The Certificate and the registrant's By-Laws further provide for indemnification of the registrant's directors and officers to the fullest extent permitted by Section 145 of the Delaware General Corporation Law, including circumstances in which indemnification is otherwise discretionary. Insofar as indemnification for liabilities arising under the Securities Act of 1933 may be permitted to directors, officers and controlling persons of the registrant, pursuant to the foregoing provisions, or otherwise, the registrant has been advised that in the opinion of the Securities and Exchange Commission such indemnification is against public policy as expressed in the Act and is, therefore, unenforceable. In the event that a claim for indemnification against such liabilities (other than the payment by the registrant of expenses incurred or paid by a director, officer or controlling person of the registrant in the successful defense of any action, suit or proceeding) is asserted by such director, officer or controlling person in connection with the securities being registered, the registrant will, unless in the opinion of its counsel the matter has been settled by controlling precedent, submit to a court of II-1

appropriate jurisdiction the question whether such indemnification by it is against public policy as expressed in the Act and will be governed by the final adjudication of such issue. Item 16. Exhibits Exhibit No. Description ----------- ---------------------------------------------------------------- 3.1 Restated Certificate of Incorporation of the Company, as amended (included as Exhibit 3.1 to Amendment No. 3 of the Company's Registration Statement on Form S-1 (No. 33-13402), and incorporated herein by reference).* 3.2 Certificate of Amendment to Restated Certificate of Incorporation, as amended, dated June 22, 1993 (included as Exhibit 3.2 to the Company's Quarterly Report on Form 10-Q filed on June 18, 1996, and incorporated herein by reference).* 3.3 Certificate of Amendment to Restated Certificate of Incorporation, as amended, dated August 8, 2002 (included as Exhibit 3.3 to the Company's Quarterly Report on Form 10-Q filed on September 17, 2002, and incorporated herein by reference).* 4.1 Form of Warrant to Purchase Common Stock.* 4.2 Form of Non-Qualified Stock Option Grant Agreement.* 5.1 Opinion of Kramer Levin Naftalis & Frankel LLP. 23.1 Consent of Ernst & Young LLP. 23.2 Consent of Kramer Levin Naftalis & Frankel LLP (included in Exhibit 5.1). 24.1 Power of Attorney.* * Previously filed. Item 17. Undertakings (a) The undersigned registrant hereby undertakes: (1) To file, during any period in which offers or sales are being made, a post-effective amendment to this registration statement; (i) To include any prospectus required by Section 10(a)(3) of the Securities Act of 1933; (ii) To reflect in the prospectus any facts or events arising after the effective date of this registration statement (or the most recent post-effective amendment thereof) which, individually or in the aggregate, represent a fundamental change in the information set forth in this registration statement. Notwithstanding the foregoing, any increase or decrease in volume of securities offered (if the total dollar value of securities offered would not exceed that which was registered) and any deviation from the low or high end of the estimated maximum aggregate offering range may be reflected in the form of prospectus filed with the Commission pursuant to Rule 424(b) if, in the aggregate, the changes in volume and price represent no more than 20 percent change in the maximum aggregate offering price set forth in the "Calculation of Registration Fee" table in the effective registration statement; (iii) To include any material information with respect to the plan of distribution not previously disclosed in this registration statement or any material change to such information in this registration statement; II-2

provided, however, that paragraphs (a)(1)(i) and (a)(1)(ii) do not apply if the information required to be included in a post-effective amendment by those paragraphs is contained in periodic reports filed by the registrant pursuant to Section 13 or 15(d) of the Securities Exchange Act of 1934 that are incorporated by reference in this registration statement. (2) That, for the purpose of determining any liability under the Securities Act of 1933, each such post-effective amendment shall be deemed to be a new registration statement relating to the securities offered therein, and the offering of such securities at that time shall be deemed to be the initial bona fide offering thereof. (3) To remove from registration by means of a post-effective amendment any of the securities being registered which remain unsold at the termination of the offering. (b) The undersigned registrant hereby undertakes that, for purposes of determining any liability under the Securities Act of 1933, each filing of the registrant's annual report pursuant to Section 13(a) or 15(d) of the Securities Exchange Act of 1934 (and, where applicable, each filing of an employee benefit plan's annual report pursuant to Section 15(d) of the Securities Exchange Act of 1934) that is incorporated by reference in the registration statement shall be deemed to be a new registration statement relating to the securities offered therein, and the offering of such securities at that time shall be deemed to be the initial bona fide offering thereof. II-3

SIGNATURES Pursuant to the requirements of the Securities Act of 1933, the registrant certifies that it has reasonable grounds to believe that it meets all of the requirements for filing on Amendment No. 1 to Form S-3 and has duly caused this registration statement to be signed on its behalf by the undersigned, thereunto duly authorized, in the City of Canton, Commonwealth of Massachusetts, as of the 22nd day of November, 2004. CASUAL MALE RETAIL GROUP, INC. By: /s/ Dennis R. Hernreich ----------------------------------------- Name: Dennis R. Hernreich Title: Executive Vice President, Chief Operating Officer, Chief Financial Officer and Treasurer (Principal Financial Officer and Principal Accounting Officer) Pursuant to the requirements of the Securities Act of 1933, this Amendment No. 1 to the Registration Statement has been signed by the following persons as of November 22, 2004 in the capacities indicated below. Signatures *_________________________ Director, President and Chief Executive David A. Levin Officer (Principal Executive Officer) /s/ Dennis R. Hernreich Executive Vice President, Chief Operating - -------------------------- Officer, Chief Financial Officer and Dennis R. Hernreich Treasurer (Principal Financial Officer and Principal Accounting Officer) *_________________________ Chairman of the Board of Directors Seymour Holtzman *_________________________ Director James Frain *_________________________ Director George T. Porter, Jr. *_________________________ Director Joseph Pennacchio

*_________________________ Director Alan S. Bernikow *_________________________ Director Jesse H. Choper *_________________________ Director Frank J. Husic *_________________________ Director Robert L. Sockolov /s/ Dennis R. Hernreich - -------------------------------------------- * By Dennis R. Hernreich, attorney-in-fact

EXHIBIT INDEX Exhibit No. Description ----------- ----------- 3.1 Restated Certificate of Incorporation of the Company, as amended (included as Exhibit 3.1 to Amendment No. 3 of the Company's Registration Statement on Form S-1 (No. 33-13402), and incorporated herein by reference).* 3.2 Certificate of Amendment to Restated Certificate of Incorporation, as amended, dated June 22, 1993 (included as Exhibit 3.2 to the Company's Quarterly Report on Form 10-Q filed on June 18, 1996, and incorporated herein by reference).* 3.3 Certificate of Amendment to Restated Certificate of Incorporation, as amended, dated August 8, 2002 (included as Exhibit 3.3 to the Company's Quarterly Report on Form 10-Q filed on September 17, 2002, and incorporated herein by reference).* 4.1 Form of Warrant to Purchase Common Stock.* 4.2 Form of Non-Qualified Stock Option Grant Agreement.* 5.1 Opinion of Kramer Levin Naftalis & Frankel LLP. 23.1 Consent of Ernst & Young LLP. 23.2 Consent of Kramer Levin Naftalis & Frankel LLP (included in Exhibit 5.1). * Previously filed.

                                                                     Exhibit 5.1


                                     November 22, 2004

Casual Male Retail Group, Inc.
555 Turnpike Street
Canton, Massachusetts 02021

            Re:   Registration Statement on Form S-3
                  ----------------------------------

Ladies and Gentlemen:

      We have acted as special counsel to Casual Male Retail Group, Inc., a
Delaware corporation (the "Company"), in connection with a Registration
Statement on Form S-3, as amended (the "Registration Statement"), for the
registration of the sale from time to time by the holders thereof of 1,818,504
shares (the "Shares") of common stock, par value $0.01 per share, of the
Company.

      We have made such inquiries and reviewed such documents and records as we
have deemed necessary to enable us to express an opinion on the matters covered
hereby, and we have also examined and relied upon representations, statements or
certificates of public officials and officers and representatives of the
Company.

      Based upon and subject to the foregoing, we are of the opinion that:

1.    the issued Shares have been duly authorized and legally issued and are
      fully paid and non-assessable; and

2.    the Shares that have not yet been issued have been duly authorized and,
      when issued in accordance with the terms applicable thereto, will be
      legally issued, fully paid and non-assessable.

      We do not express any opinion with respect to any law other than the laws
of the State of New York, the Delaware General Corporation Law and the federal
laws of the United States of America. This opinion is rendered only with respect
to the laws and legal interpretations and the facts and circumstances in effect
on the date hereof.

      We hereby consent to the use of this opinion as Exhibit 5.1 to the
Registration Statement and to the use of our name under the caption "Legal
Matters" in the prospectus included in the Registration Statement. In giving
this consent, we do not thereby admit that we are within the category of persons
whose consent is required under Section 7 of the Securities Act of 1933, as
amended, or the rules and regulations promulgated thereunder.


                                    Very truly yours,
                                    /s/ Kramer Levin Naftalis & Frankel LLP
                                    Kramer, Levin, Naftalis & Frankel LLP



                                                                    Exhibit 23.1


            Consent of Independent Registered Public Accounting Firm

We consent to the reference to our firm under the caption "Experts" in Amendment
No. 1 to the Registration Statement (Form S-3) and related Prospectus of Casual
Male Retail Group, Inc. for the registration of 1,818,504 shares of Casual Male
Retail Group, Inc. common stock and to the incorporation by reference therein of
our report dated March 24, 2004, with respect to the consolidated financial
statements and schedule of Casual Male Retail Group, Inc. included in its Annual
Report (Form 10-K) for the year ended January 31, 2004, filed with the
Securities and Exchange Commission.

                                                    /s/ Ernst & Young LLP

Boston, Massachusetts
November 17, 2004