SECURITIES AND EXCHANGE COMMISSION WASHINGTON, DC 20549 ----------------- SCHEDULE 13G (Rule 13d-102) INFORMATION TO BE INCLUDED IN STATEMENTS FILED PURSUANT TO RULES 13d-1(b), (c), AND (d) AND AMENDMENTS THERETO FILED PURSUANT TO RULE 13d-2(b) (Amendment No. ___) Casual Male Retail Group, Inc. - -------------------------------------------------------------------------------- (Name of Issuer) Common Stock, par value $.01 per share - -------------------------------------------------------------------------------- (Title of Class of Securities) 148711 10 4 - -------------------------------------------------------------------------------- (CUSIP Number) September 10, 2004 - -------------------------------------------------------------------------------- (Date of Event Which Requires Filing of this Statement) Check the appropriate box to designate the rule pursuant to which this Schedule is filed: [ ] Rule 13d-1(b) [X] Rule 13d-1(c) [ ] Rule 13d-1(d)

CUSIP No. 148711 10 4 SCHEDULE 13G Page 2 of 5 Pages - -------------------------------------------------------------------------------- 1. NAME OF REPORTING PERSONS. I.R.S. IDENTIFICATION NOS. OF ABOVE PERSONS (ENTITIES ONLY) The Clark Estates, Inc. 13-5524538 - -------------------------------------------------------------------------------- 2. CHECK THE APPROPRIATE BOX IF A MEMBER OF A GROUP (SEE INSTRUCTIONS) (a) [ ] (b) [ ] - -------------------------------------------------------------------------------- 3. SEC USE ONLY - -------------------------------------------------------------------------------- 4. CITIZENSHIP OR PLACE OF ORGANIZATION New York - -------------------------------------------------------------------------------- 5. SOLE VOTING POWER 2,446,324* NUMBER OF ---------------------------------------------------------------- SHARES BENEFICIALLY 6. SHARED VOTING POWER 0 OWNED BY EACH ---------------------------------------------------------------- REPORTING PERSON WITH 7. SOLE DISPOSITIVE POWER 2,446,324* - -------------------------------------------------------------------------------- 8. SHARED DISPOSITIVE POWER 0 - -------------------------------------------------------------------------------- 9. AGGREGATE AMOUNT BENEFICALLY OWNED BY EACH REPORTING PERSON 2,446,324* - -------------------------------------------------------------------------------- 10. CHECK BOX IF THE AGGREGATE AMOUNT IN ROW (9) EXCLUDES CERTAIN SHARES (SEE INSTRUCTIONS) [ ] - -------------------------------------------------------------------------------- 11. PERCENT OF CLASS REPRESENTED BY AMOUNT IN ROW (9) 7.1% - -------------------------------------------------------------------------------- 12. TYPE OF REPORTING PERSON (SEE INSTRUCTIONS) CO - -------------------------------------------------------------------------------- * Includes 247,059 shares of Common Stock subject to warrants exercisable within 60 days of the date hereof. - --------------------------------------------------------------------------------

CUSIP No. 148711 10 4 SCHEDULE 13G Page 3 of 5 Pages ITEM 1(a). NAME OF ISSUER: Casual Male Retail Group, Inc. ITEM 1(b). ADDRESS OF ISSUER'S PRINCIPAL EXECUTIVE OFFICES: 555 Turnpike Street Canton, MA 02021 ITEM 2(a). NAME OF PERSON FILING: The Clark Estates, Inc. ITEM 2(b). ADDRESS OF PRINCIPAL BUSINESS OFFICE OR, IF NONE, RESIDENCE: One Rockefeller Plaza, 31st Floor New York, NY 10020 ITEM 2(c). CITIZENSHIP: New York ITEM 2(d). TITLE OF CLASS OF SECURITIES: Common Stock, par value $.01 per share ITEM 2(e). CUSIP NUMBER: 148711 10 4 ITEM 3. IF THIS STATEMENT IS FILED PURSUANT TO RULE 13d-1(b), OR 13d-2(b) OR (c), CHECK WHETHER THE PERSON FILING IS A: (a) [ ] Broker or dealer registered under Section 15 of the Exchange Act. (b) [ ] Bank as defined in Section 3(a)(6) of the Exchange Act. (c) [ ] Insurance company as defined in Section 3(a)(19) of the Exchange Act. (d) [ ] Investment company as defined in Section 8 of the Investment Company Act. (e) [ ] An investment adviser in accordance with Rule 13d-1(b)(1)(ii)(E). (f) [ ] An employee benefit plan or endowment fund in accordance with Rule 13d-1(b)(1)(ii)(F). (g) [ ] A parent holding company or control person in accordance with Rule 13d-1(b)(1)(ii)(G). (h) [ ] A savings association as defined in Section 3(b) of the Federal Deposit Insurance Act. (i) [ ] A church plan that is excluded from the definition of an investment company under Section 3(c)(14) of the Investment Company Act. (j) [ ] Group, in accordance with Rule 13d-1(b)(1)(ii)(J). N/A

CUSIP No. 148711 10 4 SCHEDULE 13G Page 4 of 5 Pages ITEM 4. OWNERSHIP: (a) Amount beneficially owned: 2,446,324 (b) Percent of class: 7.1% (c) Number of shares as to which such person has: (i) Sole power to vote or to direct the vote: 2,446,324 (ii) Shared power to vote or to direct the vote: 0 (iii) Sole power to dispose or to direct the disposition of: 2,446,324 (iv) Shared power to dispose or to direct the disposition of: 0 ITEM 5. OWNERSHIP OF FIVE PERCENT OR LESS OF A CLASS: If this statement is being filed to report the fact that as of the date hereof the reporting person has ceased to be the beneficial owner of more than five percent of a class of securities, check the following [ ]. N/A ITEM 6. OWNERSHIP OF MORE THAN FIVE PERCENT ON BEHALF OF ANOTHER PERSON: N/A ITEM 7. IDENTIFICATION AND CLASSIFICATION OF THE SUBSIDIARY WHICH ACQUIRED THE SECURITY BEING REPORTED ON BY THE PARENT HOLDING COMPANY OR CONTROL PERSON: N/A ITEM 8. IDENTIFICATION AND CLASSIFICATION OF MEMBERS OF THE GROUP: N/A ITEM 9. NOTICE OF DISSOLUTION OF GROUP: N/A ITEM 10. CERTIFICATIONS: By signing below I certify that, to the best of my knowledge and belief, the securities referred to above were not acquired and are not held for the purpose of or with the effect of changing or influencing the control of the issuer of the securities and were not acquired and are not held in connection with or as a participant in any transaction having that purpose or effect.

CUSIP No. 148711 10 4 SCHEDULE 13G Page 5 of 5 Pages SIGNATURE After reasonable inquiry and to the best of my knowledge and belief, I certify that the information set forth in this statement is true, complete and correct. Dated: September 10, 2004 THE CLARK ESTATES, INC. By: /s/ Kevin S. Moore -------------------------------- Name: Kevin S. Moore Title: President