UNITED STATES
                       SECURITIES AND EXCHANGE COMMISSION
                             Washington, D.C. 20549

                                  SCHEDULE 13D
                  Under the Securities and Exchange Act of 1934
                               (Amendment No. 5)*

                         Casual Male Retail Group, Inc.
- --------------------------------------------------------------------------------
                                (Name of Issuer)

                     Common Stock, $0.01 par value per share
- --------------------------------------------------------------------------------
                         (Title of Class of Securities)

                                    148711104
                    -----------------------------------------
                                 (CUSIP Number)

                         Chilton Investment Company, LLC
                        1266 East Main Street, 7th Floor
                               Stamford, CT 06902
                                 (203) 352-4000
- --------------------------------------------------------------------------------
(Name, Address and Telephone Number of Person Authorized to Receive Notices and
                                 Communications)

                                October 19, 2007
- --------------------------------------------------------------------------------
             (Date of Event which Requires Filing of this Statement)

If the filing person has previously filed a statement on Schedule 13G to report
the acquisition that is the subject of this Schedule 13D, and is filing this
schedule because of Rule 13d-1(e), Rule 13d-1(f) or Rule 13d-1(g), check the
following box |_|.

Note: Schedules filed in paper format shall include a signed original and five
copies of the schedule, including all exhibits. See Rule 13d-7 for other parties
to whom copies are to be sent.

*The remainder of this cover page shall be filled out for a reporting person's
initial filing on this form with respect to the subject class of securities, and
for any subsequent amendment containing information which would alter
disclosures provided in a prior cover page.

The information required on the remainder of this cover page shall not be deemed
to be "filed" for the purpose of Section 18 of the Securities Exchange Act of
1934 or otherwise subject to the liabilities of that section of the Act but
shall be subject to all other provisions of the Act (however, see the Notes).

- --------------------------- -------------------------- CUSIP No. 148711104 SCHEDULE 13D Page 2 of 4 Pages - --------------------------- -------------------------- - -------------------------------------------------------------------------------- 1 NAME OF REPORTING PERSON I.R.S. IDENTIFICATION NO. OF ABOVE PERSON (ENTITIES ONLY) Chilton Investment Company, LLC - -------------------------------------------------------------------------------- 2 CHECK THE APPROPRIATE BOX IF A MEMBER OF A GROUP (a) |_| (b) |X| - -------------------------------------------------------------------------------- 3 SEC USE ONLY - -------------------------------------------------------------------------------- 4 SOURCE OF FUNDS AF, OO - -------------------------------------------------------------------------------- 5 CHECK IF DISCLOSURE OF LEGAL PROCEEDINGS IS REQUIRED PURSUANT TO ITEMS 2(d) or 2(e) |_| - -------------------------------------------------------------------------------- 6 CITIZENSHIP OR PLACE OF ORGANIZATION Delaware - -------------------------------------------------------------------------------- 7 SOLE VOTING POWER 8,360,163 ---------------------------------------------------------------- NUMBER OF 8 SHARED VOTING POWER SHARES BENEFICIALLY 0 OWNED BY ---------------------------------------------------------------- EACH 9 SOLE DISPOSITIVE POWER REPORTING PERSON WITH 8,360,163 ---------------------------------------------------------------- 10 SHARED DISPOSITIVE POWER - -------------------------------------------------------------------------------- 11 AGGREGATE AMOUNT BENEFICIALLY OWNED BY EACH REPORTING PERSON 8,360,163 - -------------------------------------------------------------------------------- 12 CHECK IF THE AGGREGATE AMOUNT IN ROW (11) EXCLUDES CERTAIN SHARES |_| - -------------------------------------------------------------------------------- 13 PERCENT OF CLASS REPRESENTED BY AMOUNT IN ROW (11) 20.1% - -------------------------------------------------------------------------------- 14 TYPE OF REPORTING PERSON (See Instructions) IA - --------------------------------------------------------------------------------

- ---------------------------- ------------------------- CUSIP No. 148711104 SCHEDULE 13D Page 3 of 4 Pages - ---------------------------- ------------------------- ITEM 1. Security and Issuer. Pursuant to Rule 13d-2(b) of Regulation 13D-G under the Securities Exchange Act of 1934, as amended, the Schedule 13G initially filed on November 10, 2005, by Chilton Investment Company, LLC (the "Reporting Person") with respect to the Common Stock, $0.01 par value (the "Common Stock"), of Casual Male Retail Group, Inc., a Delaware corporation (the "Schedule 13G"), as amended by Amendment No. 1 to the Schedule 13G filed on February 14, 2006, replaced by the Schedule 13D filed on May 9, 2006 (the "Schedule 13D"), as amended by Amendment No.1 to the Schedule 13D filed on June 9, 2006, Amendment No. 2 to the Schedule 13D filed on June 29, 2006 and Amendment No. 3 to the 13D filed on April 13, 2006 and Amendment No. 4 to the 13D filed on July 19, 2007, replaced by Amendment No. 2 to the Schedule 13G filed on July 20, 2007 is hereby replaced by this Amendment No. 5 to the Schedule 13D to report a change in beneficial ownership. The Schedule 13D is amended as follows: ITEM 5. Interest in Securities of the Issuer. Item 5(a) is hereby supplemented by the addition of the following: (a) As of the date hereof, the Reporting Person is deemed to be the beneficial owner (through its control of the Client Accounts) of 8,360,163 shares representing 20.1% of the Issuer's Common Stock. The foregoing calculations are based on 41,561,080 shares of Common Stock issued and outstanding as of August 17, 2007 according to publicly available filings of the Issuer. Item 5(c) is hereby supplemented by the addition of the following: (b) The number of shares and the price per share of all transactions effected by the Reporting Person during the last sixty days are as follows: o On August 27, 2007, 4,800 shares of Common Stock were sold through Weeden Securities at a price per share of $11.0000. o On August 27, 2007, 97 shares of Common Stock were sold through B-Trade at a price per share of $10.0400. o On August 27, 2007, 3 shares of Common Stock were sold through Credit Suisse at a price per share of $10.0300. o On September 4, 2007, 5,782 shares of Common Stock were sold through Bank of America at a price per share of $10.1421. o On September 4, 2007, 84,332 shares of Common Stock were sold through Knight Securities at a price per share of $9.9322. o On September 5, 2007, 2,168 shares of Common Stock were purchased through JP Morgan at a price per share of $9.7762.

- ---------------------------- ------------------------- CUSIP No. 148711104 SCHEDULE 13D Page 4 of 4 Pages - ---------------------------- ------------------------- o On October 1, 2007, 31,787 shares of Common Stock were purchased through Credit Suisse at a price per share of $9.6334. o On October 2, 2007, 357 shares of Common Stock were purchased through Credit Suisse at a price per share of $9.7844. o On October 3, 2007, 3,929 shares of Common Stock were sold through Credit Suisse at a price per share of $9.1949. o On October 3, 2007, 640 shares of Common Stock were purchased through Credit Suisse at a price per share of $9.3219. o On October 4, 2007, 1,601 shares of Common Stock were purchased through Lehman Brothers at a price per share of $9.2408. o On October 5, 2007, 75,400 shares of Common Stock were purchased through ITG Channel at a price per share of $8.8440. o On October 5, 2007, 125,000 shares of Common Stock were purchased through Jefferies & Co. at a price per share of $9.0040. o On October 5, 2007, 41,300 shares of Common Stock were purchased through Lehman Brothers at a price per share of $9.2008. o On October 8, 2007, 25,000 shares of Common Stock were purchased through Jefferies & Co. at a price per share of $8.9739. o On October 8, 2007, 1,100 shares of Common Stock were purchased through Lehman Brothers at a price per share of $8.8982. o On October 8, 2007, 785 shares of Common Stock were purchased through Credit Suisse at a price per share of $8.9511. o On October 10, 2007, 4,932 shares of Common Stock were sold through Credit Suisse at a price per share of $8.4672. o On October 11, 2007, 32,200 shares of Common Stock were purchased through Credit Suisse at a price per share of $8.5424. o On October 17, 2007, 829 shares of Common Stock were purchased through Credit Suisse at a price per share of $8.0080. o On October 17, 2007, 2,413 shares of Common Stock were sold through Bank of America at a price per share of $7.9844. o On October 19, 2007, 307,000 shares of Common Stock were purchased through Susquehanna at a price per share of $7.5517. SIGNATURE After reasonable inquiry and to the best of my knowledge and belief, I certify that the information set forth in this statement is true, complete and correct. Date: October 23, 2007 Chilton Investment Company, LLC By: /s/ James Steinthal ----------------------------------------- James Steinthal Managing Director