It's an expensive place to visit, but there are ways to cut costs. Essentially you have 3 options: 1: Book a hotel in the city and use public transportation to get around. 2: Book a hotel outside of the 5 boroughs but in a place where getting into the city won't be difficult. 3: Book an accommodation on a site like Air BnB and have access to a kitchen and public transportation. With option 1, you're probably looking at somewhere around $250 to $350 per night. With option 2 you might be able to find something for around $150 per night. With option 3 you could negotiate for a cheaper rate per night depending on how far out you're willing to travel. But if you book into a place in Westchester County, Fairfield County, New Jersey or Long Island, transportation costs per day will be extortionate. And even if you were to rent a car and pay for parking and pair that with PATH, LIRR or MetroNorth, it would be expensive. There are no "cheap" options, and cheaper means more of a headache.
Option 1 would run you between $2,000 and $2,500 for the week for accommodation, between $700 and $1,000 for the week for meals, and probably $200 to $500 on transportation. So all in you're talking $3,000 before you start thinking about entry fees, souvenirs or miscellaneous.
With option 2 you could shave that down by $500, but you'd be spending hours on trains and buses every day, so in the end, because you've only got a week, it would really put a dent in your time.
With option 3 you could stay in an actual apartment and cook your own breakfasts and use buses and subways, but a two fare zone is almost as inconvenient as being in a commuter town, and some of those neighbourhoods are not very nice.
Drop the extra coin and stay in Midtown like all the other tourists do. You'll be in the thick of it all. Eat where locals eat, take advantage of the plethora of free stuff available in New York City, eschew cab rides for the subway and you'll be fine.
If you want cheap, go to Erie, PA. If you want to spend a week in New York City, it's going to cost you. It's that simple.