1000% in agreeance - and I know you will enjoy the layout more
Just quickly, I started out with an 8' X 4' HO sectional layout, and it was fun, my first so I was excited just to have a train running. Then I joined this place - what a mistake that was! I saw all of the possibilities and potential for a layout and what could be done - darn I thought to myself - what am I doing with this little 8' X 4' thing when I could have - heck a 9' X 5" ?????? So I added an additional 8' to the 8' X 4' and built my second layout.
I then stumbled across this thing called N Scale ... hmmm, smaller stuff - thinking thinking thinking - got it!! Smaller stuff in the same area means a lot more stuff and prospects - heck yeah!!!!!!! So I ripped up my (then third layout I think) added an extra 6' X 4" to the end of the bench work and put in my first N Scale!
Thankfully, I have run out of room in my Train Room so no more expansions, but I think you get my point. It is all too easy to get carried away and end up with a mess and a whole lot of frustration. As many have already said - keep it manageable and it will remain enjoyable