When you make changes to your ecommerce site, how do you test or follow up to ensure that the changes are positive? Obviously you don't want to make changes that cause a loss of income, but how are you sure?
If you are against that, if you have some sort of metrics in place to measure you traffic I would use that system. Judge over say a period of a week to see if you get more traffic, convert more sales, etc.
How if you were selling one time products so that the visitors and customers purchasing your products are new people who never seen the previous look of your website?
I agree with you though to test the newly changed website in certain time span, to see if it performs better or worse than the old one does.
The other issue there being that, if you have seasonal changes to your sales then you won't know really if the changes caused the drop or increase in sales. It seems hard to get a definitive answer.