Being able to perform split A/B Testing is something that is vital for ecommerce stores if they wan to improve conversions.
Currently this is something that's really hard to achieve. Being able to select things like displaying a side bar or not, testing different sidebars. Changing the colours of buttons (altering the class names on tags), different layouts (using templates) would be useful for example.
Seconded. I use GA experiments, but it would be good to have this built in. At the very least, have a option to not have certain products show up in search results and listings, so I can make duplicates and test.