Another problem was that we keep the sax in my car so she can play with it on car rides. When it is in the back seat or on the floor, it frequently starts playing when I hit a bump. My first thought was to remove the batteries (that was actually my second thought -- my first was to throw it out the window, but I didn't want to litter), but my daughter seems to enjoy it, and it at the very least keeps her entertained in the car.
My solution:
1. I took the sax apart, which involved removing 7 philips head screws
2. I found a spot where an SPST switch would fit (I had one on hand, but they're only about $3 at Radio Shack, and worth every penny)
3. I drilled a hole in the plastic and mounted the switch
4. I desoldered the end of the red wire connected to the positive terminal on the battery compartment
5. I soldered that wire to the outside terminal of the switch
6. I cut a short length of red wire (I happened to have some 22 gauge wire handy) and connected the middle terminal to the positive battery terminal
I put everything back together, and now we have a sax that turns off.