I suspect that this is going to be distinctly non-trivial. Filtering out high-frequency noise to get the low-frequency signal is tricky, and can make the output feel a bit laggy. Look
here and
here for some things to try.
One potentially crutch that you have is that you know, or can find out, the frequency and amplitude of the noise signal. It probably won't just be the frequency that the speaker is playing, as the vibrations picked up by the accelerometer will depend on the design of the phone. You could try playing a range of different frequencies on through the speaker, analyse the resulting accelerometer signal, and try and compute some relationship between speaker signal and accelerometer output. With this knowledge, you have a much better idea of the characteristics of the unwanted accelerometer noise and thus a better chance at filtering it.
However, this is a lot of work and probably not practical for the dalvik VM to be bothering with at runtime.