If your just box filtering, then the previous level is what you want...otherwise it depends on your filter.
What do you mean by box filtering?
My main usage is the manipulation in 3d of photos and text (where each letter is a texture in alpha mode...)
For text, the two fields that matter are:
1- small text (i.e. tiny mipmap levels) should be crisp
2- the transition between levels (i.e. when zooming in/out) should be seamless
I don't mind spending time on computation, as long as the mipmap generation is fine tuned for these needs...