Your ESPboy2 has the same problem mine (an earlier revision) does; the battery is slightly thicker than the case section that goes around it, so the back of the case bulges out in the middle.
When I get a chance, I’m going to cut a thin sheet of plastic to the same shape as the battery case section, to add in to fix this.
I don’t like that you can’t adjust the speaker volume with the unit assembled. With the original ESPboy, I was able give access to the volume control by drilling a hole in the back panel. I don’t think this is possible with the ESPboy2.