The problem is that on mobile displays (issue also present in the chrome mobile emulator) the keys stay inset after tapping, instead of returing to raised position, even though each key is being blurred on touchend.
I played around with your code a little bit and had a hard time fixing your problem. I can clearly see that mouseleave resets the button image, however touchleave does nothing.
I personally would try using event handlers on functions that change the button images. Instead of using CSS blur and focus, how about a touchstart event that changes the button image and a touchleave event that resets the image? If that doesn’t work you can always just use setTimeout for like a few hundred millisecs when you touch a button to reset the image. Even your click event isn’t currently perfect; ideally you would want the button to change only on mousedown, and reset on mouseup, just like a real life calculator.