The on-screen compass is meant to be like a real compass. That is, it does not correspond to the map view shown, but rather points the real direction of north. This is for convenience so you don’t have to switch back and forth between PocketEarth and the Compass app. If it is not correctly pointing to north, then most likely the compass needs calibration (by waiving your device in the funny figure 8 motion that Apple recommends).
The GPS arrow also by default uses the compass to show you the direction on the map that your device is pointing. In PocketEarth v1.6 you can now change this to instead show the direction of motion by going to Settings > Advanced > GPS Arrow and changing it to Heading.