It turns out you can find out exactly what info google collects on you, organized by day. Log into your google account and go to http://maps.google.com/locationhistory and you will see a map of everywhere you (or at least your phone) went that day. Here is the map that comes up for a recent random day in my life.
One thing you will notice is that I appeared not to follow the roads! I assure you I was not drunkenly driving across people's lawns all day. My best guess is that they only take data at certain times so the exact route you take from point A to B is lost.
Google offers you the option to delete your history for that day or delete all your history. I am not sure how "deleted" your history really is when you choose this option. You can export your history as a KML file (if you know how to analyze them) and see time stamps of every data point they have.
I don't know if Apple Maps has a similar feature, but I bet they collect similar info if you allow it.
I wonder how much data is collected even if you turn off all location services? I know there are useful services that are enabled by this data yet there are definitely lots of reasons to be wary of abuse by both government and corporate interests.
Now you know why Walter White always used burners!