Gruv Glide is great but for really dirty records you want Groovy Record Cleaner. Or just soap and water. But for everyday use I use a dry carbon fiber brush so I can play the record right away. This one here is the bomb: Amazon.com
You can also clean dry with a lint roller (not the tape kind but the kind with a gooey layer of rubber crap that sticks to things); this sort of thing: Amazon.com
Nagaoka used to make a lint roller style thingie just for records but I don’t think you can find those anymore. But I got a lint roller for five bucks at the drug store and it works fine.
And yes it’s good to clean your stylus once in a while because they do get coated with gunk. Q-Bert shows how to clean one stylus with another stylus but I find it only works if you have bionic vision. Look for this sort of thing: Amazon.com
There are different ones out there but you want to make sure you get one with a brush with lots of short bristles packed into a tiny space rather than long bristles loosely packed.
Mr Clean Magic Eraser for your needle cleaning needs, one box of three will do you years.
As for record cleaning solutions, I’m a bit wary of them, sometimes I swear the surface noise of the record has increased after what looked like a good cleanup. On my control vinyl I don’t care so much but… I’ve pulled quite a few handfuls of hair trying to get a cleaner record for digitizing.