You always want latency or jitter as low as possible. Latency adds up in the chain. Instrument/Sequencer (2ms) -> Some digital effect (3ms) -> Digital Mixer (3ms) -> In-Ear monitor bridges an air gap with processing (6ms) Suddenly you have 14ms latency. Bad, but reality. So, every ms less is better.

Regarding jitter, this is the worst, because the brain cannot adapt to the changes, whereas constant latency can be regulated somehow by the brain.