Red diesel
Seriously, if its just dust then I would suggest wire brush. Of course you could goto the extreme of using brick acid.
I would not think of putting any sealer on them at all. Its like painting your pebble dash white - once you do it - your retreating it every year to a fresh coat.
Its stone - its great as it is.
Thats said - I did throw a jar of mayonnaise at the wall many years ago in temper - the wall look great where it hit - cant wash it out at all - its loverly and damp shiney looking.