Ooh, a little unfair Rhiannon.
I don't think Mustard is hinting that he is 'special' and therefore his after hours excursion would be ok 'because it's him'.
I think he's simply saying that the people who care and love the place may do it and people that don't, or are just happy to wander around doing the tourist thing, won't.