This man Irving though, is a supporter of the Third Reich and an admirer of Hiter. He would like to restore a Nazi Europe. That isn't slightly dodgy views anymore than Hitler's were slightly dodgy.
Do you give freedom of speech to someone, if the result that they are ultimately looking for, from their speech, is the destruction of millions of us?