Substitution failure is not an error is bad programming, in my opinion.
It seems to me that if you are making special cases for different classes in a template, then you’ve clearly missed the point of templates and are using them incorrectly. Templates are supposed to apply a concept or algorithm uniformly to a class. A vector, or a hash, work on objects of any type, uniformly.
If you’re SFINAEing, then what you really want to be doing is make a base class and derive other classes from it, each having traits specific to that class. That’s the very definition of what object oriented programming is for.
By taking advantage of a hack to cover a language flaw that serves no purpose but to supply entries for ‘the most heinous error message to come out of a c++ compiler’ contest, you’re being cool, but you’re not being a good programmer.