I feel that yes the constitution should be referred to, but that we need to stop being ashamed by the "living document" idea because its been around for 230 years and new issues arise that could theoretically invalidate some of the amendments. I feel that when people say "is it in the constitution" that they are effectively trying to end the argument and, to quote Molyneux, is "not an argument".
Does this mean we could have slavery?
The reason slavery is bad has nothing to do with its prohibition in the constitution.