I tried this in one of my stories. Not knowing much about modern slavery, I just made the argument.
The society was formed out of artists/philosophers/scientists after the collapse of an empire. As such, they value logic, and see purity as the ultimate expression of logic. Therefore, the majority (which supports slavery) sees half-breeds and half-creatures as inferior, because they are impure... Since their society is dedicated to upholding logic, and these people are inherently illogical, they must be inferior.
I don't like the argument, but until I do more research I can't come up with anything better.