Why should all human beings have certain rights simply by virtue of being human? One justification is an appeal to religious authority. However, in increasingly secular societies this approach has its limits. An alternative answer is that human rights are justified through human dignity. This paper argues that human rights and human dignity are better separated for three reasons. First, the justification paradox: the concept of human dignity does not solve the justification problem for human rights but rather aggravates it in secular societies. Second, the Kantian cul-de-sac: if human rights were based on Kant’s concept of dignity rather than theist grounds, such rights would lose their universal validity. Third, hazard by association: human dignity is nowadays more controversial than the concept of human rights, especially given unresolved tensions between aspirational dignity and inviolable dignity. In conclusion, proponents of universal human rights will fare better with alternative frameworks to justify human rights rather than relying on the concept of dignity.