Both the title of a 1946 motion picture and American standard song, the phrase to each his own generally means each person is entitled to his or her own personal tastes and opinions. This could apply to a person's choice of romantic partner or musical preference or political leanings or any other pursuit. After all, what might appear offensive or unappealing to some may be quite the opposite for others.
There are a few other common sayings that also cover this non-judgmental territory, including whatever floats your boat, different strokes for different folks, and the Biblical aphorism judge not, lest ye also be judged. In this sense, any of these terms might be used to sign off on an improbable romance or unexpected affiliation.
The phrase itself, or at least a popular variant, can be traced back to ancient Greece, where it became a popular sentiment in the legal profession. In order for a society to work well together, there had to be a certain level of permissiveness and tolerance. This is the key principle behind the reluctance of lawmakers to legislate morality. What may be offensive or immoral to one group may be perfectly permissible in another, so the concept discourages efforts to create artificial boundaries where personal freedoms are concerned.
There are also several Biblical references to the concept. Jesus' parable concerning servants given various sums of money (talents) according to each servant's value to his master is one example of each individual being rewarded or punished according to his or her own merits. The Apostle Paul also refers to believers in Christ receiving gifts of the spirit according to his or her own virtues and abilities.
There some highly controversial uses of the phrase as well. Karl Marx wrote in his preface to the Communist Manifesto that each worker should receive compensation according to his own contributions to the State. This concept can be traced back to ancient Grecian philosophy as well. When the Nazi regime built the concentration and work camps to hold Jewish prisoners, the gates were often emblazoned with two popular expressions: Arbeit Macht Frei (Work will make one free) or Jedem das Seine, which literally translated to "To each his own," but idiomatically meant "Each person gets what he deserves."
The sentiment behind the modern expression suggests an admirable level of tolerance for those who find love or happiness or contentment in ways that do not necessarily mesh with our own beliefs or preferences.