The concept I label “moral contextualism” is, in a sense, more or less a direct continuation of cultural relativism; a generalization.
Cultural relativism argues that you should judge a person by the culture they inhabited; insofar as you make moral judgments, they should be by the moral structures in place in the society in which they existed.
Moral contextualism does not argue that you should judge a person by their standards of their culture - it doesn’t actually specify how you judge a person at all. You can pair it with any ethical system you like; in utilitarianism, it creates a “0 score”, the “score” that the perfectly average person in your context would have; not good, not bad, just average (yes, I know, utilitarianism practiced properly doesn’t keep score - so much the worse for utilitarianism as an ethical framework for people, who do keep score). In virtue ethics, your virtue is relative to the average person. And insofar as it addresses deontology, it mostly says a good person follows the rules more than average, and a bad person less.
So - if you’re a poor person living in hard conditions, you’re a good person if you do better than other poor people living in hard conditions. The standard for good and bad is, in some sense, adjusted for your circumstances. Live in a society in which slavery is well-regarded and common, and being a kind slaver might be sufficient to be a good person; live in our society, and being the best and kindest slaver still makes you a monster.
If this seems bizarre or counterintuitive, consider a society in which the economically well-off face few hard moral dilemmas, and can afford to pay the higher premium for the more moral option, while the economically worse-off face many hard moral dilemmas, and cannot afford morality much of the time. I know, terribly unrealistic, but bear with me. Suppose you’re in charge of deciding who goes to heaven, and who goes to hell. How do you judge people fairly?
John never did anything bad to anyone. Mary stabbed and killed a man. Who the bad person is seems fairly obvious. But nothing bad ever happened to John, and the man Mary stabbed looked exactly like a stranger who strangled her older brother in front of her when she was six years old. Now it’s unclear if either are really bad people; Mary now seems like a sympathetic character, not bad but tragic. Context matters.
Of course, you can see how people will actually apply this: They’ll get increasingly specific about how they are, identifying with those things that justify their bad behavior, so they can eke out a happy “I’m a decent person, even if I’m not a saint”. Yeah, I’m an alcoholic who yells at my kids, but at least I don’t beat them like my parents did.
But notice something. That is in fact how people actually apply this. They do this. They do this constantly - and not just for themselves. Mary, who stabbed and killed a man who happened to have to misfortune of looking like the man who strangled her brother, is a sympathetic tragic figure. Anne who stabbed and killed a man for no reason at all just seems like a bad person.
Look past the -reasons- you come up with, and examine the general framework which permits these reasons. You already believe in moral contextualism; you already believe that Mary is a better person than Anne. Why doesn’t your codification of morality have this in it? Shouldn’t it?
Look at cultural relativism with fresh eyes, if it was something you found abhorrent before.
Insofar as ethics is a search for something in us, as people, as opposed to an external objective thing outside who we are - insofar as ethics is the practice of discovering what we think is ethics, and it must all add up to normality - then this is in fact a key part of ethics.