Most of the software that influences the behavior of human beings wasn’t created with strong ethical constructs around it. Software developers should ask themselves ethical questions like: "who does this affect?", "who could get hurt by this?", and "who does this disadvantage or advantage?". Developers shoould try to answer these questions, and also be comfortable with questions they can’t answer yet.
Theo Schlossnagle, CEO of Circonus, spoke about professional ethics for software developers at QCon London 2018. InfoQ is covering this conference with Q&As, presentations, summaries, and articles.
InfoQ interviewed Schlossnagle about the importance of ethics, what developers can do to incorporate ethical considerations, and asked him what the consequences of unethical software should be.
InfoQ: What makes ethics important for software developers?
Theo Schlossnagle: If you look around, the vast majority of people who are working today in our industry, writing code, making decisions that impact users, haven’t had an intense ethics course in their life. They haven’t taken an ethics course in high school, they haven’t taken an ethics course in college. It doesn’t mean that they don’t know ethics; ethics are pretty innate in human beings.
There’s a playbook for discussing ethics; there’s a playbook for contemplating them; there’s not a playbook for answering them. The question is what your mental model is for making yourself answer those questions instead of just avoiding them and pretending they don’t exist.
We have 30 years of software development, and the last 10 to 15 of those have been hyper-accelerated software development. We have software all over the place that influences the behavior of human beings, and we didn’t create that software with strong ethical constructs around it.
InfoQ: Can you give us an example of ethical considerations that a developer might face?
Schlossnagle: Sure. There are certain soap dispensers in airports that are automatic, that detect when your hand is under it unless you’re African-American, because then it doesn’t see your hand. One needs to hold a white piece of paper under the soap dispenser to get it to fire. The reason that’s there is not because a group of engineers thought that would be really funny to disenfranchise African-American people and people with dark skin. That is a layered cake of a lack of ethical questioning. It’s simply because people chose not to ask the questions, not because they asked them and were like "let’s do evil."
It’s important that we have a framework for asking questions. Open-ended questions; ethical questions are almost always open-ended. Who does this affect? What could go wrong with the software? Not in bugs, not in lines of code, not in crashing, but in who will this not service? Those are hard questions, and we’re going to screw it up. What I want people to do is to ask the questions, and have that as part of their software development cycle.
In the context of the detector of skin color, I don’t know what technology is in it at all. But if you have a simple computer vision system you’re probably using a computer vision library, perhaps there should be a small ethical discussion in the computer vision library about what the consequences of training this on a wrong model are? For example, you could also detect cookies with rat poop in them as if they were chocolate chips. There are other consequences, and you might disenfranchise a minority if you don’t train it on a diverse set of human beings.
InfoQ: What can developers do to incorporate ethical considerations into their work?
Schlossnagle: I think that software developers have come a long way, and they try to think about security as they develop code now. However, they’re not asking themselves, what are the ethical consequences of this? Who could get hurt by this? Who does this enable over another person? Who does this disadvantage or advantage? They’re not asking those questions. My goal is to have that part of the natural sequence of developing software.
Someone should be asking those questions. Every developer coming to your other question of who this is for, every line-level developer who writes a line of code should have the capacity to incorporate those questions into what they do and it should go all the way up the stack to grand vision.
InfoQ: What type of questions should developers be asking?
Schlossnagle: It’s highly dependent on what they do. What type of software they work on. I think that the one thing to ask is a simple question of how could this harm someone. I would argue that no one ever writes software without that software eventually interfacing with a human being. We are the ultimate consumers of software. We build software to make human lives better. It’s not like we’re going to make robot lives better. If we do it’s because robots make human lives better. There’s a chain there that always ends up with the human as the as the benefactor of technology. Just asking the question of how is my code going to be used by humans?
InfoQ: What can the software community do to support coping with ethical issues?
Schlossnagle: They can encourage the ad hoc addition of ethical questioning into their development process. Perhaps start by adding an ethicalconsiderations.md to their repositories with questions and answers and encourage pull requests.
InfoQ: Should there be consequences of unethical software?
Schlossnagle: Yes, I would argue there should be possible consequence of justice when you do things unethical. But ethics are driven not by punishment, but by mutual societal benefit. At university I learned how to ask questions, even the hard ones, and be comfortable with questions I couldn’t answer yet. And ethical questions should be part of it.
InfoQ have previously conducted a podcast with Theo Schlossnagle on Software Ethics and the Presence of Doing Good, and also interviewed him about Better Engineering via Better Discourse.