Published on April 22nd, 2021 📆 | 5119 Views ⚑
0Ethics: University of Minnesota’s hostile patches
The University of Minnesota (UMN) got into trouble this week for doing a study where they have submitted deliberately vulnerable patches into open-source projects, in order to test whether hostile actors can do this to hack things. After a UMN researcher submitted a crappy patch to the Linux Kernel, kernel maintainers decided to rip out all recent UMN patches.
Both things can be true:
- Their study was an important contribution to the field of cybersecurity.
- Their study was unethical.
Itâs like Nazi medical research on victims in concentration camps, or U.S. military research on unwitting soldiers. The research can simultaneously be wildly unethical but at the same time produce useful knowledge.
In addition, the sorts of bugs it exploits shows a way forward in the evolution of programming languages. Itâs not clear that a âsafeâ language like Rust would be the answer. Linux kernel programming requires tracking resources in ways that Rust would consider inherently âunsafeâ. Instead, the C language needs to evolve with better safety features and better static analysis. Specifically, we need to be able to annotate the parameters and return statements from functions. For example, if a pointer canât be NULL, then it needs to be documented as a non-nullable pointer. (Imagine if pointers could be signed and unsigned, meaning, can sometimes be NULL or never be NULL).
So Iâm glad this paper exists. As a researcher, Iâll likely cite it in the future. As a programmer, Iâll be more vigilant in the future. In my own open-source projects, I should probably review some previous pull requests that Iâve accepted, since many of them have been the same crappy quality of simply adding a (probably) unnecessary NULL-pointer check.
However, I think IRB sign-off for computer security research is stupid. Things like masscanning of the entire Internet are undecidable with traditional ethics. I regularly scan every device on the IPv4 Internet, including your own home router. If you paid attention to the packets your firewall drops, some of them would be from me. Some consider this a gross violation of basic ethics and get very upset that Iâm scanning their computer. Others consider this to be the expected consequence of the end-to-end nature of the public Internet, that thereâs an inherent social contract that you must be prepared to receive any packet from anywhere. Kerckhoffâs Principle from the 1800s suggests that core ethic of cybersecurity is exposure to such things rather than trying to cover them up.
The point isnât to argue whether masscanning is ethical. The point is to argue that itâs undecided, and that your IRB isnât going to be able to answer the question better than anybody else.
But hereâs the thing about masscanning: Iâm honest and transparent about it. My very first scan of the entire Internet came with a tweet âBTW, this is me scanning the entire Internetâ.
A lot of ethical questions in other fields comes down to honesty. If you have to lie about it or cover it up, then thereâs a good chance itâs unethical.
The above research is based on a lie. Lying has consequences.
But at the same time, itâs obviously noise and bad output. If the researcher were developing a static analyzer tool, they should understand that this is crap noise and bad output from the static analyzer. They should not be submitting low-quality patches like this one. The main concern that researchers need to focus on for static analysis isnât increasing detection of vulns, but decreasing noise.
In other words, the debate here is whether the researcher is incompetent or dishonest. Given that UMN has practiced dishonesty in the past, itâs legitimate to believe they are doing so again. Indeed, âstatic analysisâ research might also include research in automated ways to find subversive bugs. One might create a static analyzer to search code for ways to insert a NULL pointer check to add a vuln.
Now incompetence is actually a fine thing. Thatâs the point of research, is to learn things. Starting fresh without all the preconceptions of old work is also useful. That researcher has problems today, but a year or two from now theyâll be an ultra-competent expert in their field. Thatâs how one achieves competence â making mistakes, lots of them.
But either way, the Linux kernel maintainer response of âwe are not part of your research projectâ is a valid. These patches are crap, regardless of which research project they are pursuing (static analyzer or malicious patch submissions).
Conclusion
I think the UMN research into bad-faith patches is useful to the community. I reject the idea that their IRB, which is focused on biomedical ethics rather than cybersecurity ethics, would be useful here. Indeed, itâs done the reverse: IRB approval has tainted the entire university with the problem rather than limiting the fallout to just the researchers that couldâve been disavowed.
The natural consequence of being dishonest is that people canât trust you. In cybersecurity, trust is hard to win and easy to lose â and UMN lost it. The researchers should have understand that âdishonestyâ was going to be a problem.
Iâm not sure there is a way to ethically be dishonest, so Iâm not sure how such useful research can be done without the researchers or sponsors being tainted by it. I just know that âdishonestyâ is an easily recognizable issue in cybersecurity that needs to be avoided. If anybody knows how to be ethically dishonest, Iâd like to hear it.
Update:Â This person proposes a way this research could be conducted to ethically be dishonest:
By asking the top boss if itâs okay if you lie to their team, a la an authorized penetration test.
In this case that might still not be ethical, because while the top guy can answer for the /project/ he canât answer for the other /people/, who are volunteers and not employees.
â Random of Eddie (@random_eddie) April 21, 2021
*** This is a Security Bloggers Network syndicated blog from Errata Security authored by Robert Graham. Read the original post at: https://blog.erratasec.com/2021/04/ethics-university-of-minnesotas-hostile.html
Gloss