This presentation was recorded at YOW! Perth 2023. #GOTOcon #YOWhttps://yowcon.comHillel Wayne - Chief Everything Office at Windy Coast Consulting @hillelway...
TLDW. But I think it’s better to treat programming as being more like math, where you need sound inference rather than mere experimental observation to believe things work.
Example: if you drive a 10 ton truck over a bridge and the bridge doesn’t collapse, that’s encouraging. If you drive a million of those trucks over the bridge and the bridge still doesn’t collapse, that’s sufficient engineering certainty that you can declare the bridge safe to handle a 10 ton load. But software surviving a million test cases (which could happen in just 1 second of operation) is far less meaningful. Weird corner cases matter a lot more. Attackers actively seeking inputs to break the program will uncover and explit far more bugs than normal variations in test inputs will find (fuzzing helps). Changes in the build environment or library versions can break stuff that depended (without your realizing it) on undocumented behaviour of the surrounding system. This stuff happens all the time. People put way too much trust in software that passed some tests but wasn’t really thought through.
Tldr: you have to think things through at a level closer to that of mathematics than to that of traditional engineering.
I think that’s a helpful analogy and comment. Please remember this while I go on to nitpick. I’m aiming at in both fields, there may be more math-leaning scientists and concrete-leaning workers, with the engineer being somewhat in the middle.
Declaring bridges safe probably involves a lot of math and tables in the background. I guess we don’t actually run a million trucks but estimate the safety theoretically, with a few experimental tests. Likewise, a security specialist can define the edge cases against which the tests should be run. That may be the same person who also implements the test, but I want to emphasize it’s two different roles. And we might consider one more of a scientist, and the other more of a worker.
So how much your activity resembles that of a mathematician, or a traditional engineer probably depends on your specific task, and how much your team requires you to generalize or specialize.
TLDW. But I think it’s better to treat programming as being more like math, where you need sound inference rather than mere experimental observation to believe things work.
Example: if you drive a 10 ton truck over a bridge and the bridge doesn’t collapse, that’s encouraging. If you drive a million of those trucks over the bridge and the bridge still doesn’t collapse, that’s sufficient engineering certainty that you can declare the bridge safe to handle a 10 ton load. But software surviving a million test cases (which could happen in just 1 second of operation) is far less meaningful. Weird corner cases matter a lot more. Attackers actively seeking inputs to break the program will uncover and explit far more bugs than normal variations in test inputs will find (fuzzing helps). Changes in the build environment or library versions can break stuff that depended (without your realizing it) on undocumented behaviour of the surrounding system. This stuff happens all the time. People put way too much trust in software that passed some tests but wasn’t really thought through.
Tldr: you have to think things through at a level closer to that of mathematics than to that of traditional engineering.
I think that’s a helpful analogy and comment. Please remember this while I go on to nitpick. I’m aiming at in both fields, there may be more math-leaning scientists and concrete-leaning workers, with the engineer being somewhat in the middle.
Declaring bridges safe probably involves a lot of math and tables in the background. I guess we don’t actually run a million trucks but estimate the safety theoretically, with a few experimental tests. Likewise, a security specialist can define the edge cases against which the tests should be run. That may be the same person who also implements the test, but I want to emphasize it’s two different roles. And we might consider one more of a scientist, and the other more of a worker.
So how much your activity resembles that of a mathematician, or a traditional engineer probably depends on your specific task, and how much your team requires you to generalize or specialize.