People who have been diagnosed for hypertension or diabetes are most of the times nervous to get on the machine that would test them and give a new reading. This causes elevated readings. So, doing a test disturbs a system. Be it human body or software.
I was the proud tester of the famous Moto Razr V3. This phone was a dream phone to many in their past lives. The era before iOS and Android. It was a Flip Phone and Sleek. The iPhone 8 of those days 🙂 As a tester, anytime I found a crash, I had to enable the logger and repeat the tests so that I could provide the developers with logs to fix the bug. The moment I enabled the logger – the phone slowed down. I found new crashes and sometimes could not even get to the point of reproducing the crash I found without the logger enabled. Sad story. The logger disturbed the System Under Test.
Today. How do we test without instrumenting the app? We can. Letting a human test the app. That way – the app remains in a state the user would use it and we know how the non instrumented app works. Instrumentation can help in automation. However, most instrumentation practices are bad. They don’t disturb the system too much but they don’t add much value either.
Today’s world is built for developers to become more cozy. Investors ask me – how do we build a test platform where developers don’t don’t need to do anything? I wish someone asks- how do we build a test system that developers do the right things?
We can’t test an app to a good enough extent without instrumenting it because of the complexity of the environment in which the app has to work and the kind of details we would need to be able to come up useful reports.
I thought through deeply about what investors asked me. They are right. We need to make developers cozy. Cozy enough that testing is not an overhead for them. As they write code – the tests should run. The right ones. How hard!