Category Archives: TestInsane

SOFTWARE TEST AUTOMATION – THE WHYs, WHATs, WHENs and HOWs?

Let me cut the crap and jump over to the actual topic. That’s the least I can do for you J

Why should I think of automating the tests?

The product development has reached a milestone and yet to be enriched with more features. We wish to make sure that the new changes don’t break the existing features. Yes, we can always test them manually during each release. However, there are two major demerits.

Let us assume the releases to the test team are located far apart. During the time gap the product might be updated with many changes. In an agile environment we would like to verify often or when the changes made to the product, that the changes have not broken the existing features than waiting for the next test cycle. It would be impossible to verify often all test cases manually. So a right approach will be to automate the tests for the existing features and run them and report the deviations every release. Verify only those test cases manually which are difficult to automate and we do not wish to automate.

Let us assume the test cycles are not far apart. Over a period the manual test efforts will keep on growing as the more features Test Automation - TestInsaneadded. If we have to keep the test cycle short and unchanged then we would need more manual test hours. While we plan everything in terms of cost and resources (a human is treated as a resource), we forget that it’s living humans who are testing the nonliving product. The tester who has to repeat the same tests every test cycle starts to feel monotony and may not test the product with same keen and attentive state of mind thereby failing to make sure the quality of test. A hard truth! That’s why many complain software testing is boring and non-challenging as there is less time available for him/her to acquire new skills and use them to test the product with fresh mind-set. The solution is to automate them and manually test only the areas which are not covered by the automation tests. The test engineer can now explore new ways to test the earlier features at the same time focusing more on testing the new features introduced.

What if we intend to repeat each test several times to test for the reliability? A tester will die of boredom or lack of sleep. But machines can run all day and they can do repetitive task uninterrupted at better speed than a human can. So we have hired machines to work longer and faster to do repetitive and boring tasks.

The take-away from automating the tests are,

  • Save the cost of delivering stable product by reducing the manual efforts in relatively quick time compared to manual tests.
  • Make testing less boring and monotonous for our engineers so that they can focus on acquiring new skills to test the product better.
  • Consistency in test quality as there is no human intervention (Remember if the automation tests are not written rightly then the purpose of the automation may not be met).
  • Immediate feedback of the changes to the developers in CI environment so that they can fix the changes immediately than waiting for the next test cycle.
  • Improved speed in achieving the better quality and reduced test cycle duration.

 

What and When to automate?

It’s not practical to automate everything and it can be done. But, it is very challenging to deliver the best test results which a skillful tester can deliver as one can think of better permutations and combinations of actions and creative test ideas which may not be automated for valid reasons. I am not saying we cannot. There are products which have more test code than the product code and that’s a sign of good product. But, as mentioned before we can cover major use cases of all important features which make sure the greater stability of the product with good volume of test data (of differing combinations and permutations).

  • We should automate test cases which are difficult to be tested by the human as we are limited by speed and ability to generate huge test data and test for all the cases in limited time.

 

  • We should not automate the test cases which add little value and the more cost in automating. Say, CAPTCHA string recognition in automating login feature. Rather I would disable the CAPTCHA in the development server and automate the login feature (unless the CAPTCHA generation is purposefully weakened in test environment and their order is made known to enable the automation in development environment). Otherwise, I would need a module which would do CAPTCHA image decoding, which is an overhead mostly. Say, the charts are generated and the numbers are to be verified against the chart values. We would need to do chart image decoding. The easier option is to do image comparison to expected image if we know the test data input. Imagine testing a photo organizing app which organizes photos belonging to a person as album. How do we automate the test case to verify if the photo belonging to a person are grouped as an album? Unless we have predefined data and we know the expected result to compare against, the test case cannot be automated for any test data which is dynamically generated. However, it still helps to verify that the feature is not fully broken by checking it to work with the small test data we have (and we know the expected results).

 

  • If certain features of the product are expected to behave consistently when the same tests are performed repeatedly for long hours, several times as required by the product requirements, the automation is the only way forward. An example would be channel change in Television to work without failure for several times which is typically high.

 

  • We should automate when most of the feature implementation is complete, requirements almost frozen and stable. Otherwise, we end up maintaining the automation suite often than expected.

 

How to build a good automation framework?

  • Carefully identify test cases keeping automation as the end goal to assert the stability of the application under test. The test cases should cover most important use cases with a goal to uncover bugs and assure the features work across releases meeting least required quality.

 

  • The automation scripts and test data separation is a must. This demands the need to write generic code which enables testing for wide range of test data and test cases without a need to change or add the code or for a developer to get involved. Ability to change or edit the test data should be easier and adaptable to the changes in the minor changes or enhancements to the product features.

 

  • Avoiding dependency between test cases as much as possible unless the test case and test data get too big. Think of an application which allows the user to navigate to other pages/screen and use them only if they complete registration form. Let’s assume there are 5 features to automate and registration being one of them. Let’s assume we have coded and automated 5 test cases where first test case executed is registration. All other tests cases are executed if the first test case which is registration pass. If the registration fails for some reason rarely (unless failing always) then all other test cases are skipped or fail. I do not like that. So I would make registration as one of the step to be executed in every test case. Even if the registration fails in one of the test case, rest of the test cases executed successfully.

 

  • Choosing an automation software and the binding (programming language) which has immense support for other plugins which are needed and enhance the automation framework. The tool selected should be stable, not too difficult to learn, has good technical documentation and support, the fixes for serious bugs are delivered quick enough and lastly should have the features to automate almost everything that we have and we will have in the product to be automated. If the tool supports multiple bindings I would prefer a binding which has wider support for third party plugins to support the features which are most needed in an automation framework (Say, test results reporting, test case management, and spreadsheet read/write plugins such as JUNIT, TestNg and apache POI etc. TestNg is supported in java, python does not support it). If there are multiple bindings which are equally good, I shall prefer the one in which I am more good at and comfortable than learning the new programming language. But, there have been organizations who prefer a binding(programming language) which is not having support for plugins as much as other bindings for a reason that they are mostly comfortable with that binding and do not intend to migrate to new language which would demand time to learn and code proficiently. Change is always a difficult option!!

 

  • Implementing test cases which are easy to scale and maintain. If I have to give an example, think of a login page in a web application which needs to be automated. We write tests which automate entering the username and password text fields and click on “submit” button. The script works as expected now. Tomorrow the username and password text fields order are exchanged. Unless the developer has though over this and has coded the scripts with locator which are robust enough to rightly locate the username and password fields irrespective of the order is bound to fail.

 

  • Implementing test cases with readability and maintainability as one of the necessity. The automation scripts should reuse the code. So always write helper functions common across all products and specific to a product and reuse them to reduce code clutter and to enable readability and modularity. Say an application has number of pages where we need to enter the mailing address having fields first name, last name, building number, street name, city, state and pin code. Instead of repeating the same lines of code for test cases for different pages which prompt for mail address, I will add a helper function which accepts all fields as arguments and edits the fields. Now each test case need to reuse this function with the right data passed. The result is scripts are more readable. Imagine there is a bug or a new field has been added to “mail address” block say, landmark. All we need to do is extend the helper function. So easy to maintain.

 

  • Apply and enforce coding guidelines so that it’s easier to understand and modify the scripts over a long time. Even if the original developer of the script has quit the organization or during original developers absence the other developer does not find difficulty in understanding the implementation to fix or to change existing scripts to address new requirements.

 

  • Ability to generate a test report which would give enough information about why the test case failed and other useful statistics (without a need of human presence as the automation suite is run). This demands ability to embed extra intelligence which would attempt to diagnose to zero-in on the cause for failure and other supporting details.

 

  • It should enable test case management, so that we can choose the test cases to skip in easier way instead of getting into at the code level.

 

  • Ability to support continuous integration and test result reporting by mediums such as email or SMS (as an SOS for test case failure) to the stakeholders to get a quick feedback and act so. Triggering automation run needs to be time driven (say every day at 10:15PM) and event driven (some code changes are committed to code repository or new build generated in the build server triggered by code change commits).

 

  • The automation framework should enable necessary level of customization by means of configuration files which can either be edited directly or by using a GUI tool than getting into the code and other technical aspects. The other aspect is to apply configuration changes without having to rebuild the automation framework unless it hits the performance of the automation in action or it’s highly difficult and value added is not worth.

 

  • Speed of the test execution is crucial when we intend to test large number of test cases with even larger amount of test data. Should avoid fixed delays except rare cases and design should also be taken care at function level for speed. It would be great if the automation tool supports running the tests in parallel on multiple machines with different environment/platform to speed up the tests and to verify cross platform compatibility.

 

  • Provide test coverage report at test case and code level (at code level it’s called code coverage).

 

  • Ideally should be test platform/environment agnostic (OS, browser, machine architecture, time zone etc.) as far as possible. This needs intelligently designing the framework.

 

These are purely my opinions about a good automation test framework based on my experience and learning in software automation and software development in general. A constructive feedback or questions or discussion are welcome. Good day!

ABOUT SANDEEP TUPPAD (VP, Mobile Apps Testing and Automation)

SandeepTuppad - VP, Mobile Apps TestingSandeep Tuppad has a decade of experience in software development (Drivers & Application Software) for Consumer Electronic Devices such as Digital Television and other display devices. He has prior experience of implementing automation framework and utilities aiding software testing. At TestInsane, he is responsible for exploring and implementing test automation framework, utilities, gathering knowledge and mentoring the team to enable “quality mobile apps testing”. He loves to travel and pursue outdoor adventure sports.

 

Try our web and mobile test automation services – Buzz me at sandeep (at) testinsane (dot) com.

TestInsane - Web and Mobile Test Automation

TestInsane – Web and Mobile Test Automation

My journey to become wacky tester has begun!

While I started my career as a Tester, I thought  I wanted to be unique from others but I knew I had to go miles for that, but I didn’t lose hope. I started surfing through the web and I went  through the blogs of some famous Testers. Finally, I found someone interesting; yeah that was Mr. James Bach. I browsed his blog and found it as interesting and impressive.

On that day, I came to know there were lot of things I have to learn apart from the things what I had done so far on Testing. I was interested to learn about the new concepts in Software Testing like Rapid Software Testing,Context Driven Testing.

That was the  time I felt I was lucky I noticed the Skype ID of Satisfice INC. In addition to that, I also happen to find James Bach’s Skype ID too. I was excited and requested him to add me in his contacts and waited for the response. Yeah, finally I was connected with James Bach. I was really happy and I thought that I attained the first step of success.

Suganthi SelvaKumar

Suganthi SelvaKumar

Then I discussed with James about what actually Testing was? Then James explained some concepts like Mental-Model approach and Oracle-Problem solving method. James introduced me some famous Indian women Testers like Parimala Hariprasad and Smita Mishra. Not to forget that he also mentioned about Lalit Kumar Bhamare. And also he shared the articles with me which he has published. Thereafter, I decided in order to attain my uniqueness, this is the correct time that I have to go for a better team to work with which will add value to my life as a software tester.

After that, I decided to quit my job from my current company and planned to move to an start-up which will add value to my future. I discussed about this with James Bach, then he introduced me to Test Insane and spoke about Santhosh Tuppad, provided with the Skype ID of Santhosh Tuppad. I surfed about Test Insane, and then I was interested on watching their mindmapping approach and Heuristic table of testing. They gave importance to the skills of a tester and the kind of values they live by was fascinating. I decided that TestInsane was a perfect testing start-up in which I can be able to learn a lot of things that will add value to my knowledge and can explore my ideas in testing the software better.

Then immediately, I approached Santhosh Tuppad and showcased my interest in being part of Test insane and finally Santhosh Tuppad provided me an opportunity to work with TestInsane. Yeah, am an member of Test insane now I felt happy on writing this.

My first day experience was different because while entering into TestInsane itself I found out that Test insane was not like other organizations, but unique. From my first day itself I came to know a lot of new things from Santhosh Tuppad like Mind map approach and Heuristic Table of Testing, he explained with clear examples. Now, am interested in the processes of creating the mindmap approaches. I am sure that Santhosh and his team will guide me in  best way to attain my success. Working with Test insane will be the second step of my Success. I would like to mention that Santhosh Tuppad is great in thinking differently & in a unique way. And I am very much sure that TestInsane will help me become a better tester through this journey. Thanks James and Santhosh.

This was the perfect moment that I started my career with Test insane and it added the confidence in me that I will be a unique tester who provides a best Quality Assurance testing services to the software and be a freethinker with indomitable spirit.

I’m not a slave to 5 Lakhs!

A fine line differentiates ‘living’ from ‘surviving’. It’s funny how one’s life could revolve around these two words that easily make or break one’s peace of mind ‘coz they both depend merely on the choices made in life. As of today, Sandeep Hiremath a.k.a Freakish Tester at TestInsane Technologies, is living and declares that he’s not a slave to 5 lakhs!

A star software tester, an optimist and a free thinker – Sandeep Hiremath’s story would definitely  force you to reflect on whether your job is one that makes you happy or just one that makes you go on and on until one fine day you say “That’s enough, I need a break!”

Attending interviews had become a hobby of sorts for Sandeep who started off at a BPO providing technical support for Adobe and later hopped jobs between data entry and software testing while finally ending up at TestInsane, after gaining work experience for about 1.5 years. “I don’t remember there being any interview at TestInsane. I was asked 3 technical questions and then shared a casual conversation with Santhosh Tuppad (founder, TestInsane) about my likes and dislikes. Before I knew it, I was hired as an intern. Since then, life’s just been rocking!” he
recalls with a smile.

There’s a difference of worlds between working for the sake of money and working for the sake of happiness! Sandeep found his calling when, after joining TestInsane he received a call for an interview from a third party recruitment firm. Upon answering a few questions over the phone, he waited for them to get back with the result. When he finally got the call, the company (hiring for Accenture) was willing to offer him a salary of INR 5 Lakhs per annum. “When they asked me if I’d like to take it, I said ‘No’. They were shocked and asked me the reason. So, I asked them if they had any strict timings of work, if they expected the employee to abide by a particular dress code, or if they’d want me to work only from the office at all times. When their answers to all my questions were ‘Yes’, I knew then that I would never be a slave to 5 lakhs if it meant that I’d have to do the same old thing day after day and at the end just lose myself and my peace of mind. Rejecting that offer gave me the confidence to never regret having done it!”

Sandeep Hiremath

Sandeep Hiremath

From drafting simple e-mails to clients, publishing blog posts that translate his experiences at TestInsane, writing elaborate bug reports, to leading a project; it’s a learning engagement all the time which according to Sandeep has a value higher than 5 Lakhs!

“At the end of each day, it’s about enjoying a good night’s sleep,” says Sandeep, and that’s what he gets by doing what he loves the most and working at a place that lets him be who he is – a freakish tester!

Maybe it’s not always about the money, maybe it’s not always about doing the ‘right’ thing. Sometimes it’s also about listening to what you want, exploring a little and taking a few risks. If you get lucky, you might understand what it’s like to not be a ‘slave’.

TestInsane Team

TestInsane Team

For those yet to embark on their careers, instead of using Sandeep’s story as an inspiration, take it as a lesson – that it’s not necessary you need to start the conventional way. Follow what makes you happy because it’s very easy to find a well paying job but it’s equally hard to find one that makes you fall in love with what you do!

So, here’s to TestInsane’s Freakish Tester – Sandeep Hiremath – May the spirit, the love and the rockstar in you, never die!

Please See: “To know more about Sandeep Hiremath – TestInsane’s rockstar tester, visit his blog at http://freakishtester.blogspot.in/

A thousand words are not enough… ~ Deepika Burli

Posted on by 5 comments

Deepika Burli is hired by TestInsane to help us with anything related to content-writing which includes Press Releases, Blogging, Drafting stories of team members and more. Here is her experience that she shares in her own writing style. I personally loved the way she writes. Enjoy reading!


Trying to kill time after your undergrad, taking a break to realize your options and trying to do stuff that would just keep you occupied can actually work. At least it did for me. I never had a clue that writing content for just one company would open doors to meeting and writing for people (firms) who would change the way you perceive so many things!

About a month after I started writing content for a friend’s start up, I received a mail from Santhosh Tuppad. I didn’t gather much from the term Exploratory Software Testing, but I did gather this much that he wanted me to write about it, about his company Test Insane and probably other stuff. My first expression was “How the fuck am I supposed to write about anything related to software?” My friend sitting next to me at that time said “If it’s knocking your door, then just cock-up and take it. You’ll anyway learn in the process.”

Deepika Burli - The Content Ninja!

Deepika Burli – The Content Ninja!

Honestly, I thank my stars that I always listen to her, for I found myself sitting in front of Santhosh for about 2 and a half hours at Matteo, talking about everything else but software. That was when I knew I’d be an idiot to not try this out!

Exactly a week later, I landed up in this small, endearing workplace in Jayanagar (a place I’ve never braved exploring before… but by now I was convinced that there’s always a first time!). “So this is where brainiacs decode the world of bugs” was the only ignorant thought running through my mind while I shook hands with Sales Daddy Srinivas “Srini” Bharadwaj (I call him that ‘coz he loves his job and he loves being a dad) and Insane tester Sandeep Hiremath, who brags about doing the most fortunate thing by joining Test Insane.

While having a rather long and chatty conversation with them we were joined by Sandeep Tuppad, another obsessive tester and a passionate traveler (a cyclist and mountaineer to be specific). Although it took him a cup of tea to actually open up his Pandora’s box – a silent player’s always the one with crazy ideas!

While in conversation with Pranav KS probably the youngest in the team, I realized that this company doesn’t work on the basis of how experienced one is but on the basis of how passionate one is, to learn, explore and improve. Pranav, without doubt is a rockstar in the making.

While I didn’t have much of a conversation with the VP of Test Insane, Karthik Kini (again, a very silent player), I did find a connection with his hometown and fell in love with his idea of establishing the same organization back there only because of the serenity one can work with in a place as tranquil as Udupi!

I’ve been told there’s another member whom I unfortunately couldn’t meet, Ajay S Menon. He’s been bragged about by his team mates to be a star creative head at Test Insane, who works from anywhere he likes.

Finally, coming to the man himself, Santhosh Tuppad can hold the most uncommon conversations which end up making sense all the time. He eased me into the ideology of his entire establishment, his victories, his career history that he has learnt a lot from and more than anything, his outlook about everything big or small defines not just him but also the company he so fondly founded.

When asked for an interview as a new member (although on a freelance basis) of Test Insane, I thought it would be fitting to pour my first impression of the company out as a blog post. For a little about me, I’m a Graduate with a degree in Journalism and a passion for writing which still needs a lot of improvisation! I’m on a short break from everything related to studies to just explore stuff (like meeting insane, crazy testers ;-)) that I probably wouldn’t have if I’d been sitting in a classroom right now.

So here I am, enjoying writing every word of this insanely huge post (sorry for that) leaving no page unturned in expressing how delighted I am to join Test Insane as their Content Writer.

INDIA TESTING AWARDS: TestInsane is TESTING STARTUP OF THE YEAR 2015

We always knew that we were doing great work in software testing as a start-up and we believed in our work. I happened to receive call from Unicom and also an e-mail about the nomination of TestInsane for “Testing start-up of the year 2015” category. I did not respond to the e-mail, but again I received a call from Unicom to know about my decision. I spoke to one of the person from Unicom and I said, “We don’t want to participate in this because we already know that we are insanely awesome in our work and wouldn’t be nice if we don’t win the award”. But, again I gave it a thought and said “YES” for the nomination. We had to talk about our various work at TestInsane so that the jury members could decide on one winner. I was pretty confident that we shall win this award for sure because we had done great work for our customers and also software testing community in terms of open-source contribution by developing apps for testers to help them test better.

And on the day 24-July-2015, at 1700 hours India Testing Awards 2015 event started. It started with awards for various people in different categories of awards and when the time was to announce “TESTING STARTUP OF THE YEAR 2015”, I could feel goosebumps when TestInsane was announced as winner. I was very happy to receive this award and also we were very quick in getting here.

Thank you Unicom & Next Gen Testing for this award. This is special for us. We continue to do great work and amaze the world. Keep showing us the love :-)

Last, but not least I thank all our team members for making us capable of getting this award. Special thanks to Karthik Kini, Sandeep Tuppad and Ajay S Menon. We are awesome as a cool team.

Below are some pictures taken by me :-)

 

TESTING STARTUP OF THE YEAR 2015 Award

TestInsane - TESTING STARTUP OF THE YEAR 2015

TestInsane – TESTING STARTUP OF THE YEAR 2015 (Srinivas on the left is our sales bragger) And the beard guy is Santhosh Tuppad!

TestInsane is the TESTING STARTUP OF THE YEAR 2015