July 2, 2018

Stop Compromising Quality: Setting Automation Up for Success - Angie Jones, Ministry of Testing - Boston

Missed last Wednesday's Ministry of Testing - Boston Meetup, hosted and sponsored by SmartBear? Where Angie Jones, SDET at Twitter, provides metrics that help her decide what to automated at Twitter?

Have no fear! SmartBear live-streamed the conference. It starts at the 52 minute mark.

Stop Compromising Quality: Setting Automation Up for Success https://youtu.be/WAu1gWmKp5w


Thank you so much Brianne Cordima and Bria Grangard of SmartBear for setting up this event as the Ministry of Testing - Boston's event organizers!


Slides:Which Tests Should We Automate? - Angie Jones




Setup


Let's say you were presented at Twitter with ten features that may need automated tests written. How do you decide what to automate?


  • Add a Tweet
  • View Tweets
  • Pin a Tweet
  • Follow a User
  • Set a Handle / Username
  • Set Location
  • Update Handle
  • Block a User
  • View Analytics of a Tweet
  • Balloons Appear When It's Someone's Birthday

At the Meetup, Angie had us all pick a partner to analyze these features together.

Step One: What is Your Gut Reaction?


Check off which features, in column "G", before you start analyzing the features, which ones may need automated tests written.


Step Two: Measure the Risk


There are two metrics to consider. On a scale of 1 to 5, what value should each metric be?

  • Impact: If broken, what's the impact to customers?
  • Probability: Frequency of use by customers.

Multiply the two values you came up with. Write the value in column "R".


Step Three: Measure the Value

On a scale of 1 to 5, what values would each feature have?

  • Induction to Action: How quickly would this failure be fixed?
  • Distinctness: Does this test provide new info?

Multiply the two values you came up with. Write the value in column "V".


Step Four: Cost Efficiency


On a scale of 1 to 5, what values would each feature have?

  • Simplicity: How easy will it be to script this?
  • Time: How quickly can this be scripted?

Multiply the two values you came up with. Write the value in column "C".



Step Five: History


Let's say the following areas have these many bugs:

  • Setting location: 51
  • Updating handle: 27
  • Pinning Tweet: 2
  • Viewing Tweets: 10
... Wow! Geolocation has so many bugs associated with it!


On a scale of 1 to 5, what values would each feature have?

  • Frequency of Breaks: Volume of historical failures for this test. Geolocation services would definately get a "5" for that. Balloons? It would get a "0" since there are not any bugs associated with it. 
  • Similarity to Weak Areas: Volume of Historical Failures in related areas.

Multiply the two values you came up with. Write the value in column "H". 


Step Six: Tally Up The Scores





Thank you so much, Angie Jones, for running this workshop.

Happy Testing!

-T.J. Maher
Sr. QA Engineer, Software Engineer in Test
Meetup Organizer, Ministry of Testing - Boston

Twitter | YouTube | LinkedIn | Articles
Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)