Webinar: Joe Colantonio talks about Test Automation Trends for 2016

Test Automation Trends for 2016 and Beyond

Staying Employable In Changing Times

Sauce Labs Webinar
1/12/2016: 2:00 pm: 
Joe Colantonio, TestTalks.com

Coding isn't the hard part of my role of being a brand-new automation engineer. Although I'm in my forties getting into development for the first time, I am actually doing okay -- in spite of always stressing out about the fact that grad school was a decade ago. I’ve now been writing automation scripts in Java and WebDriver for a year coming this March. Do anything for a year and you start becoming more comfortable with it.

It's keeping up with the latest trends in automated software testing, now that is the hard part!



It was only a year ago that I had the nagging fear that because my sole knowledge base was manual testing, my entire software testing career was dead, dead, dead. This has since been replaced with other nagging fears: Can I keep up with the industry standard? Is what I am doing now the industry standard? Is there, in fact, an automated testing industry standard? I'm not privy to what other companies are doing. I’ve never worked as an automation developer for any other companies besides the one I am working for now. It is hard to plot out a graph with only a single point of data.

That is why I was so excited to hear from the Sauce Labs mailing list that they would be hosting a webinar on test automation trends for 2016, to be given by Joe Colantonio from TestTalks.

After doing a bit of research about the speaker, I was quite excited to hear what Joe had to say about software testing, and registered for the Sauce Labs Webinar immediately.

About The Talk

http://info.saucelabs.com/TestAutomationTrends2016_RegLP.html

"Don't let your testing efforts fall behind — its time to get ready for the new year. In this session, Joe shares his insights about upcoming trends in testing gleaned from his popular TestTalks podcast interviews. He will discuss the skills, best practices and frameworks you need to know to stay employable in a changing dev/test world". 

About Joe Colantonio

Joe Colantonio a prolific writer and blogger who —  according to his LinkedIn page — has been in the software testing field for the past fifteen years at places such as Verisign and GE Healthcare. His main site, “Joe Colantonio: Automation Awesomeness” at http://www.joecolantonio.com/ connects to:

• TestTalks:  Mr. Colantonio's podcast is now up to the 85th episode, which interviewing Dave Haeffner about his Selenium Guidebook - Java edition. Sauce Labs is the exclusive sponsor of TestTalks... I wonder how long it would take to listen to all of them on my commute to and from work?
• Automation Awesomeness: Mr. Colantonio's blog has been going since July 4, 2010, when he talked about Mercury WinRunner and Quick Test Pro (QTP) 

I was so excited about the talk, I booked a conference room at my workplace and streamed the presentation so the rest of the automation team could listen in.

Sauce Labs Webinar: Test Automation Trends for 2016:

Slides: Test Automation Trends for 2016

Test Automation Trends for 2016 and Beyond from Sauce Labs

My Notes on the Talk:

Joe Colantonio started the talk by joking that the reason he has his blog is so that he can stay employed. It is so tough to keep up with the latest trends, so now with his blogs and his podcasts, he is forced to stay in the loop with current trends.

Staying in Touch With Current Trends

How Mr. Colantonio stays in tune with the current trends:

Indeed.com Trends:
http://www.indeed.com/jobtrends

• You can plug any job search keywords in and see how the buzzwords are trending upward or downwards. 

Google Trends:
https://www.google.com/trends/

• Take a look at what other people have been searching for over time. 

Github:
https://github.com/

• Have an open-source product? Everyone stores their source code here, free to explore. You have Angular 2, Node.js  and of course Selenium. It even has the W3C spec for WebDriver API and Wire Protocol at https://github.com/w3c/webdriver.

World Quality Report 2015-2016
https://www.capgemini.com/thought-leadership/world-quality-report-2015-16

• From Capgemini Consulting: "The speed of digital transformation and the reduction of time-to-market life cycles for products and services raise the importance of Quality Assurance (QA) and Testing for organizations worldwide. How have they been adapting to meet the needs of today’s digital environment?" A four minute intro video and a summary is on the site. You can sign up to get an email link to download their free 80 page PDF.

Zephyr Annual Report

• "Zephyr is an on-demand, cloud based Test Management Platform with innovative applications and unparalleled, metrics based visibility via real time dashboards." Zephyr also publishes an annual PDF, "How The World Tests" at  http://go.getzephyr.com/HowTheWorldTestReports. If you do not want to sign up for the PDF it also has an Insights section talking about trends in the testing industry. 

State of Testing Reports

• Every year, PractiTest on their QABlog at http://qablog.practitest.com/state-of-testing/ asks testers to fill out a survey. You can get the results of the 2015 survey in PDF  format. Take the State of Testing 2016 survey! 

Testing Has Changed. Testing Keeps Changing. 

On my blog, I've mentioned about Waterfall + QA, Before Waterfall and Agile Software Development.

Before: Joe Colantonio pointed out, that with Waterfall, we were all siloed: From Business Analysts creating the requirements to Developers writing the code, to Testers / QA. It was a process in serial, slowly making its way from the left to the right.

Joe Colantonio, We're All Testers Now

Now: we are developer tests and ops / production.

 In the Future: DEV + Test + Ops will all be working together

Joe quoted a guest on his show, Paul Gerrard, a software testing consultant who demoed at EuroSTAR 2015, "A New Model For Testing":

“Testing is not a role - its an activity with a deliverable.” - Paul Gerrard

... Whether the developer or tester is doing it, testing needs to be done.

SHIFT +LEFT

Mr. Colantonio introduced the new trend of "Shifting Left". With Waterfall, everything used to be siloed Business Analysts did their work, Developers did theirs, and Testers did theirs. With Agile work, there Job responsibilities had to shift... and they did. They shifted left. Everyone needs to be a tester.  Testing is up front, not just at the end.

I found another article explaining it further: What is Shift Left Testing?

"In the past, most traditional test automation focused on the UI and not the back-end. This was a leftover mindset from the days of 'just throw it over the wall'. With the current shift in how teams are organized, the skillset for testing is evolving. Dev/Ops, continuous delivery, mobile, cloud and the Internet of Things is forcing a rapid pace of development. Now, QA professionals and business analysts are more ingrained in the overall development process from start to finish. Now that software development for different platforms are using a 'shift-left' approach (which involves moving testing to earlier in the software development lifecycle), it is requiring test automation engineers to enhance their technical skills" From https://www.brighttalk.com/webcast/8653/145355

Mr. Colantonio mentioned other trends he had noticed were Using DevOps principles, and Configuration Management Automation with tools such as with Chef and Puppet. I did a bit of independent research on these terms:

• About Puppet: https://puppetlabs.com/about

"Puppet Labs is the leader in IT automation. Our software helps sysadmins automate configuration and management of machines and the software running on them. With our software, businesses can make rapid, repeatable changes and automatically enforce the consistency of systems and devices, across physical and virtual machines, on prem or in the cloud".

• About Chef: https://www.chef.io/about/

"We are Chef. We are IT automation for speed and awesomeness. We give you a model for automating IT infrastructure and applications that drive self-reliance across your development and operations teams. We are the Chef Community. We are tens of thousands strong. We are helping your businesses become faster, safer and more flexible, so you win in today's 24x7 digital economy. Join our movement today".

• About DevOps: https://en.wikipedia.org/wiki/DevOps

"DevOps (a clipped compound of "development" and "operations") is a culture, movement or practice that emphasizes the collaboration and communication of both software developers and other information-technology (IT) professionals while automating the process of software delivery and infrastructure changes. It aims at establishing a culture and environment where building, testing, and releasing software, can happen rapidly, frequently, and more reliably.  

"[...] At the Agile 2008 conference, Andrew Clay Shafer and Patrick Debois discussed 'Agile Infrastructure'. The term 'DevOps' was popularized through a series of devopsdays starting in 2009 in Belgium. Since then, there have been devopsdays conferences held in many countries worldwide. 

"Tools such as Docker and Jenkins have automated more quality assurance processes and releases by development team. Puppet and automated configuration tools like Vagrant have also been used and frequently referenced in DevOps discussions."

• About Continuous Integration, according to Thoughtworks: 

"Continuous Integration (CI) is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early. 

"Because you’re integrating so frequently, there is significantly less back-tracking to discover where things went wrong, so you can spend more time building features. 

"Continuous Integration is cheap. Not continuously integrating is costly. If you don’t follow a continuous approach, you’ll have longer periods between integrations. This makes it exponentially more difficult to find and fix problems. Such integration problems can easily knock a project off-schedule, or cause it to fail altogether".

Shift + Right

Mr. Colantonio pointed out that not only is it Shift + Left, where everyone is a tester. It is is also Shift + Right, where everyone is a developer.

Automation should be as fast as possible. Going through text based reports? Joe suggested writing python scripts to go through the reports.

Other Trends

TDD/BDD 39%. We want to make sure that products and code wows our customers. Using behavior driven development should not be just an automation tool. It speaks about requirement as early as possible. When I ama user, I should go to the page and get this result. You can use an English syntax, and talk about the requirements.

• About Test Driven Development, from Wikipedia:

"Test-driven development (TDD) is a software development process that relies on the repetition of a very short development cycle: first the developer writes an (initially failing) automated test case that defines a desired improvement or new function, then produces the minimum amount of code to pass that test, and finally refactors the new code to acceptable standards. Kent Beck, who is credited with having developed or 'rediscovered' the technique, stated in 2003 that TDD encourages simple designs and inspires confidence. 

"Test-driven development is related to the test-first programming concepts of extreme programming, begun in 1999, but more recently has created more general interest in its own right."

• Behavior Driven Development example:

Story: Returns go to stock 

In order to keep track of stock 

As a store owner
I want to add items back to stock when they're returned 

Scenario 1: Refunded items should be returned to stock
Given a customer previously bought a black sweater from me
And I currently have three black sweaters left in stock
When he returns the sweater for a refund
Then I should have four black sweaters in stock

With TDD you need to get developers who love to test.

Vendor Tools

Mr. Colantonio noted that Selenium WebDriver is THE tool that people are using, around 45%.
QTP / UFT (Quick Test Pro /  Unified Functional Test): Only 10% are using. Want more info? See on HP's site, UFT is the new QTP.

Selenium is morphing with Vendor tools, such as SmartBear TestComplete, HP LeanFT, Teller Test Studio. Vendor Tools act as a wrapper around Selenium. These tools import the Selenium WebDriver API.

Joe Colantonio, Selenium and Vendor Based Tools

• About SmartBear TestComplete: http://smartbear.com/product/testcomplete/overview/

"TestComplete is a functional automated testing platform developed by SmartBear Software. TestComplete gives testers the ability to create automated tests for Microsoft Windows, Web, Android (operating system), and iOS applications. Tests can be recorded, scripted or manually created with keyword driven operations and used for automated playback and error logging. [...] TestComplete is broken out into three modules: Desktop, Web, Mobile [...]"

• About HP LeanFT: Press release on June 2, 2015:

"Businesses today must act with speed and agility, and continuously capitalize on opportunities to deliver new solutions and experiences, business models, and revenue streams. In software development, this has fueled the rise of Agile methodology, which embraces a continuous delivery process, rather than a phased, sequential approach. For software testing, this means developers and testers seek to 'shift left,' and continuously test their applications much earlier in the application development lifecycle. 

"HP LeanFT embraces the shift left concept by leveraging the key tools of the modern Agile developer ecosystem. Powerful, lightweight, and cost-effective, HP LeanFT is built specifically for continuous testing and continuous delivery, and fits naturally into existing ecosystems (such as Microsoft TFS, GIT, and Subversion) and frameworks that support test driven and behavior driven development". 

• Teller Test Studio: http://www.telerik.com/teststudio

From Wikipedia: "Telerik Test Studio is a Windows-based software testing tool for web and desktop, functional testing, software performance testing,load testing and mobile application testing developed by Telerik. The tool ships with a plugin for Visual Studio and a standalone app that use the same repositories and file formats. Test Studio supports HTML, AJAX, Silverlight, ASP.NET MVC, JavaScript and WPF. Test Studio supports test execution in Internet Explorer, Firefox, Safari and Chrome".

Why all these vendor tools wrapping around Selenium? I had understood that the W3C is working on a web driver standard, but what I hadn't realize was that once web driver is a standard, all companies need to support it. Since WebDriver is nothing more than an API, other programs can use it, placing it in a user-friendly interface.

Gartner's Magic Quadrant for Software Test Automation 2015

If you are looking for industry standards, Joe Colantonio brought up The Gartner's Report, Magic Quadrant for Software Test Automation 2015:

Gartner (December 2015)

Companies and Toolsets mentioned in the report:

• Testing Anywhere, by Automation Anywhere 
• Silk Portfolio, Silk Mobile and Silk Central Connect, by Borland, now owned by Micro Focus. 
• Hewlett Packard Enterprise: As of 1 November 2015, HP separated into two independent publicly traded companies: Hewlett Packard Enterprise (HPE) and HP Inc.
• IBM Rational Suite
• Oracle
• Original Software
• Progress Telerik  Test Studio
• Ranorex
• SmartBear TestComplete
• TestPlant 
• Tricentis 
• Worksoft 

... and there are a few more pages of notes that I need to transcribe...

Thank you very much, Joe! We loved your talk! We're so glad we attended this session. Where else would we have found so much information in one place!

-T.J. Maher
 Sr. QA Engineer, Fitbit
 Boston, MA

// Automated tester for [ 10 ] months and counting!

Please note: 'Adventures in Automation' is a personal blog about automated testing. It is not an official blog of Fitbit.com.