SDET Prep: Data Structures: Arrays, Hashmaps and how to implement them in Java.

This blog post is part of a series as I research how to move from Automation Development to being a Software Developer in Test. We covered some basic algorithms, before. These next few posts will deal with data structures, illustrating them with Java. 

As an automation developer, I've been focusing on browser testing with Selenium WebDriver and Java, being able to draw from my decades of experience testing web applications. When it comes to various data structures, I use arrays, lists, arraylists, and the occasional hashmap in my day-to-day work, but that is about it.

A Software Developer in Test (SDET) isn't that far removed from a Software Development Engineer (SDE), relying on data structures and algorithms I once studied as a Computer Sci major back at Bridgewater State. Note: Each link below goes to the respective Harvard University CS50 video, if any.

• Data Structures: Arrays, Singly Linked Lists, Doubly Linked Lists, Binary Trees, Stacks, Queues, ArrayLists, and Hash Tables
• Algorithms: Breadth First Search and Depth First Search (not on CS50), Binary Search, Insertion Sort, Merge Sort, Quicksort. 
• Concepts such as Recursion, Bit Manipulation (not on CS50), Pointers, Dynamic Memory Allocation and Structures 
• Design Patterns like: Singleton Design Pattern, Factory Design Pattern
• Storing Memory in Stack vs. Heap

If you are a manual tester or automated tester and want to shift to software development, Gayle Laakmann McDowell's Cracking the Coding Interview is an excellent summation of all terms listed above. It seems to be a good resource for even Senior developers, who may be far removed from college.

SDET Prep: Mergesort: Walkthrough and sample code from "Cracking the Coding Interview"

This blog post is part of a series as I research how to move from Automation Development to being a Software Developer in Test. These next few posts will deal with algorithms.

I've often found good technical videos as difficult to find as good documentation, and for the same reasons... they are geared more towards developers who use the technology day-to-day. The last time I used sorting algorithms on a regular basis such as Insertion and Selection Sort, Bubble Sort, Mergesort, and Quicksort was when I was a Comp Sci Major back in the 1990s. I haven't really seen them while I have been an automation developer.

Luckily, there is Harvard University's CS50 at https://www.youtube.com/user/cs50tv

With our last blog post, we covered the sorting algorithm Insertion Sort, good for sorting small amounts, but not very efficient for large amounts. In this blog post, we are going to be covering an algorithm that is faster for larger amounts... though it does take up a bit of space.

Mergesort

Mergesort is a "divide-and-conquer" algorithm. According to Wikipedia,  "In computer science, divide and conquer (D&C) is an algorithm design paradigm based on multi-branched recursion. A divide and conquer algorithm works by recursively breaking down a problem into two or more sub-problems of the same or related type, until these become simple enough to be solved directly. The solutions to the sub-problems are then combined to give a solution to the original problem".

Stanford University  professor emeritus Donald Knuth -- author of the 1960s saga "The Art of Computer Programming" -- cites the mathematician and computer scientist John von Neuman as inventing the Mergesort in 1945.

SDET Prep: Insertion Sort and Learning By Video

This blog post is part of a series as I research how to move from Automation Development to being a Software Developer in Test. These next few posts will deal with algorithms.

When I find my programming skills are a bit rusty from disuse, I find watching educational videos on the internet is the next best thing to face-to-face instruction in a classroom.

With online lessons, I can repeat an example as many times as I want, rewind the lesson, and I can pause the screen to type out the code they are using, playing around with it immediately -- the best way to learn code.

Within the last couple of blog posts, when we started talking about measuring the efficiency of algorithms using Big-O notation, I displayed Eric Drowell's Big-O Complexity Chart of commonly used sorting algorithms.

We are going to cover three of those sorting algorithms in a bit more detail: Insertion sort, mergesort, and quicksort, starting with Insertion sort.

As a reference, we'll use lessons from Khan Academy, and from Harvard's CS 50.

SDET Prepwork: What is Big O notation and Logs? Comparing Growth Rate in Algorithms

This blog post is part of a series as I research how to move from Automation Development to being a Software Developer in Test. These next few posts will deal with algorithms.

What is the difference? ... Well, the main thing is that as an automated tester, I haven't needed to worry about Big-O notation and sorting algorithms, such as the ones found on Eric Drowell's Big-O Cheat Sheet.

Big-O Complexity Chart

Excellent

Good

Fair

Bad

Horrible

Array Sorting Algorithms

Algorithm	Time Complexity			Space Complexity
	Best	Average	Worst	Worst
Quicksort	O(n log(n))	O(n log(n))	O(n^2)	O(log(n))
Mergesort	O(n log(n))	O(n log(n))	O(n log(n))	O(n)
Timsort	O(n)	O(n log(n))	O(n log(n))	O(n)
Heapsort	O(n log(n))	O(n log(n))	O(n log(n))	O(1)
Bubble Sort	O(n)	O(n^2)	O(n^2)	O(1)
Insertion Sort	O(n)	O(n^2)	O(n^2)	O(1)
Selection Sort	O(n^2)	O(n^2)	O(n^2)	O(1)
Tree Sort	O(n log(n))	O(n log(n))	O(n^2)	O(n)
Shell Sort	O(n log(n))	O(n(log(n))^2)	O(n(log(n))^2)	O(1)
Bucket Sort	O(n+k)	O(n+k)	O(n^2)	O(n)
Radix Sort	O(nk)	O(nk)	O(nk)	O(n+k)
Counting Sort	O(n+k)	O(n+k)	O(n+k)	O(k)
Cubesort	O(n)	O(n log(n))	O(n log(n))	O(n)

The Big O Cheatsheet By Eric Drowell

In this blog post, we are going to be focusing on walking through the sorting algorithms Insertion Sort, Mergesort, and Quicksort, and implementing them in Java code.

First things first, though...

What is Big O Notation? 

Big O notation is how computer scientists compare and contrast the efficiency of sorting and searching algorithms, the tried-and-true recipes used to write programs that search through massive amounts of data.

The concept of Big O notation was first introduced and the popularized by the German mathematicians, first by Paul Gustav Bachman, in his book on analytical number theory, Analytische Zahlentheorie, and then by Edmund Landau.

The "O" symbol mentioned in the book was initially an omicron, the 15th letter of the Greek alphabet, showing a formula to show the formula of a rate of growth.

SDET Prepwork: What is an SDET? Intro to the Efficiency of Sorting Algorithms: Insertion sort vs Merge sort

This blog post is part of a series as I research how to move from Automation Development to being a Software Developer in Test. These next few posts will deal with algorithms.

What is the Difference between Automation Developers and Software Developers in Test?

Automation Developers Use Code to Create Tests

Have you ever had to perform browser testing on your web application just before a release to production, taking a week or so to do regression testing, making sure that after the new functionality was added that all of the old stuff still worked? And you have ended up staring at the same twenty pages in a variety of Internet Explorer browsers, Firefox, and Chrome, and attempted to make sure the elements on the page are there? That you can still log in? That you can still perform the same actions in all of the browsers? Don't you wish that someone could just write a computer program to do this testing for you?

Create Browser User Interface Tests: When it comes to browser testing, automation developers solve this problem, by pairing Selenium WebDriver with a computer language such as Java to write clean, readable test code that uses basic object-oriented design principles such as:

• Factoring out any commonalities in your tests into separate methods so it is easier to maintain.  
• Grouping the web elements and methods that interact with the web elements on a page into Page Objects
• Creating wrappers for Selenium objects to provide better diagnostics when something goes wrong... Or using the Page Factory to lazy load the web elements initializing them just before you need them.
• How to use RemoteWebDriver and implement different browsers into your testing with Selenium Grid, Browser Stack or Sauce Labs.

Write Performance Tests: An Automation Developer deals with Stress Testing, Performance Testing and Load Testing using tools such as Apache JMeter.

Script Database Tests: Want to make sure that the data entered in the browser user interface is stored correctly in the database? Use libraries in your favorite programming language that handle opening connections to the database, and SQL scripts that query that the data was placed in the correct table.

As an Automation Developer, you might be working with Arrays, Lists, and ArrayList to assemble and store data, for loops and foreach loops, but their focus is on creating the tests.

Software Developers in Test Use Code to Write Test Frameworks

Software developers in Test take what the Automation Developer does one step further.

From the Microsoft Developer article, What is an SDET? 

"The SDET role can be compared with that of the SDE (Software Development Engineer) role. The latter is what most people would think of when they say 'Developer'. An SDE (depending on level and seniority) will Architect, Design, and Implement software systems that will be used by the target end users. SDEs are responsible for the quality of their software and will implement testing (often unit tests), as well as seek reviews for both their designs and their code. SDEs will often have influence on the software processes employed, and be responsible for following best practices in software development.

"The SDET is also a developer. The SDET must have knowledge of what represents good software design. The SDET must be able to create high quality, maintainable [...] code. The code generally created by the SDET however are for automated test cases and the frameworks to execute and report them. An SDET’s knowledge of software design is often focused on testability, robustness, and performance, and they usually play a contributory or reviewer role in the creation of designs for production software. An SDETs skill set will often include more experience in software processes and how to test software. Testing software is generally  done so as to assess and mitigate risk, and SDETs need to be expert in this. So an SDET can be summarized as having development skills and software quality skills. While SDEs should have this also, SDEs balance more towards the former while SDETs balance more towards the latter".

From talking to other Software Developers -- both in Test and in general -- I've gathered that if you want to make the switch, you really have to start thinking more about performance.

They need to focus on:

• Figuring out how to store data more efficiently in data structures such as stacks, queues, linked lists, hash maps, heaps, and binary search trees.
• Learning what a linear sort is and why it is inefficient, and figuring out different ways to search through data studying known search algorithms and sorting algorithms such as binary search, quicksort or mergesort. 
• Understanding where you want to focus on: If memory is at a premium, you want your solution to use the minimum amount of space possible. If time the major factor, you want to make your solution as quick as possible. Your solution should take into consideration these space complexities or time complexities. 
• Comparing and contrasting how fast your solution is using Big-O notation, and figuring out if your solution you are using will scale.

Test The-Internet 2.0: Setting up Selenium Grid: RemoteDriver and DesiredCapabilities

Recap: Since May 2016, we've been blogging about forming a new automated test framework using Dave Haeffner's test site, The-Internet.

Picking Gradle as the build tool, we walked through how to create unit tests, just to make sure everything is still working as we make changes to the project, setting them up in JUnit and Hamcrest in both IntelliJ and Eclipse.

Like the two frameworks we built last year, we are using CSS Selectors to find the web elements on a page. We grouped together pages using the Page Object Model and PageFactories. After walking through the reader with experimenting with first Selenium Grid and then Docker, we set up Selenium Grid using official Docker images from Docker Hub.

... I should call this blog post, "How I spent my summer vacation". Each time we rewrite our test framework, I experiment on this blog with tools and technologies we are using at work. Instead of spending four hours reading a technical manual, I'm spending the same amount of time getting my hands dirty, taking notes all along the way. It takes no time at all to turn all that research I collected into a blog post.

With this blog post, we will be using RemoteWebDriver to bring in the browser nodes listed in our Selenium Grid, connecting them with DesiredCapabilities to help run our tests.

Introduction to API Testing with Postman and Newman

Postman ( https://www.getpostman.com/ ) is tool they use at work to test APIs... along with Swagger, Pact, and Java with Apache HTTPComponents. Postman comes in two versions ... as a Chrome app or a Mac app. They seem similiar, except the Mac app "is packaged with add-ons that make request capturing and cookie handling seamless" according to Postman's documentation.

What is an API?

"Application program interface (API) is a set of routines, protocols, and tools for building software applications. An API specifies how software components should interact and APIs are used when programming graphical user interface (GUI) components. A good API makes it easier to develop a program by providing all the building blocks. A programmer then puts the blocks together". - Webopedia

Need to get up to speed on APIs?

REST API Concept and Examples:



Brought to you by JelledTV's WebConcepts YouTube channel, "This video introduces the viewer to some API concepts by making example calls to Facebook's Graph API, Google Maps' API, Instagram's Media Search API, and Twitter's Status Update API.

• "Youtube's Facebook Page via the Facebook Graph API  
• "Google Maps Geocode API call for the city of Chicago  
• "Apigee Instagram API console  
• "HTTP Request Methods 
• Twitter's Status Update documentation.  

If you want to find many other sites that use APIs, go to ProgrammableWeb.com.

... They also mention how to use this new tool that came out... Postman: https://www.getpostman.com/

Using Docker + Jenkins for Continuous Integration: Assorted Links

So much to do! So little time! There are so many pieces I need to add to my new little automated test framework, Test_TheInternet_2.0 found at https://github.com/tjmaher/Test_TheInternet_2.0. You can see what is on my "wish list" at the end of this blog post.

But what I really want to do is experiment with setting up my own Jenkins environment on my home system, something DevOps does at my workplace. I create, at work, Jenkins jobs to run tests, but it is not the same thing as being able to tinker with Jenkins Plugins, set up the system, etc.

I found out the other day that on Docker Hub, Jenkins has its own Docker image! https://hub.docker.com/_/jenkins/! That is definitely something I really want to tinker with.

• Not familiar with Docker? Read more about Docker on this blog!
• Not familiar with Jenkins? Read more about Jenkins on this blog!

Instead, here are some links that I was planning on checking out that I wanted to share with you. I probably won't be able to get to this for another few months or so, at the rate I am going:

• Access The State of Docker, Jenkins and Continuous Delivery survey infographic: https://pages.cloudbees.com/rs/083-PKZ-512/images/Docker-Jenkins-Survey-Infographic.pdf
• Get the whitepaper Jenkins, Docker and DevOps: The Innovation Catalysts: https://pages.cloudbees.com/Website_Docker-Jenkins.html?lsd=WebsiteForm_DockerJenkins
• Learn more about Continuous delivery with Docker: https://www.cloudbees.com/continuous-delivery/jenkins-docker
• Watch the webinar Continuous Delivery with Jenkins Pipeline and Docker Explained: https://pages.cloudbees.com/2016-0120-Jenkins-Workflow-Explained-Registration-Landing-Page.html

The plan is to finish off the automated test framework we started: Test_TheInternet_2.0. You can see the source code at https://github.com/tjmaher/Test_TheInternet_2.0

• RemoteWebDriver: Taking our DesiredCapabilities we touched upon, and expand it. 
• Docker Compose: https://docs.docker.com/compose/: Bundle together scaling up or down Selenium Grid browser nodes, downloading Selenium Grid, starting up a Hub, and starting up various nodes.
• MicrosoftWebDriver: ( Link ): How to incorporate Microsoft Edge into your tests.
• Microsoft Virtual Images: https://developer.microsoft.com/en-us/microsoft-edge/tools/vms/ add IE8, IE9, IE10, IE11 on Windows 7 or IE 11 and Microsoft Edge to Windows 10 nodes to your Selenium Hub! Just use VirtualBox, Vagrant, HyperV (Windows), VMWare, or Parallels (Mac). Take a snapshot of the images because the originals are only good for 90 days. (What, no Docker solution? Gee, Microsoft, do you always have to be different?)
• Selenium Grid Extras: https://github.com/groupon/Selenium-Grid-Extras

... I also want to check out what Sauce Labs has for its own sample test framework in Java-Junit-Selenium: https://github.com/saucelabs-sample-test-frameworks/Java-Junit-Selenium

As always, Happy Testing!

-T.J. Maher
Sr. QA Engineer,
Fitbit-Boston

// BSCS, MSE, and QA Engineer since Aug. 1996
// Automation developer for [ 1.5 ] years and still counting!
// Check out Adventures in Automation and Like us on Facebook!

Notes: Introduction to Artifactory, build tools, binary files, and binary repositories

With this blog post, I wanted to talk about a tool we are using at work, JFrog Artifactory, https://www.jfrog.com/artifactory/  to store our dependencies and our binary files. Whenever I start learning a new tool, I always search for online demos of the new tool, introductory YouTube videos, and free training classes and webinars. Sometimes, I am pleasantly surprised...

If you are a newly minted automation developer, you might not know that there is a lot more to the software development process than just writing code:

• A Selenium WebDriver / Java test depends on the Java bindings for Selenium (the selenium-java library) to be downloaded and usable for your Selenium / Java project. 
• The tests need to be built and compiled into some type of form before they can be run. Build artifacts, such as JAR (Java Archive) files, need to stored somewhere until they are ready to be used.

Artifactory helps by providing a way to store both the dependencies and the files produced. And their webinar might help put technical concepts in context, things that a manual tester may not be familiar with: build tools, Maven, Gradle, binary files, blobs, diff tools, and source control systems.

Build Tools -- such as Maven and Gradle -- automate the creation of executable applications from source code. The build process incorporates downloading dependencies -- such as Selenium-Java in a Selenium WebDriver / Java project -- compiling, linking and packaging the code into a usable or executable form -- such as a JAR (*.jar) file in a Java project.

Setting Up Selenium Grid with Chrome and Firefox browser nodes from Docker-Selenium

Please Note: This blog post will be using the official SeleniumHQ's Docker-Selenium project, not Leo Gallucci's Docker-Selenium project

We will be experimenting, in this blog post, with installing Docker, deploying Selenium HQ's various Chrome and Firefox Docker images, regular versions, and using them with their Docker image of Selenium Hub.

Later on, we will tie this in with my BasicWebDriverFramework_Gradle, including into the project our work with PageFactories, and adding RemoteWebDriver and DesiredCapabilities functionality. In the post after this, we will build on this information, using Docker Compose to scale Selenium Grid.

Docker: https://www.docker.com/
SeleniumHQ Images: https://hub.docker.com/u/selenium/
Docker-Compose: https://docs.docker.com/compose/

• Want to tinker a bit with setting up a virtual environment with VirtualBox, Vagrant, and Docker first? 
• Want a practice exercise setting up Docker?
• Unfamiliar with the command line? Want a practice exercise setting up Selenium Grid? 

Why install Selenium Grid in a container system like Docker? Docker is built to help make configuring systems easier, helpful if you are setting up Selenium Grid with one Selenium Hub and multiple Chrome and Firefox nodes.

Normally, to set up Grid, you would:

• Download Selenium-StandAlone-Server
• Start up the Selenium Hub: java -jar selenium-server-standalone-2.53.0.jar -role hub
• Start up various nodes: java -jar selenium-server-standalone-2.53.0.jar -role node -hub http://localhost:4444/grid/register -browser browserName=firefox,maxInstances=2 -browser browserName=chrome,maxInstances=2

... Then the Selenium Grid would be ready to use at http://localhost:4444. 

With Docker, it is still that same three step process... except we can bundle all of those three steps into one with Docker Compose. That is for later blog posts. First, though, let's talk about Docker...

Selenium Conf India 2016 Talks: Placing Selenium Grid in a Docker Container

Selenium Conference India 2016 was held June 24, 2016 to June 26, 2016 in Bangalore. It's like Christmas for me, after every conference when they release the presentation slides the presenters used, indexing them all on the schedule at https://confengine.com/selenium-conf-2016/schedule. Simon Stewart, creator of WebDriver, gives his State of the Union. Automation developers showcase their work. You get to see what other automation developers are doing with the tool.

Videos usually are released a few weeks after the event.

I noticed that there were two talks and a workshop that dealt with a technology I was tinkering with for the past few months: Setting up Docker Containers. They were pairing Selenium with Docker to help them with testing.

Setup of Selenium Grid for Beginners: Brace yourself! We're Using the Command Line

Are you a manual tester who really wants to tinker with setting up Selenium Grid with your automated test framework, but is skittish about using a Command Line Interface? This blog entry is an attempt to walk you through it, every step of the way. Is there a detail I missed? Anything too vague? Please let me know in the comments section, below. Using the Command Line is tough, but it is something every tester needs to get used to... with time, you will have fun using it. (Or that is what I keep telling myself! :) )

• Don't have an automated test framework set up? Take a look at a basic one I have for IntelliJ and Eclipse.

For this example, I will be setting up Selenium Grid a Windows 10 box, what I use at home. Feel free to set it up on whatever Mac or Linux/ Unix operating system you have. We aren't going to be doing a deep dive on any of the moving parts. That will come later. This article is focusing on the setup.

Directions can be found on SeleniumHQ's Grid 2 GitHub site: https://github.com/SeleniumHQ/selenium/wiki/Grid2

Setting up Docker the "easy" way on Mac or Windows: The Docker Toolbox and the Get Started Guide

It's the Docker Toolbox for Easy Installation!

I've been on a quest since I became an automation developer, documenting everything I have been doing adding to the automated test framework at my workplace. Because I didn't start at square one, there are things I wonder about...

The Past: Browser images on Virtual Machines

When performing browser testing around a decade ago, I used to use VMWare to setup numerous virtual machine images for my testing team. The images contained the multitude of browsers and platforms configurations we needed. I would configure and place them the shared drive at whatever company I was working on. If we wanted to see how our web application looked on a clean install of the now defunct Netscape Navigator and Windows NT, and compare that with IE6, IE7, and IE8, our entire testing team could blow away whatever previous virtual machines they had on their desktop, and copy whatever was needed that day to their computer.

I am on a quest to find that type of ease-of-use.

Blogging about automated testing frameworks

When I first started this blog, my undergrad and grad school degrees in computer science and software engineering were gathering dust! What a difference a few years makes.

This blog started way back when I was still trying to break into the automated testing field. It became more than that once I was hired at Fitbit-Boston, embedded in the team developing automated test frameworks. I started writing about:

• Setting up a development environment with IntelliJ / Maven, then with IntelliJ / Gradle, then Eclipse / Gradle. 
• Taking a deeper dive into Gradle, exploring Groovy, using it to run simple tests.
• Using both TestNG and JUnit4 ways of making assertions. Suddenly, Hamcrest.
• Abstracting the page using the Page Object model.
• Using CSS Selectors to find and manipulate web elements on the page.
• How to organize the web element selectors: first in enums, then in By statements.
• Wrapping Selenium calls, placing them in a CommonUtils library.
• Doing away with wrapping Selenium calls and just using Page Factory.
• Performing API testing with Apache's HTTP Components and cURL.
• Passing data to tests using pojos and builder patterns,
• Showing examples of good logging for tests.
• Experiments with setting up a virtual dev environment with VirtualBox, Vagrant, and Docker.
• Coming up with various programming projects, and uploading the source code to GitHub.

As a manual tester, I loved getting paid to explore and understand new technologies, see how they do (or don't) work. As an automated developer, there are so much more technologies to explore!

Selenium Conference 2016

Ugh! I always feel so far behind... Selenium Conf 2016 happened in Bangalore, India back in 6/24/2016 to 6/26/2016, a whole four days ago, and I still haven't had any time to check out their schedule...

I love that they turn their schedule posted on the web as a way to archive the presenters videos and slides. https://confengine.com/selenium-conf-2016/schedule

What I am excited about is:

• Selenium Grid Workshop
• Testing as a Container : Using Docker with selenium and friends to ship fast
• Cross-platform, Multi-device Instant Communication Testing in Parallel using Appium and Docker

This is exactly what I was looking for!

... I wonder if over the Fourth of July Weekend I can fit in a spare day to watch these videos, take notes, and turn them into new blog entries?

As always, Happy Testing!

-T.J. Maher
Sr. QA Engineer,
Fitbit-Boston

// QA Engineer since Aug. 1996
// Automation developer for [ 1.5 ] years and still counting!
// Check out Adventures in Automation on Facebook!