Skip to main content
Frameworks^3
by Dave Haeffner
ARRgyle.com
Frameworks Agenda
What it is
Why it's worth it
What should go into one
What's available today
Q&A
What is...
an acceptance testing framework?
Elizabeth Hendrickson says...
1. Defines the format in which to
express expectations
2. Creates a mechanism to hook into, or
drive, the application under test
3. Executes the tests
4. Reports the results
[link]
Why is...
an acceptance testing framework...
worthwhile?
The Standard Evolution of
Selenium automation at an organization
1 - Record tests with Selenium IDE
2 - End up with a brittle suite of tests
3 - Export to code, get them working
4 - Rewrite them so they're DRY
5 - Scale, hit performance issues
6 - Monkey-patch as you go
A - Ignore it and go back to manual testing
B - Throw it out and rewrite
Standard Selenium Evolution
Hmm...
What really makes up...
an acceptance testing framework?
(especially one that uses Selenium)
Framework anatomy
Current
Runner
Integration
Configurable
Reporting
Logging
Performance
DSL
Taggable
But the real value isn't realized...
until you do something with it.
Why Feedback loop for the team
How Integrate with CI
What Timely, in-band notifications
Frameworks in the wild...
Open Source
Py.Saunter (Python)
SaunterPHP (PHP)
ChemistryKit (Ruby)
Geb (Groovy)
SST (Python)
ScalaTest (Scala & Java)
Baby Framework (Java)
JDriver (Java)
Open Source cont'd
Robot Framework (Python & Java)
Fitnesse (Java & .NET)
Cucumber (Ruby)
Cucumber-jvm (Java)
Specflow (.NET)
Turnip (Ruby)
Proof
Titanium xAFT
Commercial Offerings
SauceLab Partners
QAOnDemand
TFT Consulting
Dynacron Group
Softcrylic
ThinkSys Inc
Hindsight Software
QASource
Recap
● An 'acceptance testing framework' that is
built for successful Selenium has a lot of
parts
● A framework will help you get started, grow,
and avoid pitfalls
● There are many "frameworks" (both open
source and commercial)
● But there are few that come with everything
you need out of the box
● The real value comes from using it within
your development workflows
Office Hours (30 min, FREE)
sohelpful.me/tourdedave
Blog & Newsletter (FREE)
arrgyle.com
dave@arrgyle.com@TourDeDave
Additional Resources
Selenium HQ's outdated framework page
Agile Alliance Functional Testing Tools
spreadsheet (outdated-ish)
wiki (should be the record of truth)
List of available Selenium Resources
blog post on arrgyle.com
SF Selenium Meetup's Testing Tool Demo Night videos