Ask The Tester – January 2013

In November 2011, I was fortunate to meet someone who would change my writing and speaking career forever.  Matt Heusser was writing a column called “Ask The Tester” for ST&QA Magazine – a magazine that so many testing professionals follow.  Matt wanted to interview me for his column.  We worked together, and I was fortunate to be featured in the March 2012 edition.

After a wonderful 2012, writing articles for the magazine and speaking for their conference (STP Con) in October, 2012, I was asked by Matt to take over the article in 2013.  What an honor and a privilege.

So now comes the part where I need your help.  If you’re a test practitioner, leader, or simply have testing related questions for my January 2013 candidate, please submit a comment to this blog and I will review the top 10 comments with our candidate.  If selected, your question will be published in our January 2013 article of ST&QA Magazine (you can view the latest online copy at

I plan to pull together all questions by early next week (Jan 7, 2013-Jan 9, 2013) – so submit your comment / question as soon as possible and increase your chances of being part of our article!!  So let me tell you a little about John……

Meet John Ruberto

When it comes to financial software, Intuit is one of the leaders in the space.  From TurboTax to QuickBooks and others, each have been a part of our lives for many years.  They offer financial products that assist consumers and companies with taxes, finances, bookkeeping, accounting, payroll, and many other features.

I introduce to you John Ruberto from Intuit.  Their team assists with functional testing, performance testing, as well as automation.  John has been developing software in a variety of roles for over 25 years. Currently, he is the Quality Leader for QuickBooks Online, a web application that helps small business owners manage their finances.   Before joining Intuit, John was the test manager for element and network management systems at Alcatel-Lucent. He has also held development manager roles for Phoenix Technologies, and was a software engineer for Boeing.

For the past 13 years, John has held test & quality leadership roles. As such, he is known for career development for quality professionals, quality strategy, and bringing innovation to testing organizations.

If you have a question, please submit as a comment to this blog, and provide the following:

      • Your Name
      • Your Title
      • Your Company (if applicable)
      • Your Geographic Location

Thanks in advance for your questions everyone!

Author: Mike Lyles

I am an International Speaker, writer, author, blogger, and Director of QA & Project Management with over 25 years of experience. I have always had an intense passion to mentor, coach, and guide others to grow professionally. It is fulfilling to witness the look on people’s faces when you are able to share with them something that is enlightening. I want to be remembered for the way I made others feel when they were around me! I can be followed at the following additional social networks – Please feel free to add me! Connections are our best way to help others! Twitter: @mikelyles ……. follow me and I’ll follow you! Facebook: ……. send a friend request! LinkedIn: ……. send a connection request! Learn more about me at:

23 thoughts on “Ask The Tester – January 2013”

  1. QUESTION: How do you formulate and update the testing criteria (requirement, objectives or “desirements”) for performance – please describe in terms of what a performance requirement looks like at Intuit, the cadence/frequency for updating and staying current with performance objectives and any challenges you’ve overcome. (Question emailed to me from Mark Tomlinson, Independent Performance Consultant & Conference Speaker)

  2. QUESTION: Is the Intuit business management prioritizing “product performance” in a competitive way? Do they see performance, capacity, scalability and reliablity – as “presumed” or “inherent” in the product…or do they call it out as a competitive differentiation? (Question emailed to me from Mark Tomlinson, Independent Performance Consultant & Conference Speaker)

  3. QUESTION: Have you ever gotten support from the Intuit business to stop a release due to poor performance? (Question emailed to me from Mark Tomlinson, Independent Performance Consultant & Conference Speaker)

  4. QUESTION: Given your product’s focus on the enterprise, can you provide examples of quality assurance practices you employ specifically to meet the needs of enterprise users and how they may differ from consumer facing products? (Question emailed to me from Seth Eliot, Senior Knowledge Engineer in Test, Microsoft)

  5. QUESTION: For your website and online offerings, can you share what quality assurance and validation practices take place after deployment to production? (Question emailed to me from Seth Eliot, Senior Knowledge Engineer in Test, Microsoft)

  6. QUESTION: For your website and online offerings, what practices do you employ to ensure you have adequate capacity and scalability for known crunch times like end of quarter or tax time? How do you ensure such capabilities while avoiding wasting resources (over-capacity) at non-crunch times? (Question emailed to me from Seth Eliot, Senior Knowledge Engineer in Test, Microsoft)

  7. QUESTION: How do you measure the effectiveness and efficiency of your testing processes, and how do you institute improvements? How do you measure stakeholder satisfaction with the work your group does? (Question emailed to me from Rex Black, President of RBCS)

  8. QUESTION: How do you manage the individual and collective skills of your testers, so that you can optimize the test team’s fit with Intuit’s needs? (Question emailed to me from Rex Black, President of RBCS)

  9. QUESTION: When you are hiring testers, what specific traits do you look for in interviews? What skills to you look for on a person’s resume, and how to you evaluate those skills during interviews.(Question emailed to me from Rex Black, President of RBCS)

  10. What lifecycle do you use? Are you transitioning to agile/have you transitioned? Can you tell us the story of that and what happened to your test team? (That might be more than one column’s worth of answers right there!)

  11. For improving things (quality, effectiveness of QA) what measures had he (have they) taken? Did he find anything (process related things) that was required to be changed? If yes, how did he manage that change? How do they tackle differences between business and development/QA team ( on occasions where QA has it’s own opinion/stance/standards/policies).

  12. On a past testing effort while testing tax software, one of our challenges was tax law interpretation. At times we would have three tax accountants in the room and get three different interpretations of new tax laws. How do you handle that type of “testing” challenge at Intuit?

  13. I see you moved from testing network management embedded devices to testing web applications. What was most different between these context? What helped in the switch, and what made the transition difficult?

  14. Do you have any mobile testing practices? Do you have a separate mobile testing team or do you combine desktop/web testing practices with your mobile testing? Do you combine your mobile testing projects with your desktop and web application projects? Do you have a software test team with specific mobile testing skill sets?

  15. There are many different views on test automation, its effectiveness as well as ROI. In your practical experience whats your take on test automation? At Intuit what measures you follow to ensure desired results are achieved from test automation?

    Name: Sudhir Patil
    Your Title: Director
    Your Company: Qualitia Software
    Your Geographic Location: India

  16. Hi Mike. Best of luck on your new endeavor!

    Here are the questions for John:

    QUESTION #1: I read your blog post and PNSQC 2010 presentation proceedings on Customer-Driven Quality. Very impressive! How long is this framework in place? Can you share any “aha moments” resulted from implementing this program?

    QUESTION #2: What is in your title’s name? Do your responsibilities differ from QA Manager/Test Manager? If yes, how?

    Anna Royzman
    QA Manager
    Liquidnet Holdings, Inc.
    New York

  17. What level of expectations you set from your team of test leads and testers whether they are with in the organizations. Or from vendors

  18. Do you use any of the cloud providers premises for testing your application? if yes what type of application? was they process smooth?how do they charge you? If no. are you considering taking some of you testing tasks to the cloud?

  19. John – in terms of bringing innovation to testing organizations, can you describe any of the innovations you’ve managed to introduce at Intuit? I’m interested in how open such a financial software business is to more modern testing approaches (e.g. exploratory testing).

  20. What’s the core competency of Intuit when doing automation? How testing outsourcing plays role in your business model and what’s the trend of automation testing in next five years?

    Thanh Huynh
    Tester at FPT Software

  21. Mr. Ruberto,
    1- As the QA profession has evolved for the past 20+yrs., have QA managers seen the benefits and ROI for the corporations in out sourcing QA jobs?
    2- How do you perceive QA’s role with Big Data Testing Frameworks?

  22. • How many teams are involved in performance testing a product like Quickbooks Online, and how are teams’ responsibilities segregated? What are the challenges you face in managing and coordinating the efforts of those teams?
    • What key factors must be considered, and/or challenges overcome, when setting up your test environment?
    • Are image-based testing tools currently used in your automated testing process? If not, what would it take for them to be considered for use in the future?

    Thank you,

    Sam Benihya
    Director of IT
    NRG Global, Inc.
    Los Angeles, CA

  23. You have held a variety of leadership roles across different industries. How would you say your leadership style has evolved over the years? Have you found you needed to shift your style for different roles or industries? What have you found to be different about leading a group of testers versus a group of programmers (or any other group of people)? Do you have any tips for up and coming test leaders?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s