Acceptance Tests Patterns

Author - Alex Sharov

What we want?

• Execution Speed
• Maintain Complexity

1) Execution Speed

Why tests are slow

• Check payment by card: go home, go random category, put something in cart, push button checkout, register new customer, pay by card
• Check payment by bank: go home, go random category, put something in cart, push button checkout, register new customer, pay by bank
• Check payment by cash: go home, go random category, put something in cart, push button checkout, register new customer, pay by cash

1 test for 1 test case

• Test 1 - check adding product to cart - and nothing more
• Test 2 - check registration - and nothing more
• Test 3 - check pay by card - and nothing more

Point 1: Keep tests simple

How we can skip some steps?

Point 2: Application must have QA API

QA API

• Fast data preparing
• If you need create new user, or put products to cart - it can help fast
• Universal access to data. [Get/Edit/Rest to default] user data
• If it's parallel running tests - they should to lock users

Point 3: Standardise QA API

Don't sleep!

https://svn.rocket-internet.de/svn/SHOP/cucumber_save/profiles/lazada/step_definitions/checkout.rb

find(:css, '#tab_IPay88_option').click
# This sleep is necessary due o new 3 step checkout cart
# When we change netween payment methods, min-cart does a ajax update
# And if we try to proceed to checkout when mini cart is till updating
# A error message appears, so we need to wait a little for it,
sleep 5

Point 4: You want sleep - you need QA API

Want do double cycle in test - use QA API

#go to each product line in page
for i in 1..product_group.length
  product_in_group = page.all(:xpath, "//*[@id='productsCatalog']/li[#{i}]/ul/li")
  #go to each product on product line
  for j in 1..product_in_group.length
    #select random product in product group
    find(:xpath, "//*[@id='productsCatalog']/li[#{i}]/ul/li[#{j}]/*/a").click
    stock_hint = product_has_stock()
    #If product has stock add to cart and stop loop
    if stock_hint
      click_button('AddToCart')
      items_added +=1
      if items_added == items.to_i
        break
      else
        visit path_to "/all-products/?price=#{Fixtures.instance['price range']}&sort=priceasc&dir=asc&page=#{@page}"
      end
      #if product does not has stock, go back all product page and choose another product
    else
      visit path_to "/all-products/?price=#{Fixtures.instance['price range']}&sort=priceasc&dir=asc&page=#{@page}"
    end
  end
end
    

You should can run test isolated

• Otherwise you never parallel it
• Otherwise fail of test will not show you what broken in application

Point 5: Keep tests isolation

2) Maintain Complexity

BDD

  Scenario: Register new user
    When I click on login button
    Then I should see login overlay
    When I click on new user button
    And I enter new user information
    Then I should be logged in
    When I click on logout button
    Then I should be logged out
    When I go to the homepage
    When I click on login button
    Then I should see login overlay
    And I reuse new user credentials
    Then I should be logged in
            

Very simple outside

BDD - inside

• IDE - become useless
• Regular Expressions
• Just moving complexity from one point to another

Point 6: BDD does not simplify your tests

BDD made for marketing, not for testing

Acceptance tests related with markup

Split: "logic of UI" and "logic of App" Then on change markup your tests will possible to repair in one place: class which contains "logic of UI"

Point 7: Use Pattern PageObject

Don't use XPATH

https://svn.rocket-internet.de/svn/SHOP/cucumber_save/profiles/lazada/step_definitions/cart.rb

#select random product in product group
using_wait_time 10 do
find(:xpath,
    "//*[@id='productsCatalog']/li[#{random_product_group.to_s}]/ul/li[#{random_product_in_group.to_s}]/div/a[contains(@class,'itm-link')]/span[@class='itm-productInfo']").click
end
            

Alleluia that we have comment

Point 8: Css selectors readable than XPATH

Don't use content

https://svn.rocket-internet.de/svn/SHOP/cucumber_save/profiles/lazada/step_definitions/homepage.rb

when 'MY'
  find(:xpath,"//li/a[contains(@href,'order-tracking')]").click
when 'PH'
  find(:xpath,"//li/a[contains(@href,'orderstatus')]").click
when 'ID'
  find(:xpath,"//li/a[contains(@href,'order-status')]").click
when 'TH'
  find(:xpath,"//li/a[contains(@href,'order-tracking')]").click
when 'VN'
  find(:xpath,"//li/a[contains(@href,'kiem-tra-don-hang')]").click
end

Point 9: Markup much stable then content

You can use special classes for tests

They are more stable than markup, just need sometimes to kick markup-guys

Point 10: Yes, we have Alice_Component_Selenium!

3) Other stuffs

Collecting js errors

Just create global object and put there all client errors, after test get it and check.

Point 11: Client side must have QA API

Coverage

• Acceptance test can't has coverage of code, but it can has coverage of All Features
• One way is to put test by folders like 'SEAAMZ-1024'

Point 12: Measure Coverage

Tests are profitable only if they fast

• Don't sleep
• Parallelise
• QA API
• Decrease queue in Jenkins

Glossary

• Point 1: Keep tests simple
• Point 2: Application must have QA API
• Point 3: Standardise QA API
• Point 4: You want sleep - you need QA API
• Point 5: Keep tests isolation
• Point 6: BDD does not simplify your tests
• Point 7: Use Pattern PageObject
• Point 8: Css selectors readable than XPATH
• Point 9: Markup much stable then content
• Point 10: Yes, we have Alice_Component_Selenium!
• Point 11: Client side must have QA API
• Point 12: Measure Coverage
• Point 13: Keep tests fast

Thanks

• Presentation Link: http://slides.lzd.co/acceptance_patterns
• Source: https://github.com/nizsheanez/lzd-acceptance-patterns.git

Author - Alex Sharov