Test Automation and Trading systems – It’s the only way forward!

By Kerry McKenna

The benefits of automation are widely applicable and proven. At the highest level it comes down to one fundamental advantage: efficiency. As companies strive to become more efficient, automation – wherever and whenever possible – is the clear answer.

Automated testing is no exception to the rule. As many of us in the trading technology world know (especially in the Openlink ecosphere), manual testing is time consuming and expensive. At KWA, our common experience at many clients tells us that most are faced with the same issues around manual testing:

• Cost: If we could do it in less time, we would save money.
  
• Quality: We test manually but we make mistakes from time to time.
  
• Coverage: We have a limited time allocated for testing and so we are forced to select which tests to run.
  
• Productivity: If we could do it in less time, we would be able to start regression testing later and have a greater payload in our releases.
  
• Predictability: If we could do it in less time, we would use the time saved to be able to respond to any problems found.
  

To sum it up, manual testing is inefficient and more expensive in the long run.

In the Openlink world, thorough and reliable testing is essential. Openlink systems bring us the benefit of straight through processing – one system which integrates front, middle, and back office. The downside of the equation is that the system is massively complex. One bug in the code can have widespread impacts and a bug-fix or enhancement in one area could potentially break functionality in a completely different area of the system that a manual tester may not even consider. This means that every code change, configuration change, any change, requires robust regression testing. But at some point, we have to make a decision – how much time and resources can we afford to spend on regression testing every change we make? Automation is the only option where we’re not put in a position to make that decision each time we introduce a new change. There is an up-front cost to automated testing which perhaps dissuades companies from investing – but this is true for all automation – and the consensus from companies around the world is this – it’s worth it.

Our experience across many clients shows the value that can be generated from this investment, especially when considering Openlink upgrades and maintenance releases. The current Openlink release schedule of a new maintenance release every two months for the latest versions, means that it is imperative to identify and raise defects in the first weeks of the release cycle. If a defect is raised after this point it is likely to miss the code cutoff date for the next release and therefore the fix will only be in place for a subsequent release. If the defect is critical then this can result in a delay of several months to the upgrade project and a parallel rise in costs. Establishing a comprehensive automated regression test suite as the first stage of an upgrade project is imperative. There will be an element of manual testing required, but this should concentrate on high value- added tasks, rather than commonly executed test cases.

There are several different approaches to test automation. The traditional approach has been to use GUI driven testing tools such as “TestPartner”. The problem with this approach is that the Openlink GUI is not a fully Windows compliant GUI. Items which a user would expect to be a window, are not. This results in many workarounds and a fragile automation framework. This can also result in testing the GUI, rather than the important underlying functionality of the system.

At KWA our automated testing services are focused on behaviour driven testing, using APIs and standard tools such as “Fitnesse”. But what is behaviour driven testing? It goes hand-in-hand with the more commonly known concept of behaviour driven development. Behaviour driven development is a key concept in the agile development methodology – development is driven by system agnostic business rules. Given a set of business circumstances, this is the output I expect to get. We then develop to the requirement.

Behaviour driven testing is leveraged by agile development teams, but it can also be applied independently. The overarching concept is the same – test cases are defined as business rules, and tests pass when the output from the system is matching my expectation from a business perspective. In the Openlink world, this translates as automating the business processes (e.g. invoicing), rather than individual elements of the GUI. Behaviour driven testing is especially valuable because it strengthens communication across teams – managers don’t need to be system experts to understand what is being tested, or which scenarios are passing and failing tests; developers can appreciate what is important from a business perspective; users can communicate what they expect to put in – and get out – of the system.

Hence, we are supporting behaviour driven testing in our testing services, but what is a testing framework to begin with? It is the framework within which we create automated tests – it provides a means to enter test cases, methods to execute tests and compare expected vs actual results, and a way to view and analyse the outcome of the tests. As part of our QA services at KWA, we work with clients to build business-driven test cases and integrate those test cases into their underlying applications (i.e. Endur, Findur).

Test Framework components

In our first automated testing project, a less than 3 months’ investment resulted in 160 repeatable tests covering 3 different outbound (deal and invoice) extracts across many deal types including FX, Physical deals, Metal Swaps, Loans, Futures deals, Cash deals. The scenario coverage included:

1. Buy/sell different metal and financial currencies with tax and non-tax
   
2. Physical deals with shipping charges – freight/dispatch/flat/fees/tax
   
3. Metal transfers from a range of accounts to external accounts
   
4. Loans and deposits across a range of internal and external accounts
   
5. Metal swaps – partial fixing vs no fixing vs fully fixed
   
6. Amendments on trades prior to invoice generation
   
7. Trade cancellations
   
8. Same day invoice cancellations
9. Etc.

The tests take approximately 15 minutes to run. Executing these tests manually might take a couple weeks, not to mention if an issue is found and fixed, the right thing to do to ensure good quality is to execute all of the test cases again.

These are the types of projects where KWA’s automated behaviour driven testing services and tools will have the best return on investments:

• Regression Testing
  
• Version Upgrade Projects such as OpenLink from v12.0 to v14.0 or v16.0
  
  • Including Endur Delivery Ticket to Parcel Pricing upgrades
    
• Migration Projects where Data is transferred from a Legacy application to a new system
  
• Business driven development testing for new enhancements

Companies who invest in automated testing realize that time and money previously spent on manual QA can be invested in new ways – new technologies, innovation… the possibilities are endless. Welcome to 2018!