Banking application tests: know-how is king!
If you think that any testing team can verify a banking application, think again! Approving banking applications for release requires not so much an army of testers, but a team of analysts with in-depth domain knowledge.
Increasing the level of customer experience and expanding their offer banks have become producers of IT solutions that match their application ecosystem. And since banking system errors have far-reaching consequences, not only in terms of the bank’s image, but also compliance, the stakes for the reliability of banking IT solutions are extremely high.
Outsourcing tests to specialized partners is not yet popular in the banking sector. The obvious safety concerns are the main drag, but there are also concerns that the most pressing problem in testing is more complex: the lack of appropriate test staff. Or perhaps it is also a matter of good process organization and understanding of the ins and outs of tests in the banking sector?
Arkadiusz Stanimir and Agnieszka Suder, CCA Europe experts with long-standing experience in banking applications introduce the subject of software testing and explain why it is worth entrusting this task to external specialists.
Arkadiusz, Agnieszka, please share what your roles in the testing process are. What exactly do you do?
Arkadiusz Stanimir: I am Chief Analyst, I also supervise the Testing Department. In a project, my role is that of a Test Master.
Agnieszka Suder: Depending on the client’s chosen approach, I am a Project Manager or a Scrum Master. My role is primarily to plan individual project phases, i.e. to prepare test scenarios, run tests or provide post-deployment support, as scheduled by the client. I also ensure that the team has a clear vision of the project, remove obstacles as they appear and bring the project to a successful closure, i.e. a positive recommendation for business implementation.
Let’s start with the basics: how is banking application testing different from testing other business solutions?
Agnieszka: Contrary to “classic” applications, broadly understood banking tests require domain knowledge, i.e. how banking processes work and how they are interconnected. This knowledge is, in fact, necessary to prepare test cases that are to comprehensively cover the area of critical functionality, which will be affected by the change introduced as part of the project.
Arkadiusz: The key element of testing a banking application is the analysis phase. This is when specific problems and areas are identified, followed by actions defined to ensure that the application works properly. In other words, the bank does not tell us in advance what and how to test. They only present documentation (functional or technical) of the changes introduced to the system. It is us, the analysts, who must develop a process that ensures maximum test coverage with the least expenditure, and then carry out the appropriate actions and present the results and recommendations.
What if a team with no banking experience got onto designing such a test? If they hadn’t asked the right questions?
Agnieszka: Without expert banking know-how, the entire testing process will be at enormous risk, mainly of test incompleteness, which will, unfortunately, be quickly and painfully verified in the production phase. In-depth knowledge of the banking process and its connections is necessary to scrupulously analyze the scope of tasks and prepare test scenarios to detect all potential errors.
Arkadiusz: In banking, the test area is not defined by an application, since the bank processes involve a number of applications and the core system. A bank’s architecture comprises several dozen or even several hundred different applications. In the analytical phase, it is necessary to determine which architectural elements, e.g. applications or databases, the process involves, and which access channels. It cannot be done without domain knowledge and project experience. Even if the system’s change involves a new application that supports a specific business area, this application always functions in the context of the entire banking application architecture and its ecosystem of connections within specific business processes.
So in banking tests, the point is not to “click through” an application, but to ensure the entire banking process’ correct operation. How do you approach it? What is the first step when a request for proposal comes in?
Agnieszka: We start by reading the documentation, discussing the potential scope and listing issues that require clarification. We determine the tests’ potential scope and determine what applications and databases it encompasses, those we know, or some new ones? All of this will impact on the duration and cost of the project. To clarify the issues, we contact the business owner, as the key to success is to fully understand the topic. There is no room for understatement or guesswork here. We prepare a proposal only when all issues have been identified.
Arkadiusz: An integral part of the design phase is discussing the necessary access to applications, tools, databases or log servers. In banking, access to these resources is highly protected and must be well justified in the documentation. Our role is to specify the resources to which we need access and its scope and level, together with our clients’ justification. The bank is accountable to its clients, and also amenable to control institutions for ensuring the security of access to its structures.
Testing a complex banking process or a completely new application in all access channels entails considerable costs. Is there any way to optimize costs if the proposed scope of tests exceeds the specified budget and timeline?
Agnieszka: When the client decides to reduce tests’ scope, we propose the most representative sample of test cases. This way, we comprehensively cover the most important functionalities and still fit into the schedule and costs. Excessive quantitative and qualitative cuts cannot be accepted, because only partial testing sometimes renders the project completely unjustified. After all, by taking up a testing project, we vouch for its proper functioning. As mentioned before, it is the skills of the final testers, and the domain knowledge of analysts that count in banking tests.
Arkadiusz: If the bank’s budget allows for testing only a part of the proposed scope,this part should be precisely defined. You can’t just take the cases as they appear on the list, and discard the rest. What should be selected is the most important and representative ones, e.g. on the statistical analysis of the popularity of specific banking services in specific access channels. Of course, we will select for testing the most popular channels, with the highest number of operations of a given type. The criteria also depend on the functional scope and characteristics of the project itself. It is only seemingly a cost issue – optimization must be approached analytically, from a business point of view.
All right, suppose you have successfully divided the tests into areas, provided comprehensive test case coverage, and successfully conducted the tests. What’s the next step?
Agnieszka: If the testing phase is successfully completed, we prepare a final report with a “go” recommendation. The document is not limited to template formulas, because we want to convey the acquired knowledge as accurately as possible. We summarize the functional or environmental errors, the identified problems and the related business decisions, if any, but also lessons learned, which will allow for even more effective actions in the future. In addition, we discuss the project knowledge gained during a summary workshop so that the client has a feeling of full control over the project.
Arkadiusz: Not every test project is completed as fully successful. Sometimes some functionalities do not work, and some have not been tested in the prescribed time. However, knowing the consequences and our recommendations, the bank may decide which features to release to end users. Of course, the correct operation of the so-called critical functional paths should be validated. Still, auxiliary or substitutable processes can be implemented even if they haven’t been tested in full. If there are any untested cases left, during the summary workshop we provide tips on what to do in case of specific errors, what alternative processes can be used, and which functions to disable until the tests are completed.
CCA Europe specializes not only in manual but also automated tests. What is the benefit of test automation, and when do you use it?
Agnieszka: Test automation allows primarily to run several tests simultaneously, which speeds up the testing project as a whole as compared to manual testing. Of course, this involves initial investment, because the preparation of automated test scripts can be quite laborious. However, their subsequent reusability quickly compensates for the cost and effort. Additionally, such tests can be conducted regardless of the time of day and to a large extent, eliminate human error. As for its use, the above advantages make this form of testing the most suitable for repetitive operations, i.e. regression, performance or load tests.
Arkadiusz: Since the banking IT architecture is changing dynamically and must be subject to continuous validation in the spirit of Continuous Integration, there is a global increase in the significance of automated regression tests. If there is a chance that the script will be used repeatedly, as is the case with regression testing, it is worth investing the time, to later run tests for free. What is also important is the sheer volume of data. What is impossible to test manually, can be tested automatically.
And how was the year 2020 in terms of tests?
Agnieszka: 202O has kept us busy indeed, and in all likelihood, we will enter 2021 with just as much involvement. One of the most important projects was the next stage of development of the SME/AGRO Credit GO! Sales platform for Credit Agricole. We are glad that the client sees us as a partner in the further development of his product and that we can enhance our competences in script automation, in addition to functional tests. Although we had to switch to completely remote work almost overnight, I am proud to admit that it went smoothly and proved that regardless of the place of work, the CCA team is equally committed to their projects.
Arkadiusz: In the last quarter, we have integrated the proprietary PATT application with one of the most popular solutions for testing web applications, Selenium. The integration allows not only to accelerate the work of testers but also to reduce business costs. Most importantly, however, extensive validation of the system, ensures a reliable representation of the banking process from the end user’s point of view. We expect to leverage its potential next year.
Looking forward to news of your new successes, thank you for your time!