Selecting an application testing provider: Getting the right blend of skills and business sense

Selecting an application testing provider: Getting the right blend of skills and business sense
John Steven is internal CTO of Cigital.

(c) Melekhin

Partnering with the right vendor to test your applications is a critical decision and the investment you make has consequences not only for the security and integrity of your apps, but for the reputation of your business.  You need to have confidence that the testing provider you select has the right blend of skills and tools, as well as a wider understanding of the business and compliance issues at play, so the breadth and scale of tests exactly match your requirements. 

It is not always clear what type of testing support you’ll get unless you know the right questions to ask. To avoid costly mistakes, there are fundamental ideas that can enable you to find the partner that best matches your risk profile and is most qualified to help fix your vulnerabilities and scale with your needs.

Breadth and depth

Your vendor should have the expertise to apply different testing strategies according to the risk level and requirements specific to each of your apps. The right vendor will create a complete inventory of your apps, order them based on their security risk, and devise an appropriate testing plan that matches depth of test to the risk level of each application.

It’s important to bear in mind automated testing on its own isn’t enough to provide a complete picture of your vulnerabilities. In order to defend against multi-step attacks – or ones that involve social engineering – your vendor should be able to conduct in-depth manual testing to mirror the perspective of a hacker. Therefore, vendors should have the ability to employ multiple best-of-breed tools, with customisations that match your business needs.

With depth addressed, your attention should turn to breadth. Double-check that your vendor has a coverage model which lets you test your full portfolio. Recent breaches have proven that limited testing breadth to only high-risk applications creates fatal testing gaps that are readily exploited. Even if you’re not testing each app to the same extent, it’s key to have a full inventory of your apps and a consistent testing schedule so nothing is missed.

Fixing what is found

It’s not just about running tests anymore. Now it’s important for vendors to take a holistic approach: providing remediation guidance so you can address the issues and fix the causes so fewer security issues reach the testing phase.

Vendors should always review their findings with you, detail causes of vulnerabilities and offer remediation advice. After the initial test read-out your vendor should continue to provide live remediation support on demand. Too many consultants offer suggestions that only work in theory. A provider that has worked as a developer, or has experience as an in-house security leader, and understands real-life pressures and working environments can become an asset and partner to your team.

Visibility and control

Application Security Testing is not a trivial exercise, so turning over testing to a provider places a premium on your visibility into the process and your control through interactions with the vendor. You’ll want to know how easy it is to schedule a test on your apps; whether this needs to be done in advance to confirm the testing resources are available, and what types of tests you want run. It’s also good to know which team members can request tests.

You will want to see the results in a way that is pragmatic, concise, and immediate. Does the vendor have security experts that review the findings and eliminate false positives? Do they include remediation advice? Can you get an immediate sense of my progress and potential problems at-a-glance? Can access to the test results be limited to specific roles or groups? You want to be assured that you have complete and frictionless visibility into the process and results.


Business requirements and priorities evolve. New threats emerge. Your service should allow you the flexibility to change and pivot to address these circumstances. What happens if your business grows, or your organisation is part of an acquisition or merger, or your customers ask you to test apps in a different way to meet their security requirements? Your testing vendor must provide flexibility to manage your evolving portfolio – while continuing to help you manage costs. You don’t want to be penalised for changing testing focus to a different app, or testing to a different depth.

Demonstrating success

Selecting the right vendor is critical for your organisation, so you need a method to demonstrate if the choice was a success. This begins with more visibility into the broader testing program with basic reporting capabilities that can capture a macro view. To further judge whether the app security testing has been worth the investment you’ll have to consider your vendor’s speed, efficiency and accuracy. Other things to consider: are developers improving over time? What is the defect density of the new code? Do they now have time for other projects?

Selecting the right vendor to test your apps is one of the most important decisions you’ll make in protecting the integrity of your valuable business assets. Going through this checklist systematically should provide a good grounding for picking your potential partner. You should never be afraid to put your vendors through their paces to be sure that they can provide you with what you need to move forward with confidence. in hearing industry leaders discuss subjects like this and sharing their use-cases? Attend the co-located IoT Tech Expo, Blockchain Expo, AI & Big Data Expo and Cyber Security & Cloud Expo World Series with upcoming events in Silicon Valley, London and Amsterdam and explore the future of enterprise technology.

View Comments
Leave a comment

Leave a Reply

Your email address will not be published. Required fields are marked *