Security Testing FAQ
1. Why is certification a good idea?
Even if you already care strongly about security, and diligently test the security of your network, sites and applications from both inside and outside your router, the chances are that few people outside your own organisation know whether your network is secure or not.
The Audited by Secure Application seal is your opportunity to demonstrate the care and attention that you take in your internet security to the people who matter most: the people who actually use your site. This means that these people can use your services with a much greater degree of assurance than if they know nothing at all about your approach to security. Generating a higher degree of trust amongst consumers of your services means that all aspects of your internet presence will be more effective.
2. Could displaying a seal make me a more likely target for attackers?
It's unlikely. The largest numbers of remote compromises come from very widespread attacks by worms or attackers who try a specific exploit across very large numbers of machines. Typically, these attackers will launch their attack against the whole of the internet, or a large IP address space, rather than a single site, and then review what machines they have control of. People targeting specific sites may infer from the seal that the site is up to date against well known vulnerabilities, and target an uncertified site instead.
3. Why Secure Application?
Secure Application has one of the best pedigrees in security testing and network exploration of any company on the internet. Secure Application has been providing network security services, including application testing, code reviews, and automated penetration testing since 1994. Additionally, Secure Application has explored the internet providing research data and analysis on web servers, operating systems, hosting providers, ISPs, encrypted transactions, electronic commerce, scripting languages and content technologies on the internet. This gives Secure Application a unique bird's eye view of what is happening on the internet, and direct access to nearly all of the world's leading web technology organisations.
Secure Application is funded through retained profit and has a cosmopolitan client list, spread through the USA, the UK, mainland Europe, the Middle East, Asia Pacific and Latin America. Clients include British Telecom, Capita, John Lewis, Microsoft, the 2010 and 2012 Olympic Games, Rackspace, Skype, Symantec, Cisco and Softlayer.
4. What does "Audited by Secure Application" actually mean?
"Audited by Secure Application" does not mean that the site or network is impregnable, but shows that the site is actively maintaining its security against remote compromise from the internet. Each site or network displaying the "Audited by Secure Application" seal is keeping the audited site or network address space free from well known remote attacks, and has an internet's eye view with which to manage its external security. The "Audited by Secure Application" seal is served dynamically and shows the date of the last clean test, when no serious well-known vulnerabilities that could permit remote compromise were detected.
5. We already have a certificate from an SSL certificate authority. Why should we certify our site twice?
Using an SSL certificate means that the traffic to and from your site using the SSL service is encrypted. It says nothing about the security of the site or its web applications. Many SSL sites have serious vulnerabilities that could be identified and fixed by using the "Audited by Secure Application" service.
6. I run a shared hosting system. Is it possible to get the Secure Application seal not only for myself but also my customers?
Yes, it is. Please contact us with details of your system, specifically your network address space, and the hostnames of the virtual hosts you would like us to test. We can arrange a process for adding and removing virtual hosts from the test set.
7. I am a security expert. Couldn't I test my own network?
We encourage clients to test their own networks and many of our clients have very experienced security teams. However, relying on your own testing can be like marking your own examination paper. Companies handling credit card data are recommended - or required, for larger merchants - to be regularly scanned by an external vendor, under the Payment Card Industry Data Security Standard.
By using Secure Application, you get access to people who test thousands of networks each year, not just one - a genuine internet's eye view of your network from outside of your own firewall — and direct access to a professional second opinion.
8. We have a firewall. I thought that meant that we are secure?
A correctly configured firewall can eliminate attacks against services that you do not need to make visible to the internet. However, many attacks are against critical services such as HTTP, SSL, SMTP, and DNS, which you need to allow through your firewall in order to operate your normal business.
Additionally, when you need to make changes to your firewall configuration, testing will give you confidence that you have not opened up more of your services than you intended.
9. How does the Audited by Secure Application process work?
After signing up for the "Audited by Secure Application" service, we will test the network address space or web site being audited. Tests take in the region of four to five hours elapsed time per 10 hosts tested, and include a full TCP and UDP port scan to determine which services are available to the internet. Each service is tested for information leaks, configuration errors and outright vulnerabilities. The HTTP and SSL page trees where those services are present are inspected for attributes which may indicate risks. Mistakes even as simple as broken links are highlighted. Secure Application tests multiple servers in parallel to reduce the risk of load on any one particular server, and the elapsed time for a network with 10 visible servers is not significantly different from the elapsed time for a single server.
Once the tests are complete we will contact you with a url and username/password to your report. If vulnerabilities are found, urls to advisories describing the problem and how to remedy are provided, so that you can fix the problems and retest your sites. Support by email and telephone is included within the service.
In some cases - for example in the case of a buffer overflow exploit, where we cannot directly test a vulnerability without risking crashing the server - it is possible for a false positive to be generated. If you are a customer who has a security seal you can mark and sign for these false positives. Only once the report is clean will the "Audited by Secure Application" security seal update to show the new scan date.
As time goes on, you will make changes to your configurations and new vulnerabilities in services you use will be discovered. When a change is discovered in your network or site's internet profile, you will be alerted, and you can use the information in the advisory to fix the problem.
10. As a credit card merchant or service provider, we are required to have regular external scans by an approved vendor. Can Secure Application provide this?
Yes - Secure Application is an Approved Scanning Vendor in the PCI DSS scheme. Our reports will highlight vulnerabilities that need to be resolved to achieve PCI compliance, and we will produce quarterly reports to show your PCI compliance status. If you require scans for the purpose of PCI compliance, please mention this when asking about scans and we can help you determine the right networks and servers to be included in the scan to achieve PCI compliance.
11. How do you price Audited by Secure Application?
We price the "Audited by Secure Application" service based on the size of the ip address range we need to test, and the number of machines visible to the internet. We will confirm the ip address ranges with you, and quote a price on this basis.
12. Can you give 100% assurance that all security problems have been found?
No. By definition a testing service can only find vulnerabilities and cannot prove the absence of vulnerabilities. That said, our reports clearly show our methods and test scope, so a person with reasonable security experience can gauge the thoroughness of the tests. Secure Application has the custom of an impressive list of clients, with several well-known companies renewing their security testing contracts with us for over five years.
13. Could the tests crash my services?
In theory this is possible, but in practice it is unlikely as we take great care to avoid damage to the services we test. We are very experienced at testing business-critical live services. The load on the test site is typically small and should not disrupt other users. Denial of services exploits are detected by passive methods only, and buffer overflow exploits are not attempted.
14. Even if you don't crash any services, what is the load likely to be, and how long will it last?
Secure Application's testing is comprehensive, and a test of a single machine is likely to last around 4-5 hours, and may retrieve in the region of 50M of data. This allows for a full TCP and UDP port scan, applying several thousand tests against http and SSL ports, and retrieval of part of the web page tree from those services. We make the tests as well behaved as possible, by using generous timeouts on requests, and testing multiple machines in parallel to avoid hitting a single server intensively over too short a period of time.
15. Will reading all the reports and advisories and fixing the vulnerabilities be a lot of extra work?
No. The Secure Application auditing process will save a lot of time, as it removes the research effort required to find out about vulnerabilities and determine which are relevant to your own installation. The advisories collect information from multiple resources and give succinct instructions on patching, together with useful background information on each potential problem.
16. What about false positives?
Where it is not possible to confirm a potential vulnerability without risking disruption to your services, - such as Denial of Service attacks, or buffer overflows -vulnerabilities may be flagged on the basis of a software version number rather than generating the actual error condition. This can lead to false positives, whereby a vulnerability that has already been fixed is reported on the basis of the software version. We continually improve our tools to try and eliminate as many false positive issues as possible, and additionally make it possible for our customers who have a security seal to mark vulnerabilities as patched. The person marking a vulnerability as a false positive effectively signs for this, making accountability possible within large organisations.
17. I have four servers behind one external load balanced address. How do you test them?
If we can only see one IP address externally, we treat that as one machine, and tests will be executed on whichever of the machines responds to our requests. If you want us to test all four, the simplest scenario is to make them individually visible to us on separate IP addresses. For larger configurations, we can arrange for an on-site testing point.
18. I also have a reactive intrusion detection system. What are the implications?
Secure Application's tests include a comprehensive TCP and UDP port scan, and tests for large numbers of well known vulnerabilities which will trigger a reactive IDS. We will notify you of the ip address ranges we conduct our tests from, in order that the full set of tests will run against your network.
19. So much for well known vulnerabilities. Will you also certify our own web applications?
Secure Application will certify web applications that it has tested, showing the date tested, and the number of days spent testing the application. Application testing is very important because even machines that are well administered, and correctly port filtered with no well known vulnerabilities, can be vulnerable to a direct attack on the application's own functionality.
Whereas automated testing is good at finding common well-known (published) exploits, a consultancy audit can additionally find faults unique to your site, caused by application programming and design errors, as well as more complex configuration errors. A consultancy test can also interpret and exploit leaked information and give constructive advice on solutions.
For more information on web application security testing, see the Web Application Testing page
20. Do you find vulnerabilities in third-party software, and what do you do if you find them?
Yes, often. If the vulnerability is new and not specific to your servers, and will affect others, then we work with you and the third-party vendor to find a solution before public announcement.
21. How do you charge for auditing web applications?
We charge for web application testing on a time and materials basis. Please contact security-sales@Secure Application.com for further details.
22. Can you also provide performance monitoring and outage alerting for my web sites?
Yes. Secure Application has a network of ten performance monitoring points around the world (currently London, New York, Pennsylvania, San Jose, Phoenix, Italy, Romania, Zurich and Amsterdam) and makes http requests every fifteen minutes from each measurement point. Outage notifications are sent by email when all measurement points are unable to reach the site. Detailed performance graphs are made available with a 31 day data history.
For more information on performance monitoring, see the Performance Monitoring page.
23. Can you also provide testing for internal networks and extranets not generally visible to the internet?
Yes; these will normally involve site visits at the start of the project, but in other respects the process can be very similar. Testing machines can be positioned on internal networks, and provisioned with updates in the same way as our internet testing machines.