Measure and Monitor Application Security
Sivarama Subramanian,
CISM, is senior architect
of technology at Cognizant
Technology Solutions, where
he is currently overseeing
the security initiatives for
retail and retail e-commerce
engagements. Subramanian
is a member of the ISACA
Chennai, India, Chapter
and can be reached at
sivaramasubramanian.
kailasam@cognizant.com.
In the increasingly digitally connected world,
information is the most valuable asset. Web
applications are the gateway to access information,
and they are no longer confined to a simple browser
interface invoked from a laptop or desktop. With
smartphones and smart TVs (televisions with a
Wi-Fi or network connections) and appliances,
applications are available everywhere.
This thrusts a lot of responsibilities on the
security community to safeguard the interests of
stakeholders and the information that is exchanged
via web applications. The security community1
has published best practices, guidelines and
checklists to embed security into web applications.
For example, one of the best practices is to call
out specifically how to handle the SQL injection
or to handle the cross-site scripting in the design
document itself so that when the developers
implement the design, the security is inherent in the
code. Securing the system development life cycle
(SDLC) is no longer a separate activity.
How does one ensure the effectiveness of
application security? How are the security
initiatives that minimize the risks and threats
from hackers measured? This article attempts to
define metrics that measure the effectiveness of
application security in an organization.
DEFINE THE METRICS
Metrics are the prime indicators of management
initiatives in any organization. Organizations
witness a slow, but steady, increase in need for
information security within. In many organizations,
information security has attained a fairly
considerable level of maturity. Web application
security, as a part of the overall information
security program, plays a major role in protecting
valuable information. Therefore, the best time to
define a metric is at the start of the application
security program. The best possible approach is to:
• Identify the metrics.
• Identify the data-collection techniques.
• Obtain agreement from key stakeholders.
• Report the metrics to key stakeholders at the
agreed-upon time interval.
The identified metrics should be useful to
measure the effectiveness of the security
program as well as to identify the gaps for
future improvement.
There are two broad categories of metrics
that can be captured for application security.
The first set of metrics is for incidents and
vulnerabilities (figure 1), and the second set is for
the application security program itself (figure 2).
Metric
Number of incidents reported
Do you have
something
to say about
this article?
Visit the Journal pages
of the ISACA web site
( www.isaca.org/journal),
find the article, and
choose the Comments
tab to share your
thoughts.
Figure 1—Metrics for Incidents and Vulnerabilities
Purpose
Represents the number of incidents reported or discovered in the measurement
window and helps identify the up/down trend of incidents
Number of incidents resolved Helps identify the up/down trend of resolutions. Downtrend can be investigated, and
timely corrective action can be taken.
Number of vulnerabilities reported Represents the number of vulnerabilities reported or discovered in the measurement
window and helps identify the up/down trend of vulnerabilities
Number of vulnerabilities resolved Helps identify the up/down trend of resolutions. Downtrend can be investigated, and
timely corrective actions, such as awareness training or focused reviews,
can be taken.
Total security effort Helps identify the effort spent on all the security activities. The idea is to reduce the
effort by following a secure SDLC program.
Helps identify where the effort is being spent for resolving the incidents and
vulnerabilities. The idea is to reduce the effort by following a secure SDLC program.
Measures the defect leakage of the security program. If the value of the ratio is more
than one, the program is not effective.
Average effort—vulnerability
assessment
Effective ratio (number of
reported vulnerabilities/number
of found vulnerabilities)
