quantitatively measure the performance, dependability, perfomability and security

A unified approach for specifying measures of performance, dependability and performability

--W.H.Sanders

Using some mathematical structure, it is also possible to measure cps security in terms of the amount of "reward(or some other term)" during a specified interval of time, or the rate of accumulation of reward at a specified instant of time or in steady state. It is an evaluation of performance, perfomability based on stochastic activity network model, which could be expanded to security region.

--However, it is hard to understand this reward model, need more thoughts here!

Quantitative Evaluation of Information System Security

--R. Ortalo

This paper first specify the security policy, describe the vulnerability of target system (or organization), and then a quantitative evaluation approach based on the privilege graph model (could be some other model). This method then applied to a security-critical real organization: a medium size bank agency. Also, it is important to illustrate the security measures.

Overall, the goal is to maintain a satisfactory level of security, without impeding the operation of the system. System is still running (well) under malicious attack.

Usual security evaluation methods: evaluation criteria (ITSEC, Information Technology Security Evaluation Criteria, 1991) or risk analysis (Anderson, Comparing Risk Analysis Methodologies, 1991)

Cons: Previous work only focus on the information system design, rather than on the actual system operation (case study).

The method proposed in this paper is also model-based evaluation approach.

1)definition of security policy: security objectives and security rules (specification language)

2)modeling vulnerabilities of the organization, adopted from Dacier and Deswarte, 1994, called a privilege graph. The arcs, the nodes, and the transformation

Using model to quantitatively measure the security of cps for sure would be an promising field. to my best knowledge, I have seen two model, one is use Petri-net with stochastic model, and the other one is use privilege graph. Soon, I will get details for such method and finally come up with my own method.

prospective research

I found that no system-level methodology currently exists that can quantify the amount of security provided by a particular system-level research. Most of security methods have been qualitative. We may find a way that can quantitatively rate the security for cps, and give them a score just like what the security software does to our windows system. It is therefore useful to categorize measures of system behavior. There are some previous works related with "reward models" using the amount of reward accumulated during a specified interval of time, or a "reward rate" at a specified instant of time in steady state.

an idea on intrusion response

Intrusion response, as it said, it is kind of a reaction to some attacks happening (ongoing) in the system already. It effects after the adversary successfully attacked the system. The goal for such research is to bring an insecure network (CPS) under ongoing attacks to its normal operational mode with the minimum possible cost.

For example, if our CPS is under light DoS attack, controller and plant could still talk with each other, but there is large delay on the network caused by DoS attack. The UAV plant may not track the signal well. This is the cost, if we could define it formally, it would be much better. Then how could we response/react to this insecure network to make the possible cost minimum, which strategy should we choose to response/react based on which kind of selection algorithm.

But if we do so, that will make this problem as mitigating the attack.

Intrusion response and recovery

RRE: A Game Theoretic Intrusion Response and Recovery Engine (good paper)

--Saman Zonouz, Himanshu Khurana

This paper has implemented RRE on top of Snort, which is an open-source signature based IDS. RRE employs a game-theoretic response strategy against adversaries modeled as opponents in a two-player Stackelberg stochastic game. An important term used here is Attack-Response Trees (ARTs) to analyze undesired security events. By solving a partially observable competitive Markov decision process that is derived from attack-response trees, RRE will choose an optimal response actions. Basically, it depends the detection of Snort (alerts from here), using its alert to figure out respective response.

RRE is based on automated cost-sensitive model. What's great in this paper is that it modeled the security maintenance of computer networks as a two-player game in which the attacker and response engine try to maximize their own benefits by taking optimal adversary and response actions, respectively. Using ART, RRE explicitly takes inherent uncertainties into account along with alerts from Snort.

It is good method for small network (LAN) IDs.

Automated Response Using System-Call Delays

--Anil somayaji

This paper developed a system called pH (for process homeostasis), which can detect and stop intrusions before the target system is compromised. It monitors every executing process on a computer at the system-call level, and responds to anomalies by either delaying (slowing down) or aborting system calls. Normal behavior is determined by the currently running binary program.

pH is implemented as a patch for the Linux 2.2 kernel. They modified the system call dispatcher so that it calls a pH function prior to dispatching the system call. Basically, the author insert a independent process into Linux to monitor the entire process and detect the abnormal process.

It is good for single-host IDs.

Cooperating Security Managers: A Peer-Based Intrusion Detection System

--Maj. Gregory B. White, Eric A. Fisch

This paper designed and implemented CSM, which could perform a larger network IDS. Individual CSM works on each individual hosts. There is a security manager working to cooperatively and autonomously communicate with them and determine the current state of a system.

The prototype of CSM was developed using a Sun SPARC-station LX running SunOS v5.3. Basically, CSM is a package patched in OS to detect intrusive activities. Applied to a network, CSM is designed to perform intrusion detection and reporting functions in a distributed environment without requiring a designated central site or server to perform the analysis of network audit data.

Toward Cost-Sensitive Modeling for Intrusion Detection and Response (good paper)

--Wenke Lee@gatech

This paper builds a cost-sensitive intrusion detection model, including development cost, operational cost, damage cost and the cost of manual and automated response to intrusions. this kind of cost-sensitive machine learning techniques can produce detection models that are optimized for user-defined cost metrics.

The experiment uses the data from a military network with a wide variety of intrusions injected into the network over a period of 7 weeks. The data was divided into two parts: training set and test set.

The main objective in applying such a model is to compare intrusion damage and response cost to

Here is what I thought:

As we have seen above, almost all the previous work has developed a tool/package/system aiming at specific environment or they have data set with intrusions injected already. Since we don't have such kind of data set/trace, we could try to take advantage of the unique environment --- our cps simulation environment to develop a similar detection and response engine, as a goal. One problem is that this may not be easily ported to other internetworked environment. But one good thing is that it could be used to test different kinds of response strategy used to pick out a best response against the adversaries.

Control techniques in NCS

Paper Reading: Control methodologies in networked control systems

After reading paper “Simulation of Network Attacks on SCADA system”, I was thinking maybe I could do a series of simulation to analyze the effects of network attacks on NCS. Different from previous work, I was thinking to make two situations, one with full-fledged DoS attack on routers so that the network will be essentially broken at that point and result in a loss of regulatory function of the controller, complete loss of the communication between controller and plant; the other with only attacking some factors inside plant, not the whole plant, so that the controller is not blind to any of the required sensors, but its regulation function could be still hampered by it not being able to control all the factors in the plant. This requires a deep understanding on the NCS we are using in NCSWT. This is the idea coming from control perspective. Maybe I could make some change to the NCS to output some more factors or let it have some more input values besides x, y, id we already had. This is why I read thoroughly over control technique.

Some valued points got from this paper:

In the NCS research field, regardless of the type of network used, the overall NCS performance is always affected by network delays. This network delay (time-varying, constant, periodic) still significantly affects the close-loop system. And it requires an advanced control methodology. (Before that, I was thinking with the development of network today, network speed, network bandwidth is making this delay almost ignorable. It may not be necessary to make network delay as research object since it may not have a significant effect on NCS performance. I was wrong.)

This network delay is composed of sensor-to-controller delay and controller-to-actuator delay for all the NCS, including hierarchical structure (multiple control system with one or multiple main controller).

To be noted that, for the effects of delays in the close-loop control system, it only has two main effects. One is widely known to degrade system performances of a control system, such as higher overshoot and the longer settling time when the delays are longer than expected. The other one is to destabilize the system by reducing the system stability margin. There have been several studies to derive stability criteria for an NCS in order to guarantee that the NCS can remain stable in a certain condition. However, there is no generic stability analysis that can be applied on every NCS. I guess this could still use the frequency domain analysis for checking stability when the delays are added into the system.

For the control techniques used to solve this network delay problem, you have to maintain the stability of the system first and then try to maintain the performance of the system. Two methods were caught into the eyes:

Sampling time scheduling methodology, to appropriately select a sampling period for an NCS such that network delays do not significantly affect the control system performance;

Event based methodology, instead of using time, this method uses a system motion as the reference of the system.

To be continued…

Research fields in CPS

During my survey last week, I attempt to have a comprehensive study of current research on CPS area. Reading papers from ICCPS'10, I found that CPS covers a wide area, such as foundation of CPS, the CPS's application, CPS testbed, CPS security, and CPS design, CPS resource allocation. I will focus on the CPS security survey.

A brief history about CPS could make us have a good understanding on it. NSF awarded a five-year $5 million grant to a research project titled “ Science of Integration for Cyber-Physical Systems,”. This project is a joint effort of many research centers (Vandy, ND, General Motors Research and Development Center, UCB, Memphis). Vanderbilt will lead the project. It aims to develop the theory, methods and tools to build cps, by combining seamlessly the necessary heterogeneous computational and physical components. The Notre Dame team will be using the theoretical concepts such as passivity and symmetry to address system uncertainties and the interdependence of design concerns.

Also, by looking into the paper, I found that nowadays all the CPS research has a great real-world testbed, not just simulation. Even model designed CPS research has tried to build their testbed so as to evaluate its performance.

Two papers were taken a lot of thought when I'm reading.

A Testbed for Secure and Robust SCADA Systems

Simulation of Network Attacks on SCADA Systems

both of them were written by guys from ISIS. They claim that it is good to use C2WT as testbed for evaluation of effect by network attack over SCADA. By doing this, you have to know more about control system used in C2WT, how plant and controller are related. Considering NCS we are using is from Emeka's model, I'm asking for further reading from Emeka about this NCS.

Paper Reading on: Modeling Load Redistribution Attacks in Power Systems

This paper introduces a special type of false data injection attack, called load redistribution attacks, defined by the authors themselves. It is very similar to what I did as Deception Attack in CPS. However, it is also very specific to Smart Power Grid System, to affect the outcome of the state estimation and then further mislead the operation and control functions of Energy Management System. With some assumptions, this paper come up with a unique attack type, LR, increasing load at some buses and reducing loads at other buses while maintaining the total load unchanged. In this kind of attack, only load bus injection measurements and line power flow measurements are attackable. It can mislead the state estimation process without being detected by any of the existing techniques for bad data detection. (Not quite understand about this. It cannot be detected? Really?)

And then, the author quantitatively analyzed its damage to system operation using 6 different LR attack case. By increasing the magnitude of attack, the system operation cost is increased. From the damaging effect analysis, the author differentiate two attacking goals: immediate attacking goal and delayed attacking goal. This is a good selling point for this paper. Do a damage effect analysis using a bi-level model and a KKT-based method is used to identify the most damaging attack from an attacker's perspective. It is aiming to maximize the operation cost immediately after the attacks. The upper level represents the attacker, and the lower level represents the reactor, subject to some constraints that make it as LR attack.