Constructing performance sensitivities with sample paths in continuous-time Markov systems

Fang Cao; Xi Ren Cao

doi:10.1109/CDC.2006.377690

Constructing performance sensitivities with sample paths in continuous-time Markov systems

Fang Cao^*, Xi Ren Cao

^*Corresponding author for this work

Department of Electronic and Computer Engineering

Research output: Chapter in Book/Conference Proceeding/Report › Conference Paper published in a book › peer-review

4 Citations (Scopus)

Abstract

Sensitivity analysis plays an important role in performance optimization of stochastic systems. It provides a unified view to different areas such as perturbation analysis, Markov decision processes, and reinforcement learning. Furthermore, with the sample path based construction of sensitivity this approach leads to some new research directions such as the event-based optimization approach [5]. The previous results are on discrete-time Markov chains [4] and in this paper, we extend the sample path based construction approach to continuous-time Markov processes. The complexity involved is that in continuous-time Markov processes the transition rate also changes in addition to the changes in the transition probability matrix.

Original language	English
Title of host publication	Proceedings of the 45th IEEE Conference on Decision and Control 2006, CDC
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1069-1074
Number of pages	6
ISBN (Print)	1424401712, 9781424401710
DOIs	https://doi.org/10.1109/CDC.2006.377690
Publication status	Published - 2006
Event	45th IEEE Conference on Decision and Control 2006, CDC - San Diego, CA, United States Duration: 13 Dec 2006 → 15 Dec 2006

Publication series

Name	Proceedings of the IEEE Conference on Decision and Control
ISSN (Print)	0743-1546
ISSN (Electronic)	2576-2370

Conference

Conference	45th IEEE Conference on Decision and Control 2006, CDC
Country/Territory	United States
City	San Diego, CA
Period	13/12/06 → 15/12/06

Keywords

Continuous-time markov systems
Discrete event dynamic systems
Performance sensitivity
Perturbation analysis
Potentials

Access to Document

10.1109/CDC.2006.377690

Cite this

Cao, F., & Cao, X. R. (2006). Constructing performance sensitivities with sample paths in continuous-time Markov systems. In Proceedings of the 45th IEEE Conference on Decision and Control 2006, CDC (pp. 1069-1074). Article 4177537 (Proceedings of the IEEE Conference on Decision and Control). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CDC.2006.377690

@inproceedings{34179812de954ff8bf34c5d08a608297,

title = "Constructing performance sensitivities with sample paths in continuous-time Markov systems",

abstract = "Sensitivity analysis plays an important role in performance optimization of stochastic systems. It provides a unified view to different areas such as perturbation analysis, Markov decision processes, and reinforcement learning. Furthermore, with the sample path based construction of sensitivity this approach leads to some new research directions such as the event-based optimization approach [5]. The previous results are on discrete-time Markov chains [4] and in this paper, we extend the sample path based construction approach to continuous-time Markov processes. The complexity involved is that in continuous-time Markov processes the transition rate also changes in addition to the changes in the transition probability matrix.",

keywords = "Continuous-time markov systems, Discrete event dynamic systems, Performance sensitivity, Perturbation analysis, Potentials",

author = "Fang Cao and Cao, \{Xi Ren\}",

year = "2006",

doi = "10.1109/CDC.2006.377690",

language = "English",

isbn = "1424401712",

series = "Proceedings of the IEEE Conference on Decision and Control",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1069--1074",

booktitle = "Proceedings of the 45th IEEE Conference on Decision and Control 2006, CDC",

note = "45th IEEE Conference on Decision and Control 2006, CDC ; Conference date: 13-12-2006 Through 15-12-2006",

}

Cao, F & Cao, XR 2006, Constructing performance sensitivities with sample paths in continuous-time Markov systems. in Proceedings of the 45th IEEE Conference on Decision and Control 2006, CDC., 4177537, Proceedings of the IEEE Conference on Decision and Control, Institute of Electrical and Electronics Engineers Inc., pp. 1069-1074, 45th IEEE Conference on Decision and Control 2006, CDC, San Diego, CA, United States, 13/12/06. https://doi.org/10.1109/CDC.2006.377690

Constructing performance sensitivities with sample paths in continuous-time Markov systems. / Cao, Fang; Cao, Xi Ren.
Proceedings of the 45th IEEE Conference on Decision and Control 2006, CDC. Institute of Electrical and Electronics Engineers Inc., 2006. p. 1069-1074 4177537 (Proceedings of the IEEE Conference on Decision and Control).

Research output: Chapter in Book/Conference Proceeding/Report › Conference Paper published in a book › peer-review

TY - GEN

T1 - Constructing performance sensitivities with sample paths in continuous-time Markov systems

AU - Cao, Fang

AU - Cao, Xi Ren

PY - 2006

Y1 - 2006

N2 - Sensitivity analysis plays an important role in performance optimization of stochastic systems. It provides a unified view to different areas such as perturbation analysis, Markov decision processes, and reinforcement learning. Furthermore, with the sample path based construction of sensitivity this approach leads to some new research directions such as the event-based optimization approach [5]. The previous results are on discrete-time Markov chains [4] and in this paper, we extend the sample path based construction approach to continuous-time Markov processes. The complexity involved is that in continuous-time Markov processes the transition rate also changes in addition to the changes in the transition probability matrix.

AB - Sensitivity analysis plays an important role in performance optimization of stochastic systems. It provides a unified view to different areas such as perturbation analysis, Markov decision processes, and reinforcement learning. Furthermore, with the sample path based construction of sensitivity this approach leads to some new research directions such as the event-based optimization approach [5]. The previous results are on discrete-time Markov chains [4] and in this paper, we extend the sample path based construction approach to continuous-time Markov processes. The complexity involved is that in continuous-time Markov processes the transition rate also changes in addition to the changes in the transition probability matrix.

KW - Continuous-time markov systems

KW - Discrete event dynamic systems

KW - Performance sensitivity

KW - Perturbation analysis

KW - Potentials

UR - https://www.scopus.com/pages/publications/39649119386

U2 - 10.1109/CDC.2006.377690

DO - 10.1109/CDC.2006.377690

M3 - Conference Paper published in a book

AN - SCOPUS:39649119386

SN - 1424401712

SN - 9781424401710

T3 - Proceedings of the IEEE Conference on Decision and Control

SP - 1069

EP - 1074

BT - Proceedings of the 45th IEEE Conference on Decision and Control 2006, CDC

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 45th IEEE Conference on Decision and Control 2006, CDC

Y2 - 13 December 2006 through 15 December 2006

ER -

Constructing performance sensitivities with sample paths in continuous-time Markov systems

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this