Sir James Murray in his Scriptorium
Oxford English Dictionary

Cyclostationarity Definition

Cyclostationarity (1): (noun) Statistical cyclicity (defined below) of numerical time-series data (discrete- or continuous-time). The time-series property of having temporal statistics, such as temporal—or Fraction-of-Time (FOT, defined below)—mean, variance, autocorrelation, cumulative distribution, etc. that cycle with time. The defining property of a cyclostationary time series. Details provided on Page 3.

Fraction-of-Time (FOT) Probability: (noun) The FOT probability of an event in a time series is the fraction of time, over the lifetime of the time series, that this event occurs. For example, the FOT cumulative distribution function evaluated at the numerical value x is the FOT that the times series takes on values less than or equal to x. Also referred to as temporal probability. Details provided on Page 4.

Statistical Cyclicity: (noun) Cyclicity of statistics; periodic dependence of statistics on time, such as periodically time-varying mean. The concept of a time average that varies periodically with time is at the core of the empirical theory of cyclostationarity and generalizes to time averages that vary with multiple incommensurate periods, as explained on Page 3. In this more general setting, statistical cyclicity is cycling of statistics with one or more sine-wave components with frequencies that may or may not be harmonically related. (See the classical theory of Fourier Analysis.) As defined here, the cyclicity is regular. (Irregular cyclicity is defined below.)

Irregular Statistical Cyclicity: (noun) Statistical cyclicity with periods that vary with time in an unpredictable manner. A time series exhibits irregular statistical cyclicity if and only if there exists a nonlinear time-warping function that renders the statistical cyclicity regular. In practice, this property may have to be discovered empirically using data-adaptive property-restoral algorithms that learn the warping function that produces cyclostationarity, as explained in JP65. Not all non-regular statistical cyclicity is irregular as defined here. For example, some time-series of mechanical vibration data is non-regular, such as the sequence of damped oscillations, each of which is initiated by a rotating bearing striking a fault in a bearing race under the condition of irregular rotational speed. The time variation on the speed can be removed by time warping, but this warps the damped oscillations in a non-regular manner. Details provided on Pages 3 and 4.

Cyclostationary (1): (adj.) A time series that exhibits cyclostationarity is said to be a cyclostationary time series. A modifier that applies strictly to only time series of numerical (real- or complex-valued) time-series data. Sometimes used inappropriately to indicate periodicity of things other than time-series data, like the system function for a linear periodically time-varying transformation—sometimes referred to as an operator on time series referred to as vectors—or like an algorithm designed to exploit cyclostationarity of time series data; parameters of such algorithms may vary periodically with time, but an algorithm is not a time-series of data.

Cyclostationarity (2): (noun) Same as cyclostationarity (1) defined above except with the more concrete (empirical) temporal probability replaced with the abstract stochastic probability; that is, the underlying empirical time series is replaced with an abstract (intangible) stochastic process, which is a time-series of abstract random variables, not empirical numerical values. More specifically, the temporal statistics (mathematically idealized in the FOT theory by using the limit, as averaging time approaches infinity) are replaced with abstract expected values defined relative to abstract probability measures. The fundamental duality, between the temporal (FOT) theory for time series and the stochastic (classical probabilistic) theory for stochastic processes, of cyclostationarity is a key theme of the graduate-level text book Introduction to Random Processes, with Applications to Signals and Systems.

Cyclostationary (2): (adj.) Same as cyclostationary (1) definition except with the cyclostationarity (1) definition replaced with the cyclostationarity (2) definition.

Polycyclostationarity: (noun) Cyclostationarity with statistical cyclicity containing some sine-wave components that are not harmonically related; that is, the cyclicity consists of multiple (finite in number) incommensurate cycles.

Almost Cyclostationarity: (noun) A generalization of Polycyclostationarity that accommodates infinitely many incommensurate cycles. That is, the statistical cyclicity takes the form of statics that are almost periodic functions of time. (See the classical theory of Almost Periodic Functions.)

Cyclostationary Time Series (TS) (or Stochastic Processes (SP))—Generic: (noun) Times series (or Stochastic processes) exhibiting any one of the many specific types of cyclostationarity encompassed by the above definitions, all—with the exception of the simplest case of regular cyclostationary stochastic process with exactly periodic statistical cyclicity—originally introduced by the WCM in Introduction to Random Processes, with Applications to Signals and Systems and Statistical Spectral Analysis: A Nonprobabilistic Theory (and earlier in journal papers referred to in these books). This generic term also includes Generalized Almost Cyclostationary processes, originally introduced by William A. Brown in On the Theory of Cyclostationary Signals; more extensive development of GACS was achieved by Antonio Napolitano, who also introduced the generalization called Spectrally Correlated Processes, and presented these generalizations in the book Generalizations of Cyclostationary Signal Processing. Some of these generalizations of cyclostationary stochastic processes do not have time-series counterparts and are therefore more abstract and more loosely tied to empiricism. The most comprehensive treatment in one source of all the above types of time series and stochastic processes, as of 2019, is Antonio Napolitano’s 2020 book Cyclostationary Processes and Time Series: Theory, Applications, and Generalizations. These latter two book also include several additional generalizations of cyclostationary stochastic processes.

Cycle: (noun) A series of events that are regularly repeated in the same order

(adj.) This noun can be used as an adjective. For example, the frequency of a cycle can be referred to as the cycle frequency.

Cyclic: (adj.) Occurring in cycles; regularly repeated. The statistics of a cyclostationary time series are cyclic. The frequencies of the cycles are cycle frequencies, but they are not cyclic frequencies. The aliasing phenomenon of cyclostationarity, in which measurement of one cycle is affected by another cycle, is called cycle aliasing, but it is not cyclic aliasing.

Cyclicity: (noun) the quality or state of something that occurs or moves in cycles. Cyclostationary time series exhibit cyclicity (of their statistics).

PURPOSE OF THIS WEBSITE

This website has been established for the sole purpose of supporting students and users of the statistical theory and methodology of cyclostationarity, including researchers and practitioners in academia and industry—engineers, scientists, and other researchers working with time-series data representing cyclic phenomena. The primary objective is to assist those trying to learn the fundamentals of the existing body of knowledge on this topic, but a selection of new research results also is expected to be included, although the scope of this secondary part of the website remains to be determined. More specifically, this website is a study guide and overview of cyclostationarity, a subfield of statistical signal processing theory and methodology, which provides recommended study materials including narratives, expository commentary, essays, and linked references to monographs and expository treatises, all intended to elucidate, illuminate, explicate, and otherwise critique the subject of cyclostationarity to the best of my ability as WCM -- William A. Gardner, Website Content Manager. For a glimpse of this website’s content, go to the Table of Contents Page.


June 2020 comment from the WCM:

In the two years since construction of this website began, I have increasingly been taking license to include autobiographical material. When this site was initiated, the objective was for it to be a purely tutorial website addressing only the technical subject of cyclostationarity. As the writing of content proceeded, autobiographical remarks began creeping in and now, two years later, these two topics have merged—more so on some pages than others but, overall, to such an extent that I felt it should be briefly addressed here at the outset. It’s unlikely that I would ever devote the time and effort required to write a book-length autobiography, but as long as I see ways for autobiographical content to contribute to the teaching of cyclostationarity—its origins, the motivations for its development, the impediments to its development, the debates it has engendered, aspects of humanity that it has exposed etc., as well as its purely technical content—I shall continue to blend these two topics together. This personal flavor is unusual for a tutorial treatment of such a highly technical subject, but—as readers will find—the history of this subject has been a personal journey for me and the conflict that has defined this journey is an important lesson in itself about those aspects of human nature that limit scientific progress.

UBIQUITY OF CYCLICITY

This website is motivated by observations regarding the Ubiquity of Cyclicity in time-series data arising in science and engineering dating back to my (WCM) doctoral dissertation from the University of Massachusetts, Amherst, under the direction of Professor Lewis E Franks, reporting on my research initiated in 1969—just after leaving Bell Telephone Laboratories, half a century before the construction of this website.

This Ubiquity of Cyclicity exists throughout what some refer to as God’s Creation: the World comprised of all natural phenomena on our planet Earth, our Solar System, our Galaxy, and the Universe; and it also exists throughout much of the machinery and process comprising mankind’s creation: technology in the form of electrical, mechanical, chemical, etc. machinery and processes. Because of this Ubiquity of Cyclicity, we find that a great deal of the observations, measurements, and other time-series data that we collect, analyze, and process in science and engineering exhibit a form of cyclicity. In the simplest cases, this cyclicity is simply periodicity—the more-or-less-exact repetition of data patterns; but it is far more common for the cyclicity to be statistical in its nature. By this, it is meant that appropriately (WCM’s Note: this is a critical modifier, to be explained in this website) calculated time averages of the data produce periodic patterns that are often not directly observable in the raw (non-averaged) data. In some cases, the averaging may be performed over the members of a preferably-large set of individual time series of data arising from some phenomenon such as might be obtained by repetition of some experiment, rather than over time (appropriately), but this is most often not the case for empirical data.

In many cases, it is found that the statistical cyclicity is regular (the statistics obtained by averaging (appropriately) long enough are essentially exactly periodic) and, in this case, the time-series is said to be cyclostationary. But in many more cases the cyclicity is irregular. Roughly speaking, this means the period of the cyclicity of the statistics, such as short-term empirical means and variances, and correlations, etc., of the time-series data changes over the long run in an irregular manner, which makes it quite difficult to perform averaging over the long term in the appropriate manner. The level of complication in time-series analysis and processing caused by irregular cyclicity was only recently, in 2015, reduced by the origination in Statistically Inferred Time Warping of theory and method for converting irregular cyclicity in time-series data to regular cyclicity. This recent breakthrough opens the door, for many fields of science and engineering, to much broader application of the otherwise now-firmly-established theory and method for exploiting regular cyclostationarity. Nevertheless, it does not address non-regular statistical cyclicity that is not irregular, as defined in this website: Irregular Statistical Cyclicity.

That being said, what exactly is meant by “exploiting cyclostationarity”? As explained in considerable detail in this website, this means using knowledge of the cyclic statistical character of otherwise erratic or randomly fluctuating time-series data to achieve higher performance in various tasks of statistical inference than could otherwise be obtained; that is, making more precise and/or more reliable inferences about the physical source of time-series data on the basis of processing that data in various ways generally referred to as “signal processing”. Such inferences may consist of detection of the presence of signals in noise, estimation of parameters of such signals, filtering such signals out of noise, identifying signal types, locating the source of propagating signals, etc.

As an indication of how widespread exploitation of cyclostationarity in time-series data has become since its inception 50 years ago, a web search using Google Scholar was performed and reported in JP65 in April 2018, This search was based on just under 50 nearly-distinct applications areas in science and engineering, and the search terms were chosen to yield only results involving exploitation of cyclicity in time-series data. By “nearly distinct”, it is meant that the search terms were also selected to minimize redundancy (multiple search application areas producing the same “hits”). As shown in Table 1, the search found about 136,000 published research papers.

As another measure of the impact the cyclostationarity paradigm has had, Professor Antonio Napolitano, in Chapters 9 and 10 of his 2019 book Cyclostationary Processes and Time Series: Theory, Applications, and Generalizations, surveys fields of application of the cyclostationarity paradigm, and identifies on the order of 100 distinct applications and cites about 500 specific published papers addressing these applications; his carefully selected bibliography on primarily cyclostationarity includes over 1500 published papers and books.

Table 1 Nearly Distinct Application Areasa

Serial NumberHeadingNumber
1"aeronautics OR astronautics OR navigation" AND "CS/CS"3,190
2"astronomy OR astrophysics" AND "CS/CS"864
3"atmosphere OR weather OR meteorology OR cyclone OR hurricane OR tornado" AND "CS/CS"2,230
4"cognitive radio" AND "CS/CS"8,540
5"comets OR asteroids" AND "CS/CS"155
6"cyclic MUSIC"512
7"direction finding" AND "CS/CS"1,170
8"electroencephalography OR cardiography" AND "CS/CS"742
9"global warming" AND "CS/CS"369
10"oceanography OR ocean OR maritime OR sea" AND "CS/CS"3,060
11"physiology" AND "CS/CS"673
12"planets OR moons" AND "CS/CS"274
13"pulsars" AND "CS/CS"115
14"radar OR sonar OR lidar" AND "CS/CS"5,440
15"rheology OR hydrology" AND "CS/CS"639
16"seismology OR earthquakes OR geophysics OR geology" AND "CS/CS"1.090
17"SETI OR extraterrestrial" AND "CS/CS"83
18autoregression AND "CS/CS"2,040
19bearings AND "CS/CS"3,980
20biology AND "CS/CS"2,030
21biometrics AND "CS/CS"309
22chemistry AND "CS/CS"2,020
23classification AND "CS/CS"10,900
24climatology AND "CS/CS"811
25communications AND "CS/CS"21,200
26cosmology AND "CS/CS"172
27ecology AND "CS/CS"356
28economics AND "CS/CS"2,050
29galaxies OR stars AND "CS/CS"313
30gears AND "CS/CS"2,000
31geolocation AND "CS/CS"676
32interception AND "CS/CS"2,270
33mechanical AND "CS/CS"4,770
34medical imaging OR scanning AND "CS/CS" 1,370
35medicine AND "CS/CS"2,990
36modulation AND "CS/CS"17,000
37physics AND "CS/CS"4,539
38plasma AND "CS/CS"542
39quasars AND "CS/CS"47
40Sun AND "CS/CS"4,320
41UAVs AND "CS/CS"238
42universe AND "CS/CS"209
43vibration OR rotating machines AND "CS/CS"3,240
44walking AND "CS/CS"990
45wireless AND "CS/CS"15,100
TOTAL135,628

a “CS/CS” is an abbreviation for “cyclostationary OR cyclostationarity”

At Issue

Considering that tutorials on this topic have been appearing in published form (journals, magazines, text books, reference books, etc.) and on websites, like Wikipedia more recently, for over thirty years now—since publication of the first comprehensive text-book treatment, Statistical Spectral Analysis: A Nonprobabilistic Theory, Part II, Periodic Phenomena, Prentice-Hall, 1987—the issue being addressed with this website is not a lack of sources, but rather a perceived need to recapture the original perspective from which this subject was first developed in earnest: a perspective that avoids unnecessary abstraction and introduces concepts in a carefully chosen manner that follows a step-by-step method that avoids conceptual leaps that too often span gaps that are larger than those students can comfortably jump across. This original perspective and the conceptual clarity it provides has come to be masked by the unnecessary abstraction of the stochastic process promulgated in many cases by authors who’s own training was unfortunately based on this mathematical construct to the exclusion of a more straightforward empirically motivated approach. The stochastic process was invented by mathematicians for mathematicians to facilitate developing/proving theorems at the often hidden or at least "glossed-over" expense of not being directly related to empirical time-series data. This unfortunate development began in the 1940s and rather quickly led to its wholesale promotion by mathematicians and its resultant adoption in the 1950s and 1960s by engineers and scientists who were not forewarned of the absence of any practical necessity for this particularly abstract mathematization of the theretofore empirical subject of time-series analysis as initially developed by empirically minded scientists and engineers prior to this transition, as explained in Statistical Spectral Analysis: A Nonprobabilistic Theory, Part I, Constant Phenomena, where more of the history of this unfortunate paradigm shift is addressed.

As discussed in considerable detail on Page 4, one can argue quite convincingly that, from a scientific and engineering perspective, a wrong step was taken back around the middle of the 20th Century in the nascent field of time-series analysis (more frequently referred to as signal processing today) when the temporal counterpart referred to here—introduced by Norbert Wiener in his 1949 book, Extrapolation, Interpolation, and Smoothing of Stationary Time Series, with Engineering Applications—was rejected by mathematicians in favor of Ensemble Statistics, Probability, and Stochastic Processes. This step away from the more concrete conceptualization of statistical signal processing that was emerging and toward a more abstract mathematical model, called a stochastic process, is now so ingrained in what university students are taught today, that few STEM (Science, Technology, Engineering, and Mathematics) professors and practitioners are even aware of the alternative that is, on this website, argued to be superior for the great majority of real-world applications—the only advantage of stochastic processes being their amenability to mathematical proof-making, despite the fact that it is typically impossible to verify that real-world data satisfies the axiomatic assumptions upon which the stochastic process model is based! In essence, the assumptions pave the way for constructing mathematical proofs in the theory of stochastic processes, not—as they should in science—pave the way for validating applicability of theory to real-world applications.

Historical Perspective

Time-Series Analysis is the designation given to the broad field of study of theory and method for analyzing data that is in chronological order: time-indexed series of numerical observations/measurements. Such data arises in essentially all fields of empirical science and engineering, manufacturing, economics, and all other fields of quantitative historical analysis, and is of crucial importance in carrying out the scientific method applied to the empirical study of dynamic phenomena. So called science without empirical study is, in fact, not science at all—it is comprised of only theoretical research, only logical processes, and is consequently highly speculative and dangerous when used as a replacement for science. It exists outside the realm of realism. The development of today’s world of High-Technology has been made possible by various key disciplines, but real science and associated engineering based on the discipline of empirical time-series analysis is certainly among them.

As explained in Statistical Spectral Analysis: A Nonprobabilistic Theory, Part I, Constant Phenomena, the seminal work on empirical time-series analysis was done prior to 1950 and produced the roots of what has become the modern-day field of statistical time-series analysis, often referred to more recently as statistical signal processing because of the early major contributions to this field by electrical and computer engineers who did a great deal of the seminal work on digital implementations of algorithms for statistical analysis and data processing, gave birth to information theory, and applied this to develop the statistical theory of communications, which deals in signals that carry information and are stored and retrieved and transmitted from one location and received at another. The pioneering work, prior to1950, did not—for the most part—include the concepts of probability and stochastic process. These abstractions did not begin to permeate the field until around 1950 and, thereafter, its rapid growth in popularity among mathematicians had the unfortunate impact of rendering subsequent theory of statistical time-series analysis unnecessarily abstract and difficult for empirically-minded students and practicing analysts to grasp, to intuit, and to apply in a practical manner.

Most likely it is because probability plays a central role in information theory that the stochastic process formulation of signal processing replaced the original temporal (FOT) formulation from time-series analysis. To be sure, stochastic processes have their place in mathematical theories that permeate many fields of quantitative study of dynamical systems. And, in many applications, it is essential that the stochastic processes are nonstationary and non-cyclostationary. But it is a shame that the stochastic theory of stationary processes was adopted in place of the FOT theory that preceded it and that, as cyclostationarity came on the scene, practicing researchers and university professors, all steeped in the stochastic theory of stationary processes, to the exclusion of the less abstract FOT theory, preferred (out of ignorance it would seem) the more familiar stochastic process framework to study and teach this emerging field despite the sound arguments against this proffered by the field’s leading pioneer (WCM). Paradigms of thought are not easily shifted. Many examples in science are cited on Page 11.

As discussed in Statistical Spectral Analysis: A Nonprobabilistic Theory, Part I, Constant Phenomena, the habit of relying on the stationary stochastic process model for thinking about signal processing is in part responsible for the relatively late discovery of the crucial role that could be played by cyclostationarity in signal processing, because it was recognized early on that modeling the time origin of a cyclostationary stochastic process by a random variable uniformly distributed over one period produced a stationary stochastic process—a familiar and therefore comfortable model. This website is the WCM’s final effort to motivate those who use and teach statistical signal processing to recognize the cost of sticking with the status quo: the stochastic process theory of cyclostationarity. 

Scope

This website presents an introduction to the fundamental concepts, history, basic theory, and applications of cyclostationarity and its exploitation for purposes of statistical inference (information extraction from time-series data) and it includes a bibliography that directs users and provides links to carefully selected reference sources for expansions on all the theoretical and methodological topics addressed here, as well as on the practice of exploiting cyclostationarity—a practice that is essentially defined by the signal processing algorithms used. (The terms data and signal are typically used interchangeably when the data is an information-bearing time-series.)

PRIMARY SOURCE

The Content Manager of this website is the author of the great majority of website content and linked material and has published a considerable amount of explanatory material on the topic of cyclostationarity since 1971, including a number of seminal contributions in research journals, graduate-level textbooks, and professional reference books that introduce and develop a comprehensive statistical theory and methodology for understanding and utilizing this special property of time-series data from cyclic phenomena. This explanatory material includes original published research results spanning nearly half a century that establish the conceptual and mathematical foundations of the subject, tutorial treatments, and the philosophical considerations that motivated the author of these publications to discover and teach the duality of two distinct models for conceptualizing and mathematizing the statistical nature of cyclostationarity: stochastic and non-stochastic models—the latter of which is also called fraction-of-time (FOT) probabilistic models and sometimes function or functional models. This published material also includes comprehensive histories, research reviews, and bibliographies on the topic of cyclostationarity.

INFORMATION CONSOLIDATION

Because the publication-industry’s commercial interests often create economic impedance to potential users’ access to previously published material—see Library Genesis and Sci-Hub—it is hoped that this website will circumvent this impedance by bringing much of the author’s seminal work and subsequent complementary work on cyclostationarity by other experts on this subject together in one place for educational purposes, and by providing users with perspective and careful guidance for gaining a command of this body of knowledge or at least those parts of this body that may serve individual users’ more specific purposes.

FOCUS

In order to maintain a close link between physical reality and mathematical models concerning cyclostationarity, the classes of continuous-time, non-stochastic, scalar-valued time-series exhibiting regular cyclostationarity or regular polycyclostationarity are preferred by the WCM as vehicles for tutorial purposes, although the WCM’s work and this website address to varying degrees other classes listed here. However, because digital computers require that time be quantized, the algorithms produced by the methodology of cyclostationarity can be implemented on digital computers only in terms of discrete-time models and processing, as seen in the algorithm-oriented material addressed herein. For a deeper discussion of the pedagogical value of the focus chosen for this website, the reader is referred to the introductory discussion on Page 4 prior to section 4.1.