On this page, the core seminal contributions to the theory of cyclostationarity are identified and concisely described. To avoid unnecessary redundancy and also to enhance readability for many visitors to this website, theorem statements in all their sometimes-gory detail are not included. Rather, statements in English of the essence of theorems are provided, and original sources and/or locations in the encyclopedic book [B2] are cited. There are two salient contributors who are the sources of the great majority of these contributions, and the contributions of each occur almost entirely in two distinct periods of time. The first period is the mid-1970s to the mid-1990s, which is when most of William Gardner’s contributions were made, and the second period is the mid-1990s to the present, which is when most of Antonio Napolitano’s contributions were made. The smaller number of contributions from others occurred primarily in the later of these two periods. Most recently, there has been renewed attention from both these contributors to FOT-Probability models for cyclostationary signals as a superior alternative for many applications to the more abstract stochastic process models. Two tightly coupled tutorial articles from 2022 (also containing novel contributions) are made available here on pages 3.2 and 3.3.
The most comprehensive treatise on the subject of cyclostationarity written to date is the recently published 2020 book entitled Cyclostationary Processes and Time Series: Theory, Applications, and Generalizations, written by the most prolific contributor to this subject in this 21st Century, Professor Antonio Napolitano [B2]. Because this encyclopedic book is the most current and authoritative and scholarly treatment of this subject, it has been used here as the primary source for identifying many of the core seminal contributions made by the several key contributors included below in Sections 9.1, 9.2, and 9.3. This book is comprised of 720 pages, comprehensively containing citations of about 1400 distinct research publications on cyclostationarity (a substantial portion of which are theoretical contributions), including 582 citations of Gardner’s publications (some of which are multiple citations of single works, like books).
The use on this page of the term seminal contribution is based on the standard definition:
Def. Seminal: adjective
(Of a work, event, moment, or figure) strongly influencing later developments;
Similar terms: influential, formative, groundbreaking, pioneering, original
The meaning of seminal favors Gardner’s contributions over those of others over the last 40 years because he “got there first” and because refinements and deeper mathematization and translation from Gardner’s FOT-probability versions to stochastic-process versions of already-formulated concepts do not qualify as seminal, though they certainly can be important contributions to the theory.
The conceptual and theoretical foundation and framework of William Gardner’s Fraction-of-Time (FOT) Probabilistic Theory of Cyclostationarity for Time Series is succinctly captured by the set of 19 numbered theorems listed below, following Gardner’s 20 basic Cyclostationarity Definitions. All definitions and theorems were originated by Gardner. The 19 theorems are partitioned into three categories, Probabilistic and Statistical Functions of Time and Frequency, Transformations of Probabilistic and Statistical Functions by Signal processing Operations, and Ad Hoc and Optimum Statistical Inference.
(Personal note: Although some of these theorems collected on this page have begun to be referred to as “Gardner’s . . . Theorem” or “Gardner’s . . . Relation” (e.g., see [B2]), there is a saying among scholars that was recently brought to my attention by Professor Thomas Kailath: “Whatever the name attached to a theorem, someone else was probably the actual originator.” To avoid this pitfall, I’ve not included my name in any of the theorems I believe—based on my scholarly searches of the literature—I originated. But this is not to say that I doubt Professor Napolitano’s scholarship leading to his attachment of my name to some of theorems included here.
For the sake of parsimony, the terms originally defined by Gardner are only listed. Instead of providing the actual definitions, citations to original sources are given. This also provides context for the origin of these terms. For the same reasons, the list of theorems provided includes only descriptive theorem labels, but this list is followed by notes that include some expansion and citations to original sources. These notes are followed by commentary on these contributions from experts in the field made around the time the contributions were made.
Each of the theorems listed have various alternative versions for which the differences lie in the details of assumptions made and, correspondingly, the strength of results obtained. This is typical for many if not most theorems in mathematics. Nevertheless, for pragmatic users of the results provided by theorems, the practical essence of each theorem is often unique. For example, the exact nature of the convergence of an infinite series depends on the particular properties of the series. The formulator of a theorem may have flexibility in the choice of assumptions made about properties. Generally, the more restrictive the assumptions made, the stronger the results that follow. In many applications, what’s important to users is the fact that specific types of series do converge in some sense and others diverge in every useful sense.
In the earliest statements of many of the theorems included here, detailed assumptions are not made. Rather, it is taken for granted that all mathematical quantities referred to in a theorem statement exist under some reasonable assumptions. For example, it is assumed that for whatever statistical moments or cumulative distributions or the like that are referred to in a theorem, there exist practical signal models for which these statistical functions do indeed exist. In fact, many examples of formulaic models are given and the statistical functions are calculated from these models. The essence of the theorems lies in the relations among these functions. These relations are generally expressed in the form of equations. The particular sense in which equality holds in these equations typically depends on detailed assumptions. An equation among time functions may hold for all time, or for almost all time (meaning all time except for a set of times of Lebesgue measure zero), or in temporal mean square, or with fraction-of-time-probability equal to one, etc. More often than not, when empirical quantities are being dealt with, these distinctions become irrelevant. Similar remarks apply to functions of multiple time variables, or frequency variables, or both types of variables, and in some cases, spatial variables.
In summary, existence of statistical functions is not the focus in these theorems; the focus is the relations among the functions (when they exist)—this is what matters in these theorems. Nevertheless, readers are forewarned that, when the objective is to further develop the underlying theory of cyclostationarity, explicit statements of the details of assumptions under which the results of theorems are valid may become important. This is found in the encyclopedic treatment [B2] in contrast to the original sources, such as [Bk2].
Terminology (citations yet to be included)
Theorems from Gardner’s Theory
CATEGORY 1: Probabilistic and Statistical Functions of Time and Frequency
Theorem 1: Fundamental Theorem of Sine-Wave Component Extraction – Original definition of the linear operator that extracts finite-strength additive sinewave components from any well behaved function of a persistent times series and proof that this operator is a nonstochastic expectation operator with respect to the generally almost period temporal (non-stochastic) cumulative probability distribution function, which also was originally defined by Gardner and proven to satisfy the defining properties of probability distribution functions in the case for which the set of sinewave frequencies comprises all harmonics of any set of incommensurate fundamental frequencies included. For one fundamental frequency, the distribution function provides a cyclostationary model for time series; for a finite set of multiple fundamental frequencies, it produces a poly-cyclostationary model, and for a countable infinity of fundamental frequencies, it produces an almost cyclostationary model.
Theorem 2: Non-Stochastic Moment Expansion Theorem for Sine Waves Extraction – Original theorem statement and proof that the set of sine waves to be extracted from any function of a time series that admits a generalized Volterra series representation can be expressed as a linear combination of the sine waves contained in each of all the moments of the time series or, equivalently, all the cumulants.
Theorem 3: Generalizations of the Fundamental Theorem of Sine Wave Component Extraction to extraction of subsets of sine waves and to Estimation on Finite-Time intervals – Completely analogous to Theorem 1, which applies to infinite intervals and complete sets of harmonically related sine waves. Because sine waves with arbitrary frequencies are not generally orthogonal to each other on finite intervals, this theorem deals with estimation instead of extraction.
Theorem 4: Sine-Wave-Extraction Derivation of the Non-Stochastic Temporal Cumulant Function – Original definition and derivation outside the framework of probability.
Theorem 5: Relation between non-stochastic Higher-Order Cyclic Moments and Cyclic Cumulants – Original derivation of the non-stochastic counterpart of the Leonov/Shiryaev Relation between stochastic moments and cumulants and its decomposition into the entirely new relation among cyclic moments and cyclic cumulants for a non-stochastic time series exhibiting cyclostationarity.
Theorem 6: Relation between Non-Stochastic Spectral Correlation and Cyclic Temporal Autocorrelation – Generalization of the Wiener Relation between non-stochastic average power spectral density function and temporal autocorrelation function: Original derivation.
Theorem 7: Relation between Non-Stochastic Temporal and Spectral Higher-Order Moments: Original derivation.
Theorem 8: Cyclic-Periodogram/Correlogram Relation and its Higher-Order Counterpart – Original definitions and derivations.
Theorem 9: Synchronized Averaging Identity for Non-Stochastic FOT-Probabilistic Functions – Original derivation.
CATEGORY 2: Transformation of Probabilistic and Statistical Functions by Signal Processing Operations
Theorem 10: Non-Stochastic Spectral-Correlation Input/Output Relations for Key Signal Processing Operations: 10.1, sampling & aliasing; 10.2, multiplication; 10.3, convolution & band limitation.
Theorem 11: Non-Stochastic Higher-Order Spectral-Moment and Spectral Cumulant Input/Output Relations for Key Signal Processing Operations: 11.1; 11.2; 11.3.
Theorem 12: Derivation of Signal Selectivity of Non-stochastic Cyclic Cumulants
CATEGORY 3: Optimum Statistical Inference
Theorem 13: Non-Stochastic Spectral Correlation Theory of Optimum Almost-periodically Time-Variant Filtering of Almost-Cyclostationary Signals: Generalization of the theory of non-causal Wiener filtering from stationary to almost cyclostationary time series. Includes the special cases of optimum poly-periodic and optimum periodic filtering of poly-cyclostationary and cyclostationary signals, respectively.
Theorem 14: Non-Stochastic Spectral Correlation Theory of Optimum Detection of Cyclostationary Signals: Maximum-SNR and Maximum-Likelihood spectral-line regenerators.
Theorem 15: Non-Stochastic Spectral Correlation Theory of N-th Order Nonlinear Synchronizers
Theorem 16: Non-Stochastic Spectral Moment and Cumulant Theory of Cyclostationary Signal Classification
Theorem 17: Non-Stochastic Theory of Almost-Periodically Time-Variant Linear System Identification
Theorem 18: Non-Stochastic Theory of Time-Invariant Volterra Nonlinear System Identification
Theorem 19: Non-Stochastic Theory of Periodically and Poly-Periodically Time-Variant Nonlinear Volterra System Identification
Notes and References for Theorems:
Applications of Theorems
Early Quotations from Commentary on Gardner’s Seminal Contributions (late 1980s to late 1990s)
Professor Enders A. Robinson, Columbia University and Member of the National Academy of Engineering, states in a letter of reference on behalf of Dr. Gardner:
From time to time it is good to look back and see in perspective the work of those people who have made a difference in the engineering profession. One of the important members of this group is William A. Gardner.
Professor Gardner has the ability to impart a fresh approach to many difficult problems. William is one of those few people who can effectively do both the analytic and the practical work required for the introduction and acceptance of a new engineering method. His general approach is to go back to the basic foundations and lay a new framework. This gives him a way to circumvent many of the stumbling blocks confronted by other workers . . .
I am particularly impressed by the fundamental work in spectral analysis done by Professor Gardner. Whereas most theoretical developments make use of ensemble averages, he has gone back and reformulated the whole problem in terms of time-averages. In so doing he has discovered many avenues of approach which were either not known or neglected in the past. In this way his work more resembles some of the outstanding mathematicians and engineers of the past. This approach took some courage, because generally people tend to assume that the basic work has been done, and that no new results can come from re-examining avenues that had been tried in the past and then dropped. William’s success in the approach shows the strength of his engineering insight. He has been able to solve problems that others have left as being too difficult. It is this quality that he so well imparts to his students, who have gone forth and solved important and far-reaching problems in their own right.
Professor Bernard C. Levy, Chairman of the Department of Electrical & Computer Engineering at the University of California, Davis, states in a nomination letter:
Dr. Gardner’s random processes textbook has several original features which make it stand out among all other textbooks in the same general area. First, it contains a chapter on cyclostationary processes, which have been one of the main topics of research for Dr. Gardner throughout his research career. These processes play a key role in the study of digital communications systems, and virtually all recent digital communications textbooks refer to Dr. Gardner’s random processes book as well as to his research papers on cyclostationary signal processing. Another original feature of Dr. Gardner’s random processes book is its detailed development of the time-average approach for evaluating the statistics of random signals. This approach provides the theoretical underpinning for the textbook Statistical Spectral Analysis: A Nonprobabilistic Theory which was written by Dr. Gardner for his Spectral Analysis course (ECE 262). Because of its revolutionary time-average approach (which can be traced back in part to the pioneering work of Norbert Wiener on generalized harmonic analysis), this textbook has been the subject of entertaining exchanges in the Signal Processing Magazine of the IEEE Signal Processing Society. As a consequence of Bill Gardner’s courage and vision in pursuing a radically new path, based on the eminently sensible view that the analysis of random signals should be based on statistics extracted from the observed data, this book has had a huge impact on modern spectrum analysis practitioners.
Professor Lewis E. Franks, previous NSF program director and previous chairman of the Department of Electrical & Computer Engineering at the University of Massachusetts, Amherst, states:
I believe I have read a major portion of Gardner’s papers and textbooks. I feel that a unique feature of all these publications, compared to other engineering documents of a similar nature, is the presence of a strong scholarly style. Previous contributions to the topic are meticulously sought out and referenced. It’s not just a matter of being polite to colleagues or avoiding confrontations over omitted citations; but a genuine attempt to establish an important historical context for new results or interpretations. The relevance of prior contributions to the topic is carefully laid out and unified… On the topic of cyclostationary processes, I feel that he has, almost single-handedly, developed the theoretical and applied engineering aspects of the topic to the point of today’s widespread recognition of its utility.
Dr. Nelson Blachman, well known communication systems author, writes:
My interest in Dr. Gardner’s research is concerned with the advances in cyclostationary signal processing that has been his greatest contribution to electrical engineering research. In fact, Professor Gardner is “Mr. Cyclostationary”, the promoter and leading international researcher in this important signal processing area, with two textbooks, numerous papers, and a federal government subsidized workshop to his credit. As a scientist involved with Department of Defense signal processing research aimed at threat analysis of signals related to national security interests, I can indicate to you that Dr. Gardner’s work has had profound impact on the analysis of these signals, but classification of the analysis has kept the importance of his work from being known to the general public and others in academia. There is another attribute of Dr. Gardner’s research and tutorial material that makes him stand out among so many of my other academic colleagues, and that is his depth of research (especially historical and mathematical detail) and his attention to precision and detail in his writings. I have always found it difficult to find errors and to take issue with any of Professor Gardner’s papers because he has meticulously done his research; this is in contrast to so many other academics who tend to be more sloppy in their mathematical precision and who do not always thoroughly check the technical literature in depth. I believe this high degree of professional research has contributed greatly to the widespread acceptance of Dr. Gardner’s technical writings as being the preeminent authority on cyclostationary signal processes and their exploitation.
Dr. Akiva Yaglom, Mathematician and Physicist, USSR Academy of Sciences, wrote in a book review published in Theory of Probability and Its Applications:
It is important . . . that until Gardner’s . . . book was published there was no attempt to present the modern spectral analysis of random processes consistently in language that uses only time-averaging rather than averaging over the statistical ensemble of realizations [of a stochastic process] . . . Professor Gardner’s book is a valuable addition to the literature”
Professor James Massey, information theorist and cryptographer, Professor of Digital Technology at ETH Zurich, member of the National Academy of Engineering, wrote in a prepublication book review in 1986:
I admire the scholarship of this book and its radical departure from the stochastic process bandwagon of the past 40 years.
Dr. Bart F. Rice, of Lockheed Research, past chairman of the Santa Clara chapter of the IEEE Signal Processing Society, in 1992 letters of reference on behalf of Dr. Gardner states:
Gardner’s crowning achievement is the development of the theory of spectral correlation and cyclostationary signal processing and analysis. It is hard to overstate the importance of this work. Like many important theoretical developments, the theory is ‘unifying’ in that it brings into a common, cohesive framework results that previously seemed unrelated, or whose relation was not completely appreciated or understood. The consequences have been new insights and new results. In the not-too-distant future, it will constitute part of the core graduate curriculum in signal processing and in communications . . . And, as with much seminal work of a profound nature, Gardner’s theories have spawned a large amount of activity and new ideas and applications by others.
Recent Quotations Addressing Gardner’s Contributions (2020)
In addition to the early quotations (excerpts from letters) given above, some new quotations—excerpts taken from the preface of the 2020 book [B2]—which include bibliographical notes and references, are provided below, and apply to Theorems 1 – 19.
Preface, p. xxii, “Fundamental noise and interference immunity and signal-separability properties of CS and ACS processes [and time series] discovered in the early 1980s by Gardner (e.g., (Gardner,1985) [Bk3] and (Gardner, 1987f) [Bk2]), and theoretically developed in his seminal work performed primarily during the subsequent decade provided the foundation for the last 30 years of exploitation of almost-cyclostationarity in the design and analysis of signal processing methods and algorithms for communications, telemetry, and radar systems.”
Preface, p. xxiv, “The FOT probability framework for CS and ACS [time series] is introduced in (Gardner, 1985) [Bk3] and treated in depth in (Gardner, 1986c) [JP15], (Gardner, 1987f, Part II) [Bk2], (Gardner and Brown, 1991) [JP34], and (Gardner, 1991a) [JP36] with reference to continuous-time signals and in (Gardner, 1994) [JP53] for both continuous- and discrete-time signals; the case of complex signals is considered in (Gardner, 1987f, Part II) [Bk2] and (Brown, 1987) [p.8.3].”
Preface, p. xxvi, “The first fundamental treatment on cyclostationary processes in a book can be found in (Gardner, 1985, Chap. 12) [Bk3]. Several books are entirely dedicated to cyclostationarity. The extensive treatment in Part II of (Gardner, 1987f) [Bk2] contains the foundations of cyclic spectral analysis in the FOT approach.”
The conceptual and theoretical foundation and framework of William Gardner’s Theory of Cyclostationarity for Stochastic Processes [Bk3] is similar to that described in Section 9.1.1 for non-stochastic time series; however, it is less extensive because of Gardner’s preference for the non-stochastic theory and his tenet that the non-stochastic theory is of considerably higher conceptual and utilitarian value for practitioners. Almost every theorem cited in Sections 9.1.1 has its counterpart in the theory for stochastic processes. The reader is referred to the book [Bk1] for the earliest compilation in book form, and the book [B2] for the most comprehensive (by far) treatment. The book [Bk5, Chap. 1] is complementary in the extent of its treatment of the contrasting features of the two theories.
