Table Of Contents

9. Seminal Contributions to Basic Theory and Methodology of Cyclostationarity

On this page, the core seminal contributions to the theory of cyclostationarity are identified and concisely described. To avoid unnecessary redundancy and also to enhance readability for many visitors to this website, theorem statements in all their sometimes-gory detail are not included. Rather, statements in English of the essence of theorems are provided, and original sources and/or locations in the encyclopedic book [B2] are cited. There are two salient contributors who are the sources of the great majority of these contributions, and the contributions of each occur almost entirely in two distinct periods of time. The first period is the mid-1970s to the mid-1990s, which is when most of William Gardner’s contributions were made, and the second period is the mid-1990s to the present, which is when most of Antonio Napolitano’s contributions were made. The smaller number of contributions from others occurred primarily in the later of these two periods. Most recently, there has been renewed attention from both these contributors to FOT-Probability models for cyclostationary signals as a superior alternative for many applications to the more abstract stochastic Stochastic | adjective Involving random variables, as in stochastic process which is a time-indexed sequence of random variables. process models. Two tightly coupled tutorial articles from 2022 (also containing novel contributions) are made available here on pages 3.2 and 3.3.

The most comprehensive treatise on the subject of cyclostationarity written to date is the recently published 2020 book entitled Cyclostationary Processes and Time Series: Theory, Applications, and Generalizations, written by the most prolific contributor to this subject in this 21st Century, Professor Antonio Napolitano [B2]. Because this encyclopedic book is the most current and authoritative and scholarly treatment of this subject, it has been used here as the primary source for identifying many of the core seminal contributions made by the several key contributors included below in Sections 9.1, 9.2, and 9.3. This book is comprised of 720 pages, comprehensively containing citations of about 1400 distinct research publications on cyclostationarity (a substantial portion of which are theoretical contributions), including 582 citations of Gardner’s publications (some of which are multiple citations of single works, like books).

The use on this page of the term seminal contribution is based on the standard definition:

Def. Seminal: adjective

(Of a work, event, moment, or figure) strongly influencing later developments;
Similar terms: influential, formative, groundbreaking, pioneering, original

The meaning of seminal favors Gardner’s contributions over those of others over the last 40 years because he “got there first” and because refinements and deeper mathematization and translation from Gardner’s FOT-probability versions to stochastic-process versions of already-formulated concepts do not qualify as seminal, though they certainly can be important contributions to the theory.

  • 9.1 Seminal Contributions of William A. Gardner
    9.1.1 Non-Stochastic Cyclostationarity Theorems

    The conceptual and theoretical foundation and framework of William Gardner’s Fraction-of-Time (FOT) Probabilistic Theory of Cyclostationarity for Time Series is succinctly captured by the set of 19 numbered theorems listed below, following Gardner’s 31 basic Cyclostationarity Definitions. All definitions and theorems were originated by Gardner. Users are also referred to the Glossary of Notations and Terminology at [Bk2, pp. xxv-xxvi]. The 19 theorems are partitioned into three categories, Probabilistic and Statistical Statistical | adjective Of or having to do with Statistics, which are summary descriptions computed from finite sets of empirical data; not necessarily related to probability. Functions of Time and Frequency, Transformations of Probabilistic and Statistical Functions by Signal processing Operations, and Ad Hoc and Optimum Statistical Inference.

    (Personal note: Although some of these theorems collected on this page have begun to be referred to as “Gardner’s . . . Theorem” or “Gardner’s . . . Relation” (e.g., see [B2]), there is a saying among scholars that was recently brought to my attention by Professor Thomas Kailath: “Whatever the name attached to a theorem, someone else was probably the actual originator.” To avoid this pitfall, I’ve not included my name in any of the theorems I believe—based on my scholarly searches of the literature—I originated. But this is not to say that I doubt Professor Napolitano’s scholarship leading to his attachment of my name to some of the theorems included here.)

    For the sake of parsimony, the terms originally defined by Gardner are only listed. Instead of providing the actual definitions, citations to original sources are given. This also provides context for the origin of these terms. For the same reasons, the list of theorems provided includes only descriptive theorem labels, but this list is followed by notes that include some expansion and citations to original sources. These notes are followed by commentary on these contributions from experts in the field made around the time the contributions were made.

    Each of the theorems listed have various alternative versions for which the differences lie in the details of assumptions made and, correspondingly, the strength of results obtained. This is typical for many if not most theorems in mathematics. Nevertheless, for pragmatic users of the results provided by theorems, the practical essence of each theorem is often unique. For example, the exact nature of the convergence of an infinite series depends on the particular properties of the series. The formulator of a theorem may have flexibility in the choice of assumptions made about properties. Generally, the more restrictive the assumptions made, the stronger the results that follow. In many applications, what’s important to users is the fact that specific types of series do converge in some sense and others diverge in every useful sense.

    In the earliest statements of many of the theorems included here, detailed assumptions are not made. Rather, it is taken for granted that all mathematical quantities referred to in a theorem statement exist under some reasonable assumptions. For example, it is assumed that for whatever statistical moments or cumulative distributions or the like that are referred to in a theorem, there exist practical signal models for which these statistical functions do indeed exist. In fact, many examples of formulaic models are given and the statistical functions are calculated from these models. The essence of the theorems lies in the relations among these functions. These relations are generally expressed in the form of equations. The particular sense in which equality holds in these equations typically depends on detailed assumptions. An equation among time functions may hold for all time, or for almost all time (meaning all time except for a set of times of Lebesgue measure zero), or in temporal mean square, or with fraction-of-time-probability equal to one, etc. More often than not, when empirical Empirical Data | noun Numerical quantities derived from observation or measurement in contrast to those derived from theory or logic alone. quantities are being dealt with, these distinctions become irrelevant. Similar remarks apply to functions of multiple time variables, or frequency variables, or both types of variables, and in some cases, spatial variables.

    In summary, existence of statistical functions is not the focus in these theorems; the focus is the relations among the functions (when they exist)—this is what matters in these theorems. Nevertheless, readers are forewarned that, when the objective is to further develop the underlying theory of cyclostationarity, explicit statements of the details of assumptions under which the results of theorems are valid may become important. This is found in the encyclopedic treatment [B2] in contrast to the original sources, such as [Bk2].

    Narrative Description of Gardner’s Seminal Work

    A unique aspect of Gardner’s contributions to knowledge is both their unity and their unifying impact on previous knowledge. While many academic contributors to the field of statistical signal processing often develop a variety of bits and pieces of the overall body of knowledge, by using available tools for addressing specific problems such as algorithm development, analysis, and testing for specific applications, Gardner—as a fresh PhD graduate in 1972—set his sights on proving his early conviction that cyclostationary signal models in general were of high importance to the field of statistical signal processing throughout many application areas and required a comprehensive foundational theory of their own, analogous to that developed in earlier decades for stationary signals. As a young faculty member, he built a program at UC Davis that pioneered this development. Despite an initial lackluster response from publication and grant application reviewers during his first 15 years in academia, Gardner persisted with his mission which initially culminated in his now classic 1987 book [Bk2]. This advanced-level text/reference book presented his comprehensive wide-sense theory of cyclostationarity, associated methodology, and many resultant novel signal-processing algorithms for a variety of applications. 

    During the first half of this period, Gardner hypothesized that the disinterested response from reviewers was at least in part due to the masking of the essence of cyclostationarity by the substantial abstraction of the nonstationary stochastic process model, which was the tool of the day for studying randomly fluctuating unpredictable data, such as noise and information bearing signals in communications systems. This field was Gardner’s primary source of motivating applications, having earlier been a member of the technical staff at Bell Telephone Laboratories. Gardner discovered that the essence of cyclostationarity is captured by two characteristic properties, neither of which were stochastic in nature. The first is the property that finite-strength additive sine wave components (with associated spectral lines) can be generated from cyclostationary data by subjecting the data to a nonlinear transformation like a squaring operation or a lag product. He dubbed the frequencies of such generated spectral lines “cycle frequencies”. The second is the property that the time fluctuations of the data that occur in one sub-band of the spectrum, after down-conversion to what is called “baseband”, are temporally correlated with the baseband time fluctuations from some other down-converted sub-band, provided that these sub-bands are separated by a cycle frequency. 

    Gardner’s wide-sense theory of cyclostationarity that captures these defining properties of cyclostationarity addresses quadratic nonlinearities and pairs of temporally correlated sub-bands. His higher (n-th) order theory addresses higher-order homogeneous polynomial nonlinearities (such as cubic and quartic lag products) and mutual temporal correlation among all members of sets of n sub-bands. And his strict-sense theory unifies these nested (the characteristics of order n are a special case of those of order n+1) theories under the umbrella of what he called Fraction-of-Time (FOT) Probability theory, which introduces periodically time-variant FOT Probabilities. These probabilities, however, are NOT those upon which stochastic process theory is based. Rather these are an invention of Gardner’s, for the purpose of creating a sound mathematical basis—a comprehensive theory—for the empirical subject of time series analysis of data exhibiting cyclostationarity, that is, a mathematical theory that describes all the properties of data associated with sine-wave generation and spectral correlation of all orders. Gardner specialized and generalized this theory to data exhibiting stationarity and poly-cyclostationarity and almost-cyclostationarity.

    Gardner also proved and demonstrated in his 1985 book [BK1], [Bk3] that his comprehensive theory has an almost obvious dual theory based on stochastics processes but, in his 1987 book (see especially Pages 3.1-3.3 herein for completion of his argument), he argued that this substantial addition of abstraction offers nothing of practical value unless the user wishes to study non-ergodic or non-cycloergodic models, for which the FOT-Probabilities change from one sample path of the process to another. And such models also mask the defining properties of cyclostationarity that are relevant to empirical signal processing and that provide the primary motivation for using cyclostationarity models to solve problems. The formulation of the stochastic process theory has nothing to do with sine-wave generation or temporal correlation among the contents of sub-bands, though it does involve ensemble correlation at distinct sets of frequencies by invoking the probabilistic law of large numbers to approximately represent ensemble correlations by expected values. Nevertheless, ensembles of signals are rarely available in real-world scenarios, though they are created on computers for Monte Carlo simulations (yet these ensemble members are actually time-segments of a single signal, meaning they produce time averages).

    During the next stage, following the mid-1980s, lasting about 10 years, Gardner focused on developing both the higher-order and strict-sense theories of cyclostationarity, again producing a comprehensive foundational theory with associated methodology and many more resultant novel algorithms, especially for signal classification applications. During this latter 10-year period, the statistical signal processing community rapidly came on board with work on diverse applications and further algorithms development. Applications include cellular communications, cognitive radio, blind channel equalization, direction finding, spatial filtering, signal interception, joint transmitter/receiver optimization for communications, and many others. 

    In addition, some theoreticians who felt the dual theory based on stochastic processes merited more attention than Gardner had given it, or simply preferred keeping with the tool they had been indoctrinated in, published work on signal processing based on this alternative model, which Gardner had introduced in his 1985 book, in many cases giving short shrift to the earlier and often equivalent work based on the dual FOT-Probability model. 

    The profound value of this new branch of statistical signal processing was seen first in cellular and other multiuser communications systems development and, separately and quietly (with regard to the open literature) in government national defense programs on signals intelligence. The significant growth in both these fields is a direct result of Gardner’s insightful work, which was both pioneering and comprehensive. Even 20 years following this latter 10-year period, Gardner is still making fundamental contributions to further extend and generalize this discipline, including (1) extending applicability from primarily engineering to science (2018) where cyclostationarity is more often irregular, and (2) adding deeper mathematical underpinning to the important FOT-Probability theory that he originated for cyclostationarity in its various forms and further advancing our understanding of limitations of stochastic process models and their ergodicity properties (2022)—see Pages 3.1-3.3 cited above.

    Another area of engineering that has seen significant growth as a result of this new branch of statistical signal processing is that of machine diagnostics based on mechanical vibrations. Gardner’s early work played a key role in initiating the substantial body of work on cyclostationarity in mechanical vibrations, which started around the year 2000. The following brief summary uses a particularly popular review article by Jerome Antoni to illustrate these origins: “Cyclostationarity by Examples”, Mechanical Systems and Signal Processing 23 (2009) 987–1036, Invited Review https://www.sciencedirect.com/science/article/pii/S0888327008002690. This article’s acknowledgement of Gardner’s seminal contributions is illustrated below with a selection of quotations, indexed by page number. Each referenced publication by Gardner is identified with a link to that publication in this website.

    p. 1016. The design of estimators in the cyclostationary context is a vast subject beyond the scope of the present paper. Interested readers are invited to consult Refs. [7,16,26]. [JP16]

    p.1015. One has no other choice than relying on the assumption of cycloergodicity which stipulates that the expected value E{ . } returns the same result as . . . the P-operator. This is thoroughly discussed in Ref. [8]. [Bk2]

    Proposition 9 (Cyclic Wiener-Khinchin relation). The spectral correlation density is the Fourier transform of the cyclic autocorrelation function…Elaborating on this very simple relationship, direct connexions with the Wigner-Ville spectrum, the PSD and the classical autocorrelation function are also possible…More details can be found in Refs. [10,26]. [Bk1]

    p. 1028. 4.3.3. Blind identification. Under some conditions on the excitations, the cyclostationarity property can be exploited to identify a system from the measurement of its responses only, as first observed in the seminal work [12] [JP39]

    Page 990: … works in cyclostationarity date back to the early sixties, but it is truly since the eighties that cyclostationarity has become a subject of active research. Most of the precursory works pertain to the field of communications, where it was recognised that the process of modulating a signal for Hertzian transmission naturally led to a cyclostationary behaviour. A great tribute is due to William A. Gardner who first established many of the theoretical foundations, laid down the currently used terminology, and also foresaw many applications. As a matter of fact, W. A. Gardner was probably the first to recognise that the cyclostationary framework is appropriate for any physical phenomenon that gives rise to data with periodic statistical characteristics: ‘‘in mechanical-vibration monitoring and diagnosis for machinery, periodicity arises from rotation, revolution, and reciprocating of gears, belts, chains, shafts, propellers, bearings, pistons, and so on’’ [10]. [Bk1]

    p. 1004. …pure nth-order cyclostationarity refers to any hidden periodicity that can be evidenced by using a non-linear transformation of order n after all cyclostationary components of lower orders (< n) have been removed. The study of pure higher-order cyclostationarity is outside the scope of this paper and the interested reader is invited to consult Refs. [18,19]. [JP55]

    It is rare to see a single individual dominate the development of a field of study from its inception through its various stages of maturation over half a century. But this is indeed what Gardner has done. To put a capstone on this body of work, Gardner has recently independently developed this comprehensive educational website on cyclostationarity that is intended to address the challenge of rapidly bringing young scholars up to speed on this body of advanced knowledge, so that they can efficiently become productive scholars in their own right, and also to support applied engineers and scientists with presentations emphasizing the critical statistical concepts in contrast to abstract treatments based on stochastic process models.

    Authorities in the fields of signals intelligence, statistical signal processing for communications and geological exploration (Blachman, Levy, Massey, Rice, Robinson, cited below on this Page 9.1) describe his contributions as “courageous”, “radically new”, “revolutionary”, and “profound”, and characterize him as a “pioneer” and a “preeminent authority”.

    Historical Perspective on Cyclostationarity

    A very broad perspective of where Gardner’s contributions fit into the long history of modeling cycles in time-series data is provided on Page 4.1, where it is observed that modeling of cycles in time series progressed through 3 stages: (1) “Hidden Periodicities (initiated by Euler and Lagrange in the 1700s), which are modeled as periodic signals (or single sine waves) in broadband additive noise (initially (a) non-stochastic time-series noise and later—1930s/40s—(b) stationary stochastic noise), (2) Disturbed Harmonics (initiated by Yule in 1927), which are modeled as narrowband autoregressive models driven by broadband noise (initially (a) non-stochastic time series and later (b) stationary stochastic processes), and (3) Cyclostationarity (initiated by Gardner in 1985-1987), which is modeled as either (a) non-stochastic time-series with Fraction-of-Time probability distributions and/or non-stochastic moments that vary in time periodically or almost-periodically (multiple incommensurate periods), or (b) stochastic processes with (almost) periodically time-varying moments and/or probability distributions (over ensembles of sample paths). This third phase of development of models and analytical tools represents a huge step in sophistication, which revolutionized algorithm development for statistical signal processing of time-series data exhibiting cyclicity, including both ad hoc techniques and optimum statistical inference and decision making. Between the 1950s and the introduction of Gardner’s theory in the mid-1980s, the focus was on stationary stochastic processes of types (1) and (2). This has since given way in Gardner’s theory of cyclicity to the dual models referred to as cyclostationary non-stochastic time series and cyclostationary stochastic processes. Gardner uses the term cyclostationary to mean purely cyclostationary, poly-cyclostationary (finite number of incommensurate periods), almost cyclostationary (countably infinite number of incommensurate periods), and irregular cyclostationarity (time-warped cyclostationarity). Prior to Gardner’s comprehensive theory, there were a few isolated narrow-scope papers during the two immediately preceding decades, which considered only stochastic processes. That work did not lead to Gardner’s work, which focused primarily on the more innovative non-stochastic models (and thereby generalized Norbert Wiener’s theory of generalized harmonic analysis), the results of which easily translate in most cases to the dual theory based on the more abstract and less empirical stochastic process models.

    Terminology

      1. Cycle Aliasing [Bk2, p. 403, 528]
      2. Cycle Detector [Bk1, p.352], [Bk2, pp. 497-503], [JP27]
      3. Cycle Frequency [Bk1, p. 303], [Bk2, p. 385]
      4. Cycle Leakage [Bk2, p. 528]
      5. Cycle Resolution [Bk2, p. 388]
      6. Cycle Spectrum [Bk1, p. 304], [Bk2, p. 392]
      7. Cyclic Autocorrelation [Bk1, p. 303], [Bk2, p. 3]
      8. Cyclic Expectation [Bk2, pp. 517-519]
      9. Cyclic Correlogram [Bk1, p. 309], [Bk2, p. 386]
      10. Cyclic Cumulative FOT Distributions and Densities [Bk2, pp. 511-515]
      11. Cyclic FOT Probability [Bk2, pp. 511-515]
      12. Cyclic Periodogram [Bk1, p. 309], [Bk2, p. 385]
      13. Cyclic Polyspectrum [JP55]
      14. Cyclic Spectral Density [Bk1, p.304], [Bk2, p. 559]
      15. Cyclic Spectrum [Bk1, p. 304], [Bk2, p. 365]
      16. Cyclic Temporal Cumulants [JP55]
      17. Cyclic Temporal Moments [JP55]
      18. Cyclic Wiener Filter [JP48], [Bk2, p. 482]
      19. Cyclic Wiener Relation [JP36, p. 22], [Bk2, p. 390]
      20. Cycloergodicity [Bk1, pp. 435-349], [JP11]
      21. Poly-periodic Component Extraction Operator [JP34, pp. 282-284]
      22. Poly-periodic Cumulative Probability Distribution [Bk1, p.348], [Bk2, pp. 512], [JP34, p. 283]
      23. Pure n-th Order Cycle Frequency [JP55]
      24. Purely Cyclostationary [Bk2, p. 392]
      25. Sine-Wave Component Extraction Operator [Bk2, pp. 517-519], [JP34, pp. 280-284]
      26. Spectral Autocoherence [JP15, p. 20], [Bk2, p. 366]
      27. Spectral Correlation Density Function [Bk1, p. 304]
      28. Spectral Cumulants [JP55]
      29. Spectral Line Generation [Bk2, p. 359-369]
      30. Spectral Moments [JP55]
      31. Synchronized Average [Bk1, p. 311], [JP15, p. 17]

     

    Theorems from Gardner’s Theory

    CATEGORY 1: Probabilistic and Statistical Functions of Time and Frequency

    Theorem 1:  Fundamental Theorem of Sine-Wave Component Extraction – Original definition of the linear operator that extracts finite-strength additive sinewave components from any well behaved function of a persistent times series and proof that this operator is a nonstochastic expectation operator with respect to the generally almost period temporal (non-stochastic) cumulative probability distribution function, which also was originally defined by Gardner and proven to satisfy the defining properties of probability distribution functions in the case for which the set of sinewave frequencies comprises all harmonics of any set of incommensurate fundamental frequencies included. For one fundamental frequency, the distribution function provides a cyclostationary model for time series; for a finite set of multiple fundamental frequencies, it produces a poly-cyclostationary model, and for a countable infinity of fundamental frequencies, it produces an almost cyclostationary model.

    Theorem 2: Non-Stochastic Moment Expansion Theorem for Sine Waves Extraction – Original theorem statement and proof that the set of sine waves to be extracted from any function of a time series that admits a generalized Volterra series representation can be expressed as a linear combination of the sine waves contained in each of all the moments of the time series or, equivalently, all the cumulants.

    Theorem 3: Generalizations of the Fundamental Theorem of Sine Wave Component Extraction to extraction of subsets of sine waves and to Estimation on Finite-Time intervals – Completely analogous to Theorem 1, which applies to infinite intervals and complete sets of harmonically related sine waves. Because sine waves with arbitrary frequencies are not generally orthogonal to each other on finite intervals, this theorem deals with estimation instead of extraction.

    Theorem 4:  Sine-Wave-Extraction Derivation of the Non-Stochastic Temporal Cumulant Function – Original definition and derivation outside the framework of probability.

    Theorem 5:  Relation between non-stochastic Higher-Order Cyclic Moments and Cyclic Cumulants – Original derivation of the non-stochastic counterpart of the Leonov/Shiryaev Relation between stochastic moments and cumulants and its decomposition into the entirely new relation among cyclic moments and cyclic cumulants for a non-stochastic time series exhibiting cyclostationarity. 

    Theorem 6:  Relation between Non-Stochastic Spectral Correlation and Cyclic Temporal Autocorrelation  – Generalization of the Wiener Relation between non-stochastic average power spectral density function and temporal autocorrelation function: Original derivation.

    Theorem 7:  Relation between Non-Stochastic Temporal and Spectral Higher-Order Moments: Original derivation.

    Theorem 8:  Cyclic-Periodogram/Correlogram Relation and its Higher-Order Counterpart – Original definitions and derivations.

    Theorem 9:  Synchronized Averaging Identity for Non-Stochastic FOT-Probabilistic Functions – Original derivation.

    CATEGORY 2: Transformation of Probabilistic and Statistical Functions by Signal Processing Operations 

    Theorem 10:  Non-Stochastic Spectral-Correlation Input/Output Relations for Key Signal Processing Operations: 10.1, sampling & aliasing; 10.2, multiplication; 10.3, convolution & band limitation.

    Theorem 11:  Non-Stochastic Higher-Order Spectral-Moment and Spectral Cumulant Input/Output Relations for Key Signal Processing Operations: 11.1; 11.2; 11.3.

    Theorem 12:   Derivation of Signal Selectivity of Non-stochastic Cyclic Cumulants

    CATEGORY 3: Optimum Statistical Inference

    Theorem 13:  Non-Stochastic Spectral Correlation Theory of Optimum Almost-periodically Time-Variant Filtering of Almost-Cyclostationary Signals: Generalization of the theory of non-causal Wiener filtering from stationary to almost cyclostationary time series. Includes the special cases of optimum poly-periodic and optimum periodic filtering of poly-cyclostationary and cyclostationary signals, respectively. 

    Theorem 14: Non-Stochastic Spectral Correlation Theory of Optimum Detection of Cyclostationary Signals: Maximum-SNR and Maximum-Likelihood spectral-line regenerators. 

    Theorem 15: Non-Stochastic Spectral Correlation Theory of N-th Order Nonlinear Synchronizers

    Theorem 16: Non-Stochastic Spectral Moment and Cumulant Theory of Cyclostationary Signal Classification

    Theorem 17:  Non-Stochastic Theory of Almost-Periodically Time-Variant Linear System Identification

    Theorem 18:  Non-Stochastic Theory of Time-Invariant Volterra Nonlinear System Identification

    Theorem 19: Non-Stochastic Theory of Periodically and Poly-Periodically Time-Variant Nonlinear Volterra System Identification

     

    Notes and References for Theorems:

    • Re: Theorems 1 – 19 — Origin of the Fraction-of-Time (FOT) Theory of ACS Time-Series, which is dual to the Theory of ACS Stochastic Processes which reflects the Gardner Isomorphism which is the ACS counterpart of the Wold Isomorphism for stationary stochastic processes. See p. 62 in [B2] and Page 3.2 herein for CS; for ACS, see [JP34] and pp. 519-520 in [Bk2] and Page 3.2 herein.
    • Re: Theorem 1 — Origin of the Theorem of Temporal Expectation for ACS time-series based on the Sine-Wave-Component-Extraction Operator. This is the FOT dual to the standard Fundamental Theorem of Expectation applied to ACS stochastic processes. See pp. 43-50, 137-138 in [B2] and pp. 517-519 in [Bk2]. The proof of the theorem given on pp. 11 -13 in [Bk5] is as elegant as possible, consisting of two simple steps, and this novel method of proof also extends to the classical Fundamental Theorem of Expectation for stochastic processes.
    • Re: Theorem 2 — Origin of the Moment Expansion Theorem. This is formulated and proven on pp. 11 – 13 in [Bk5], where the most elegant proof possible, consisting of two simple steps, is given.
    • Re: Theorem 3 — Origin of the fact that the theory of cyclostationarity (including poly-cyclostationarity and almost-cyclostationarity), which is based on infinite limits of time average operations—non-empirical quantities—has a counterpart based on empirical finite-time averages. (See Page 3.5 herein.)
    • Re: Theorem 4 — Origin of the cumulant solution, in a non-probabilistic setting, to a problem concerning sine-wave generation by nonlinear transformation of persistent time series, see p. 146-149, 510 in [B2]. The solution provided on p. 3396 in [JP55] was proposed by Gardner and verified by C.M. Spooner. This solution was then characterized by Gardner and Spooner in terms of the cyclic cumulative distribution function and cyclic moments on pp. 3397-3399 in [JP55].
    • Re: Theorem 5 — Origin of the Cyclic Moment/Cyclic Cumulant Relation for times-series exhibiting cyclostationarity: This is a non-stochastic counterpart of the Leonov/Shiryaev Relation between moments and cumulants of a nonstationary stochastic process, pp. 147-148 in [B2]. This counterpart decomposes the relation for a stochastic processes into a set of relations among the cyclic components of the moments and cumulants for the cases of processes and time series exhibiting cyclostationarity [JP55].
    • Re: Theorem 6 — Origin of the Relation between the Cyclic Autocorrelation Function and the Spectral Correlation Function (originally dubbed Cyclic Wiener Relation by Gardner because it is an extension and generalization of the Wiener Relation, a term Gardner introduced to distinguish this relation from the Wiener-Khinchin Relation for stochastic processes). See page 390 in [Bk2], and pp. 10, 20, 56, 57, 139 in [B2]. 
    • Re: Theorem 7 — Origin of the Relation between Higher-Order Temporal and Spectral Moments of time-series, pp. 138,139 in [B2].This is the non-stochastic counterpart of what is called the Shiryaev-Kolmogorov Relation which is the higher-order generalization of the 2nd order Wiener-Khinchin Relation, see [JP55]. This relation is derived from the empirically-motivated definition of the spectral moment as the limit of the joint moment of finite-time Fourier transforms of a time series, rather than—as in [B2]–obtained by defining the spectral moment to be the Fourier transform of the cyclic temporal moment function. 
    • Re: Theorem 8— Origin of the Cyclic Periodogram/Correlogram Relation for time-series (see p. 57 in [B2] and pp. 385-386 in [Bk2]) and it’s nth-order counterpart (see p. 3419 in [JP56]), which is the finite-time counterpart of Theorem 7. 
    • Re: Theorem 9 — Origin of the Synchronized Averaging Identity for functions containing an additive almost period component. This identity decomposes that component into a sum of periodic components each derived directly from the time series and decomposes each periodic component into a sum of sinusoidal components each derived directly from the time series, pp. 485-486 in [B2]. See also pp. 362-365 and 511-515 in [Bk2] and pp. 332-334 in [Bk3].
    • Re: Theorem 10, 11 — Origin of the Spectral Correlation Characteristics of basic signal processing operations (time-sampling & aliasing, multiplication, convolution, and band limitation), pp. 82-109 in [B2] and [JP15]; and Generalization to Higher-Order Moments/Cumulants, [JP56], pp. 133- 149 in [B2], and Chap. 2 in [Bk5].
    • Re: Theorem 13 — Invention of Cyclic Wiener Filtering Theory, also called FRESH Filtering, and proof that Fractionally-Spaced-Equalizers are Cyclic Filters with Subsampled Outputs, pp. 330-333 in [B2]. See also pp.482-485 in [Bk2] and [JP48]. This seminal work gave rise to significant progress in multi-user receiver filter optimization and especially joint receiver/transmitter filter optimization, Article 1 in [Bk5]. It also provides the theoretical basis for separation of spectrally overlapping signals exhibiting cyclostationarity by exploitation of spectral redundancy.
    • Re: Theorem 14 — Invention of the Single-Cycle Detector and proof that it is a Maximum-Signal-to-Noise-Ratio sine-wave generator, and original derivation of the decomposition of the maximum-likelihood detector for weak ACS signals into a coherent sum of Max-SNR Cycle Detectors, pp. 286-290 in [B2]. See also pp, 497-502 in [Bk2] and [JP27], [JP41], and [JP49]. 
    • Re: Theorem 15 — Origin of the central role that Nth-order spectral correlation plays in the operation of Nth-Order Nonlinear Synchronizers for ACS signals, pp. 333-335 in [B2]. See also Article 2 in [Bk5] and [JP17].
    • Re: Theorems 12, 16 — Original discovery of the signal selectivity property of cyclic temporal and spectral cumulants, and demonstration of utility for classification of spectrally overlapping signals and spectrum sensing for cognitive radio, pp. 322-324, 328-329 in [B2]. See also pp. 371-375 in [Bk3], pp. 8-9, 65-66, 115 in [Bk5], and pp.3399-3400 in [JP55]. This separability property of cyclic correlations has also spawned an important new class of Super-Resolution Direction Finding Algorithms, the first of which were Cyclic MUSIC and Cyclic ESPRIT. Surveys provided in [BkC1] and Chap. 3 in [Bk5].
    • Re: Theorem 17 — Original discovery that Blind Phase-Sensitive Channel Identification/Equalization is made possible with only 2nd-order statistics by exploiting the cyclostationarity of channel-input signals, p.343 in [B2]. This discovery, first reported in [JP39], did not include a particularly attractive algorithm to demonstrate this new capability, but it immediately gave rise to a flurry of contributions by other researchers to blind-adaptive channel equalization using only 2nd-order cyclic statistics. See Articles 4 and 5 in [Bk5]; surveys provided in [BkC2] and Chap. 3 in [Bk5].
    • Re: Theorems 11, 17-19 — Original discovery of Input/Output-Corruption-Tolerant Linear System Identification Methods, which are made possible by exploiting cyclostationarity of an input-signal component, pp. 335-337 in [B2] (see also [JP31]); and original extension and generalization of Volterra Nonlinear System Identification methods by exploiting cyclostationarity of the excitation, p. 344 in [B2]. Benefits for time-invariant nonlinear system identification are demonstrated mathematically in the originating paper where the methods are derived [JP50] and what are apparently the first-ever methods proposed for periodic and poly-periodic nonlinear system identification are demonstrated mathematically in the paper where they are derived [JP59].

     

    Applications of Theorems

    • The above theorems gave rise to the discovery, for CS and ACS time series, of the fundamental noise and interference tolerance properties in statistical inference and the Signal-Separability, Spectral Correlation Separability, and more generally Cyclic Temporal and Spectral Cumulant Separability properties, and demonstration of applicability to design and analysis of signal processing methods and algorithms for communications, telemetry, and radar systems. This body of work has demonstrated that substantial improvements in system performance can be obtained in various signal processing applications involving multiuser communications and interference-limited environments, such as detection, estimation, and classification of signals, by exploiting cyclostationarity—that is, by recognizing and modeling signals as CS and ACS instead of using the classical stationary-process models. Google Scholar identifies more than 50 of Gardner’s published research papers in peer-reviewed journals, and books, in which these achievements are reported, and identifies tens of thousands of research papers that cite this work.
    • These theories and methods have provided the basis for the establishment of the core of a major part of RF signals intelligence algorithm development throughout government laboratories & agencies and industrial government-contractors in the US and cooperating nations since the mid-1980s. Most of this work is not published in the open literature. (See SSPI Reports on Page 6 and Page 12 herein, and also see the Quotations from Nelson Blachman and Bart Rice below.)
    • These theories and methods have provided the basis for new interference-tolerant signal processing techniques of signal-presence detection, parameter estimation, system identification, modulation classification, signal-source location and signal extraction; for example, these theories and methods have been adopted as the basis for spectrum sensing in crowded RF environments upon which the entire operation of cognitive radios relies. They have been used to develop a variety of substantive new families of algorithms for signal direction-of-arrival estimation, blind adaptive channel identification and equalization, blind adaptive spatial filtering, single-sensor cochannel signal separation, multi-user joint receiver/transmitter filter optimization, and nonlinear system identification. (See Page 2.5 herein and the reference lists in [JP64] and in the various chapters and articles in [Bk5] and [B2].)
    • These theories and methods have also provided the basis for improved time-series analysis in a variety of other fields of science and engineering, such as Econometrics, Biology, Climatology, Acoustics and Mechanical Vibrations, and Electrical Circuits, Systems, and Control. See Page 6 herein, and the pages in [B2] that are cited on Page 6. In fact, major advances in rotating machinery monitoring and early diagnosis of machine faults, such as gear and bearing degradation have been based on cyclostationarity exploitation, pp. 360-362 in [B2].
    • Development of the ad hoc concept of time de-warping into the basic theory of converting irregular cyclostationarity into regular cyclostationarity has served as a means for rendering the extensive and powerful tools of cyclostationary signal processing technology applicable to data exhibiting irregular cyclicity—when the rate of change of cycle frequencies is not too fast—which pervades essentially all fields of science. (See Page 4.2 herein.)

     

    Early Quotations from Commentary on Gardner’s Seminal Contributions (late 1980s to late 1990s)

    Professor Enders A. RobinsonColumbia University and Member of the National Academy of Engineering, states in a letter of reference on behalf of Dr. Gardner:

    From time to time it is good to look back and see in perspective the work of those people who have made a difference in the engineering profession.  One of the important members of this group is William A. Gardner.

    Professor Gardner has the ability to impart a fresh approach to many difficult problems. William is one of those few people who can effectively do both the analytic and the practical work required for the introduction and acceptance of a new engineering method. His general approach is to go back to the basic foundations and lay a new framework. This gives him a way to circumvent many of the stumbling blocks confronted by other workers . . .

    I am particularly impressed by the fundamental work in spectral analysis done by Professor Gardner. Whereas most theoretical developments make use of ensemble averages, he has gone back and reformulated the whole problem in terms of time-averages. In so doing he has discovered many avenues of approach which were either not known or neglected in the past. In this way his work more resembles some of the outstanding mathematicians and engineers of the past. This approach took some courage, because generally people tend to assume that the basic work has been done, and that no new results can come from re-examining avenues that had been tried in the past and then dropped. William’s success in the approach shows the strength of his engineering insight. He has been able to solve problems that others have left as being too difficult. It is this quality that he so well imparts to his students, who have gone forth and solved important and far-reaching problems in their own right.

    Professor Bernard C. Levy, Chairman of the Department of Electrical & Computer Engineering at the University of California, Davis, states in a nomination letter:

    Dr. Gardner’s random Random | adjectiveUnpredictable, but not necessarily modeled in terms of probability and not necessarily stochastic. processes textbook has several original features which make it stand out among all other textbooks in the same general area. First, it contains a chapter on cyclostationary processes, which have been one of the main topics of research for Dr. Gardner throughout his research career. These processes play a key role in the study of digital communications systems, and virtually all recent digital communications textbooks refer to Dr. Gardner’s random processes book as well as to his research papers on cyclostationary signal processing.  Another original feature of Dr. Gardner’s random processes book is its detailed development of the time-average approach for evaluating the statistics of random signals. This approach provides the theoretical underpinning for the textbook Statistical Spectral Analysis: A Nonprobabilistic Theory which was written by Dr. Gardner for his Spectral Analysis course (ECE 262). Because of its revolutionary time-average approach (which can be traced back in part to the pioneering work of Norbert Wiener on generalized harmonic analysis), this textbook has been the subject of entertaining exchanges in the Signal Processing Magazine of the IEEE Signal Processing Society. As a consequence of Bill Gardner’s courage and vision in pursuing a radically new path, based on the eminently sensible view that the analysis of random signals should be based on statistics extracted from the observed data, this book has had a huge impact on modern spectrum analysis practitioners.

    Professor Lewis E. Franksprevious NSF program director and previous chairman of the Department of Electrical & Computer Engineering at the University of Massachusetts, Amherst, states:

    I believe I have read a major portion of Gardner’s papers and textbooks.  I feel that a unique feature of all these publications, compared to other engineering documents of a similar nature, is the presence of a strong scholarly style.  Previous contributions to the topic are meticulously sought out and referenced.  It’s not just a matter of being polite to colleagues or avoiding confrontations over omitted citations; but a genuine attempt to establish an important historical context for new results or interpretations.  The relevance of prior contributions to the topic is carefully laid out and unified… On the topic of cyclostationary processes, I feel that he has, almost single-handedly, developed the theoretical and applied engineering aspects of the topic to the point of today’s widespread recognition of its utility.

    Dr. Nelson Blachman, Office of the Chief Engineer, Electronic Defense Laboratories, GTE Government Systems Corporation; well known communication systems author (see article), writes:

    My interest in Dr. Gardner’s research is concerned with the advances in cyclostationary signal processing that has been his greatest contribution to electrical engineering research. In fact, Professor Gardner is “Mr. Cyclostationary”, the promoter and leading international researcher in this important signal processing area, with two textbooks, numerous papers, and a federal government subsidized workshop to his credit. As a scientist involved with Department of Defense signal processing research aimed at threat analysis of signals related to national security interests, I can indicate to you that Dr. Gardner’s work has had profound impact on the analysis of these signals, but classification of the analysis has kept the importance of his work from being known to the general public and others in academia. There is another attribute of Dr. Gardner’s research and tutorial material that makes him stand out among so many of my other academic colleagues, and that is his depth of research (especially historical and mathematical detail) and his attention to precision and detail in his writings. I have always found it difficult to find errors and to take issue with any of Professor Gardner’s papers because he has meticulously done his research; this is in contrast to so many other academics who tend to be more sloppy in their mathematical precision and who do not always thoroughly check the technical literature in depth. I believe this high degree of professional research has contributed greatly to the widespread acceptance of Dr. Gardner’s technical writings as being the preeminent authority on cyclostationary signal processes and their exploitation.

    Dr. Akiva Yaglom, Mathematician and Physicist, USSR Academy of Sciences, wrote in a book review published in Theory of Probability and Its Applications:

    It is important . . . that until Gardner’s . . . book was published there was no attempt to present the modern spectral analysis of random processes consistently in language that uses only time-averaging rather than averaging over the statistical ensemble of realizations [of a stochastic process] . . . Professor Gardner’s book is a valuable addition to the literature”

    Professor James Massey, information theorist and cryptographer, Professor of Digital Technology at ETH Zurich, member of the National Academy of Engineering, wrote in a prepublication book review in 1986:

    I admire the scholarship of this book and its radical departure from the stochastic process bandwagon of the past 40 years.

    Dr. Bart F. Rice, of Lockheed Research, past chairman of the Santa Clara chapter of the IEEE Signal Processing Society, in 1992 letters of reference on behalf of Dr. Gardner states:

    Gardner’s crowning achievement is the development of the theory of spectral correlation and cyclostationary signal processing and analysis. It is hard to overstate the importance of this work. Like many important theoretical developments, the theory is ‘unifying’ in that it brings into a common, cohesive framework results that previously seemed unrelated, or whose relation was not completely appreciated or understood. The consequences have been new insights and new results. In the not-too-distant future, it will constitute part of the core graduate curriculum in signal processing and in communications . . . And, as with much seminal work of a profound nature, Gardner’s theories have spawned a large amount of activity and new ideas and applications by others.

     

    Recent Quotations Addressing Gardner’s Contributions (2020)

    In addition to the early quotations (excerpts from letters) given above, some new quotations—excerpts taken from the preface of the 2019 book [B2]—which include bibliographical notes and references, are provided below, and apply to Theorems 1 – 19.

    Preface, p. xxii, “Fundamental noise and interference immunity and signal-separability properties of CS and ACS processes [and time series] discovered in the early 1980s by Gardner (e.g., (Gardner,1985) [Bk3] and (Gardner, 1987f) [Bk2]),  and theoretically developed in his seminal work performed primarily during the subsequent decade provided the foundation for the last 30 years of exploitation of almost-cyclostationarity in the design and analysis of signal processing methods and algorithms for communications, telemetry, and radar systems.” 

    Preface, p. xxiv, “The FOT probability framework for CS and ACS [time series] is introduced in (Gardner, 1985) [Bk3] and treated in depth in (Gardner, 1986c) [JP15], (Gardner, 1987f, Part II) [Bk2], (Gardner and Brown, 1991) [JP34]and (Gardner, 1991a) [JP36] with reference to continuous-time signals and in (Gardner, 1994) [JP53] for both continuous- and discrete-time signals; the case of complex signals is considered in (Gardner, 1987f, Part II) [Bk2] and (Brown, 1987) [p.8.3].” 

    Preface, p. xxvi, “The first fundamental treatment on cyclostationary processes in a book can be found in (Gardner, 1985, Chap. 12) [Bk3]. Several books are entirely dedicated to cyclostationarity. The extensive treatment in Part II of (Gardner, 1987f) [Bk2] contains the foundations of cyclic spectral analysis in the FOT approach.”

     

    9.1.2 Stochastic Cyclostationarity Theorems

    The conceptual and theoretical foundation and framework of William Gardner’s Theory of Cyclostationarity for Stochastic Processes [Bk3] is similar to that described in Section 9.1.1 for non-stochastic time series; however, it is less extensive because of Gardner’s preference for the non-stochastic theory and his tenet that the non-stochastic theory is of considerably higher conceptual and utilitarian value for practitioners. Almost every theorem cited in Section 9.1.1 has its counterpart in the theory for stochastic processes. The reader is referred to the book [Bk1], for the earliest compilation in book form (see also the 2nd ed. [Bk3]), and the much later book [B2] for the most comprehensive (by far) treatment. The book chapter [Bk5, Chap. 1] is complementary in the extent of its treatment of the contrasting features of the two theories.

    There are, however, some seminal contributions based on stochastic process models that either do not involve cyclostationarity or involve it only tangentially. The most noteworthy is the “Radically Different Method of Moments” introduced in the 1976 paper [JP4] and re-introduced in 2022 on page 11.4. This is an entirely different method from the classical one, introduced by Karl Pearson over a century ago, that provides for estimating parameters of probabilistic models of data. The tangential connection is the fact that the process of converting a multivariate (vector) model of observations with multiple observed sample vectors to a finite-length time-series model with a single sample path produces a cyclostationary stochastic process model for the data.

  • 9.2 Seminal Contributions of Antonio Napolitano

    –     List in preparation using [B2] ; sources being taken from Page 8.2     –

  • 9.3 Seminal Contributions of Others

    –     List in preparation using [B2]; sources being taken from Page 8.3     –