Noncentral limit theorem for the generalized Rosenblatt process

Abstract

We use techniques of Malliavin calculus to study the convergence in law of a family of generalized Rosenblatt processes Zγ with kernels defined by parameters γ taking values in a tetrahedral region Δ of $\RR^q$. We prove that, as γ converges to a face of Δ, the process Zγ converges to a compound Gaussian distribution with random variance given by the square of a Rosenblatt process of one lower rank. The convergence in law is shown to be stable. This work generalizes a previous result of Bai and Taqqu, who proved the result in the case q=2 and without stability.