Audio Processing is likely one of the most necessary utility domains of digital sign processing (DSP) and machine studying. Modeling acoustic environments is a necessary step in creating digital audio processing techniques resembling: speech recognition, speech enhancement, acoustic echo cancellation, and so forth.
Acoustic environments are stuffed with background noise that may have a number of sources. For instance, when sitting in a espresso store, strolling down the road, or driving your automobile, you hear sounds that may be thought of as interference or background noise. Such interferences don’t essentially observe the identical statistical mannequin, and therefore, a mix of fashions will be helpful in modeling them.
These statistical fashions may also be helpful in classifying acoustic environments into completely different classes, e.g., a quiet auditorium (class 1), or a barely noisier room with closed home windows (class 2), and a 3rd possibility with home windows open (class 3). In every case, the extent of background noise will be modeled utilizing a mix of noise sources, every occurring with a special likelihood and with a special acoustic degree.
One other utility of such fashions is within the simulation of acoustic noise in numerous environments primarily based on which DSP and machine studying options will be designed to unravel particular acoustic issues in sensible audio techniques resembling interference cancellation, echo cancellation, speech recognition, speech enhancement, and so forth.
A easy statistical mannequin that may be helpful in such situations is the Gaussian Mixture Model (GMM) through which every of the completely different noise sources is assumed to observe a selected Gaussian distribution with a sure variance. All of the distributions will be assumed to have zero imply whereas nonetheless being sufficiently correct for this utility, as additionally proven on this article.
Every of the GMM distributions has its personal likelihood of contributing to the background noise. For instance, there might be a constant background noise that happens more often than not, whereas different sources will be intermittent, such because the noise coming via home windows, and so forth. All this must be thought of in our statistical mannequin.
An instance of simulated GMM knowledge over time (normalized to the sampling time) is proven within the determine beneath through which there are two Gaussian noise sources, each of zero imply however with two completely different variances. On this instance, the decrease variance sign happens extra usually with 90% likelihood therefore the intermittent spikes within the generated knowledge representing the sign with larger variance.
In different situations and relying on the appliance, it might be the opposite method round through which the excessive variance noise sign happens extra usually (as will probably be proven in a later instance on this article). Python code used to generate and analyze GMM knowledge will even be proven later on this article.
Turning to a extra formal modelling language, let’s assume that the background noise sign that’s collected (utilizing a high-quality microphone for instance) is modeled as realizations of impartial and identically distributed (iid) random variables that observe a GMM as proven beneath.
The modeling drawback thus boils all the way down to estimating the mannequin parameters (i.e., p1, σ²1, and σ²2) utilizing the noticed knowledge (iid). On this article, we will probably be utilizing the method of moments (MoM) estimator for such function.
To simplify issues additional, we will assume that the noise variances (σ²1 and σ²2) are identified and that solely the blending parameter (p1) is to be estimated. The MoM estimator can be utilized to estimate a couple of parameter (i.e., p1, σ²1, and σ²2) as proven in Chapter 9 of the e-book: “Statistical Sign Processing: Estimation Idea”, by Steven Kay. Nonetheless, on this instance, we are going to assume that solely p1 is unknown and to be estimated.
Since each gaussians within the GMM are zero imply, we are going to begin with the second second and attempt to get hold of the unknown parameter p1 as a operate of the second second as follows.
Observe that one other easy methodology to acquire the moments of a random variable (e.g., second second or larger) is through the use of the second producing operate (MGF). An excellent textbook in likelihood idea that covers such subjects, and extra is: “Introduction to Chance for Information Science”, by Stanley H. Chan.
Earlier than continuing any additional, we wish to quantify this estimator by way of the basic properties of estimators resembling bias, variance, consistency, and so forth. We’ll confirm this later numerically with a Python instance.
Beginning with the estimator bias, we will present that the above estimator of p1 is certainly unbiased as follows.
We will then proceed to derive the variance of our estimator as follows.
It is usually clear from the above evaluation that the estimator is constant since it’s unbiased and likewise its variance decreases when the pattern measurement (N) will increase. We will even use the above components of the p1 estimator variance in our Python numerical instance (proven intimately later on this article) when evaluating idea with sensible numerical outcomes.
Now let’s introduce some Python code and do some enjoyable stuff!
First, we generate our knowledge that follows a GMM with zero means and normal deviations equal to 2 and 10, respectively, as proven within the code beneath. On this instance, the blending parameter p1 = 0.2, and the pattern measurement of the information equals 1000.
# Import the Python libraries that we'll want on this GMM instance
import matplotlib.pyplot as plt
import numpy as np
from scipy import stats
# GMM knowledge era
mu = 0 # each gaussians in GMM are zero imply
sigma_1 = 2 # std dev of the primary gaussian
sigma_2 = 10 # std dev of the second gaussian
norm_params = np.array([[mu, sigma_1],
[mu, sigma_2]])
sample_size = 1000
p1 = 0.2 # likelihood that the information level comes from first gaussian
mixing_prob = [p1, (1-p1)]
# A stream of indices from which to decide on the element
GMM_idx = np.random.alternative(len(mixing_prob), measurement=sample_size, substitute=True,
p=mixing_prob)
# GMM_data is the GMM pattern knowledge
GMM_data = np.fromiter((stats.norm.rvs(*(norm_params[i])) for i in GMM_idx),
dtype=np.float64)Then we plot the histogram of the generated knowledge versus the likelihood density operate as proven beneath. The determine reveals the contribution of each Gaussian densities within the general GMM, with every density scaled by its corresponding issue.
The Python code used to generate the above determine is proven beneath.
x1 = np.linspace(GMM_data.min(), GMM_data.max(), sample_size)
y1 = np.zeros_like(x1)
# GMM likelihood distribution
for (l, s), w in zip(norm_params, mixing_prob):
y1 += stats.norm.pdf(x1, loc=l, scale=s) * w
# Plot the GMM likelihood distribution versus the information histogram
fig1, ax = plt.subplots()
ax.hist(GMM_data, bins=50, density=True, label="GMM knowledge histogram",
coloration = GRAY9)
ax.plot(x1, p1*stats.norm(loc=mu, scale=sigma_1).pdf(x1),
label="p1 × first PDF",coloration = GREEN1,linewidth=3.0)
ax.plot(x1, (1-p1)*stats.norm(loc=mu, scale=sigma_2).pdf(x1),
label="(1-p1) × second PDF",coloration = ORANGE1,linewidth=3.0)
ax.plot(x1, y1, label="GMM distribution (PDF)",coloration = BLUE2,linewidth=3.0)
ax.set_title("Information histogram vs. true distribution", fontsize=14, loc="left")
ax.set_xlabel('Information worth')
ax.set_ylabel('Chance')
ax.legend()
ax.grid()After that, we compute the estimate of the blending parameter p1 that we derived earlier utilizing MoM and which is proven right here once more beneath for reference.
The Python code used to compute the above equation utilizing our GMM pattern knowledge is proven beneath.
# Estimate the blending parameter p1 from the pattern knowledge utilizing MoM estimator
p1_hat = (sum(pow(x,2) for x in GMM_data) / len(GMM_data) - pow(sigma_2,2))
/(pow(sigma_1,2) - pow(sigma_2,2))In an effort to correctly assess this estimator, we use Monte Carlo simulation by producing a number of realizations of the GMM knowledge and estimate p1 for every realization as proven within the Python code beneath.
# Monte Carlo simulation of the MoM estimator
num_monte_carlo_iterations = 500
p1_est = np.zeros((num_monte_carlo_iterations,1))
sample_size = 1000
p1 = 0.2 # likelihood that the information level comes from first gaussian
mixing_prob = [p1, (1-p1)]
# A stream of indices from which to decide on the element
GMM_idx = np.random.alternative(len(mixing_prob), measurement=sample_size, substitute=True,
p=mixing_prob)
for iteration in vary(num_monte_carlo_iterations):
sample_data = np.fromiter((stats.norm.rvs(*(norm_params[i])) for i in GMM_idx))
p1_est[iteration] = (sum(pow(x,2) for x in sample_data)/len(sample_data)
- pow(sigma_2,2))/(pow(sigma_1,2) - pow(sigma_2,2))Then, we verify for the bias and variance of our estimator and evaluate to the theoretical outcomes that we derived earlier as proven beneath.
p1_est_mean = np.imply(p1_est)
p1_est_var = np.sum((p1_est-p1_est_mean)**2)/num_monte_carlo_iterations
p1_theoritical_var_num = 3*p1*pow(sigma_1,4) + 3*(1-p1)*pow(sigma_2,4)
- pow(p1*pow(sigma_1,2) + (1-p1)*pow(sigma_2,2),2)
p1_theoritical_var_den = sample_size*pow(sigma_1**2-sigma_2**2,2)
p1_theoritical_var = p1_theoritical_var_num/p1_theoritical_var_den
print('Pattern variance of MoM estimator of p1 = %.6f' % p1_est_var)
print('Theoretical variance of MoM estimator of p1 = %.6f' % p1_theoritical_var)
print('Imply of MoM estimator of p1 = %.6f' % p1_est_mean)
# Beneath are the outcomes of the above code
Pattern variance of MoM estimator of p1 = 0.001876
Theoretical variance of MoM estimator of p1 = 0.001897
Imply of MoM estimator of p1 = 0.205141We will observe from the above outcomes that the imply of the p1 estimate equals 0.2051 which could be very near the true parameter p1 = 0.2. This imply will get even nearer to the true parameter because the pattern measurement will increase. Thus, now we have numerically proven that the estimator is unbiased as confirmed by the theoretical outcomes achieved earlier.
Furthermore, the pattern variance of the p1 estimator (0.001876) is nearly an identical to the theoretical variance (0.001897) which is gorgeous.
It’s at all times a contented second when idea matches follow!
All photographs on this article, until in any other case famous, are by the writer.
Source link
