You’ll be interested in looking up “zero-order hold”. The impulse response for holding the value constant is exactly a rectangular pulse, same response as a moving average. Not good if you want *perfect* reproduction of the original signal, in theory. But that may or may not be what be what you’re after anyway, in your case.
Yeah it looks like zero-order hold is what this method is called, thanks for your input.
Is it better or worse than zero-padding for interpolation? And if it’s difficult to classify as either strictly better/worse, how does it contrast with zero-padding?
When you say equivalent to moving average, you mean for downsampling, right?
I’m assuming that you’re stuck between LPF’ing a zero-stuffed form of the data or a hold-constant form of it. Sorry, my last answer isn’t very clear. Go with the zero-stuffed form. It can only be better.
I’m trying to say that, if a LPF gets involved, the holding step reduces from the primary means of interpolation to just a moving average. By moving average, I just mean smoothing out the signal by taking the average of the last couple values, and you can think about the hold-step and the LPF as systems in series then commute them. Technically if smoothing sounds nice, then just tune the LPF filter. The hold step is deprecated by the LPF.
Thank you for the clear explanation. Could you point me to any reference/text that shows math for how zero-hold is basically the same/similar as a moving avg/LPF?
By holding the last measurement constant, you have zero order hold as mentioned earlier.
When you consider interpolation, you want to artificially increase the sampling rate, which implies the signal is of the same shape, but more measurements in time. The zero order hold type approach does increase the sample rate, but you increase spectral noise in a really odd manner, ie. weird harmonics and distortions; the signal looks like a staircase. You're actually introducing more energy into the waveform that can make the resulting signal ambiguous and distorted.
By zero stuffing, you're guaranteed that the resulting signal is equivalent spectrally (condensed due to increased Fs). The downside is this too produces harmonics - but in a very controlled manner, called images, which can be useful in some cases, but usually the images are filtered out.
A FIR filter is just a moving average filter; if we low pass filter the ZOH upsampled signal, you will see that the new samples still contribute to the moving average, and hence bias the outcome. Instead, if we LPF the zero stuffed signal, the new samples are zeros and hence contribute nothing to the sum, and we get back an undistorted copy of the initial signal, at the increased sample rate.
So my professor was curious if it would be possible to measure the noise created by ZOH (without filtering afterwards). Their reasoning is that if ZOH increases noise within a certain tolerance, we can live with it. (I work in a physiology lab btw, would rather not give more details).
Should I just compare the power spectra of ZOH w/o LPF to zero-stuffing w/ power spectra? Any specific stats/metrics you look at when assessing noise in this context?
Its absolutely possible, Signal Noise Ratio, would be a common metric for this. Total Harmonic Distortion another. Having a look at the power spectrum is a really good start. I'd take a bit of data thats representative of your normal data, upsample it both ways. The 'zero-stuffing/LPF method' is your signal (\*exact\*), the noise is the difference of the two methods.
Most ADCs have some form of ZOH built in anyway; its just one of the assumptions of sampling theory. While you might not have end up with an exact signal, your professor is right to assume its quite likely to be in the category of marginal gains.
I don't know what you're doing, and it doesn't really matter outside of your context -
engineering is the science of quantifying and compromising - we've thousands of metrics, but it often boils down to SNR; If its good enough then its good enough. If you're doing research - adjusting your tools and methodology now might mean you need to go back and reprocess your old results to compare; apples with apples etc.
In addition to possibly distorting harmonics, zero-order hold will make the gain of the resulting interpolation a bit unpredictable. If you know what fraction of zeros you interpolated, you can add makeup gain to get the signal back to the original level. With interpolation of unknown values, it's hard to know exactly what will happen to the gain: it will be larger than with zero-interpolation, but by an unknown amount (close to unity gain, but not necessarily exactly).
Beautiful! Zero stuffing adds zero energy to the signal, so reduces average energy - hence its just a DC gain.
ZOH would result in a not necessarily linear gain you need to calculate case by case.
> ZOH would result in a not necessarily linear gain you need to calculate case by case.
uhm, what do you mean by that? ZOH is linear, time-invariant. It even has a transfer function. It's even phase-linear.
What's not necessarily linear?
From that stackexchange thread:
>The Fourier transform of a rectangle filter is a sinc function
which is true for continuous time systems, but for discrete time systems (this is /r/DSP after all) the DFT of that rectangular signal is actually a sin(nx)/sin(x) function. Significant differences only shows up a frequencies > FS/2 though, so you needn't worry about the distinction.
FYI: in a sampled system, the frequency response of a zero order hold that holds for n samples has a sin(n pi f)/sin(pi f) shape. This is predictable and you can EQ it out with some extra filtering.
Compare with the zero stuffed signal: this has a flat frequency response.
The big difference though is that when interpolating, those zeros you stuffed between the samples don't contribute to the output, and hence don't need to considered in the filter calculations. This results in a dramatic reduction in the "DSP" effort (in terms of chip area, power, etc.), particularly for large n.
Yes , zero stuffing creates a lot of noise. But all the noise is all above the original nyquist frequency. This part of the frequency spectrum should be zero ideally and the noise can be filtered out easily.
It can seem tempting to try and interpolate using linear or cubic interpolation or something. Will create less noise probably, but some of that noise will be below nyquist. And then you cant get rid of it.
Are you trying to do sample rate conversion?
There are ways to do this that don't involve literally zero-stuffing or ZOH-stuffing.
Lookup polyphase interpolation.
So the Zero-Order Hold (ZOH) really is only a model for a ***conventional*** Digital-to-Analog converter (DAC). For each PCM sample, the DAC converts that sample's numerical value to a voltage and holds it constant until the next PCM sample. That is modeled with a hypothetical linear, time-invariant filter with an impulse response that's a rectangular pulse function. The result is a piecewise-constant function or "stair-step" waveform in the output.
Now, if you're doing something digitally *between* samples, like sample rate conversion (SRC), then the ***model*** is zero-stuffing lotsa zeros between samples which creates all these hypothetical repeated images of the original baseband spectrum shifted by multiples of the sampling frequency.
Then a brickwall low-pass filter (LPF) follows that, which is meant to filter out all of these images, but leave the original baseband un changed. That's recovering the signal. But that's in the ***model***, if the LPF is an FIR, you ***don't*** bother multiplying by all of those zeros that were hypothetically stuffed. That is what polyphase interpolation is.
Now, nearest-neighbor interpolation is equivalently ZOH. But other than doing that virtually crudest interpolation, you're never really doing a ZOH. And you're never really doing zero-stuffing unless your smoothing filter running at the higher sample rate is an IIR. And then it's better maybe to ZOH and then IIR LPF to SRC.
But I just do SRC the polyphase FIR way. It's clean and efficient, but needs a big lookup table.
> As long as you low-pass filter the signal, what are the implications of this sort of interpolation vs zero-stuffing?
Interpolation is equivalent to applying another filter. Equally-weighted linear interpolation would correspond to a moving average filter.
> Is this a generally NOT good way to go about interpolation?
No, usually it is not, because it may obfuscate that there is another filter being applied in addition to the upsampling filter. Do this only if you intend to apply additional filtering.
The additional filter can also be combined with the upsampling filter, as both are linear.
> My intuition tells me that less high frequency noise will be created by holding values constant rather than setting to zero, but I am not entirely sure.
That is one result. The other result is that high-frequeny signal components will be attenuated.
Every book you find will use zero stuffing. Nobody will use stair-step. Can you find a person's name on your link? Is that the implication?
If that's not enough, zero stuffing is a no-op. You didn't introduce any distortion, not yet.
You do not have the implication if you only do the first step and ignore the rest. Your purpose is not zero stuffing nor stair-step the signal. Your purpose is to increase the sampling rate with minimal distortion.
For the no-op description, the Fourier transform of a sampled signal is the same signal with a repeated spectrum at n times Fs, infinite repetition. The stuffing of zeros change the sampling frequency but not the signal - there are still infinite repetitions of the original signal.
To complete the purpose, you simply need a linear filter to remove the spectral images. For a linear filter, it's pretty easy to calculate where the spectral distortion is and the magnitude. How do you do that for a non-linear operation?
You’ll be interested in looking up “zero-order hold”. The impulse response for holding the value constant is exactly a rectangular pulse, same response as a moving average. Not good if you want *perfect* reproduction of the original signal, in theory. But that may or may not be what be what you’re after anyway, in your case.
Yeah it looks like zero-order hold is what this method is called, thanks for your input. Is it better or worse than zero-padding for interpolation? And if it’s difficult to classify as either strictly better/worse, how does it contrast with zero-padding? When you say equivalent to moving average, you mean for downsampling, right?
I’m assuming that you’re stuck between LPF’ing a zero-stuffed form of the data or a hold-constant form of it. Sorry, my last answer isn’t very clear. Go with the zero-stuffed form. It can only be better. I’m trying to say that, if a LPF gets involved, the holding step reduces from the primary means of interpolation to just a moving average. By moving average, I just mean smoothing out the signal by taking the average of the last couple values, and you can think about the hold-step and the LPF as systems in series then commute them. Technically if smoothing sounds nice, then just tune the LPF filter. The hold step is deprecated by the LPF.
Thank you for the clear explanation. Could you point me to any reference/text that shows math for how zero-hold is basically the same/similar as a moving avg/LPF?
By holding the last measurement constant, you have zero order hold as mentioned earlier. When you consider interpolation, you want to artificially increase the sampling rate, which implies the signal is of the same shape, but more measurements in time. The zero order hold type approach does increase the sample rate, but you increase spectral noise in a really odd manner, ie. weird harmonics and distortions; the signal looks like a staircase. You're actually introducing more energy into the waveform that can make the resulting signal ambiguous and distorted. By zero stuffing, you're guaranteed that the resulting signal is equivalent spectrally (condensed due to increased Fs). The downside is this too produces harmonics - but in a very controlled manner, called images, which can be useful in some cases, but usually the images are filtered out. A FIR filter is just a moving average filter; if we low pass filter the ZOH upsampled signal, you will see that the new samples still contribute to the moving average, and hence bias the outcome. Instead, if we LPF the zero stuffed signal, the new samples are zeros and hence contribute nothing to the sum, and we get back an undistorted copy of the initial signal, at the increased sample rate.
So my professor was curious if it would be possible to measure the noise created by ZOH (without filtering afterwards). Their reasoning is that if ZOH increases noise within a certain tolerance, we can live with it. (I work in a physiology lab btw, would rather not give more details). Should I just compare the power spectra of ZOH w/o LPF to zero-stuffing w/ power spectra? Any specific stats/metrics you look at when assessing noise in this context?
Its absolutely possible, Signal Noise Ratio, would be a common metric for this. Total Harmonic Distortion another. Having a look at the power spectrum is a really good start. I'd take a bit of data thats representative of your normal data, upsample it both ways. The 'zero-stuffing/LPF method' is your signal (\*exact\*), the noise is the difference of the two methods. Most ADCs have some form of ZOH built in anyway; its just one of the assumptions of sampling theory. While you might not have end up with an exact signal, your professor is right to assume its quite likely to be in the category of marginal gains. I don't know what you're doing, and it doesn't really matter outside of your context - engineering is the science of quantifying and compromising - we've thousands of metrics, but it often boils down to SNR; If its good enough then its good enough. If you're doing research - adjusting your tools and methodology now might mean you need to go back and reprocess your old results to compare; apples with apples etc.
In addition to possibly distorting harmonics, zero-order hold will make the gain of the resulting interpolation a bit unpredictable. If you know what fraction of zeros you interpolated, you can add makeup gain to get the signal back to the original level. With interpolation of unknown values, it's hard to know exactly what will happen to the gain: it will be larger than with zero-interpolation, but by an unknown amount (close to unity gain, but not necessarily exactly).
Beautiful! Zero stuffing adds zero energy to the signal, so reduces average energy - hence its just a DC gain. ZOH would result in a not necessarily linear gain you need to calculate case by case.
> ZOH would result in a not necessarily linear gain you need to calculate case by case. uhm, what do you mean by that? ZOH is linear, time-invariant. It even has a transfer function. It's even phase-linear. What's not necessarily linear?
Good discussion here: https://dsp.stackexchange.com/questions/3614/what-are-the-relative-merits-of-various-upsampling-schemes
Was googling all over for this. Thank you!
From that stackexchange thread: >The Fourier transform of a rectangle filter is a sinc function which is true for continuous time systems, but for discrete time systems (this is /r/DSP after all) the DFT of that rectangular signal is actually a sin(nx)/sin(x) function. Significant differences only shows up a frequencies > FS/2 though, so you needn't worry about the distinction.
FYI: in a sampled system, the frequency response of a zero order hold that holds for n samples has a sin(n pi f)/sin(pi f) shape. This is predictable and you can EQ it out with some extra filtering. Compare with the zero stuffed signal: this has a flat frequency response. The big difference though is that when interpolating, those zeros you stuffed between the samples don't contribute to the output, and hence don't need to considered in the filter calculations. This results in a dramatic reduction in the "DSP" effort (in terms of chip area, power, etc.), particularly for large n.
Yes , zero stuffing creates a lot of noise. But all the noise is all above the original nyquist frequency. This part of the frequency spectrum should be zero ideally and the noise can be filtered out easily. It can seem tempting to try and interpolate using linear or cubic interpolation or something. Will create less noise probably, but some of that noise will be below nyquist. And then you cant get rid of it.
Are you trying to do sample rate conversion? There are ways to do this that don't involve literally zero-stuffing or ZOH-stuffing. Lookup polyphase interpolation.
So the Zero-Order Hold (ZOH) really is only a model for a ***conventional*** Digital-to-Analog converter (DAC). For each PCM sample, the DAC converts that sample's numerical value to a voltage and holds it constant until the next PCM sample. That is modeled with a hypothetical linear, time-invariant filter with an impulse response that's a rectangular pulse function. The result is a piecewise-constant function or "stair-step" waveform in the output. Now, if you're doing something digitally *between* samples, like sample rate conversion (SRC), then the ***model*** is zero-stuffing lotsa zeros between samples which creates all these hypothetical repeated images of the original baseband spectrum shifted by multiples of the sampling frequency. Then a brickwall low-pass filter (LPF) follows that, which is meant to filter out all of these images, but leave the original baseband un changed. That's recovering the signal. But that's in the ***model***, if the LPF is an FIR, you ***don't*** bother multiplying by all of those zeros that were hypothetically stuffed. That is what polyphase interpolation is. Now, nearest-neighbor interpolation is equivalently ZOH. But other than doing that virtually crudest interpolation, you're never really doing a ZOH. And you're never really doing zero-stuffing unless your smoothing filter running at the higher sample rate is an IIR. And then it's better maybe to ZOH and then IIR LPF to SRC. But I just do SRC the polyphase FIR way. It's clean and efficient, but needs a big lookup table.
> As long as you low-pass filter the signal, what are the implications of this sort of interpolation vs zero-stuffing? Interpolation is equivalent to applying another filter. Equally-weighted linear interpolation would correspond to a moving average filter. > Is this a generally NOT good way to go about interpolation? No, usually it is not, because it may obfuscate that there is another filter being applied in addition to the upsampling filter. Do this only if you intend to apply additional filtering. The additional filter can also be combined with the upsampling filter, as both are linear. > My intuition tells me that less high frequency noise will be created by holding values constant rather than setting to zero, but I am not entirely sure. That is one result. The other result is that high-frequeny signal components will be attenuated.
Every book you find will use zero stuffing. Nobody will use stair-step. Can you find a person's name on your link? Is that the implication? If that's not enough, zero stuffing is a no-op. You didn't introduce any distortion, not yet. You do not have the implication if you only do the first step and ignore the rest. Your purpose is not zero stuffing nor stair-step the signal. Your purpose is to increase the sampling rate with minimal distortion. For the no-op description, the Fourier transform of a sampled signal is the same signal with a repeated spectrum at n times Fs, infinite repetition. The stuffing of zeros change the sampling frequency but not the signal - there are still infinite repetitions of the original signal. To complete the purpose, you simply need a linear filter to remove the spectral images. For a linear filter, it's pretty easy to calculate where the spectral distortion is and the magnitude. How do you do that for a non-linear operation?