Accurately recovering a low frequency bias signal from a random signal

I was trying to decide whether I wanted to pounce on that. Unless a tone starts before the big bang and continues on past the heat death of the universe its bandwidth isn't exactly zero.

But I knew what you meant.

Even a tone that's been turned on for a Really Long Time is still going to be subject to phase noise, drift, etc., all of which will broaden it's spectrum.

But I still knew what you meant.

Thanks!! :)

cheers, Jamie

Hi,

Thanks, I have a perturbation on the signal with known synchronization time but unknown effect on the signal :)

cheers, Jamie

You need to understand your signal to match a filter to the stuff you want and block the parts that you don't want. By eyeball it looks like there are components with periods in the range 10 samples and upwards.

Unclear if there are periodicities longer than 700 samples but it could be that or a drifting baseline. Low order polynomial fit or splines is the typical quick and dirty way to tidy up a dodgy baseline.

But without knowing what is signal and what is noise you are on a hiding to nothing. Low pass filtering will hide a multitude of sins.

The larger the dataset the better the signal to noise will become.

If you stick to powers of two even Excel can do the FFT for you (then IMABS() to get a power spectrum).

Hi, Martin - I did that with his data set and got a brief spike at the beginning and at the end. Pretty much flatline between. Am I doing something wrong?

No. That sounds about right for the dataset provided. There is a huge DC offset component which you should ignore and a slope/hump/drift or possible slowly varying component with a period of about 1000 samples.

Mostly it looks to me like noise with equal power per octave.

Subtract off (or ignore) the DC component and/or the best fit line and you might begin to see any signal that might be there. Better still take a hundred independent time series and average their PSF.

Any coherent signal will rise above the noise margin eventually.

I did a 3rd order polynomial fit using Excel (no discernible difference between 3rd and 4th order) and saw a nice curve with a half cycle of about 600 data points. It might be wishful thinking, though, and my eyes have become uncalibrated over the years.

Excel's formula for the curve is crap. You can't paste it into the cells for a plot comparison because it renders a flat line that fits to the average of the data.

Thanks for looking into it by the way! :)

cheers, Jamie

No thanks necessary, Jamie. I enjoy learning.

On 30/09/2015 15:03, John S wrote: > On 9/30/2015 8:41 AM, Martin Brown wrote: >> On 30/09/2015 12:54, John S wrote: >>> On 9/30/2015 6:37 AM, Martin Brown wrote: >>

Actually if it is the fit inside the charting component it is actually way way better at polynomial fit than the dodgy algorithm used by their MINVERSE code. The problem is one of display you have to set equation display significant digits to 16 or 20 to get the coefficients right.

The fit is extremely good at least up to cubics but goes a bit AWOL after that. You could break it in earlier versions by fitting a polynomial to measurements at a range of dates. The sum of powers matrix gets close to singular with a large additive constant on x/t.

Thanks for the help. I need to look at it again. I may have made a mistake or several.