i, Having found out that the avr-gcc lib supports floats and handles them quite gracefully, I tried porting my linear regression code from desktop on to the atmega 128L, more or less as is. For testing i used the same table of X and Y values as sample inputs to the algorithm.
Now I am using microsecond granularity as input to the regression engine and since the values are fairly large - and since there are sum of products- I get fairly large numbers (If i do the same on my desktop, the biggest number i get is sumXSq=7260107358192.000000). Hence i decided to use long long (64 bit!) as follows: (i know long long and int64_t are the same but i am clutching at straws) typedef struct regressTbl{ int16_t n ; long long sumXY; int64_t sumX,sumY; long long sumXSquared; }regTable;
and then I added the sum for each point from the table. It should be okay.. but it wasnt :( .. here is a manual version of the final values:
table.sumX = 12338578; table.sumY = -3336078; table.sumXSquared = 7260107358192;