Maths function question

[Chris Wilson]

The Motec data shows no sign of the lat or lon high words ever going to 0. I believe the Motec software has some filtering, but unsure how it works, this is how the GDA shows the track overlay when you use the options in Google Earth to make the line thinner, it's the white lines, showing several laps, obviously, compared to a single lap from the i2 export function.

 

http://www.gatesgarth.com/GDA/zigzag.png

[Andy Blyth]

Could you send me the .kml files you're using, both for the i2 data and the GDA data please? I think I'm going to have to look more closely at the numbers to get to the bottom of the spikes and the quantisation you're seeing.

[Chris Wilson]

I will do that with pleasure, but feel I am putting you (even taking the **** comes to mind ) Are you sure you are OK to carry on? I'll PM you something I got from the makers.... PM me your e-mail address please Andy.

[Andy Blyth]

LOL! Don't worry, I enjoy little challenges like this! At least we're getting somewhere; it could be a lot worse!

 

PM sent.

[Chris Wilson]

GEMS have drawn my attention to this in the help file:

 

Transient removal

 

After acquiring your data, you may discover that you had a dodgy connection or a damaged sensor. In the case of poor connections, there could be transients (spikes/clicks) in the data.

A faulty wheel speed sensor is an example where transients may arise. If you wish to generate a distance channel from the data, then you will probably want to remove the transients prior to integrating.

This can be accomplished using the declick function.

 

declick('Wheel Speed', 10, 10)

 

The first parameter to this function indicates the data source to de-click. In this case it is the 'Wheel Speed' channel but it could also be a maths expression.

The second parameter identifies the threshold. This determines how much the sample value must change by since the last sample in order for it to be a candidate for removal. Provided the transient has two edges (both a rising edge and a falling edge, in any order) then it can be removed.

The third parameter limits the number of samples wide that the click may be before the algorithm rejects it as valid data. i.e. if the click is less wide than this value then it will be smoothed over.

 

The smoothing simply linearly interpolates between the good samples either side of the click.

 

Conditional error removal

 

To clean up latitude and longitude data, you can use a number of methods. Its useful if you have a GPS fix / GPS valid channel (which could always be generated using maths to restrict lat/lon to a sensible range). You can then use the declick_if function:

 

declick_if('Latitude Raw', bool('GPS Valid') == false)

 

This is essentially the same as declick, except that you are responsible for the click detection phase.

 

Filtering

 

Another option may be to filter your data. If you have a large DC offset you could use a high-pass filter. If you have lots of spikes or hight frequency noise then you could use a low-pass filter to remove them.

 

The filters in GEMS Data Analysis Pro are linear phase windowed sinc filters. These have a very steep cutoff frequency and also do not adjust the time coherence of data across the frequency range. GEMS Data Analysis Pro automatically removes any time delays that would normally be imposed by these filters.

[Andy Blyth]

What update rate was your M800 sending the GPS CAN frames? Also, do you know what rate the logger was sampling the incoming frames? I may be able to explain the "zig-zig" pattern you're seeing if the rates are close to the IIRC 5Hz rate of your GPS receiver.

[Chris Wilson]

All GPS coordinates HW and LW are at 20 Hz in the logger, speed is also at 20 Hz, GPS time is logged at 10Hz. The Motec ecu logs at 20Hz for the coordinates, which is what they recommend for a 5 Hz receiver. Not sure where to look for CAN frame rates, does 1 MHz sound right?

[Chris Wilson]

CAN 1 Transfer rate is 100Hz

[Andy Blyth]

Hmm maybe not then. The rates you have look good for the 5Hz receiver. You need to ensure that you're transmitting and sampling at a minimum of double the rate of the data. I think that the zig-zag pattern may be a side effect of the latidude and longitude being sent separately (I'm assuming that they are sent in separate frames; it's been a while since I watched the webinar with the low level details of the way MoTeC uses the CAN frames). It's as if the longitude of one sample is being mixed with the latidude of the previous sample. I can't think of an easy way of getting around this at the moment.

[Chris Wilson]

I think I have sorted it, what do you think of this Andy? The programme was reading data from when the car was stationary at the start of the session and again when it was stopped at the end. The Motec stuff seems clever enough to discard this, the GEMS logs and tries to map it somehow. As the GEMS data was for a whole session rather than a lap from a session, it was having some sort of hissy fit at the start and end. I have clipped the data to remove the stationary periods and it gives this now I think a bit of filtering and it's pretty as good as the Motec i2 data? I cannot thank you enough....

[Andy Blyth]

That's looking pretty good now! I'm pleasantly surprised that it's accurate enough to see the exact line you took on the track. I thought it would only be accurate enough to see how far around you were. Were you overtaking someone on the straight behind the pit lane?

[Chris Wilson]

I think a filter to only log GPS data once moving might work? And a slight smoothing filter. Once I get this all set up I will upgrade to a 20 Hz GPS, I have my eye on one that seems to be what Motec use, but without the fancy die cast casing

 

Both Motec and GEMS show some drift, usually in a certain direction, so it can look as if you are driving on the grass. You can see the drift as the thing "warms up" the car can be stationary and the numbers in Motec's "View Sensors" page for GPS coordinates slowly drift, then after about 10 minutes seem to pretty much stabilise.

 

It is accurate enough to be potentially quite embarrassing, I'm sure Yes, I think I was overtaking a car there, I was trying to get 135 up on the banking, but decided 132 was adequate, road cars can go awfully light and funny when committed to a bend properly over about 125. The Zeus was worry free at 148, that's downforce and low C of G for you.

[Homer]

May I just add... Wow!

 

If there's a forum for functional math it's this one who'd have thunk it! Great work Andy

[Chris Wilson]

Can't thank Andy and Scott enough, this started as a quick question as to how to stitch two 16 bit channels of data together, and ended up looking like Carol Vorderman's nightmare after taking narcotics. I have learnt a lot, and admit OCD stepped in, but it works, and without you guys I'd looking at data in total spacial ignorance. Great work gentlemen, I salute and applaud you

[Andy Blyth]

You're welcome Chris. Looking back, two things would have made this very simple:

 

1> If the high word was guaranteed to be signed and the low word guaranteed to be unsigned, the combine channel would be as simple as:

 

GPS LAT COMBINE = (('GPS LAT HW'*65536)+'GPS LAT LW')/10000000

 

2> If the GDA software had a function that performed rounding towards zero for both positive and negative numbers (like the trunc() function in the C programming language), the rest of the calculation would be a simple as:

 

GPS LAT = (('GPS LAT COMBINE'-trunc('GPS LAT COMBINE'))/0.6)+trunc('GPS LAT COMBINE')

 

Much simpler! Maybe it's worth putting a request into GEMS!

[Chris Wilson]

Funnily enough GEMS have just said this to me on their forum, I think they believe it is, or should be, possible just to use the combine16 function to combine the two 16 bit channels. I get the impression they feel it's a Motec / GEMS data transfer set up issue, but as it works I am am leaving well alone, although I will try changing the LW signing in the logger, and also may put the maths functions in the logger set up, which seems possible, rather than in the data reading software.

 

##############################

 

"Good to see its accurate now. Yes you should change the setup for unsigned. The CAN output info in Motec should have displayed this.

 

IF the data provided is correct, and there is no other scaling to do, then what Motec have said (which i believe was: "The GPS Lat and GPS Long are just split in 2, nothing more challenging that

that.") should apply, which means that using just the combine16 function with a divide by 10000000 should work.

 

For the other outstanding issues, i would again check your setup and the motec output."

 

#########################################