Future Live Mode Semantics

OpenLogViewer is a cross-platform data log viewing application originally written by Gufi and new developed and maintained by Ben and Fred. Discuss all things OLV here!
Post Reply
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Future Live Mode Semantics

Post by Fred »

I've been thinking about live mode (in the last 15 seconds or so) and realised that if we allow it to eat and eat memory, then it'll interrupt the flow of the data across the screen each time it increases the array size. IE, that's unacceptable. What this means is that we have to let the user set a buffer size to use (N samples long) and treat it as a circular buffer. Ben, when you get a chance to think about that, do, and let me know what is likely to come apart at the seams if/when we enabled that mode of operation.

The other thing that that means is that after N samples we'll be throwing away data, so we'll also need to write data out to a file as we go using a buffered writer that writes fairly regularly to keep things smooth, but not too regularly or it'll eat too much CPU.

This message brought to you by a console window that said 2600 records loaded in 37000ms or, 15ms per record, or 65Hz max sample rate before host CPU saturation (with my current other loads). This is fairly worst case, the machine is being a dog at the moment, however is slower than FreeEMS pumps them out, so needs to be improved, I guess. There is an API that needs adding in your drawing classes such that we can improve the efficiency of the data classes. The change will be one that enables you to extract the data natively and display it without significant processing. At the same time I'll then be able to populate it natively and save a shit load of CPU time in the process, not to mention a factor of 8 memory in many cases which will also save time doing array resizes during load, should they be necessary. It's not a priority at all, and I'd like to be heavily involved in that change, but keep it in mind.

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
BenFenner
LQFP144 - On Top Of The Game
Posts: 360
Joined: Wed Jul 09, 2008 3:15 pm

Re: Future Live Mode Semantics

Post by BenFenner »

Off the top of my head, the size of the circular buffer simply needs to be set to the amount of data points that can currently fit in the display window (already calculated in code). This could dynamically change if someone zooms?
As data falls off the left side of the display it gets written to disc or put in another buffer that waits to be written.

Maybe we pick a certain buffer size and use as many of those (plus one) as it takes to span the entire display and write the oldest buffer to disc as it falls out of use? That keeps writes to disc a constant in size and frequency.

As for the rest of the issues with implementing this, there will be a few, but I can't exactly pin down what they will be right now.
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: Future Live Mode Semantics

Post by Fred »

Good thoughts, Ben!

How about this:

Live mode: Minimal/dynamic/fixed/chosen buffer that just keeps enough, no more.
Dead mode: The usual.

The writing to disk is already handled in my plugin (not sure why I wrote that), and can be handled by a CSV dump if a megashit plugin was ever added to handle live mode.

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
BenFenner
LQFP144 - On Top Of The Game
Posts: 360
Joined: Wed Jul 09, 2008 3:15 pm

Re: Future Live Mode Semantics

Post by BenFenner »

I have no objections.
Post Reply