MAJOR FreeEMS Binary Log Loading Performance Increase

OpenLogViewer is a cross-platform data log viewing application originally written by Gufi and new developed and maintained by Ben and Fred. Discuss all things OLV here!
Post Reply
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

MAJOR FreeEMS Binary Log Loading Performance Increase

Post by Fred »

I've not finished for the day yet, with about 6 hours to go until I go to bed and 4 or so tomorrow morning before it needs to be online for Ben, and with dinner to cook and eat and clean up after and a shower to have, that's probably 6 hours total left to do work on it, BUT, so far:

(131314 / 12073 ) * (160000 / 51173) = THIRTY FOUR TIMES FASTER :-)

Loads 160k records in 12 seconds or less compared to 51k records in 2.15 minutes before. Also, This is from a file with 207k records and the new code stays up and displays whatever it gets to, as well. The old code falls over when it runs out of memory. The new code detects memory doom and prevents it.

Not a bad bit of performance increase there :-)

Let's see what else I can achieve before I put the code to sleep tonight!

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: MAJOR FreeEMS Binary Log Loading Performance Increase

Post by Fred »

The speed and my testing has exposed a bug. I just tested the old version from before I started and it does the same thing. All it means is that you have to relaunch the app between loads of big files or it will run out of memory. IE, there is a grand scale object reference memory leak. Issue here:

http://issues.freeems.org/view.php?id=363
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: MAJOR FreeEMS Binary Log Loading Performance Increase

Post by Fred »

Bug is fixed, as are a few others, and final performance before bed is:

(207161 / 6543 ) * (160000 / 51173) = NINETY NINE TIMES FASTER :-)

3am here, I'm knackered. I also haven't finished with the code. There are a couple of niggles before it's worthy of seeing the world. It's stable and fast, though.

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: MAJOR FreeEMS Binary Log Loading Performance Increase

Post by Fred »

This is now publicly available in all three major OLV repos! Enjoy!

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
Post Reply