View unanswered posts | View active topics It is currently Tue Dec 12, 2017 5:46 am



Reply to topic  [ 4 posts ] 
MAJOR FreeEMS Binary Log Loading Performance Increase 
Author Message
Moderator
User avatar

Joined: Tue Jan 15, 2008 2:31 pm
Posts: 14909
Location: Home sweet home!
I've not finished for the day yet, with about 6 hours to go until I go to bed and 4 or so tomorrow morning before it needs to be online for Ben, and with dinner to cook and eat and clean up after and a shower to have, that's probably 6 hours total left to do work on it, BUT, so far:

(131314 / 12073 ) * (160000 / 51173) = THIRTY FOUR TIMES FASTER :-)

Loads 160k records in 12 seconds or less compared to 51k records in 2.15 minutes before. Also, This is from a file with 207k records and the new code stays up and displays whatever it gets to, as well. The old code falls over when it runs out of memory. The new code detects memory doom and prevents it.

Not a bad bit of performance increase there :-)

Let's see what else I can achieve before I put the code to sleep tonight!

Fred.

_________________
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!


Sun Oct 30, 2011 6:59 pm
Profile WWW
Moderator
User avatar

Joined: Tue Jan 15, 2008 2:31 pm
Posts: 14909
Location: Home sweet home!
The speed and my testing has exposed a bug. I just tested the old version from before I started and it does the same thing. All it means is that you have to relaunch the app between loads of big files or it will run out of memory. IE, there is a grand scale object reference memory leak. Issue here:

http://issues.freeems.org/view.php?id=363

_________________
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!


Sun Oct 30, 2011 9:10 pm
Profile WWW
Moderator
User avatar

Joined: Tue Jan 15, 2008 2:31 pm
Posts: 14909
Location: Home sweet home!
Bug is fixed, as are a few others, and final performance before bed is:

(207161 / 6543 ) * (160000 / 51173) = NINETY NINE TIMES FASTER :-)

3am here, I'm knackered. I also haven't finished with the code. There are a couple of niggles before it's worthy of seeing the world. It's stable and fast, though.

Fred.

_________________
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!


Mon Oct 31, 2011 3:07 am
Profile WWW
Moderator
User avatar

Joined: Tue Jan 15, 2008 2:31 pm
Posts: 14909
Location: Home sweet home!
This is now publicly available in all three major OLV repos! Enjoy!

Fred.

_________________
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!


Tue Nov 01, 2011 1:37 am
Profile WWW
Display posts from previous:  Sort by  
Reply to topic   [ 4 posts ] 

Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group
Designed by ST Software for PTF. ColorizeIt.