I've not finished for the day yet, with about 6 hours to go until I go to bed and 4 or so tomorrow morning before it needs to be online for Ben, and with dinner to cook and eat and clean up after and a shower to have, that's probably 6 hours total left to do work on it, BUT, so far:
(131314 / 12073 ) * (160000 / 51173) = THIRTY FOUR TIMES FASTER :-)
Loads 160k records in 12 seconds or less compared to 51k records in 2.15 minutes before. Also, This is from a file with 207k records and the new code stays up and displays whatever it gets to, as well. The old code falls over when it runs out of memory. The new code detects memory doom and prevents it.
Not a bad bit of performance increase there :-)
Let's see what else I can achieve before I put the code to sleep tonight!
Fred.
MAJOR FreeEMS Binary Log Loading Performance Increase
MAJOR FreeEMS Binary Log Loading Performance Increase
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
Re: MAJOR FreeEMS Binary Log Loading Performance Increase
The speed and my testing has exposed a bug. I just tested the old version from before I started and it does the same thing. All it means is that you have to relaunch the app between loads of big files or it will run out of memory. IE, there is a grand scale object reference memory leak. Issue here:
http://issues.freeems.org/view.php?id=363
http://issues.freeems.org/view.php?id=363
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
Re: MAJOR FreeEMS Binary Log Loading Performance Increase
Bug is fixed, as are a few others, and final performance before bed is:
(207161 / 6543 ) * (160000 / 51173) = NINETY NINE TIMES FASTER :-)
3am here, I'm knackered. I also haven't finished with the code. There are a couple of niggles before it's worthy of seeing the world. It's stable and fast, though.
Fred.
(207161 / 6543 ) * (160000 / 51173) = NINETY NINE TIMES FASTER :-)
3am here, I'm knackered. I also haven't finished with the code. There are a couple of niggles before it's worthy of seeing the world. It's stable and fast, though.
Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
Re: MAJOR FreeEMS Binary Log Loading Performance Increase
This is now publicly available in all three major OLV repos! Enjoy!
Fred.
Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!