Page 1 of 1

MAJOR FreeEMS Binary Log Loading Performance Increase

Posted: Sun Oct 30, 2011 6:59 pm
by Fred
I've not finished for the day yet, with about 6 hours to go until I go to bed and 4 or so tomorrow morning before it needs to be online for Ben, and with dinner to cook and eat and clean up after and a shower to have, that's probably 6 hours total left to do work on it, BUT, so far:

(131314 / 12073 ) * (160000 / 51173) = THIRTY FOUR TIMES FASTER :-)

Loads 160k records in 12 seconds or less compared to 51k records in 2.15 minutes before. Also, This is from a file with 207k records and the new code stays up and displays whatever it gets to, as well. The old code falls over when it runs out of memory. The new code detects memory doom and prevents it.

Not a bad bit of performance increase there :-)

Let's see what else I can achieve before I put the code to sleep tonight!

Fred.

Re: MAJOR FreeEMS Binary Log Loading Performance Increase

Posted: Sun Oct 30, 2011 9:10 pm
by Fred
The speed and my testing has exposed a bug. I just tested the old version from before I started and it does the same thing. All it means is that you have to relaunch the app between loads of big files or it will run out of memory. IE, there is a grand scale object reference memory leak. Issue here:

http://issues.freeems.org/view.php?id=363

Re: MAJOR FreeEMS Binary Log Loading Performance Increase

Posted: Mon Oct 31, 2011 3:07 am
by Fred
Bug is fixed, as are a few others, and final performance before bed is:

(207161 / 6543 ) * (160000 / 51173) = NINETY NINE TIMES FASTER :-)

3am here, I'm knackered. I also haven't finished with the code. There are a couple of niggles before it's worthy of seeing the world. It's stable and fast, though.

Fred.

Re: MAJOR FreeEMS Binary Log Loading Performance Increase

Posted: Tue Nov 01, 2011 1:37 am
by Fred
This is now publicly available in all three major OLV repos! Enjoy!

Fred.