Feedback, datalogs still max cpu, 50% python, 40% X, 10% system
IE, stop flogging my console with packet rx and processed :
Code: Select all
00:50:07 comms.handleReceivedPackets DEBUG BasicDatalog packet received and processed
00:50:07 comms.handleReceivedPackets DEBUG BasicDatalog packet received and processed
00:50:07 comms.handleReceivedPackets DEBUG BasicDatalog packet received and processed
00:50:07 comms.handleReceivedPackets DEBUG BasicDatalog packet received and processed
It's great that you are doing it, this is the default though :-) I wanna know if that doesn't happen. You could suggest changing the log level, but last time I tried that didn't work ;-)
Same for these :
Code: Select all
00:50:07 comms.handleReceivedPackets DEBUG AsyncError packet received and processed
Although that should probably get output somewhere with it's number and message, but not just that it was received, do both at once.
I am getting :
Code: Select all
#define payloadLengthTypeMismatch 0x4004
#define payloadLengthHeaderMismatch 0x4005
#define invalidPayloadID 0x4006
Those errors. The funny thing is that I only get some when I have RPM input which makes me think "bug in my code" though not sure how/why... I could load up the 36-1 code now and see if that makes the issue go away. There are definitely issues in yours though.
Code: Select all
00:50:08 comms.default.receive ERROR processReceiveBuffer could not parse buffer. Found a start byte mid-way though packet
No packet was attached! If this was for real, I would want to see the packet contents to find out what went wrong. I don't think it was for real though.
Code: Select all
00:50:08 comms.default.receive ERROR processReceiveBuffer could not parse buffer. Checksum is incorrect! Provided: 0, generated: 155
Again, no packet. It would be nice to know why these are happening be it your fault or mine :-)
One thing that is obviously missing considering the verbosity of these logs is something like :
"Received garbage between packets, dropping..."
You should be doing that a LOT with the byte 0xAA that I spuriously place in front of many packets frequency dependently. This is something else that I don't want to know about, provided the rest works well.
-------------------------------------------------------------------------
It seems to me that you aren't layering your processing properly. Trying to do it all at once in one place or something?
Your logic should be in layers :
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
Packetise stream into chunks :
receive stream watching for start byte and silently dropping all bytes until one is found.
put bytes into single shot buffer of 4200 bytes length until you find a stop byte (this length is worst case for fully escaped packet raw)
once you find a stop byte, place packet on queue and start looking again
If you find a start byte in the middle, print error, print packet received thus far and drop it
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
Then per raw packet you should :
strip out escapes and check the checksum
Do NOT be tempted to do more than this, it is a natural discreet step.
print error with packet contents for bad checksums
place OK checksums on another queue
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
Then per processed packet you should :
check validity of header vs body and place into a data structure of approx 2100 in total length
print error with packet contents for stuff that doesn't check out properly
place OK packets on appropriate queue or queues for further processing by specific code.
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
If you keep it modular and discreet like this you can eliminate bugs. These receive bugs have been around for quite a while now :-)
http://www.extremeprogramming.org/rules/optimize.html
We need it to work 100% before we go making it fast :-)
It doesn't even make sense to try to do any of those aspects at the same time as for each one you are relying on the result of the previous one to perform your work.
*********************************************************
Question, is your logging using the binary strings? or the slow python strings? or what?
Logging is the bottle neck right now despite the errors.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
idea : if you want to know when you do those basic tasks right, print a single specific char out for each time with no newline. This will significantly reduce load on X from scrolling the large xterm and still provide debug info in the short term while we need it.
how about :
'.' for datalog packets
'*' for leading garbage
debug should be a printed out line with my string
error should be a printed out line with my code and matching string
yes, it would mean legit messages were displaced to the right, but its a temp measure and the best of several evils.
Thoughts?
I'm thinking debug is getting ditched in favour of error packets and custom mapping files for the gui.
GASP :-)
Fred.
Under warranty so will take it back this week hopefully!
