FreeEMS communications stress testing

A place for small project requests from the team to exist and grow into something fully fledged and useful to the masses.
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: freeems communications stress testing

Post by Fred »

I just found this in the firmware and am deleting the file, I thought this thread was the best place for it. I will edit this post later to clean up the contents into something useful. Later is unspecified.
  • checksum match/no match
  • non special follows esc - escaping incorrect, 0xBB must be followed by 0x55 0x44 or 0x33
  • Actual length wrong for payload - field vs payload, payload vs packet type, packet type vs field
  • start follows start byte
  • ack returned is present and matches
  • source addr = 0 - should never be the case
  • replies to correct address - if addr used, reply should match
  • replies from correct address - matches config on device
  • replies to broadcast - replies to non addressed packets and broadcast packets (similar, but different)
  • no addresses
  • parity errors (simulate with 9 data bits if you can control the serial line well enough)
  • packet over max size
  • flood test
  • send odd payload id
  • receive even payload id
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: freeems communications stress testing

Post by Fred »

And another file's contents, very draft level...
Test packet variation ideas :

What about sending you bad packets to test your logic/flow? Similar set to these I reckon

Should be configuration based so I can add my own as I go
request and expected response pairs? Order important for memory operationt tests
grouping of operations into test units? Still want to know what exactly failed though. hmm

flash writing tests should be optional and in a special section because flash writes are limited!

Test each reset type by read ram, write ram, read ram, reset, read ram

Use the echo function to ensure all the header variations work as expected.
Flags payloadid
ditto + ack
ditto + length
ditto + addrs
ditto + ack and length
ditto + ack and addrs
ditto + addrs and length

addr tests :
0 for source = error
ecu addr for source = error
ecu for target works
non ecu for target fails silently

// full test of every known combo of positive responses in a non combinatorial way plus :
one of each combo of header types and look for matching responses
one of each payload id and look for correct responses
one of each location id for memory actions :
all known ram should work, only full blocks of flash and bigger should work
for writes and burns get, modify, put, get check :
write straight to ram, check only ram
write straight to flash, check ram and flash afterwards
write to ram, tell it to burn that down, check both
table writing of blocks, axis rpm, axis load, cells validation errors – all table types, main and small ones

// ensure the escaping doesn't mess up the checksum check
packet with no escaped bytes and bad checksum
packet with escaped escape and bad checksum
packet with escaped start and bad checksum
packet with escaped stop and bad checksum
packet with escaped escape and escaped start and bad checksum
packet with escaped escape and escaped stop and bad checksum
packet with escaped start and escaped stop and bad checksum

// test that escapes function and produce good data that works and don't mess up the checksum check
// use echo function for this
packet with escaped escape and good checksum
packet with escaped start and good checksum
packet with escaped stop and good checksum
packet with escaped escape and escaped start and good checksum
packet with escaped escape and escaped stop and good checksum
packet with escaped start and escaped stop and good checksum

check that one byte +1 and one -1 don't fail checksum (only error type we can't catch...)

// test that various wrong payload IDs behave as expected
unimplemented payload IDs iterrated through
unknown even payload Ids return correct error
odd payload ids return correct error

// lengths
length doesnt match length header field
length doesn't match payload id and location id for blocks (one wrong sent for each location id)
length wrong because N zero bytes inserted which allow checksum to pass still
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: FreeEMS communications stress testing

Post by Fred »

EssEss hasn't done anything on this in ages, I'm still keen for the comms subsystems to get the once over from an external app like this. If anyone wants to step up and finish what he started, that'd be great.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
EssEss
LQFP112 - Up with the play
Posts: 244
Joined: Thu Sep 10, 2009 9:23 am
Location: Dayton, OH

Re: FreeEMS communications stress testing

Post by EssEss »

bad me!
Ok, I do have this working with the raw api I just never checked it in.

Anyways, after reviewing this thread again, I have some other ideas I'd like to build upon first. They don't line up with this threads context very well but I'm thinking that I want to make a lib first. The thinking here is that if I can wrap it up nicely and expose it via a simple flat/C api then many people should be able to grab it and use in their projects whatever their language of choice is. Busy at the moment, but I'll start getting some freetime about 2 months from now.

Another thing I'd like to shoot for in this lib is utter simplicity. Make + c98 + raw api. Straight for the throat. No autotools, no OO abstractions, no platform portability. All of that can come later, and not necessarily by me.

I keep telling myself 'someday this effort is going to pay off' - Be patient, I'm a WinAPI guy looking to completely divorce myself from that platform asap. ;)
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: FreeEMS communications stress testing

Post by Fred »

"I do have this working" - Like!
"Straight for the throat" - <3
"No autotools" - Amen!
"no OO abstractions" - Hmmmm.
"no platform portability" - Meh.
"I'm a WinAPI guy looking to completely divorce myself from that platform asap." - <3 <3 <3 :-) :-) :-) :-D :-D :-D

I have no further comments!

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
EssEss
LQFP112 - Up with the play
Posts: 244
Joined: Thu Sep 10, 2009 9:23 am
Location: Dayton, OH

Re: FreeEMS communications stress testing

Post by EssEss »

Fred wrote: "no OO abstractions" - Hmmmm.
That was poorly worded. What I mean is that I want no language specific abstractions/dependencies exposed .. a flat api of pure fcalls. I've been doing lots of lib work lately and have learned to appreciate why the good libs are a flat api. Whatever was used behind the scenes is irrelevant and a non-issue - it should be.

The lowest barrier to entry should be low.
Post Reply