Page 1 of 3

FreeEMS Serial Protocol

Posted: Thu Jul 24, 2014 5:03 am
by wilbertcr
Hello Everyone,

I would like to thank Fred and Andy for their help so far. Following their advise, I'm posting this in the Forum.

I am trying to implement a small application capable of reading and displaying information from a FreeEMS and hopefully capable of speaking to it. Something along the lines of EMSTune. My Development board arrived recently and it just required changing the permissions on ttyUSB0 and was ready to go. I initially did something other than the required:

Baud Rate: 115200
Data Bits: 8
Stop Bits: 1
Parity: Odd

And hence I was getting garbage, I think... After getting advise from DeuceEFI about the correct settings, I was able to do the capture I've attached to this post. I wanted to see if we can use this post to come up with documentation for the protocol so that others like me can enhance the potential of FreeEMS via applications that connect to it.

Wilbert
raw_feed.png

Re: FreeEMS Serial Protocol

Posted: Thu Jul 24, 2014 9:35 am
by Fred
wilbertcr wrote:I would like to thank Fred and Andy for their help so far. Following their advise, I'm posting this in the Forum.
You're welcome, however seeking help privately actually damages the community and project by spending resources in a very inefficient and non-nreusable way. Please refrain from that in future, and when approached by others to help them, ask them to do the same.
My Development board arrived recently and it just required changing the permissions on ttyUSB0 and was ready to go.
Changing permissions on the device is the wrong approach. Add yourself to the dialout group, log out, log back in, and proceed as normal.
I initially did something other than the required:

Baud Rate: 115200
Data Bits: 8
Stop Bits: 1
Parity: Odd

And hence I was getting garbage, I think... After getting advise from DeuceEFI about the correct settings, I was able to do the capture I've attached to this post.
That was the first thing I told you "100% sure you have correct serial settings?", 5 days ago. I then asked you to post your query and you did not do that until now. Apparently you moved onto the next target instead. Disappointing. This would have been resolved quickly in public.

I've posted many times about how to read FreeEMS streams in a robust fashion. But here it is again:
  • Break into packets
  • De-escape escaped bytes (can be combined with above for efficiency)
  • Check checksum and discard if wrong, counting/recording/signaling an error (EDIT: can also be combined with above, at the moment)
  • Read header flag byte using flags to know what comes next
  • Read various other header bytes using them as appropriate
  • Read body of packet and process as required depending on type and desired usage
What you do with the data, and how you route/handle packets/nacks/errors/etc is up to you, and will show yourself as a good or bad developer fairly quickly :-)

Fred.

Re: FreeEMS Serial Protocol

Posted: Thu Jul 24, 2014 9:03 pm
by wilbertcr
You're welcome, however seeking help privately actually damages the community and project by spending resources in a very inefficient and non-nreusable way. Please refrain from that in future, and when approached by others to help them, ask them to do the same.
I couldn't agree more that communication is better through the open forum. In my case, it was all the secrecy you(Fred) imposed that led me to inquire via PM. The same secrecy that got me a warning for posting something I shouldn't in the Open Forum... I was just trying to play it safe :(
Changing permissions on the device is the wrong approach. Add yourself to the dialout group, log out, log back in, and proceed as normal.
Got it, thanks for that.
Apparently you moved onto the next target instead.
Incorrect, I wrote to both you and Andy the exact same day.
Break into packets
De-escape escaped bytes (can be combined with above for efficiency)
Check checksum and discard if wrong, counting/recording/signaling an error (EDIT: can also be combined with above, at the moment)
Read header flag byte using flags to know what comes next
Read various other header bytes using them as appropriate
Read body of packet and process as required depending on type and desired usage
Thanks for that, I'm sure it'll be useful down the road, however that's a bridge I haven't crossed yet. Here's what I'm trying to figure out at the moment:
According to what I read(I may be wrong, please be patient), the following is a complete packet:

AA 01 01 91 00 62 B7 CB BA 0D 03 6B 41 80 04 47 04 69 01 37 B7 CB 00 0C 00 0D 00 06 00 00 00 00
00 00 00 00 00 00 04 47 38 00 80 00 01 32 84 54 00 91 53 33 00 00 00 BC 0E 36 0E F2 00 00 47 0C D6
00 00 00 00 D0 00 00 00 00 00 00 00 00 12 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 35 19 A8 CF 00 00 F3 CC

If we break it down we get:

0xAA : Start of packet
0x01 : 7th bit is on. I know Bit 0 - Has Length, Bit 1 - Is Negative Acknowledgment, Bit 2 - Has Sequence. What are Bit 3, 4, 5,6 and 7?
0x0191 : Basic Datalog packet???
0x00 : Mode Stop

From what I've read so far, I don't know how to interpret the following stream:

62 B7 CB BA 0D 03 6B 41 80 04 47 04 69 01 37 B7 CB 00 0C 00 0D 00 06 00 00 00 00
00 00 00 00 00 00 04 47 38 00 80 00 01 32 84 54 00 91 53 33 00 00 00 BC 0E 36 0E F2 00 00 47 0C D6
00 00 00 00 D0 00 00 00 00 00 00 00 00 12 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 35
19 A8 CF 00 00

F3: Checksum
CC: End of packet

Finally, I don't want to be just a taker here. I want to offer my language skills. Spanish is my native tongue(I was born and lived in Costa Rica until my 31st birthday) and so I can help with documentation and other stuff that may require it.You're an awesome Engineer Fred, but your Spanish sucks :)

Re: FreeEMS Serial Protocol

Posted: Fri Jul 25, 2014 12:35 am
by Fred
wilbertcr wrote:In my case, it was all the secrecy you(Fred) imposed that led me to inquire via PM.
I said, very specifically:

1) Do not mention X (someone else's project!) on the forum because it would be detrimental to the purpose of that project and thus negatively affect anyone using it.
2) Do not link out of date files publicly as I do not want them becoming defacto standards.

It was perfectly possible to post tech questions about packet parsing without doing either of these things! :-p Not another word about it in this thread, get on topic, and stay there.
Apparently you moved onto the next target instead.
Incorrect, I wrote to both you and Andy the exact same day.
This is even worse and quite rude, he's a busy man trying to get out a lot of orders to a lot of people. And I'm busy too, obviously. Now dealing with this shit. Just drop it and forget it and don't do it again. End of discussion.

Thanks for that, I'm sure it'll be useful down the road, however that's a bridge I haven't crossed yet.
Thanks for that, it's useful right now as that's the bridge I'm currently crossing.
Fixed!
If we break it down we get:

0xAA : Start of packet
0x01 : 7th bit is on. I know Bit 0 - Has Length, Bit 1 - Is Negative Acknowledgment, Bit 2 - Has Sequence. What are Bit 3, 4, 5,6 and 7?
0x0191 : Basic Datalog packet???
0x00 : Mode Stop
Bit zero, not bit seven, is 1. Bit seven is zero, as are bits six through one. This is your mistake. No idea what you mean by "mode stop" that's purely erroneous. 0x0062 is hex of 98 which is packet body length.
From what I've read so far, I don't know how to interpret the following stream:

62 B7 CB BA 0D 03 6B 41 80 04 47 04 69 01 37 B7 CB 00 0C 00 0D 00 06 00 00 00 00
00 00 00 00 00 00 04 47 38 00 80 00 01 32 84 54 00 91 53 33 00 00 00 BC 0E 36 0E F2 00 00 47 0C D6
00 00 00 00 D0 00 00 00 00 00 00 00 00 12 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 35
19 A8 CF 00 00

F3: Checksum
CC: End of packet
That's because it's configurable. Look at the struct definitions and the data log definition in the configuration of the device. This is your meta data, for the time being. Use your favourite scripting lang to parse these things into a usable format and use it.
wilbertcr wrote:You're an awesome Engineer Fred, but your Spanish sucks :)
ROFL! La verdad, amigo! :-) Pero normalmente bastante, entonces no me la pela :-p

Fred.

Re: FreeEMS Serial Protocol

Posted: Fri Jul 25, 2014 2:23 am
by wilbertcr
Thank you Fred.
That's because it's configurable. Look at the struct definitions and the data log definition in the configuration of the device.
Can you be more specific, where can I find these structs?

Wilbert

Re: FreeEMS Serial Protocol

Posted: Fri Jul 25, 2014 3:50 am
by Fred
http://builds.freeems.org/firmware/dev/ ... ource.html < some of them

http://builds.freeems.org/firmware/dev/ ... 9ec83.html < data/config

http://builds.freeems.org/firmware/dev/ ... tml#l00160 < in particular
Fred wrote:
wilbertcr wrote:You're an awesome Engineer Fred, but your Spanish sucks :)
ROFL! La verdad, amigo! :-) Pero normalmente bastante, entonces no me la pela :-p
Mi otro amigo en Galicia dice que no es correcto. Entonces intentarlo otra vez:

Pero normalmente lo suficiente, entonces me la pela! :-p Mejor? :-D

Re: FreeEMS Serial Protocol

Posted: Mon Jul 28, 2014 6:48 pm
by wilbertcr
Thanks man!

This would be a "corrected" version:
"Mi otro amigo en Galicia dice que no es correcto. Entonces **voy a** intentarlo otra vez:
Pero normalmente **es** suficiente, entonces me la pela! :-p Mejor? :-D"

But yeah, better than the first one :)

Re: FreeEMS Serial Protocol

Posted: Mon Jul 28, 2014 8:16 pm
by Fred
Cheers. Progress decoding packets?

Re: FreeEMS Serial Protocol

Posted: Mon Jul 28, 2014 11:04 pm
by wilbertcr
It's going hehe. So far, I've figured out that PayloadID 0x0191 is a Basic Data Log and that this packets are created in commsCore.c, more specifically in the function populateBasicDatalog(void) --please correct me if I'm wrong so far--. Within populateBasicDatalog, I'm a bit confused regarding what it is that:

for(chunks=TablesB.SmallTablesB.loggingSettings.firstChunk;chunks<chunkLimit;chunks++){
unsigned short localSize = TablesB.SmallTablesB.loggingSettings.logChunks[chunks].size;
confSize += localSize;
if(confSize > 2048){
confSize -= localSize;
break;
}
memcpy(TXBufferCurrentPositionHandler, TablesB.SmallTablesB.loggingSettings.logChunks[chunks].address, localSize);
TXBufferCurrentPositionHandler += localSize;
}

is copying exactly, meaning that, unlike with the header, I don't know what each byte represents in the payload.

Re: FreeEMS Serial Protocol

Posted: Tue Jul 29, 2014 12:35 am
by Fred
Please, work to the interface, do not read the code. The files I linked you above tell you everything you need to know. Ask questions about them. The implementation details are none of your concern and could change in any random way at any time without affecting you in the slightest.