View unanswered posts | View active topics It is currently Thu Jul 27, 2017 5:30 am



Reply to topic  [ 28 posts ]  Go to page 1, 2, 3  Next
FreeEMS Serial Protocol 
Author Message
DIP8 - Involved

Joined: Tue Jul 08, 2014 7:44 pm
Posts: 16
Hello Everyone,

I would like to thank Fred and Andy for their help so far. Following their advise, I'm posting this in the Forum.

I am trying to implement a small application capable of reading and displaying information from a FreeEMS and hopefully capable of speaking to it. Something along the lines of EMSTune. My Development board arrived recently and it just required changing the permissions on ttyUSB0 and was ready to go. I initially did something other than the required:

Baud Rate: 115200
Data Bits: 8
Stop Bits: 1
Parity: Odd

And hence I was getting garbage, I think... After getting advise from DeuceEFI about the correct settings, I was able to do the capture I've attached to this post. I wanted to see if we can use this post to come up with documentation for the protocol so that others like me can enhance the potential of FreeEMS via applications that connect to it.

Wilbert

Attachment:
raw_feed.png
raw_feed.png [ 116.15 KiB | Viewed 1319 times ]


Thu Jul 24, 2014 5:03 am
Profile
Moderator
User avatar

Joined: Tue Jan 15, 2008 2:31 pm
Posts: 14667
Location: Home sweet home!
wilbertcr wrote:
I would like to thank Fred and Andy for their help so far. Following their advise, I'm posting this in the Forum.

You're welcome, however seeking help privately actually damages the community and project by spending resources in a very inefficient and non-nreusable way. Please refrain from that in future, and when approached by others to help them, ask them to do the same.

Quote:
My Development board arrived recently and it just required changing the permissions on ttyUSB0 and was ready to go.

Changing permissions on the device is the wrong approach. Add yourself to the dialout group, log out, log back in, and proceed as normal.

Quote:
I initially did something other than the required:

Baud Rate: 115200
Data Bits: 8
Stop Bits: 1
Parity: Odd

And hence I was getting garbage, I think... After getting advise from DeuceEFI about the correct settings, I was able to do the capture I've attached to this post.

That was the first thing I told you "100% sure you have correct serial settings?", 5 days ago. I then asked you to post your query and you did not do that until now. Apparently you moved onto the next target instead. Disappointing. This would have been resolved quickly in public.

I've posted many times about how to read FreeEMS streams in a robust fashion. But here it is again:

  • Break into packets
  • De-escape escaped bytes (can be combined with above for efficiency)
  • Check checksum and discard if wrong, counting/recording/signaling an error (EDIT: can also be combined with above, at the moment)
  • Read header flag byte using flags to know what comes next
  • Read various other header bytes using them as appropriate
  • Read body of packet and process as required depending on type and desired usage

What you do with the data, and how you route/handle packets/nacks/errors/etc is up to you, and will show yourself as a good or bad developer fairly quickly :-)

Fred.

_________________
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!


Thu Jul 24, 2014 9:35 am
Profile WWW
DIP8 - Involved

Joined: Tue Jul 08, 2014 7:44 pm
Posts: 16
Quote:
You're welcome, however seeking help privately actually damages the community and project by spending resources in a very inefficient and non-nreusable way. Please refrain from that in future, and when approached by others to help them, ask them to do the same.

I couldn't agree more that communication is better through the open forum. In my case, it was all the secrecy you(Fred) imposed that led me to inquire via PM. The same secrecy that got me a warning for posting something I shouldn't in the Open Forum... I was just trying to play it safe :(
Quote:
Changing permissions on the device is the wrong approach. Add yourself to the dialout group, log out, log back in, and proceed as normal.

Got it, thanks for that.
Quote:
Apparently you moved onto the next target instead.

Incorrect, I wrote to both you and Andy the exact same day.
Quote:
Break into packets
De-escape escaped bytes (can be combined with above for efficiency)
Check checksum and discard if wrong, counting/recording/signaling an error (EDIT: can also be combined with above, at the moment)
Read header flag byte using flags to know what comes next
Read various other header bytes using them as appropriate
Read body of packet and process as required depending on type and desired usage

Thanks for that, I'm sure it'll be useful down the road, however that's a bridge I haven't crossed yet. Here's what I'm trying to figure out at the moment:
According to what I read(I may be wrong, please be patient), the following is a complete packet:

AA 01 01 91 00 62 B7 CB BA 0D 03 6B 41 80 04 47 04 69 01 37 B7 CB 00 0C 00 0D 00 06 00 00 00 00
00 00 00 00 00 00 04 47 38 00 80 00 01 32 84 54 00 91 53 33 00 00 00 BC 0E 36 0E F2 00 00 47 0C D6
00 00 00 00 D0 00 00 00 00 00 00 00 00 12 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 35 19 A8 CF 00 00 F3 CC

If we break it down we get:

0xAA : Start of packet
0x01 : 7th bit is on. I know Bit 0 - Has Length, Bit 1 - Is Negative Acknowledgment, Bit 2 - Has Sequence. What are Bit 3, 4, 5,6 and 7?
0x0191 : Basic Datalog packet???
0x00 : Mode Stop

From what I've read so far, I don't know how to interpret the following stream:

62 B7 CB BA 0D 03 6B 41 80 04 47 04 69 01 37 B7 CB 00 0C 00 0D 00 06 00 00 00 00
00 00 00 00 00 00 04 47 38 00 80 00 01 32 84 54 00 91 53 33 00 00 00 BC 0E 36 0E F2 00 00 47 0C D6
00 00 00 00 D0 00 00 00 00 00 00 00 00 12 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 35
19 A8 CF 00 00

F3: Checksum
CC: End of packet

Finally, I don't want to be just a taker here. I want to offer my language skills. Spanish is my native tongue(I was born and lived in Costa Rica until my 31st birthday) and so I can help with documentation and other stuff that may require it.You're an awesome Engineer Fred, but your Spanish sucks :)


Thu Jul 24, 2014 9:03 pm
Profile
Moderator
User avatar

Joined: Tue Jan 15, 2008 2:31 pm
Posts: 14667
Location: Home sweet home!
wilbertcr wrote:
In my case, it was all the secrecy you(Fred) imposed that led me to inquire via PM.

I said, very specifically:

1) Do not mention X (someone else's project!) on the forum because it would be detrimental to the purpose of that project and thus negatively affect anyone using it.
2) Do not link out of date files publicly as I do not want them becoming defacto standards.

It was perfectly possible to post tech questions about packet parsing without doing either of these things! :-p Not another word about it in this thread, get on topic, and stay there.

Quote:
Quote:
Apparently you moved onto the next target instead.

Incorrect, I wrote to both you and Andy the exact same day.

This is even worse and quite rude, he's a busy man trying to get out a lot of orders to a lot of people. And I'm busy too, obviously. Now dealing with this shit. Just drop it and forget it and don't do it again. End of discussion.


Quote:
Thanks for that, I'm sure it'll be useful down the road, however that's a bridge I haven't crossed yet.
Quote:
Thanks for that, it's useful right now as that's the bridge I'm currently crossing.

Fixed!

Quote:
If we break it down we get:

0xAA : Start of packet
0x01 : 7th bit is on. I know Bit 0 - Has Length, Bit 1 - Is Negative Acknowledgment, Bit 2 - Has Sequence. What are Bit 3, 4, 5,6 and 7?
0x0191 : Basic Datalog packet???
0x00 : Mode Stop

Bit zero, not bit seven, is 1. Bit seven is zero, as are bits six through one. This is your mistake. No idea what you mean by "mode stop" that's purely erroneous. 0x0062 is hex of 98 which is packet body length.

Quote:
From what I've read so far, I don't know how to interpret the following stream:

62 B7 CB BA 0D 03 6B 41 80 04 47 04 69 01 37 B7 CB 00 0C 00 0D 00 06 00 00 00 00
00 00 00 00 00 00 04 47 38 00 80 00 01 32 84 54 00 91 53 33 00 00 00 BC 0E 36 0E F2 00 00 47 0C D6
00 00 00 00 D0 00 00 00 00 00 00 00 00 12 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 35
19 A8 CF 00 00

F3: Checksum
CC: End of packet

That's because it's configurable. Look at the struct definitions and the data log definition in the configuration of the device. This is your meta data, for the time being. Use your favourite scripting lang to parse these things into a usable format and use it.

wilbertcr wrote:
You're an awesome Engineer Fred, but your Spanish sucks :)

ROFL! La verdad, amigo! :-) Pero normalmente bastante, entonces no me la pela :-p

Fred.

_________________
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!


Fri Jul 25, 2014 12:35 am
Profile WWW
DIP8 - Involved

Joined: Tue Jul 08, 2014 7:44 pm
Posts: 16
Thank you Fred.

Quote:
That's because it's configurable. Look at the struct definitions and the data log definition in the configuration of the device.


Can you be more specific, where can I find these structs?

Wilbert


Last edited by wilbertcr on Fri Jul 25, 2014 2:25 am, edited 1 time in total.



Fri Jul 25, 2014 2:23 am
Profile
Moderator
User avatar

Joined: Tue Jan 15, 2008 2:31 pm
Posts: 14667
Location: Home sweet home!
http://builds.freeems.org/firmware/dev/ ... ource.html < some of them

http://builds.freeems.org/firmware/dev/ ... 9ec83.html < data/config

http://builds.freeems.org/firmware/dev/ ... tml#l00160 < in particular

Fred wrote:
wilbertcr wrote:
You're an awesome Engineer Fred, but your Spanish sucks :)

ROFL! La verdad, amigo! :-) Pero normalmente bastante, entonces no me la pela :-p

Mi otro amigo en Galicia dice que no es correcto. Entonces intentarlo otra vez:

Pero normalmente lo suficiente, entonces me la pela! :-p Mejor? :-D

_________________
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!


Fri Jul 25, 2014 3:50 am
Profile WWW
DIP8 - Involved

Joined: Tue Jul 08, 2014 7:44 pm
Posts: 16
Thanks man!

This would be a "corrected" version:
"Mi otro amigo en Galicia dice que no es correcto. Entonces **voy a** intentarlo otra vez:
Pero normalmente **es** suficiente, entonces me la pela! :-p Mejor? :-D"

But yeah, better than the first one :)


Mon Jul 28, 2014 6:48 pm
Profile
Moderator
User avatar

Joined: Tue Jan 15, 2008 2:31 pm
Posts: 14667
Location: Home sweet home!
Cheers. Progress decoding packets?

_________________
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!


Mon Jul 28, 2014 8:16 pm
Profile WWW
DIP8 - Involved

Joined: Tue Jul 08, 2014 7:44 pm
Posts: 16
It's going hehe. So far, I've figured out that PayloadID 0x0191 is a Basic Data Log and that this packets are created in commsCore.c, more specifically in the function populateBasicDatalog(void) --please correct me if I'm wrong so far--. Within populateBasicDatalog, I'm a bit confused regarding what it is that:

for(chunks=TablesB.SmallTablesB.loggingSettings.firstChunk;chunks<chunkLimit;chunks++){
unsigned short localSize = TablesB.SmallTablesB.loggingSettings.logChunks[chunks].size;
confSize += localSize;
if(confSize > 2048){
confSize -= localSize;
break;
}
memcpy(TXBufferCurrentPositionHandler, TablesB.SmallTablesB.loggingSettings.logChunks[chunks].address, localSize);
TXBufferCurrentPositionHandler += localSize;
}

is copying exactly, meaning that, unlike with the header, I don't know what each byte represents in the payload.


Mon Jul 28, 2014 11:04 pm
Profile
Moderator
User avatar

Joined: Tue Jan 15, 2008 2:31 pm
Posts: 14667
Location: Home sweet home!
Please, work to the interface, do not read the code. The files I linked you above tell you everything you need to know. Ask questions about them. The implementation details are none of your concern and could change in any random way at any time without affecting you in the slightest.

_________________
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!


Tue Jul 29, 2014 12:35 am
Profile WWW
Display posts from previous:  Sort by  
Reply to topic   [ 28 posts ]  Go to page 1, 2, 3  Next

Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group
Designed by ST Software for PTF. ColorizeIt.