View unanswered posts | View active topics It is currently Mon Sep 24, 2018 3:26 am



Reply to topic  [ 6 posts ] 
Generating Data Source Files 
Author Message
Moderator
User avatar

Joined: Tue Jan 15, 2008 2:31 pm
Posts: 15181
Location: Home sweet home!
Recently, with inspiration from Dave, I have developed a fairly comprehensive mechanism for live from-the-running-firmware data layout interrogation. This is fantastic, because now all it needs is the actual field by field structure to be defined outside the ECU and the tuning software has all it needs, without overly complex unmaintainable files.

The next problem is one of maintenance also:

Duplicate data definitions.

If I keep my existing C files and write matching xml/js/json files then for every change I manually have to update both locations, make no typos and get it all perfect. If not, Dave (and users) will get angry and have less faith. It is also dirty and not at all OAOO to do this. Thus, I won't be doing this.

So, theree are two options:

2) Parse the source and generate XML/js/json files from it
1) Create and maintain XML/js/json files and parse them and output valid C (and possibly stripped down XML/js/json for use by tuner authors/programs.

Given that option 1 (Intentionally mis ordered/numbered to show bias) is far less error prone and better defined and more flexible, I strongly believe that is the best approach.

So, why a thread? I want it to be cross platform so the build will work on mac/lin/win equally well. We *could* revert to cygwin for win, but I'd prefer it stayed native too.

The first thing that comes to mind is XSLT

http://en.wikipedia.org/wiki/XSLT

But that would mean using XML, which I'm not a huge fan of. I'm not a huge fan of json either, but...

Is there something like this for json... searches :

http://goessner.net/articles/jsont/
http://goessner.net/download/prj/jsont/

How would I go about integrating that into the build process (which tools!) such that the C is generated before the compilation process?

Two or three or four transformations would need to be done:

Source data description > C headers
Source data description > C data defintions
Source data description > Simplified data description (structure)
Source data description > Simplified data description (data)

The last two aren't needed if the original rich file is suitable for parsing into an app, which it probably should be. The last one isn't needed if we don't mind pulling such default data out of a live ecu instead.

I'm not sure about the best way to proceed here, so I'd like some help, please! I know what I want, but I don't know how to achieve it. Hackers, scripters, germans, help me out! :-)

Fred.

_________________
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!


Sun Jan 09, 2011 12:41 am
Profile WWW
LQFP112 - Up with the play

Joined: Sun Apr 06, 2008 6:30 pm
Posts: 100
I like option 1 better. (use some markup language and generate source from that). All of this can be done with makefiles,and fredcooke, you're probably better at it than I am.

The markup will need to be fairly descriptive and that's a battle for another day, bu the parser for it has to be cross platform. I like XML because I'm most familiar with it, (though i probably use it very incorrectly, but that can change) MTX already uses it and has libxml2 which is full featured, documented and avail on all platforms...

_________________
-- David
http://www.megatunix.com
http://megatunix.sourceforge.net


Sun Jan 09, 2011 1:08 am
Profile
LQFP112 - Up with the play

Joined: Sun Apr 06, 2008 6:30 pm
Posts: 100
Depending on your language of choice for the markup, either develop a tool in C, or in somescripting langauge (perl?) that you compile FIRST as part of the build process that can then do the parsing/code generation in the next stage, and finally compile all that into objects and distributables that the tuning software will need. Ideally I'd like to see the distributables web-available so the tuning software if on a web-enabled system can query and download them as needed, which simplifies the distribution model somewhat.

This is similar to how QT projects are made, they use "moc" (meta object compiler), to generate code from other files as part of the build process, qmake does a lot of the behind the scenes stuff to generate the makefiles and rules (good reference), but the model for freeems is similar and can work with regular makefiles without needing a tool like qmake.

_________________
-- David
http://www.megatunix.com
http://megatunix.sourceforge.net


Sun Jan 09, 2011 1:16 am
Profile
LQFP112 - Up with the play
User avatar

Joined: Thu Sep 10, 2009 9:23 am
Posts: 244
Location: Dayton, OH
I definitely vote for #1 - gen code from data + template
Have you ever tried to make a c parser ?


Sun Jan 09, 2011 4:17 am
Profile
Moderator
User avatar

Joined: Tue Jan 15, 2008 2:31 pm
Posts: 15181
Location: Home sweet home!
There is one in the bin directory that was being used briefly, but not anymore. I should probably purge it.

_________________
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!


Sun Jan 09, 2011 11:26 am
Profile WWW
Moderator
User avatar

Joined: Tue Jan 15, 2008 2:31 pm
Posts: 15181
Location: Home sweet home!
viewtopic.php?f=8&t=1007 - The data format itself. Please contribute if you have any idea what you're talking about, or not.

Fred.

_________________
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!


Sun Jan 09, 2011 10:56 pm
Profile WWW
Display posts from previous:  Sort by  
Reply to topic   [ 6 posts ] 

Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group
Designed by ST Software for PTF. ColorizeIt.