Patching MTX for our use

MegaTunix is a cross-platform tuning application written and maintained by David J. Andruczyk. Discuss all things MTX here!
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Patching MTX for our use

Post by Fred »

Dave has been doing some really good work redesigning MTX for MS2 usage. Hopefully his work has been flexible enough for us to look at using it for FreeEMS without excessive work and or changes. Key to that will be flexibility of serial communications as I really like his interface ideas and of course the cross platform nature of it. If it is relatively easy to put in some simple controls and map them to some really simple commands, tackling serial communications at some basic level is on the agenda pretty soon. It's important to avoid legal troubles that our serial protocol is unique to us. Also, we want checksumming and address-less communications. It could end up that we write a new tool in a more basic language instead depending on the work involved in stripping/changing MTX for our needs.

In the mean time for simple comms, MT will do the trick. A basic setup MAY be able to be done to communicate with this using a different set of rules, or it may not. If not, we'll have to do more work earlier.

A couple of people have commented to me that using C++ for such a tool is a gross overkill and distasteful (I would have done it in java or C++ myself, but I'm a noob ;-) ). One of them suggested wxpython as a good easy gui based language with good built in widgets etc. I had a brief play with that a month or so back and it did indeed seem easy to pop up a window and get things done without too many chores.

If we don't/can't use Dave's fine tool, we will need to design the structure of our tool in tandem with the protocol that we create.

Just a few thoughts for you all to ponder and discuss.

The list of changes in the up and coming next MTX release :-) (this link will break sooner or later as it is Lance policy to heavily prune the forums at msefi...) :

http://www.msefi.com/viewtopic.php?p=192602#192602

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
ababkin
LQFP112 - Up with the play
Posts: 215
Joined: Tue Jan 15, 2008 5:14 pm

Re: Patching MTX for our use

Post by ababkin »

C++ distasteful for this = BS
of course it depends on 1. Who is using it and 2. How it's used and 3. Coding style and how it's documented

I've used C++ and wxWidgets to write a very nice cross-platform front end to some tuning software and didn't find it gross at all. if anything, it'll be faster than java/python
Legal disclaimer for all my posts: I'm not responsible for anything, you are responsible for everything. This is an open and free world with no strings attached.
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: Patching MTX for our use

Post by Fred »

if anything, it'll be faster than java
Don't count on that, the Java VM optimises on the fly depending on which byte code chunks are being run most often etc. It can often end up faster than even well written C++ code.

A highly talented and experienced friend of mine once set about rewriting a Java tool in C++ to try to prove that he could make it run faster (Large enterprise tool). In the end, what he found that while he could make certain parts run a little faster, overall because of the on-the-fly optimisations that the VM does, the Java version was superior in execution speed.

Java has an unearned bad rep for speed that has come about from bad design and implementation on a very capable platform.

You learn something every day ;-)

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
GartnerProspect
LQFP112 - Up with the play
Posts: 160
Joined: Tue Apr 08, 2008 9:14 pm

Re: Patching MTX for our use

Post by GartnerProspect »

I'm not a coffee drinker and I still prefer my software not to be written in coffee as well... :D

Though I must say I'm a big fan of both MTX and MLV so I guess I'm an addict now. Helps that my hardware has improved in speed by a factor of 10 since the old days.
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: Patching MTX for our use

Post by Fred »

MTX = C
MLV = NON-FREE Java

I don't mind mlv and use it, but I know a few of our members are very anti it for a number of reasons.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
GartnerProspect
LQFP112 - Up with the play
Posts: 160
Joined: Tue Apr 08, 2008 9:14 pm

Re: Patching MTX for our use

Post by GartnerProspect »

Ahh. For some reason I thought MTX was Java... I guess I've been in Vegas too long.

It's been months since I even saw my own car :|
FlappySocks
QFP80 - Contributor
Posts: 41
Joined: Tue Apr 29, 2008 1:47 pm
Location: Hampshire, UK
Contact:

Re: Patching MTX for our use

Post by FlappySocks »

My vote goes to the Mono platform, and C#. It's my choice of language these days. I do Java but mostly for embedded devices.

Mono is cross platform, and supports GTK+, which is what is used on MTX, which could speed up development of an alternative application.

I'm a bit stacked up with work of my own write now, but I would be keen to look into it soon.

Stuart.
Image
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: Patching MTX for our use

Post by Fred »

Mono is fine with me :-)

Mono ~= Java anyway ;-)

I'm not going to say anything about anyone developing GPL tuning software in any language except "good on you".

If someone knows a certain language and is willing and able to do the work, let them do it :-)

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: Patching MTX for our use

Post by Fred »

Update, a clever man that goes by the name of cpuTorture has been patching mtx to work with his GPL ecu firmware project and thus making it a little more flexible and extensible. Dave took that guy's work and changed it some more and incorporated it into main line mtx. I imagine the serial side of things will still need some significant refactoring, but that is a good start. Once the git repo is up, i'll be much more tempted to play around with it :-)

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: Patching MTX for our use

Post by Fred »

For anyone interested with spare time here is a slightly edited conversation about mtx and it's structure with relation to freeems :
(01:46:57) djandruczyk: The places to look at Interrogator/comms.xml, comms.c serialio.c, threads.c and helpers.c, might also want to look at dispatcher.c as well.
(01:47:11) fredcooke: thank you :-)
(01:48:00) djandruczyk: Basiclaly it's roughly a message passing sort of design (sorta). look for io_cmd and study how it's used..
(01:48:08) fredcooke: ok
(01:48:15) fredcooke: message passing is FTW
(01:48:21) fredcooke: only way to roll, really
(01:48:25) djandruczyk: it looks kinda ugly (well really ugly if you're not familiar with it), but it is pretty flexible
(01:48:58) fredcooke: front end code could chunk up the stream into packets, pass them to something to handle them and you're away laughing, probably.
(01:49:06) fredcooke: i'll take a look when i get a chance, probably next week.
(01:50:07) djandruczyk: the dispatcher and thread related stuff is more because of GTK+isms and threads. only one thread can do GUI calls, so it uses asyncqueues (think pipe) to pass messages around between threads and gtk timeouts (think like a quickly running cron job under the gui thread), thus keeping the UI fully responsive even during long blocking calls
(01:50:44) fredcooke: beautiful :-)
(01:51:53) djandruczyk: basically, to do the sending side, you'd create an object via initialize_outputdata(), tehn bind any kind of data you want into it via OBJ_SET(), the XML determines the mapping between textual names and whatnot to "assemble" the datastream. It might need some minor additions to do the checksumming and escaping, but that should be trivial...
(01:53:10) djandruczyk: return side stuff, will just require an additional "helper" post function for your various commands to decode the returned data.., serail reader may need some work though, as mtx is a "send then get" setup, and that may need some fiddling to figure out when to stop reading,
(01:53:41) fredcooke: send then get...
(01:53:44) djandruczyk: mtx would work best for an ecu that can be polled, not streamed..., as the serial io isn't written to run in a continuous mode as MS never streamed
(01:53:58) fredcooke: that'll work fine for everything cept the datalogs - which cnan be turned off
(01:55:25) fredcooke: will the streaming screw mtx over in the first place, though, before you get a chance to send a "stop streaming" packet out?
(01:55:47) fredcooke: is the receive thing in a thread of its own?
(01:55:47) djandruczyk: fred I'd recommend you look at the XML and those files and let it sink it, the design is complex and kinda crufty (due to MS'isms)
(01:56:09) djandruczyk: not sure if it'll hose it or not..
(01:56:13) fredcooke: sure, no problem. just thought i'd keep asking while you were on a roll :-)
(01:56:44) djandruczyk: the comms are in their own thread, but not polling continually, that part is procedural, that thread is largely idle until a message comes in telling it what to do.
(01:56:54) fredcooke: it if was in its own thread and it could chuck out a new packet to the real time display any time it got new data, it'd be fine, i imagine.
(01:57:05) fredcooke: hmm
(01:57:37) djandruczyk: prob is it's incompatible with MS design, would prefer mtx kept working on both (and other ECU's, torture) if possible
(01:58:18) djandruczyk: MS design is a pure polling desing, i suppose mtx could detect the stream and send soem magic sequence to flip itto polling mode, but i'd need HW to try that
(01:58:53) djandruczyk: I should redo that funciton to read a couple times a second apart ot see if it's actually gettign a counter, would be a more accurate test
(01:59:23) fredcooke: i can change default to be off
(01:59:27) fredcooke: but its been great for testing
(02:03:34) djandruczyk: for interrogation, theres a file called Interrogator/tests
(02:04:12) djandruczyk: the interrogation though is pretty simplistic, so that may be a problem with freeems
(02:04:44) djandruczyk: as it's kind of a bitbang setup, "send this char", look for response", for ech test, then run comparisons as per interrogation profile (i.e. match, regex, whatever)
(02:05:30) djandruczyk: ii forget if the interrogator works if you send and ecu a multi-byte string or not..
(02:06:31) djandruczyk: i should rework that so the interrogator can just use io_cmd like the rest of mtx can, and thus it can use any command defined in comms.xml, will have to think about that...
(02:07:47) fredcooke: once you fail all the ms tests (no ms attached) you can "just" send a/some packet(s) (can be precanned) asking for versions etc and figure out the variant based on that.
(02:08:27) djandruczyk: that breaks the interrogators current design though (i think).
(02:09:11) djandruczyk: too bad you cant create a piece of softwar that EMULATES your ecu and i could run it and plugin my two serial ports togetehr with a null modem cable...
(02:10:03) djandruczyk: mtx sorts does that with it's network multi-user mode. IT emulates the MS serial comms as a state machine, only over a TCP/IP socket..
(02:10:16) fredcooke: becaue the device is so picky about what it recieves, the chances of getting even a single byte out of it (cept datalogs) without the right thing going into it is slim to none. for example, in wellington with aaron, i had a cheap adatper... and it was corrupting data... and the packets were ALL dropped and it was like it wasn't hooked up. cutecom showed the data there and with manual inspection it WAS corrupt.
(02:10:19) fredcooke: and yes dave
(02:10:23) fredcooke: i could definitely do that
(02:10:27) fredcooke: sounds like fun even
(02:10:51) fredcooke: along the lines of my firmware unit testing idea (cross compile the code to both pc and device)
(02:10:58) fredcooke: i'll look into it for you.
(02:11:02) djandruczyk: not to slow your dev efforts though...
(02:11:06) fredcooke: that would be very cool.
Git repo is up, and fairly up to date at :

http://github.com/fredcooke/MegaTunix

Select the pre 0 9 20 branch - it's most up to date.

In terms of breaking the stream into packets, there is example code here :

http://stuff.fredcooke.com/freeemsParser.c

And example data here :

http://stuff.fredcooke.com/comms20100421201222.bin

More detail on packets can be found in my code, in Aaron's code or maybe in some documents, though most are out of date.

Keep in mind that various aspects of comms will be changing, but, the basic start/stop/escape/crc setup won't be changing. Just maybe packet headers and what you get reply wise etc.

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
Post Reply