FreeEMS General Progress Updates

FreeEMS topics that aren't specific to hardware development or firmware development.
Locked
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

FreeEMS General Progress Updates

Post by Fred »

It occurred to me that the progress happening on this site was hidden by it's distributed nature. You really had to care and read up regularly to know what was going on. That needed to change and this is the result.

This thread is going to be the central place to get your FreeEMS updates from. I'll still jot down daily/hourly/weekly stuff in my diary threads as will Sean, Jared, Dave, Aaron, Ben and others, however I'll try to keep track of them and do a progress update here when something interesting happens. This thread will be locked, however if you think you have something important to share, I can unlock it and allow you to put your progress up yourself. It makes sense to have people take credit for their own progress as much as possible :-)

In this first post I'd like to link to the "I've used it" threads :

I've built the FreeEMS firmware!

I've run FreeEMS Tuner software!

I've run the FreeEMS firmware!

In the future I'll add the following threads :

"I've built a FreeEMS unit by hand!" (not yet available)
"I've run my engine on FreeEMS!" (not yet available)

But not quite yet :-)

Please put your comments here : http://www.diyefi.org/forum/viewtopic.php?f=41&t=554

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: FreeEMS General Progress Updates

Post by Fred »

I guess I should give this thread a base line of where we are at right now.

Firmware

Firmware is nearly ready to be released as version 0.0.18. This will come with significant enhancements not least of which is the nearly complete serial interface. It also now builds on windows 2k and XP, thanks to Daniel for his help making that work. Thanks to Sean for his assistance in getting this far, the flash stuff is great. The holds ups here are defining what that interface is in a formal programmatic way, fine tuning the flash burning and serial stuff to work perfectly and defining exactly what the jobs of the hardware guys are and what they have to work with.

Planned for 0.0.19 is even more serial functionality, changes to the protocol, changes to the internal data structures, additional datalog types and content, better overall structure in the code, more complete docs.

Planned for 0.0.20 is a solid wheel decoder for the Nippon Denso Toyota style inputs and a generic back end to that for other authors such as Sean to write code against.

Of course, nothing is certain, but that's the general goal for the firmware side of things.

Image



Tuning Software

The tuning tool is coming along very well. Congratulations to Aaron and his trusty assistant Ben for those achievements. Plans there are a release on Jan 1 consisting of speed and compatibility improvements and enough functionality to view data coming from the device and adjust/tune what is inside it. That release date, like my schedule is flexible, but that's the general plan assuming he doesn't have too good a time over the Kiwi summer :-)

Image



Hardware

In terms of hardware Jared and Dave are making leaps and bounds. They are still waiting on me to get them some solid information about the pin out mapping and core requirements to work with the firmware cleanly. I've done some work on this but need to do a bunch more too.

Image



All in all, with 6 guys putting significant amounts of their own time into this project, it is now making leaps and bounds. Thank you very much to all involved!

Look forward to more updates here! :-)

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: FreeEMS General Progress Updates

Post by Fred »

New firmware release : 0.0.18 JackTheRipper

The last two releases added real packet based serial communications with extensive error checking and verification. This isn't quite ready for the world as the tuning interface isn't quite finished just yet. It shouldn't be long before we can adjust the parameters the device is using on the fly, and when I say that, I mean almost all of them.

Enhancements over MS due to the serial architecture include, but are not limited to :
  • Datalogging at double the speed over the same speed link.
  • Reset the device via the tuning app without cycling the key.
  • What you send IS what you get (checksummed and verified).
  • Random data can not corrupt memory (no DMA setup, all by logic/lookups)
  • If it works, it works right, though bad connections will make it slow, but that is FAR better than corrupt maps running your engine!
  • Ability to burn small blocks straight to flash without sending the whole page
  • Ability to save part of a block from live tunable ram without saving the rest
Additionally, it now builds on windows 2k and XP, thanks to Daniel for his help making that work. Thanks to Sean for his assistance in getting this far, the flash stuff is great. Finally, the mathematics code is much cleaner with no reduction in functionality.

A demonstration of the serial and flash functionality is available in this video :

http://www.youtube.com/watch?v=4MS_zc8sZxM



Version Control by Git

The firmware source code is now publicly hosted on github with the following URL :

http://github.com/fredcooke/freeems-vanilla

This is an exciting move for a large number of reasons. Not least of which being the ability to fork the code easily and allow me to keep track of what you are doing with it. In this way if you don't want to discuss what you are doing with me until it is done, you don't have to. Of course, it is better if you say "I'd like to work on XYZ" so I can advise whether that is a good idea, and if so avoid that area myself. It is also possible to just download a tar ball or zip of the current repository version of the code at any time, however I urge you to fork it using github and publish your work for the world to see so we can easily integrate it back in.



Code Statistics by ohloh

With the public source code repository we now have a source of genuine statistics about the code base. These are generated by ohloh.net automatically and available for the world to see. So far the results look good. It will be interesting to see how they change over time. If you are interested, take a look here :

https://www.ohloh.net/p/freeems



Issue tracking with MantisBT

As the system grows and diversifies we will face many challenges. One of these is keeping track of bugs and feature requests from users and testers. In that direction Aaron was kind enough to set up and host an instance of Mantis Bug Tracker. This is available for use right now. Anyone can view it, but you must sign up to create bugs, at the moment an administrator needs to allow a member to create bugs, but we could change that in the future if it is too restrictive. Please use this for anything non-trivial that you notice that needs fixing or that you would like to see. Please us this service if you are planning on reviewing the code and/or testing it out on the bench and/or just want to ask for a feature. Also have a read around the forum and check for existing bugs and requests before creating your own, but don't be shy about it. The URL is here :

http://issues.freeems.org



User driven documentation on DocuWiki

Aaron has also setup a user documentation system ah la wiki. The wiki is a DocuWiki which is best suited to source code use. To prevent vandalism by those who are less than enthusiastic about this project you need to have a login. If you feel you have something to contribute (So I can keep coding) you can find it here :

http://wiki.freeems.org



Doxygen code documentation

Jared has been generating automatic documentation from the source code with Doxygen for a few releases now. If you want to get a more visual insight into the source code, you can find them at the following location :

http://docs.freeems.org



Release versions of the code

If you are interested in getting the latest official version of the code to tinker with, you can always find them on sourceforge. It is far superior to get the latest development copy from github as above, however the release versions are guaranteed to more or less just work as described. If you are conservative or want something to use on an actual car in future (once it works as a system) then get your copy from here :

http://download.freeems.org

What is planned?

Hopefully in about 48 hours Aaron will get a tar ball copy of the fully reworked Tuner code to me. This, among other things, is supposed to have threading and a working decoder for the packets, awesome!. Who knows what else it may have in it :-)

For the firmware, 0.0.19 is due out in early January with goals including :
  • A revised serial specification and implementation
  • A fully documented serial interface and data map
  • Restructuring and tidying up of the overall code base
  • Multiple datalog types including super fast scope and logic analyser functions
  • Other documentation like serial implementation, freemind mind map, readme etc etc updated
  • Documented hardware setup design
Of course, all of that is subject to change and re-evaluation, but this is the current plan of attack!

Thanks for reading and your interest in this project and indeed the whole movement!

Regards,

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: FreeEMS General Progress Updates

Post by Fred »

New firmware release : 0.0.19 Doxygenated

Two weeks ago I released the 19th version of FreeEMS into the wild with the name Doxygenated. This release contained almost no further functionality but, significantly, is fairly completely documented in the source code and other sections. I hope to grow this documentation and keep it up to date as the code matures and becomes more functional.

You can find the documentation here :

http://docs.freeems.org/

Note the related pages section which contains some good information on a various aspects of the system. Many of these pages are still in draft form, but as mentioned they will grow and mature with time.

Included in the new documentation scheme are online release notes and change logs :

Release Notes

Change Logs

All previous change log and release notes entries are available linked from the same pages :

Previous Change Logs

Which reminds me, the Makefile no longer works on Windows without cygwin (that didn't last long) AND the Makefile now builds multiple output files with a different wheel decoder each. Only two of these do anything at this time, but that number is sure to grow a bunch in the medium term. I find this situation vastly superior to having to configure one from 20 and being able to mess up the selection and configurations surrounding it. Each file is named appropriately such that you can't possibly put the wrong thing on your controller. Less configuration and smaller code size FTW!

Slow pace of development so far in 2009

Since January I've been working on a new project at work. I was keen to make an impression and do an excellent job so I've been putting in a lot of hours on it to make sure it all goes 100% right.

I've managed to get two successful releases out to the client since then and brought the project into line with all of the best practices that I know of. The release before I joined the team was generated by hand using eclipse and no versioning or tags etc. The next one was built by maven two but manually adjusted to use different library jars and a config manually updated. The last one was built by maven two including libararies with only the manual config change. The next one will be 100% automated meaning better repeatability and reliability and exposure of all metrics surrounding the code.

They seem to be liking what I am doing as I'm now in charge of all technical aspects of the system, officially in charge of 3 of the devs and unofficially in charge of another two.

Hopefully the pressure is off a bit now that we have the system more or less under control. The remaining work is BAU stuff like adding features and removing bugs and making it faster and more usable. This means more time working on FreeEMS again.

The FreeEMS development team

I'd like to welcome Philip Johnson to the project. He has already written a basic 36-1 missing tooth decoder and picked up the use of Git to achieve integration of that into the core branch. Some of you may remember Phil as the man that ported the B&G MS2 code from the broken CodeWarrior compiler to GCC forming the basis of what is now MS2Extra. We should all be thankful to Phil for that achievement as everyone knows MS2Extra J&K >>>> MS2 B&G. I'm really looking forward to working with Phil to produce a really good system that gives people no reason to not use it.

The addition of Phil to the team means that we now have :
  • 3 Firmware devs
  • 3 Tuning app devs
  • 7 People with TA card hardware
Plus the large group of interested parties contributing ideas and constructive criticism to the discussions on the DIYEFI.org forums. One further firmware developer is due to start writing Subaru code any day now and with a bit of luck AbeFM will get stuck into some quality PID code and an NB decoder too ;-)

These numbers are about to increase as four days ago the Helsinki University of Technology's Formula SAE team, "Otaniemi Flying Finns", announced that they intend to use FreeEMS to power and control their FSAE car in the 2009 UK competition at the Silverstone race track. I wish them the very best of luck with their racing endeavours and FreeEMS development too.

In other news, the network graph on github is quite interesting at the moment :

Image

Material contributions

As I'm sure you are aware, running a website and buying all sorts of parts and tools for development costs money. Additionally, my time is valuable and I contribute almost all of my free time to making FreeEMS a success. A few people have decided to acknowledge that and donate to the project cause.

One generous individual anonymously donated 500 USD to the FreeEMS project using the pledgie campaign site and for this I am most grateful. This will make getting the bits I need for development a lot easier. Thank you very much! The same person donated a TA card to our tuning author for testing purposes. Again, thanks for that, most useful!

Alan To from http://www.14point7.com donated two JAW wideband kits (1.03 and 1.041) to me for testing purposes. I intend to assemble one of these fairly shortly and install it on the test vehicle before I try to use FreeEMS to run the engine. Again, thank you very much for this, Alan, it is most appreciated.

If you don't have a clue what you are doing when it comes to software or hardware but have some spare cash and want to help the pledgie site is available to assist you doing this :
I would like to see Jared and Phil have accounts listed there too such that donations can be directed where you feel they should go.

Any funds that I receive will go towards the following things :
  • Web site running costs - domain names and hosting etc.
  • Electronic supplies - Components, PCBs and sensors etc.
  • Providing test rigs to intelligent and interested but poor potential contributors.
  • Tools - meters, scopes, logic analysers and soldering equipment etc.
  • Fuel and replacement engines (assuming the worst) for testing purposes.
In that order. In the future we could add a bounty system where you request a feature, put a price on it and someone can pick that work up and get paid to do it. The whole community benefits from that sort of interaction so it is a very good thing to do. For now though we need to focus on the basics and getting them 100% right.

The next 6 months should be a VERY exciting time for Free and Open Source Software in the automotive arena as months of hard work start to come to fruition for everyone's benefit :-)

Regards,

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: FreeEMS General Progress Updates

Post by Fred »

This thread is SO far out of date that it is not funny. I will attempt to get a post out on the net in the next few months with all the latest on FreeEMS. Rest assured that a LOT of GREAT work has been done in every area of the project! :-) 7 engines so far, more on the way.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: FreeEMS General Progress Updates

Post by Fred »

This is a short summary post of the last three years or progress! :-)

Ages ago, I ran the first ever FreeEMS engine in the USA. I'm in the process of telling that story, you can/will be able to find it here: viewtopic.php?f=41&t=1571

The reasons for no updates are many and varied, however I've been through quite a few life changing experiences in this time, that's probably the biggest one. The other is that we've all been VERY busy making history in every respect.

For those who don't know, I went through a nine month period of effectively being homeless. I started with about 1000usd of reserves and had to conserve cash. I made sure I ate a decent breakfast and light lunch every day, however most of the time I didn't eat dinner. During that time I lost 10kg/22lb of weight and got to know who were really my friends and who were not. Before I weighed in at 83kg/183lb and I'm 185cm/6'1" tall. I initially intended to get work and set myself up in a normal way, however the challenges of existing in that somewhat humiliating way meant that wasn't possible before the summer/xmas holidays a year ago. I did make it to one interview, though, after getting changed into a suit in a shopping mall disabled toilet and leaving the mighty hotel parked elsewhere so as not to be embarrassed by it. Nothing phases me.

It didn't stop me working on FreeEMS, though. Instead I spent a lot of nights parked outside the library using their wifi and idling the engine to keep the laptop alive by inverter power after letting it drain for a few hours (eees are good like that). It didn't stop me from maintaining most of my dignity and a good standard of hygiene, either. I exchanged work on cars for showers and food and when not hanging out with someone favour for favour bathed in the local lake to keep clean. I had quite a few chats with the police who are always suspicious about people parked in strange places late at night. One such chat lasted half an hour an consisted mostly of me offering him server advice and showing him what I was working on; at the end he checked my record for me, still clean, hooray.

About the time the 1000usd ran out I was explaining my situation to an internet-famous friend in the US after which he put up a sticky thread and managed to raise 300usd from about 20 small donations from other car guys. That increased my net worth by 200%. My costs were about 100nzd per week which consisted of 50nzd of 91RON fuel used sparingly mostly for power and warmth, 30nzd of cell phone credit used almost exclusively for talking to my then girlfriend in Spain, and 20nzd on food, mostly milk, cereal, peanut butter and bread. I should mention that the reason that I had only 1000usd in the first place is that i spent everything on tickets to do two things 1) Visit my girl in Spain 2) Attend a friend's wedding in Canada (BC). Love is blind, or so they say. Not more than a few days after the 300usd rolled in, I got an email from paypal and started crying. Another friend with significantly more means than those car guys had dropped me a decent amount of cash, enough to keep coding and bouncing around the city experiencing a seriously alternative life style.

The next phase of the 9 months was when I cut a deal with a friend to, gasp, install a MS2 on his car in exchange for floor, shower, inet and washing machine. He fed me pizza and Isaac's apple and pear cider from time to time too. For various reasons, mostly not my fault, that turned into a fairly extended stay as he scraped together cash to buy the parts for the car that were needed. We ended up taking my truck, which at the time had a dead engine, to his place to fix while working on his car. I put out a call to the Mazda community for a free bottom end to replace mine and a nice guy out in West Auckland came through with a 1.8 variant. Low compression, perfect for boost, or so I thought. About the time that we got the truck running again Marcos came through with the initial build of a few Pumas and I got mine into a usable state. Clearly the truck needed a new EMS. The old one had to go ;-) I got FreeEMS installed on it and running OK, things were looking up. Then I took the ex out for a blat in it to show her what it was capable of and put a rod through the side of the block. It turns out the 1.8 version has pencil rods and un-reinforced pistons, unlike the 2.0 that I was used to abusing. I was depressed for about 24 hours and then had an idea. Perhaps my mighty hotel had a hall sensor instead of points? It did! And so engine 3 was born a few days later :-)

Around this time I received an email from a certain friend asking if a fancy temperature controlled wine cellar would be enough to get me to visit. I asked if he wanted me to visit. He asked if "Gulliver was traveling". I said "if the offer is right!". And so the plan to get work changed into a plan to prepare to leave and go through the US on the way to Spain to be with my girl. Preston helped me a HUGE amount while preparing to leave, I owe him a few favours for that! My leaving party consisted of a car meet with a bunch of car guys in a carpark and Preston with engine 4 on the back of his truck blowing flames and noise into the night air. My sister and a few friends showed up too. It was a good night. The next day I got on a plane and took off not to see my beloved country again for quite some time. I got to the US and started working with Sean on getting his car up and running. Sure enough, two weeks of beating him around the head and we had engine 5. In just three months we had run four more engines! FreeEMS was on the up :-)

I mentioned my girlfriend earlier, she's now my wife. We married in Gibraltar with just a pair of strangers for witnesses, without the approval of her parents, and with everyone I know telling me that I was a fool. It's been a rocky road, with a slew of language issues and cultural differences, however we're both doing well and now living alone together in a little flat, still very much in love. Let's hope that it stays that way! :-)

Since I've been in Spain another 5 engines have run in various places, and unlike the first 5 I wasn't around to help make sure they went well. The system is starting to mature. Sadly for me and the wife, but happily for all of you reading this, the Spanish bureaucracy are glacial at best. I'm still living from donations and her fairly average wage and working 24/7 on making FreeEMS just the best it can be. Things are tight, so if you can spare a few dollars, donations are very much appreciated. I promise to spend them on wine, pizza, transistors, tools, skype calls to other developers and other essential things, like milk and bread and water! :-)

Keeping FreeEMS on the straight and narrow has been a huge challenge for me. I'm a damn good developer, however the bigger challenge, and something I'm only really learning to do, is people management. Guiding and mentoring various people with their aspects of the project takes at least half of my time. I'll say it right now, I'm hugely grateful to everyone that has helped in any way. Whether it be documentation, installing it on a vehicle, coding on a related app, testing software, designing hardware, peer review, moral support and loyalty, a few showers and hot meals, cash or physical assistance, it's been very valuable. Thanks to you all for that!

So, where does the future of FreeEMS lay? If you ask me, right now, the future is EXTREMELY bright! There is some amazing work going into the hardware designs that are coming up to say hello, the firmware is maturing into something reliable and useful, the external tools are usable and growing steadily, and the community is starting to gain strength, breadth and depth. I'm proud of you all! :-)

In terms of the firmware, the current goal is to release 0.2.0 "Enthalpy" the sister release to the earlier "Entropy" release. After that is released the focus is on (0.3.0) a solid tuning interface, and I mean that in the broadest sense, IE, making it easy for people to develop tools to interact with the firmware. After that we're going to roll out Sean's XGATE bit bang work as 0.4.0 with the name of his choice. Following that the focus will be firmly on GPIO code to allow general use of any suitable pin for any purpose. In parallel with all of that the hardware fine tuning and related tool development and stabilisation continues. I expect all of this before next year. Right now there are ELEVEN more engines that could run in the next few weeks or so!!! This year is going to be HUGE for FreeEMS, mark my words.

I hope that brings you all up to date in suitable detail!

Thanks again for your support!

Your utterly fearless leader, Fred.

Comments thread: viewtopic.php?f=41&t=554
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: FreeEMS General Progress Updates

Post by Fred »

I guess it's time for another update in this thread! Another two years have passed and a lot of good progress has been made. Progress has been made in all areas of the project, and it's been made in at least 8 countries by various people, including me in 6 of them!

In the mean time, I've moved back to New Zealand via Sweden, America, Canada, and the UK. It's about time I published something about some of that, so here's a teaser video: http://youtu.be/rGxxafir7n0

The three main areas of interest are of course:
  • Firmware
  • Software
  • Hardware
Firmware

Massive progress in both research and development, however not much of it has been published and integrated for a variety of reasons. Some good, some technical and some bad, some political. I'm working super hard to ensure the stability and growth of the community and project equally . At some point I'll roll out a lot of stuff in a big batch. Sadly, that's just the way it has to be.

Software

Lots of good progress in EMStudio, until recently. Lots of good progress in my tool set recently. And, believe it or not, that's good for anyone who wishes to use the FreeEMS project in future, as that technology flows back to make the project better for everyone. More news on this in good time.

Hardware

The hardware side of things has also been progressing with various board designs and topology optimisations appearing and being validated in the real world. As per my vision, the hardware is of a high quality, and an open and fair nature, with one exception.

So, all in all, things are looking very bright. More news as it comes to hand.

Fred
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
Locked