emstudio unit testing

Mike's cross-platform FreeEMS tuning application, written in C++ using QT. With Mike's natural coding talent and Fred's intimate requirement knowledge it went from concept to full tuning capability in just 8 weeks!
Post Reply
User avatar
EssEss
LQFP112 - Up with the play
Posts: 244
Joined: Thu Sep 10, 2009 9:23 am
Location: Dayton, OH

emstudio unit testing

Post by EssEss »

I think I'm going to have a go at getting some components of emstudio under test.

I plan on working from the dev branch and starting a new branch in the source tree at emstudio/test. In that dir will go my first commit and I'd like some feedback: https://github.com/essess/gh_markup/blo ... /README.md

If the above sounds good I'll get started. Otherwise, let me hear what you have to say.
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: emstudio unit testing

Post by Fred »

"100% coverage in not a goal" >> "100% coverage is not a goal" ?

Looks pretty good. Some generalisations while generally true, make me crave pointing out exceptions. EG one test fails post change. I have one beast method in my J code with high complexity, but to refactor would break down the coherency which is excellent as is. It has > 100 tests, and a small change could break many tests.

Are you planning to continually rebase against what Mike does, or have him integrate your changes "going forward"?

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
EssEss
LQFP112 - Up with the play
Posts: 244
Joined: Thu Sep 10, 2009 9:23 am
Location: Dayton, OH

Re: emstudio unit testing

Post by EssEss »

Oops ... I was assuming that Mike would take my stuff into his code base and they could live together in perpetuity. I just thought that's how it's done .. I guess it doesn't have to be done that way. Good point - I'll leave it up to him. The ultimate goal was to use this to help with safe refactoring. I don't think I'd want to even attempt any internal cleanup unless this 'lived with the codebase'.

The guidelines are just guidelines that I usually follow. I also wanted the guidelines to be a point of focus when a contributor is on the fence about how they approach their current test problem. I understand what you're saying .. but I'm also willing to bet that you have it noted somewhere why you took the approach you did. I know you go for 100% coverage too and I usually don't. I don't want to waste my time writing test code that depends on internals of the class under test when the code base is so volatile. Once it becomes stable, no one ever goes back and bothers to spend their time filling in more tests to reach 100% - they'd rather spend their time on features. At least that's how I've seen it done in the corporate world .. you may be completely right when it comes to the open source world and time != money. I'm still new to this but don't consider myself a noob. The corporate domains I've worked in are usually small and highly constrained on resources which include dev time and even experience (except for one place and you know what it was).

Of course, Mike is the ultimate decision on everything. This is his baby and I simply want to try my hand at working cooperatively on a project where I'm not in the drivers seat. I thought this was the best thing to do without slowing down his development. He pushes so hard and I didn't want to be a speed bump.
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: emstudio unit testing

Post by Fred »

Time still == money, even in FOSS :-p

You're right that 100% coverage is a stupid thing to aim for. The fact that I aim for it (sometimes) anyway means that I'm just a wretched horrid perfectionist.

FYI, only my lowest level libs do I have anything like 100% coverage, most higher experimental stuff has 0 to 10% coverage as required by necessity. For lower level stuff that I depend on *everywhere*, I deem 100% to not actually be overkill, as mostly the functionality is simple and generic, and the benefit of 100% certainty and reliability is huge.

I grow coverage as stability and certainty of arch sets in. While an arch is still WAY in flux, tests would just slow down dev. Yes, I know this isn't the purist attitude, but it's a pragmatic one, depending on the subject matter, problem domain and scope of work. Sometimes having or writing tests is the only good way to solve a problem. I do this too, horses for courses, right? :-)

I would really like to see Mike pull your stuff in, esp if it's not too intrusive. If not, it'd be pretty rude IMO. Have a look at Eric's branch of the fw, still not integrated (for various reasons). No reason not to pull it in once it's solid and nailed down. Will be happy to have a test dir in parallel for ever and a day.

Fred.

PS, you're not a n00b at all :-p
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
EssEss
LQFP112 - Up with the play
Posts: 244
Joined: Thu Sep 10, 2009 9:23 am
Location: Dayton, OH

Re: emstudio unit testing

Post by EssEss »

I'll wait for Mike's view on all of this and continue w/some code in the meantime.

First up is the stuff between the plugin and the device.
malcom2073
LQFP112 - Up with the play
Posts: 211
Joined: Tue May 01, 2012 4:17 pm
Location: Shrewsbury PA
Contact:

Re: emstudio unit testing

Post by malcom2073 »

I don't read these forums nearly often enough haha.

As I've mentioned to you in IRC, this is an excellent idea. When it comes to code integration, I think as long as we stay in close communication with each other it shouldn't be hard for me to continuously pull your tests and general improvements into the main branch. I think letting me know what you're working on, and me letting you know what I'm working on will help to reduce potential conflicts at merge time.

I like what you've done splitting the repo out into core, lib, plugin, and test. That's pretty much what I has envisioned when I was talking to you on IRC the other day. I should be able to test and then pull into dev as soon as you're satisfied the structure is where you want it to be. Make use of github's pull request feature. This will let me know unequivocally that you are ready for me to pull something in. Push often, and merge often. As you said the code is fairly volatile, and if I change code that you're writing api's for, we want to know sooner rather than later. Don't be afraid to push up work-in-progress stuff. I push code all the time (typically in named branches) that doesn't work, or only half works. This keeps it backed up, and lets other people look at it and provide suggestions and comments on the code itself even if the functionality isn't there yet.

When you're going through things, please feel free to send an email or irc message (I'm almost always on irc, ping malcom2073 and I'll see it) saying "Hey, you fucked up X and Y, you should've done A and B", and give me a chance to ask why and learn your reasoning. EMStudio has been a learning experience for me, and I'm sure there are a large number of things that could be done better or are just plain done wrong :)

The way I see it, as a developer, you're supposed to make me stop (collaborate and listen!), take a moment, and make sure I'm coding in the best manner possible. Tests will take some of my time, but they will lead to better and more maintainable code. If you can make me feel bad about how I've coded something, then I have something to learn and better myself with, and more importantly, better the codebase.
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: emstudio unit testing

Post by Fred »

merge >> rebase! :-)

"make me feel bad about how I've" >> "make me understand how I should have"

Otherwise awesome! :-)

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: emstudio unit testing

Post by Fred »

Thanks, Mike! Rehosted, use freely:

Image

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
Post Reply