Unit Testing

A place to discuss software and code that isn't automotive related. Free and open source preferred but not compulsory.
Post Reply
User avatar
Spudmn
LQFP112 - Up with the play
Posts: 232
Joined: Thu Feb 10, 2011 12:27 am
Location: Auckland, NZ

Unit Testing

Post by Spudmn »

I am starting to teach myself Unit Testing.

I am pretty new to the concept of TDD. I have a few questions that I hope someone can help with.

The code I am writing is for embedded microcontrolers. They range from the PIC-32MX to small 8051. I am programming them in C and have started to look at the Unity framework. http://sourceforge.net/projects/unity/

One of the problems I have is that the Unity framework uses up too much RAM on my little 8051.

I could run my unit testing on an x86 and GCC but this may not show up any bugs in the 8051 compiler.

Is it better to run the unit tests on native hardware? If so I guess that I could start hacking down Unity to try and get it to fit.

Another question is, when you are testing embedded code that writes to the registers of the micro do you simulate the register with your test rig so that you can check the value that was written.

Any other pointers, hints, tips or advice will be very much appreciated.
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: Unit Testing

Post by Fred »

EssEss is your man on this!

It's not easy, but it is possible.

I think assuming that your compiler is broken is a bad move. If you go to the trouble of making your stuff cross compilable into x86 C with proper typing and encapsulation etc, then it should be valid to run the tests on the PC. Note that you'll get one binary per thing you want to wrap, and you'll have to fake (mock) all of the firmware specific stuff in it in a reasonable way. I deemed the work involved for PC side testing too much for the time being though.

What I've done is setup a very simple function wrapper test serial call. This allows me to over-the-wire test anything with a clean params/return style. I can send all of the params in the packet, and specifiy what I expect to get back from the device as a return. This works for functions that return error codes too. This approach leverages the solid serial comms in freeems and wouldn't be as suitable without that, though would still work. I only have one testable function at the moment, but at some point it will cover all non-ISR code that doesn't interface with the hardware and all hw interface code will be thin and simple.

I had another thought about testing ISR code the other day, which I may attempt in future but isn't ready for discussion just yet :-)

Link: http://www.methodsandtools.com/archive/ ... .php?id=59

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Spudmn
LQFP112 - Up with the play
Posts: 232
Joined: Thu Feb 10, 2011 12:27 am
Location: Auckland, NZ

Re: Unit Testing

Post by Spudmn »

Thanks Fred

I am starting to get the hang of unit testing. I am cross compiling the code using GCC on a PC. After doing this for a while I can see the advantages. It speeds up development as you are not always compiling, re-flashing you hardware and testing. With the unit testing I can just run the code and see if the tests pass or not. On the PC I also have access to better debuggers. Often the micro vendors build there own debuggers that don't integrate well with Eclipse.

It also encourages you to use int-types (uint16_t) so that you are not making assumptions about integer sizes.

In the end you still need to test it on the native hardware, but you have more confidence with your unit tested functions.
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: Unit Testing

Post by Fred »

Code: Select all

FreeAir:src fred$ grep -rn " int " .
./inc/freeEMS.h:93: * int        16 bit DO NOT USE! (current compile flags make this 16 bits, but a change of flags could will change your program if you use this because they will all be 32 bit all of a sudden)
./inc/structs.h:64: * - int        16 bit DO NOT USE! (current compile flags make this 16 bits, but a change of flags could will change your program if you use this because they will all be 32 bit all of a sudden)
;-)
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
EssEss
LQFP112 - Up with the play
Posts: 244
Joined: Thu Sep 10, 2009 9:23 am
Location: Dayton, OH

Re: Unit Testing

Post by EssEss »

usually with a simulator, you have free reign over the entire address space, therefore, in setup/teardown you can prepopulate register values and check them afterwords. good for fuzz testing too.

I usually never go to this degree, only because of the time vs. reward. I use the s~*t out of asserts() too. Most of my testing efforts are spent on everything above my HAL. I tend to always run on a simulator because 1) I can redirect I/O easily 2) I want to use the exact same code generated that runs on the final platform.

I use CUnit.

And two treasured books of mine are Working Effectively with Legacy Code and xUnit Test Patterns: Refactoring Test Code.
User avatar
Spudmn
LQFP112 - Up with the play
Posts: 232
Joined: Thu Feb 10, 2011 12:27 am
Location: Auckland, NZ

Re: Unit Testing

Post by Spudmn »

Since my last post on Unit Testing I have been using it a lot more. I don't clam to be an expert but I thought that I would post how I unit test embedded code.

I use GCC or MinGW (on windows) to run the tests. I find this very useful, as debugging is easier and faster on a PC. There is no need to flash the micro and some silicon vendors don't supply very good debug tools, if any.

I use unity, which can be found here http://throwtheswitch.org/white-papers/unity-intro.html

I take the unit C file that I want to test and include it in my test C file. I use the “exclude from build” feature of eclipse to make sure the it doesn't get compiled twice. I then write mock functions and variables that allow the unit to compile without errors.

Then write your tests and use the assert macros to check you get the results you want.

I don't know how to use cmock.c yet and I am not sure if you need to have a separate executable for each unit or there is a way to test all units in one executable.

I have put an example on https://github.com/Spudmn/Missing_teeth_Unit_Test

I hope you find this useful.
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: Unit Testing

Post by Fred »

Sweet! I've still not looked at what you sent me - I guess this supersedes it?
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Spudmn
LQFP112 - Up with the play
Posts: 232
Joined: Thu Feb 10, 2011 12:27 am
Location: Auckland, NZ

Re: Unit Testing

Post by Spudmn »

Yes. It is closer to how I do Unit Testing and I added the results from the flat battery logs.

I hope that is useful to a number of people. I had trouble finding examples when I wanted to start using Unit Testing.
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: Unit Testing

Post by Fred »

That's because few do embedded unit testing! You're a pioneer ;-) Or close to it, at least. PM EssEss and get him in here to shake you up, unless I already suggested that and you already did that.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
Post Reply