Being a good down-stream author!

FreeEMS topics that aren't specific to hardware development or firmware development.
Locked
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Being a good down-stream author!

Post by Fred »

Hello people!

Your job as a down stream author/committer of code is to make your upstream author happy. There are many things involved in this, I'll try to lay out one per post, start a comments thread if you wish to discuss what is written here.

Hopefully this thread will help make life smoother for both upstreams and downstreams and maintain harmony throughout the land! :-)

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: Being a good down-stream author!

Post by Fred »

Staying up to date with your upstream author's latest work before doing any yourself, and indeed talking to them about what you intend to do before doing it, is essential.

Before doing any work, you should always get the latest from your upstream source, and ensure your head commit hash matches theirs! Thus both your local, and your remote, and their remote have the same HEAD commit hash (and probably their local too). You both see the same point of history as "latest".

Assuming that, so far, you're just tracking the upstream, it's as simple as:

Code: Select all

git fetch upstream
git merge --ff-only upstream/master
git push
There are several cases for what happens next:
  1. You do some work, push it, notify the upstream, they've done nothing since their last push, which you based your work upon, they pull your work in with a "git merge --ff-only downstream/master", they then push again - at this point, you both have the same exact HEAD commit hash both in your local repos and in both public remote repos. Just like where we started.
  2. You do some work, so does your upstream, you both push, you notify your upstream, he pulls in your stuff, does a merge, and pushes the merge up as HEAD, you pull that back in, and push it back up, and, again, are back where we started! This case *should* be rare with a single downstream and different time zones, as parallel work itself will be rare.
  3. You do some work, your upstream does some before you finish, they push, you're not ready to push yet. In this case, whether you need to bring in their work or not, you should fetch it, and rebase upon it, so as to minimise the size of future diffs/change logs and the chances of future merge conflicts.
  4. You do some work, and push, and your upstream does more work, and doesn't pull in your stuff, and you need their stuff to carry on, and are going to carry on, you can either rebase your changes, or merge the upstreams work and carry on from there. It is more polite to rebase your work than merge the upstreams work in, however a merge at the top level could be a better option IF and probably only if, you have downstreams of your own who have pulled your commits, if not, then rebasing and force pushing is the best choice.
Ways to minimise the more difficult scenarios (2 and 3) and maximise the clean easy one (1):
  • Avoid spreading work over multiple days
  • Commit often, cleanly, and with small granularity
  • Push as often as you're happy for your upstream to take the changes
  • If sharing temporary commits with some other downstream, or even the upstream for advice, use a temporary branch
Things to note:
  • It is the upstream's job to handle merges
  • It is polite as a downstream to minimise their work load by rebaseing in order to allow him to simply fast forward your work
  • Keeping a clean history is important for traceability of faults, where linear history can be kept, it should be kept
  • The upstream authors repo is the golden source
When working on others projects, i rebase and keep up to date to minimise their work load and dirty history, this is consistent with me being a good downstream author/committer.

Here is an example of bad downstream author behaviour:

Image

This is what happened, blue is upstream, black is downstream:
  1. The downstream author did some work and pushed
  2. The upstream author did some work, but didn't pull in the down stream changes for a while
  3. The downstream author did no further work, but instead of rebaseing their stuff, or waiting for the upstream author to merge it in, they did a merge, which remained private to them and caused their history to fork away from the main line of dev - this was the first mistake. Either waiting, continuing dev on top of their own commits, or rebasing were the correct options.
  4. The upstream author did a bunch more work and then eventually pulled in and merged in the downstream author's work - the upstream author did not know about the downstream merge, and shouldn't have to know about it or consider it, either.
  5. The downstream author then merged the upstream author's work into their repo, creating a second bogus commit and history dirtiness - the correct approach at this point, was to hard reset to before the previous merge that they did, and then fast forward upto where the upstream author was at. IE, making both repos match again. Also, because the downstream author had done no work, their earlier merge was completely pointless and should never have been done.
  6. The upstream then did some more work and pushed
  7. The downstream merged AGAIN instead of resetting and fast forwarding, creating another bogus commit, instead of resetting and fast forwarding.
  8. The upstream then did some more work and pushed
  9. The downstream merged AGAIN instead of resetting and fast forwarding, creating yet another bogus commit, instead of resetting and fast forwarding.
  10. The downstream then did some more work on top of that last bogus commit and requested upstream to take in the FOUR dirty pointless merge commits along with the 2 real commits at the end, which the upstream refused to do. The correct thing to do at this point was to reset back to any upstream commit hash, fast forward from there to the upstream HEAD, and then apply new commits on top of that, before asking for upstream to pull.
This situation does one or more of the following three things:
  • Creates work for the upstream
  • Creates work for the downstream
  • Creates dirty history
If the downstream had just acted appropriately earlier on, most of this would have been avoided and everyone would have had less work to do.

I hope this helps everyone grok the working relationship between up and down stream authors.

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: Being a good down-stream author!

Post by Fred »

Provide the upstream with clean and functional commits!
  • Make sure your work builds cleanly on all target build environments!
  • Test your work thoroughly before committing and pushing!
  • Maintain a close match to the upstream style used!
  • Ensure no debug or temporary comments are left in before commit! (ALWAYS review the diff IN DETAIL before committing!!!)
  • Don't work on aspects of the program that others are working on without asking their permission/advice if you want your work merged in!
  • Don't optimise anything without proper unit testing in place to check for breakage as a result of the changes!
And more, but this will do for now!

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: Being a good down-stream author!

Post by Fred »

It turns out that there is a solution to this!

For all repositories of which you are not the lead developer, that goes for everyone except me for freeems-vanilla firmware, you should use this in your master branch config:

Code: Select all

rebase = true
Such that it looks like this:

Code: Select all

[branch "master"]
  remote = origin
  merge = refs/heads/master
  rebase = true
Likewise, I should have this in my mtx, loader and olv /.git/config files too!

Two more users did this in a minor way in the last few weeks and I did a bit of digging and came up with this.

NOTE: Beware, once rebased you should review your new commits to ensure that they still make sense, as this process can screw them up, still, merging as a down-stream, unless for a good reason and/or with your up-streams agreement isn't OK, so you'd have to do something, and rebase *probably* automates that for you.

Enjoy! :-)

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: Being a good down-stream author!

Post by Fred »

One of the guys on IRC learned his lesson on rebasing being a good idea here:

Image

If you're an upstream, don't accept wildly merged code. It could contain some surprises and muddies the waters of history should you ever need to go back to see what happened and when.

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: Being a good down-stream author!

Post by Fred »

Everyone should run this when they first setup git:

Code: Select all

git config --global branch.autosetuprebase always
Or now, if you don't already have it in your .gitconfig file :-)
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: Being a good down-stream author!

Post by Fred »

A few people have the situation where they're working on more than one thing at a time.

For example you might have your as yet un-upstreamed configuration for your freshly run FreeEMS engine number ? AND you might have some hack that you need for your specific setup AND/OR you might have a new feature that you're working on.

The WRONG way to manage this is to have all of them on one branch intermixed with each other. For example if we take the above to be V = vanilla, C = config, H = hack, F = feature, then this is a wrong sequence of commits:

VVVVVCHCCFFHCCHHFFF

The end result of that is a set of code suitable for running your personal setup, and absolutely and completely useless for any other purpose.

The RIGHT way to manage this is to have four branches like so:

VVVVVCCCCC < a branch for the config
VVVVVHHHH < a branch for the hack
VVVVVFFFFF < a branch for the feature
VVVVVCCCCCFFFFFHHHH < a branch to actually use with each rebased upon the next

That way when I introduce more commits to vanilla like so and this:

VVVVV

Becomes this:

VVVVVVVVVV

Then you can easily rebase each branch individually like this:

VVVVVVVVVVCCCCC < rebased branch with the config
VVVVVVVVVVHHHH < rebased branch with the hack
VVVVVVVVVVFFFFF < rebased branch with the feature
VVVVVVVVVVCCCCCFFFFFHHHH < a branch to actually use with each rebased upon the next

This last "to use" branch can either be reconstructed from scratch by rebasing each upon the next again, or rebased itself, depending on which is more convenient.

The thing is, not only is the mixed up messed up branch more difficult to manage and maintain and rebase, it's also got a snowball's chance in hell of EVER making it into the firmware. On the other hand, the individual branches have whatever chance they have assuming they're clean and well done, etc. IE, by keeping your config and feature separate, you can more easily, more quickly and MUCH more likely get them upstreamed. Upstreamed code is MUCH easier to maintain, IE, you don't, I do.

Should you at this point need to do some more work on the hack, you don't do it on the branch you're using, you do it on the hack branch, rebase that onto the last feature commit before the hack commits, and replace the "for use" branch with that.

At this point you decide that your tune is great, and that I should pull in your config, and I agree, and it's as easy as "git merge --ff-only contrib/config", done.

I know at least two people are in messes like this, and need to clean it up as much as possible and separate it out for integration. Another may have been about to, but I circumvented it, and thought I'd put this here to circumvent any future messes ;-)

Fred.
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
User avatar
Fred
Moderator
Posts: 15431
Joined: Tue Jan 15, 2008 2:31 pm
Location: Home sweet home!
Contact:

Re: Being a good down-stream author!

Post by Fred »

Image
DIYEFI.org - where Open Source means Open Source, and Free means Freedom
FreeEMS.org - the open source engine management system
FreeEMS dev diary and its comments thread and my turbo truck!
n00bs, do NOT PM or email tech questions! Use the forum!
The ever growing list of FreeEMS success stories!
Locked