RomRaider

As far as defs are concerned it's probably best for one individual to manage them to ensure a level of accuracy. Many can contribute of course.
The database makes it easier to manage the dependencies within the definitions. I can't imagine how you'd do it any other way.
If you really want to understand the def structure we should have a chat about it.

If you send me the updates I'll incorporate them as I do for others. That's how we got an English and German version of Logger defs...
Here's the latest logger defs to work from.
viewtopic.php?t=6681

I was thinking more along the lines of:

What type of database?

Is the schema published?

Read only access on a public host such as http://www.freemysql.net/ or http://www.mysqlforfree.com/ or similar? postgresql version? Whatever.

Get the info out there so others can look at it, play with it, interact with it, hack on it, etc.

Then maybe they can send you an sql patch script that you can check and include.

You can and, I agree, should, still manage it as a single point of control, but if the info is more out there, then it's more likely people will play with it and contribute :-)

Maybe some sort of disk based db could be git controlled? sqlite for example.

Just throwing ideas around :-)

Fred.

I was considering hosting it on the RR site with the appropriate tools/web pages to query the DB. But I'm not ready for that step yet. I've given the people working on defs the info they need to send me to updates for the logger DB. It's quite simple really.
I need the parameter E#, the address and the data length, one item per line. And in less than a minute I can update the logger def and publish the new version.

I hope to do the same with the ECU def once I get the DB built.

OK, let the sparks fly. Hopefully I did it right. I think I was using the force on a couple of button clicks but something happened. Looks like my name shows up on that diagram now at least .

Dale, I took a copy of the latest experimental logger definitions, consolidated my very simple changes with that, and sent a push request. Feel free to incorporate if you like.

EDIT: I meant to add I still do not mind if the logger.xml and ecu_defs.xml get removed from the romraider master... I just thought this would be a good way to experiment with the new process since the changes are so simple.

You sent the project the pull request, consequently, it came to me, as it's still in my name for the time being. You don't send a project a pull request, a pull request is something that goes to a human being. You did the equivalent of shouting into your office "Please pull my code, <company name>" :-) Do something more akin to "Please pull my code <name of person at next desk>". In this case, no need as Dale will see these posts and fetch your change and consider integrating it (using merge --ff-only or rebase). Next time, though, send your pull request to Dale. Or just post in the related thread. A pull request is nothing to do with git, its just a convenient way to communicate a specific need :-)

In my fork I have deleted the defs completely. I'll look at your changes and most likely, if I like them incorporated them into the backend database. I have a few other additions to the Logger def to publish as well.

Currently I've been keeping the most recent logger def here with the history of changes. It's easy for people to find with the forum "search" feature using their CAL or ECU ID.

I'm waffling on whether to have a separate definition repository or continue to host a single archive of known working defs on the forum. We want to make sure that what we publish as "working" or "official" is something that has gone through a phase of review. This was why a sub-forum of "experimental" definitions was created. At some point in the future those experiment defs would graduate to official status and be included in the published archive.
I'm not keen on the idea of the defs being copied/cloned all over the place and modified without any review process.

Hmmm.

Firstly, it'll always be obvious who/where the root source of the data is, and people will go there by default, unless someone legitimately does better.

Secondly, you have to give people a little credit to decide what they want to use on their vehicle by choice rather than by "this is all there is" non-choice.

Thirdly, if someone else steps up to the challenge and starts doing a great high quality job helping with it, that will become obvious too, people will come to know that person X does a good job and can be trusted too.

Fourthly, is there any way of positively identifying a particular definition via say a hash of the data structure and content? If not, there's nothing to say they didn't get a file from their friend and load it up and claim it was yours to avoid lack of support. If not, how about adding something that's visually available in the UI for even a noob to see. "what's in the top right corner of your xyz window?" or similar.

At least if the output of your DB was git managed then you could clearly monitor changes and diffs and see exactly what others have done or not done. You could also pin it back to a specific version if they're using git to access them. "which hash are you on?" is a universal way to know exactly what someone is using.

The forum searchability thing is good, but managing the output in a solid reliable diffable way doesn't need to change that. It could/can/will remain the primary source for them anyway. You would just be giving keen hackers the option to more easily help you and lighten your load.

My 2c :-)

Fred.

I did think about the track-ability that Git would provide such that anyone could clearly see the changes over time/versions. I see that as a good thing.

They have that now from the experimental def forum. I'm thinking more for the official definitions.

Again, that is visible in the experimental def forum. Once the work has been validated then it could move to an official release.

That was something that I was considering a few months back. The question is how to implement it over the data that applies to each an every ROM definition separately and not the entire and unrelated portions of a definition.
This would be easier to implement once we move to a directory tree rather than an all encompassing def for the ECU defs. The Logger def is one file and could easily be hashed as a version stamp.

This is useful for tracking any bugs in the official defs. The source for defs is the ROM on which the def is based. People wouldn't really be changing the official def without first working through some experimental cycle first.

As for hacking... anyone working on a definition is not hacking the definition, they are hacking the ROM. What they find in the ROM is what ends up in the definition to be shared and reviewed before becoming official.

I think we are agreeing in an indirect way.

BTW: GitHub doesn't provide a view of diffs on an XML file for some reason.

Cool :-)


To me, official == in your repo or the RR repo, nothing more. Experimental in my model would be a git clone/fork on github, and some changes that sit on some random user's username account that no one looks at except by that person telling people about it, say, on the forum, in the experimental section, but managed properly. They become official when you cherry pick the commit into the official branch/repo/account. That simple. then it's a git diff hash hash to see EXACTLY what that person changed, and a gitg view to see what they based the changes on. < valuable.


This would blur the line and allow validated to be something more automatic via community review and if they did such a great job and were trusted, people would use it verbatim without it having to take up your time. If they got THAT good you could add them as a collaborator on the official account and cut your work further.


I don't understand well enough to comment, but my initial thought is "hash of the final structure post merge with dependencies". You could have a separate hash for "just this, and the references to other objects" such that you could also tell if it hadn't changed, and some bug fix in a dependency had.


Sure, of course, but they're going to be basing their work on an existing setup and changing it as they discover more stuff, right? Why not give the experimenters the power of proper change management too.


"hacking" in the loose sense of making changes to data or source to get a different result to what the original upstream developer got, whether it be inline with what the upstream wants or not. I hack on readme files too :-)


Right, and sharing and reviewing is going to be a lot easier with proper tools handling it. Not all git repos are equal. Eg, if I started publishing changes to RR and you didn't have them in yours, people don't know who the **** I am and wouldn't use it. That simple :-) IE, official == your repo or the one in the RR account, and only core people control what goes in there. People will know and respect this.


Mostly, yes. Partly, no :-)


Link?

Fred.

Like I said, let the sparks fly. This is all new to me, I've never used github.com at all.

I blindly followed your instructions from a previous post which said "Press pull request button".

When doing that I don't recall seeing any option to specify a pull request to a specific person but will watch out for it next time, it was late.

Do you mean that I should have made the pull request to Dale's cloned repo and not the main one?

EDIT: I figured it out how to ask Dale to pull to his clone instead of RomRaider:master and updated my pull request. Sorry for missing that detail.

OK, thanks Dale. I only did it this way for now to 'try out' github etc... it seems to have worked but I need to tweak my approach as pointed out by Fred already. I did notice that you have already deleted these in your fork and agree that is the correct thing to do.


Yes, that is the exact thread where I got the latest xml file to consolidate with my local changes.



Let me try to add some thoughts to the mix for you to consider...

I think that people are free to, copy, edit and distribute defs and do what they want no matter what, so it cannot be controlled.

What would help is having a) an official 'released' def version and, as you mention, b) a process to get changes incorporated into it so that one can share local changes with others, if they are deemed useful.

You currently handle 'a' with a forum thread. I think this has the downside that discussions about indiviudal, incremental changes are buried within that thread somewhere. It would be nice to be able to see the discussion for each proposed change up to version 56 but I cannot see that so easily right now.

It would be better to make a repo for them because that will automatically handle 'b' and you don't have to manually track the history as you do now. 'b' is handled because discussions about proposed changes could happen on github.com when people execute a pull request and link that to a romraider thread for discussion when the request is put in. Or we could even try and set up Gerrit reviews for such changes.

The downside to that approach is maybe not everybody will be comfortable using github, Gerrit, etc.. but if one is comfortable editing that xml then maybe it is OK... .

EDIT: I write this before reading the back and forth you had with Fred... I think he and I are agreeing. The manual process of managing the defs right now is not optimal.

Seems to work for the recent build.xml changes:

https://github.com/dnwillia/RomRaider/c ... 030b6b40ab

What is not working for you?

I 100% agree with this.


This is the situation I was in. I had some simple changes to propose to the logger.xml. Current process is basically to post them in the thread and let people discuss. A better way would be to use git and associated tools for it.


I thought I might just reply to this that the changes I proposed to logger.xml had nothing to do with hacking the ROM, just changes to logger units, but could still be useful for the community at large. Not all changes are necessarily about newly found ECU parameters.


I 100% agree with Fred here as well.

Danny, don't worry about the pull req stuff, it doesn't matter much :-) Do try to base changes on the latest from who ever is most important at the time (Currently and likely for a long time: Dale) such that A they are relevant and B they are easy to include, though :-) Good on you for getting involved!


I think you did a fine job. Sometimes I miss the bus when it comes to articulating things, missing things that I assume are obvious, but which aren't really. Thank you!


Exactly!


Yes, we are, this time! :-) And I'm glad to see it.

Great comments, Danny!

Fred.