RomRaider

Just look at the logger.xml file in the pull request, can you see a diff of that on GitHub?

I understand what you are trying to say/do.
I think you misunderstand the development of the def. 99% of the time it's new work which is then laid on top of a foundation. Sure that foundation could be a previous def but it is in many cases all new discovery work, not modifying and adding to some existing def, but creating a new one.

There's one def per ROM, changes may be bug fixes and possibly future new discovered tables. New tables causes a ton of work for any/everybody to back port those updates. This is the point of the database, so as not to manage each def in isolation and duplicate work where the common items can be defined once and be inherited as required.

Is it a better way? Think about who uses development tools?
I don't think the user community is going to be follow a discussion outside of a forum thread. And they are the ones that do most of the proofing. Remember who your audience is, who you are doing the work for.

Wait, so the assembling the final image happens at your place using SQL and not inside RR at load time? Maybe the latter isn't practical? What quantity of defs are we talking?

Even if the defs are 100% new, git can recognise copies and partial copies and treat the data as such. It's full of golden information. The fixes and so on, even if generated from your database and scattered around from a single db change, are still useful in terms of a non-you view of what changed and why.

You could/should put a full SQL backup dump into every update commit to the git repo too. That way if you vanish tomorrow someone else can carry on. Not being negative, stranger things have happened :-)

Fred.

The image of the car's ROM is on the users computer, the ECU defs help the Editor in visualizing the image in a format the user can understand and manipulate. The changes to the image are written back to the users computer for later downloading (flashing) to the car's computer, no db is used here.
Have a look here in the zip file for the defs we are discussing, one def for each ROM.
viewtopic.php?t=3808
The XML is effectively the "database" that the Editor uses. Currently RomRaider Editor uses an all in one huge XML file. The zip above is what we want to move to.

For the Logger XML def, I just normalized the XML data into an SQL db to make updates and changes much much much easier to manage with a couple of supporting Perl scripts. I'll eventually get there with the ECU defs too.

I could pop a backup of the Logger db and the Perl scripts I made in a separate repo for safe keeping.

Git is absolutely a better way, and I don't see how that moves the discussion outside of a forum thread. The only folks using the developer tools would be those that are editing xml. IMO if you can't handle git, you probably shouldn't be messing with the xml. Anyway, wouldn't it be possible for someone to just fork the git and edit through the web GUI then do a pull request?

The forum method sort of works for consolidated xml like ecu_defs and logger.xml, but it is messy.. especially so when you have multiple people contributing to the definition.

The forum method really breaks down when you have individual def files. It makes things difficult for the end user, especially tuners who are using multiple experimental definitions. Throw in experimental bases and it becomes downright confusing to some people and a pain in the ass to all... especially when you consider the alternative of a direct link to a git dump for final and experimental def packs.

I have some more things to discuss about definitions, so I'm going to start a new thread just for them: viewtopic.php?f=14&t=8443

Ya I see what you mean. Maybe it's just to many changes. I could try a smaller change to one of the other XML files and put in another push request.

I 100% agree. If a person is capable of retuning a car and manually editing an XML file then using git to post changes is not that much of a hurdle.

I'm not sure if you can edit through the web interface. Fred might know.

Yeah, you can, but I frown upon it pretty strongly for my stuff.

I need to come back and post more here later. I think there have been some misunderstandings on my part.

I guess this is the point I wasn't getting across clearly. Moving the files from a forum thread dedicated to a specific def over to Git does not solve the problem. You will still have individual def files spread around before they are confirmed working and can go into an all encompassing archive.
I'm still leaning towards a forum thread for the testing phase as it's more central to the users. Once some testing time has pasted then yes it can be archived to git.

I feel a skype call coming on :-)

Well, I don't see floating alpha defs as a problem to be solved. I see it as a part of the process.

By having tested beta definitions in a repo, you eliminate the chance that someone unknowingly picks up an alpha definition and has a problem. You also streamline the process for end-users looking for stable defs for new cars. Link the repo in the program or download page and you're done.

Also, definitions should be included with the program. Having to download after install is absurd, it should work out of the box. I can understand wanting the user to have the latest, but there has got to be a better way that isn't an inconvenience.

I see more frequent releases in the future :-)

An RR release could be something like "RR-<RR version>-<defs version>.filetype"


I'd take it a step further than there not being a problem and would say that you won't have alpha files spread around, you'd have the clearly marked in a commit with a comment and a diff that showed you what they were/weren't. Don't treat the archive like a final thing, the master branch can be the final thing, sure. Others can do their development/discovery in a traceable commentable diffable way. It's gotta be good for everything.


Same as above, git is perfectly suited to development and testing and incremental growth. Think about the tree, the leaves can go off and float around in the breeze all they like, the trunk still grows slowly and solidly all the while.

Re the database, do I (now) understand correctly that only the logger stuff is in a database? The def stuff is ONLY in a HUGE and unmanaged (with version control) file, right?

^ that I didn't realise at first. I thought you were talking database for the defs too. Or perhaps you are, but at some point it sounded like you weren't. /me is lost.

Fred.

While Fred, Merp and I all think that ecu and logger xml definitions would be better managed in git, in the end, the way this is approached is really up to you. So, take the advice for what it is too

The logger and ecu defs are the main interface to RomRaider for most people and I think it makes sense to put these on the same footing as making changes to the java code. I see no reason that they should be treated differently, they are part of it, integral to making it work, and should be treated equally.

I still think that even for the testing phase the process would benefit from the structure that git offers. Basically people would share their logger and ecu defs by pulling from each others repo's. You might make some logger changes and you say 'hey dan, fred, merp pull these changes and try them out' We pull them, test them out, tell you they are ok and then you push them to the master... life goes on.

The benefit is that all of this can all be captured as part of the development processes just like java code changes, well documeted, searchable, etc... I could go on.

Dan

I agree and disagree. I agree that they should be managed the same way. But they're not on the same footing exactly, they have a different release cycle, so there are two things:

1) Getting the latest greatest defs because there could be 50 def releases between app releases (or vice versa).
2) Version range of defs using a specific app-compatible format, if it changes, the app needs to know that it can't use older/newer ones.


Yep, and unofficial ones could live in a branch called "doNotUseTheseUnlessYourNameIsXYZ" and other creative descriptive goodness :-)


Amen. There is likely a lot of valuable info being effectively lost here. 100% agreed.