Managing Experimental Definitions with Git

dschultz · **Posted:** Wed Sep 12, 2012 9:37 am

NSFW wrote:

Edit one line, click "generate," done.

It's not about saving CPU time, it's about saving people time.

It'll take work to write the DB-to-XML dumper, but I expect that it'll be well worth the time that it saves.

I concur. For the logger.xml def I can now produce six versions (two languages and three preferred units of measure) in the "click" of a button and it takes about 15 seconds. I can guarantee that all six versions are accurate too as they all came from the same source.

Merp · **Joined:** Thu Jul 23, 2009 1:46 pm **Posts:** 863

NSFW wrote:

Merp wrote:

It may only require a one line edit for a table change, but that's only telling half the story. It still has to [...]

Sure there's lots more that needs to happen after the one-line-edit, but the rest of that work can be automated.

Edit one line, click "generate," done.

It's not about saving CPU time, it's about saving people time.

It'll take work to write the DB-to-XML dumper, but I expect that it'll be well worth the time that it saves.

An XML to XML editor accomplishes the same things with an equally information rich dataset and less complexity. (after switching to proper scaling.xml and a separate language file)

nsfw · **Joined:** Wed Nov 22, 2006 10:23 pm **Posts:** 2565

Merp wrote:

An XML to XML editor accomplishes the same things with an equally information rich dataset and less complexity. (after switching to proper scaling.xml and a separate language file)

You could also call this "normalization."

If we all agree that the data currently in ecu_defs.xml needs to be normalized, that's a step in the right direction...

And given that we'd rather be dealing with normalized data, I think it makes sense to manage it using a tool that was designed from the ground up to make it as easy as possible to manage normalized data. And a SQL database is such a tool.

No need for arguments about it though... If there's a way to generate XML from the database, and read XML into the database, then everybody's happy.

It just hit me that if we can agree on how the information in each XML file should be ordered (decide on primary keys and sort by them, basically), then we can easily examine changes using diff tools. So, XML files on github might be a totally reasonable way to manage changes.

dschultz · **Posted:** Fri Sep 14, 2012 9:56 am

NSFW wrote:

Merp wrote:

An XML to XML editor accomplishes the same things with an equally information rich dataset and less complexity. (after switching to proper scaling.xml and a separate language file)

You could also call this "normalization."

In a way the EcuFlash defs are more normalized as they use separate scalings, where the RomRaider def uses a scaling per table so there is much repetition in scaling definition in it.

NSFW wrote:

It just hit me that if we can agree on how the information in each XML file should be ordered (decide on primary keys and sort by them, basically), then we can easily examine changes using diff tools. So, XML files on github might be a totally reasonable way to manage changes.

You mean view the changes to files, management will still be done elsewhere like in a database as you mention as to keep the information normalized and easier to update than touching every XML file you wish to add/change a table in (such as adding a Fuel Pump Duty table in to correct location in the XML file for every ECU defined).

BTW: the 16BITBASE & 32BITBASE effectively define the view which makes comparison easy if it is always followed. I plan on putting an ID tag on all tables so this will help order them as well.

Fearless · **Joined:** Thu Apr 19, 2012 3:44 am **Posts:** 385

I've cloned your repo into the RomRaider account and added you as the collaborator (it's all yours), Merp.

http://devslovebacon.com/speakers/brandon-keepers

Food for thought.

Fred.

Merp · **Joined:** Thu Jul 23, 2009 1:46 pm **Posts:** 863

Awesome.

For anyone interested, my PerRomXML branch of RomRaider also has some test code for GitHub integration.

Currently, it is incomplete (no settings UI) but mostly functional. Right now it has the ability to clone the repo and check the current head of a GitHub branch and compare it to the local repo. If it doesn't match, it can automatically update to the latest commit.

nsfw · **Joined:** Wed Nov 22, 2006 10:23 pm **Posts:** 2565

Very cool.

Fearless · **Joined:** Thu Apr 19, 2012 3:44 am **Posts:** 385

Interesting use of the lib! :-)

Merp · **Joined:** Thu Jul 23, 2009 1:46 pm **Posts:** 863

Updated OP with more instructions and .gitconfig to run a rebase from GUI.

For anyone interested, I have pushed a version of romraider to https://github.com/Merp/RomRaider/tree/ ... amlMapping that utilizes the JGit library to download/update definitions. Still very much a beta implementation, but I've done basic testing on it and it seems to be working. Its OT, but it also uses the Snakeyaml lib to export settings by object mapping to yaml instead of DOM xml output, so adding something to persistent settings is as simple as adding it to the settings class and adding javabean-compliant getter/setter methods (provided the field itself is/contains only javabean compliant class).

td-d · **Posted:** Wed Mar 13, 2013 2:07 am

Excellent, keen to test drive!

Josh Long · **Posted:** Sat Apr 13, 2013 1:14 pm

So I cant believe I have not managed to stumble across this until now. I cant believe how much you have guys have done.

Looks like I have some reading to do!

Josh

DisoDisp · **Joined:** Sat Jan 21, 2012 2:06 pm **Posts:** 45

Hi I can't get this to work as per the instructions.

When trying to clone the Alpha definitions i get a https error, MerpMod defs work just fine.

Any ideas?

Merp · **Joined:** Thu Jul 23, 2009 1:46 pm **Posts:** 863

I'm not sure what you mean. Can you post the error and clarify what you mean by MerpMod defs work?

DisoDisp · **Joined:** Sat Jan 21, 2012 2:06 pm **Posts:** 45

will post when I get a chance tomorrow.

Basically, all the other defs work, but not the Alpha defs.

DisoDisp · **Joined:** Sat Jan 21, 2012 2:06 pm **Posts:** 45

Never mind, it seems to work just fine now.

RomRaider

Managing Experimental Definitions with Git

Who is online