RomRaider

While I was writing some C# to parse the EcuFlash style XML, I had some thoughts on definitions..

First of all, I do not understand why we use ECUID and CALID in the different types of definition. Perhaps I am missing something, but it seems to me like unnecessary complication.

Why in the hell are we defining all of this information when it is already defined inside the ROM? It would seem to me that the only thing is accomplishes is shifting more burden on the definition writers that could otherwise be placed upon a piece of code. It also opens up the possibility of erroneous tables working their way into the definitions unnoticed. By simply defining the lookup table address and unit scalings then reading the lookup table from ROM at runtime, you can be absolutely sure that you are editing data. Checking that the LUT at runtime is trivial whereas checking each axis, data section, etc.. of the current definition style is borderline impossible. Granted, there are benefits to having the information documented outside of the ROM, but that could easily be accomplished by having the editor output a 'verbose' xml after the data is populated. Although.. perhaps the time is better spent on a tool like ScoobyROM.

On the subject of making definition development easier, I would like to mention that I was messing around with JSON.NET and found it almost as easy to parse as XML+XPATH+LINQ, but much easier to read and manipulate as a human. I assume that being JAVAscript object notation, it would have even more powerful parsing tools in java than those for xml, no? Anyway, there is no better time to switch the syntax than the switch to per-rom ECUFlash style definitions. Which brings me to my next point.

Why are RAM and ROM definitions even separated? Consolidating RAM and ROM into one file as different trees in the definition would further simplify things, and pave the way to an elegant RAM table & tuning definition. There could also be a separate tree for SSM parameters and rom specific scalings. Again, no better time to do this than the switch to per-rom defs.

Note: AndyA had the great idea of live unit switching in the editor, similar to the logger. Having two separate Imperial/Metric definition sets only serves to complicate things IMO. Individual defs and bases should contain trees for imperial and metric, giving the user a "switch" in the GUI. Or each table & axis could use "conversions" like logger parameters with a drop down containing different units, in fact I like that much better.

One more note.. the ECUFlash structure of <table> <table/> <table/> </table> might save a line of parsing code if recursion is involved, but it is otherwise a terrible idea IMO.

As someone who has just spent their past few days digging around inside ROMs. I like this post, all of it.

The simple answer is, doing this makes RomRaider/EcuFlash 100% Subaru-centric. The way the def is written now you can define any axis, map, data item for any ROM regardless of manufacture and RomRaider/EcuFlash should be able to display the map. Have you ever looked at a Mitsu def?


The rest of this has already be asked and answered in the RomRaider dev thread that qoncept started over a year ago. I agree the RAM (Logger) parameters should be part of the ECU def and had proposed that quite a while ago. There's just a lot of work to getting it done (but the wage here is not very good )

I tend to find that putting the addresses into XML takes me just as long as actually disassembling the rom itself - very time consuming. I would certainly welcome a more automated way to generate the defs - would leave me with more time to work through the backlog, like I've been trying to do.

I like the idea of keeping Subaru-specific stuff out of RR. I'm a little surprised that there isn't a Megasquirt definition yet, and I don't want to close the door on that.

I like the idea of making it easier to create Subaru-specific definitions.

We could get both of those things if we create a Subaru-specific definition format, plus a Subaru-to-RomRaider definition converter that would read a Subaru-specific definition together with the corresponding ROM file, and spit out a RomRaider definition file.

And by "we" I mean "someone other than me."

This is looking like a task for the spreadsheet gurus with xml format.

Understandable, but I see several ways around that. A simple " if (LUT attrib exists) " statement in the XML parsing could deal with that. Or a template/class to parse a different kind of lut could be produced easily if Mitsu owners were keen on using the code.



I'll check it out and quote the important stuff here.

just to give me 2 cents to this...

I worked on diesel deffintions for RomRaider, not by pasing the dump.
I use ScoobyRom compare maps and characteristic lines and and "thougt" to have the right descritions found

The problem of useing a tool like ScoobyRom is, you only go size, factor, offset and axis of maps and characteristic lines but not names and meaning..

Even you have no chance to get single value bits..

So I realy don´t know, how to find any maps of a unknown dump without haveing a "basic deffinition" to compare to.

That the reason why you need unique Damos or A2L-file for every dump, if you want to use them in a applications tool like ETAS Inka, CANape etc...

There is no "naming" of values in a dump...

So IMA a nice idea, but non workable, if it should be done atonome and automatically.

And if you need to name the maps on your one, we are at the same point as we are now

BR

I fail to see your point. I just tore apart a previously undefined ROM, with some similarities to a known one and found tables, even one table that wasn't previously known about.

Single values do not have LUT entries, they're constants. Their true meaning can only be known through either comparison or manual decoding of the subroutines.

This is still a manual process, but its more efficient than writing this axis goes here, and this one here and that one over there, and they have this many elements... The LUT contains all that data.

I'm not sure I fully understand Jochen. Let me clarify, the idea of a LUT-address definition is only to simplify the writing process, not to auto-define new roms.


After fooling around in IDA, I noticed you can export a .MAP file that includes addresses for all tables you have named.

I was looking at ScoobyRom's source and it wouldn't be too hard to have it parse the .MAP and label any valid tables that have been found through IDA.

If most people defining defs are using IDA, it would make the process just as easy without re-writing the definition structure. Although it would require adding a 1-D table class to ScoobyRom.

the point is, that known tables change (size!) some- (most-)time in DENSO dump, even when the software depents on a "known" one.

In Diesel-dumps I got no simular one, if a software up-date is releast..
So you have to search manually if you got a legacy, or Impreza dump every time..

So your idea in theory is good, parctially it will not work, because just chancing on table size will give you "unknown" tables..


Yes, but they are very importend, because they are need to chance limits of a dump, DTC detaction or will switch on/off state-mashines.

I know no "good" tuning, without chancing any single bit value..

Don´t get me worng, I really like every idea to simplify any deffinitons build.

But when we need to handmade them after relasing a software-update, your work to chance the way to get the deffinitions at the end is a "wast of time"

Essential functionality, but the approach is bad.

What you want is a mapping file of X unit to Y unit, in all possible combinations. Then you define your table in whatever, but with a standardised key/name. Then the tool allows you to select a compatible mapping and does it for you.

Fred.

btw.:

it would be very nice, if ScoobyRom or your software will add a 1-D table class and will deffine a dump simular to a one one.

But I see some problems :
in diesel dump you have got round about 4 differend kind of boost tables depenting on regeneration or not, and 7 differend railpressure tables..

How will you deside witch one is witch, when starting adress will be chanced when only one mapsize is differend and you cannot be sure the the table itself are chanced because a newer application.

this is my point. I don´t know if the petrol dumps are so mutch easyer at this point

I'm not sure about the diesel ecu dumps, but there is a very limited set of table 'types' in all the disassemblies I've seen, which I would think could quite easily be incorporated in terms of an LUT. It would be easy enough whilst manually naming them to flag the type (i.e. 2d, multiplier, offset, etc.)

I have an couple of IDA scripts that marks and dumps tables to a file, then I have an application that reads that + a template def file and combines the two. So the table identification then becomes the most time consuming part. This is how I got to creating a def in ~4hours start to finish, with Logger parameters too.
I believe I shared it with someone but I can provide you what I have as well.
The whole process requires that you mark the tables exactly like the XmlToIdc application does, and it works best with a table dump from IDA, although you could work from a plain text file too.