RomRaider

I vote we call it "Medieval" instead. I prefer those units myself, but someone at LegacyGT.com referred to it as the Medieval system and I still think that's the best term ever. Because it's true.

Being able to select units on a per-table basis sounds good, in fact it almost sounds like a must-have feature. However I would lean toward doing that in a subsequent checkin. We've come this far with a global metric/medieval option and I don't recall hearing any complaints.

I lean toward doing highest-priority stuff first, and lower-priority stuff later, unless that approach takes far more work than doing them together.

Choosing per table isn't too big of an issue. Saving preferences will be more work.

Sorry in advance if this is taken the wrong way, or if I have misunderstood how the logger defs will be implemented in future - but I just wanted to post before time is spent on this.

Rant warning



I can understand the need to change to the same ecu def format for ease of definition creation for ECUFlash and RomRaider Editor, but why the logger?

I mean, in all honesty, what benefit is there? All it will do is increase the def file size and complexity of the def you add the required logger def code to and it'll then slow things down. From what I can tell, the idea behind the ECUFlash def format is to separate data specific to each vehicle in an effort to reduce def file sizes and so only the bare minimum def files are loaded - instead of needing to load one big def file.

If you go through the logger def file, it doesn't have anything the ecu defs need or use anyhoo, so sure the ecu defs makes sense to change to ECUFlash format, but with the logger defs, it's just more work for very little gain.

By all means you could remove the "conversion" subsections from the "parameter" and "ecuparam" sections in the the logger def file, add the "parameter" and ecuparam" sections to the sepecific vehicle defs, call the xxbitbase.xml def file and use the "scaling" sections instead in conjunction with the "parameter" and ecuparam" sections from the sepecific vehicle def, and change the tag_conversion statement to load the units, expression and format from the "scaling" section instead of the "conversion" subsection from the "parameter" and "ecuparam" sections in the the logger - but as I mentioned already, it's a lot more work for very little gain.

I honestly think it would be best to keep the logger def files totally separate to the ecu def files as they are now. Not only will it spare your time up for other dev work, but the way the unit/conversion is chosen, that is being discussed in this thread, will not need to be changed.

/rant

I think you're overestimating the impact. Speed is only an issue today because of a couple bad decisions I made back in the day. When I wrote the original definition parser I was testing with a couple ROM revision definitions that had maybe a dozen tables each. Now that I've seen we have so much more metadata to deal with and I have some xml document parsing under my belt I'm in a much better position to make it fast enough.

Java can READ a file very fast. In ms. The xml document parser can skip over unused elements in practically no time. You're right, logger metadata won't be used in the editor and vice versa (well, probably.. read on). But there will be very little negative impact. The benefit would be better cohesion. A single definition file for a single rom revision. I don't think its such a huge deal, but I think it makes the most sense.


Actually we're talking about having a separate document containing conversions now. ECU and logger defs will define whatever the base type used in the rom is and alternate scaling types will be listed in a separate document in real world values. Which I think makes sense -- it's a different type of metadata, so separate it from the rest.


Dev time is a valid argument. Switching to ecuflash's definitions seems like a no brainer since there are deficiencies in the current implementation, but I'm not sure I can justify moving the logger defs so easily. And I'm personally not familiar at all (yet) with how the logger parses and identifies roms.

So.. the benefit of combining logger and ECU metadata is minor. The disadvantage (ignoring dev time) is also minor. I can't even give an estimate of how long it would take me to develop. Do we still think its worth while?

Hi: I think you may have been misled. I believe this topic was all about the scaling to use in the ROM editor to allow users to view in units of thier preference.

I had proposed to combine all attributes of a ROM into one def file. That would be Editing and Logging definitions. I don't see it being a big problem for a couple reasons:
1) the Standard part of a Logger def is common to all cars and can exist in the base def as we have for the Editor today
2) the Extended parameters wouldn't add much more than 50 - 60 additional XML elements to the file and they are very short in length. (They contain an ID, data length, RAM address and a set of conversion, which can be relocated to the scaling def)

The Extended parameters are truly ROM dependent and that's why I see them being combined with the Editor info.
Oh and BTW, the current Logger of today reads all your Editor defs you have listed to cross reference the ECU ID to the CAL ID to identify the car. It's so fast you don't even notice it, do you?

Cheers for the reply Jared

Ah all good then I guess. If you can implement the extra logger defs into the ecu defs without actually increasing load times etc, then that is a big plus.


I totally understand the need for cohesion between the defs, and I totally understand the need to reduce number of duplications of definitions for RR Editor and RR Logger and ECUFlash.

What I meant above is that even if RR Editor is changed to use ECUFlash's def format and cohesion is found, which I think is a great idea and something that shouldn't take too much time to implement - the Logger and it's defs are totally different story.

There is no easy way to use the current ECUFlash defs the way they are now for logging purposes without adding a bunch of code in every def for every specific vehicle/ecu model. To be very specific, the ECUFlash defs have no logger support at all. They are very specific to ROM editing, and only include the required table details and scalings needed to convert units.

You could pull the scaling info for the logger from the xxbitbase.xml file, but you then still need the numerous parameter and ecuparam info for logging to work.

I thought a little more about it and you could add the parameter info to the xxbitbase.xml files as they're used by all vehicles/ecus - and then add the ecu specific ecuparam info to the relavant vehicle/ecu def that matches the ecuid defined in the ecuparam. Then you could change the logger to suit, gathering all the required info from different def files.

But this adds a HUGE amount of work to do, just separating out the ecu specific ecuparam info and adding it to the hundreds of ecu defs will take a lot of time, effort and changes itself - and that's not even taking into consideration what needs to be changed in the Logger so it parses everything correctly and retrieves the right parameter, ecuparam and switch info for the vehicle being logged.

What I was suggesting was to move RR editor to ECUFlash defs, and keep the Logger using its current logger defs until sometime in the future when devs can spare time for such things of little gain.


I definitely think moving RR editor to ECUFlash def files is worthwhile. RR doesn't have merchgod doing defs anymore, and Dale and others are stretched for time with the huge undertaking of RR dev already - and since ECUFlash and RR both use the defs, it makes a hell of a lot of sense to share dev resources between the two projects and work to use a single editor def format.

The logger on the other hand, is as explained above, a totally different beast, and I'm not sure there is any real need to change the logger defs format at this time as they do not effect or share components with the editor defs in any way.



Heya Dale.

Hmm, I think I may have been - so this is not about logger defs at all? Purely only editor defs and how they themselves deal with unit scalings/conversions?


As I replied to Jared above, changing the editor to suit ECUFlash defs is worthwhile and shouldn't be too hard.

But the logger defs is different. You can't just combine them into a single def with the ecu defs - as ECUFlash's defs are not one single file - they are hundreds, specific to each ecu.

If on the otherhand you mean combining RR's editor and logger defs, and not using the ECUFlash defs, then I can understand the suggestions - although, doing that would still not fix the current problem that editor defs are made in quadruplet - standard and metric for RR and standard and metric for ECUFlash.

If you keep the logger defs separate for a moment, you could cut that quadruplet def creation down to a single set of defs, with some changes to RR and ECUFlash editors to support user switching between standard and metric units.


I'll assume you are refering to the obd "parameter" sections? If so, then agreed, they can be added to ECUFlash's current xxbitbase.xml defs as they are generic and used by all ECUs.


You could do it that way, and change the scaling sections, but you are then changing the format of the ECUFlash defs, and
1. Colby should be in on that before any work progresses on the changes.
2. I'm not sure it is the best way, extended parameters (ie. ecuparam) are ecu specific, so it would be more consistant to add them to the specific ecu def. And if adding to the ecu def, you could leave them the same as they are now for simplicity, just adding the sections under the sections already in the ecu def files.

More work, but much better, and more consistant implementation imo.


The speed was more in relation to the editor, not the logger. But as Jared pointed out, the RR editor can be modified to improve load times and speed, so I guess that is now a moot point.




Lastly, I just wanted to say that I may voice concern loudly, but if nothing is voiced, it then nothing will be discussed and the best, most suitable direction may never be found

Whatever is agreed to for def implementation, I'm more then happy to assist, so I can help free you both for other more important dev work.

One thing I'm not sure most people realize is that the EcuFlash and RR defs are already a single effort. You have random defs here and there created by different people, but for the most part, it was Bill and now Dale controlling defs for both apps. Dale also has a relational database created with the logger metadata so the xml can be more easily generated, and (I believe) intends to do the same with editor data.

As far as EcuFlash, adding additional elements won't impact it. That's actually the point of xml -- it's extensible. Take an EcuFlash spec definition, then add a <logger> element to it, and since EcuFlash doesn't have any defined action for that element it will skip right over it. What I'm not sure yet is if the RR editor would skip over it or find some use for it.


I think you're still not seeing this the same way I am. In the logger element, we might have (and I'm changing some element names here):

Then, in conversions.xml:

Or, you know, something like that. Point is, the only scaling metadata in the editor and logger defs is to the natural internal unit the ecu uses. Converting to other units is a process external to the ecu and common to any storage method. Today we have a lot of what is basically duplication because the target boost table and map log parameter both read in psi, but both have to define expressions to convert to and from each unit we want to see. And not only does the duplication mean more work, but it means potential for inconsistency.


My impression from Dale is that making the changes to the definitions 1) isn't as big as you're thinking and 2) is something he wants to do anyway. He has mentioned creating the database for the editor defs, and in that process we want to review and normalize the data. I also plan to identify subsets of roms that use the same conversion types so we can eliminate the duplicated table names with minor differences (like an underscore at the end). (Mentioned before)


As Dale said, the logger does use the editor defs in a minor way. But I also agree that I'm not totally convinced we need to make this change, or at least right now. I'd say it's a fairly major change to the logger, but like I said, I haven't reviewed that code yet. The biggest argument I can see for doing it now (along with the editor def rehaul) is because that means one less major change in definition structure, in terms of how many times we make defs obsolete. Rather than releasing v0.6.0 and saying in the release notes "all your ecu definitions are obsolete and need to be replaced" and then saying it again in 0.7.0, we'd only have to do it once. How much that matters I'm not really sure; it's beta software for reasons like this, AND definitions are, for the most part, totally controlled by us.


Adding an new file for converting between real world units won't affect the base editor definitions. The editor defs provide no means for viewing alternate units at all. That's why there are 2 versions of the defs for each app today. Making changes to the EcuFlash spec would completely defeat the purpose of switching to them, and like I mentioned earlier, isn't a function that's ecu specific at all anyway. So what we have is a function external to the editor def processing, whose only function is to convert values to your chosen unit. No compatibility broken, no changes to EcuFlash required. We can certainly show Colby what we've done and he can choose whether to implement it.


We're listening. I'm not convinced we need to combine the defs, either. I think it does make more sense that way, its just the amount of effort.

I mentioned trying to normalize the definitions.. This isn't REAL important, but I think it would be useful (and I bet it would cut the time to open roms in half in the new parser; I don't know if you noticed in the other thread but it looks like fully parsing and opening a rom is going to take a little under a second). PM me if you'd be interested in looking at that, or anything else. Thanks!

Maybe it's just me but I never liked converting something more than once if the conversion was lossy. Wouldn't there be some side effects to converting (and then rounding) a boost map to PSI then converting (and rounding) it from PSI to some other unit?

Why not leave it raw then convert it only one time to the user desired units?

And if you do end up being able to select units per table then there should be a global way to set similar types in all tables rather than having to open each table and change it's local display units.

Thanks Jared for writing that long response. I was working one up.
Yes you are right, I have a database for the logger info and will work one up for the editor. In the end all info pertaining to one CAL ID will be accessible in the database, so dumping format x XML file or format y XML file would be trivial.

The Logger makes use of the Editor defs more than you know. Have you tried to scaling your MAF with the Logger? Have you selected the "Update MAF" button, well you just used the Editor def from the Logger. Or have you used the "Overlay Log" checkbox in the Editor. That calls the Logger into play so you can see the cells read in real-time. Then there's the Dyno tab, it uses a car_def file which could be integrated to the editor def, ROM specific. To me it just makes sense to put all the ROM, Logger and car specific stuff in one place, one file...
We can go any way we all agree on.

BTW: I like that idea of changing all related scales as a user preference with the option of still changing individual tables as the user sees fit. Such as, I like Celsius but prefer PSI over Pascals, mix and match. I'm sure dealing with all this will be a coders nightmare

bump

I have always wanted to do that.

No but seriously should I bring this back from the dead? I just found some old commented out code that I suspect didn't work because of several issues. This should now be very easy to hook up and give the user a choice of "standard/medieval/yank" vs "metric."



And no 2009 Dale this is actually already coded just needs to be hooked up. I will edit the ECU_Defs.xml and attach a screen capture.