Page 1 of 1
| Changes to loader format 06/11/2013 at 01:19 #50747 | |
GeoffM
6377 posts 
|
Most of this has been explained before but perhaps not in one, easy-to-find, location. This is my attempt to explain why the move from .exe simulations to the Loader format we have now, and what the changes are. Firstly, a bit of history. Previously simulation data was compiled with core code plus a simulation-specific code file. This produced an executable file (.exe). Whenever a new feature was introduced, or a bug fixed, it meant the simulation had to be recompiled and re-released which was, quite frankly, a bit of a nightmare with different simulations having different feature sets. Over the course of 18 months Clive and I set about turning the core code into a version that will compile and load the source data at runtime. This means that all simulations in this format have exactly the same feature set, and can all be updated much more easily. Version 3.x of this was released a few months ago. From user feedback we determined that there were a few more things we had to do which meant we had to release V4 of the core code today. Hopefully minor updates can now be achieved by users clicking on the "Check for updates" button and following the prompts, which will update both core code software and simulations. It won't update timetables as yet - they need a little more work to give them version information which the core code can read and determine whether it needs updating. Don't worry - the new format of the timetable (see below) means this can be achieved without breaking old timetables. As the Loader effectively compiles the simulation data on start-up, it is slower than an exe file - after all, it has a lot more work to do. We're aiming on improving that. The effect is worse on older machines. Saving and loading games and timetables also takes a little longer because the underlying format is XML instead of binary. The files are larger (though compressed) but the advantage is the data within is a lot more flexible with regards to different versions and expandability - something the binary format could not deal with. Adding a single object like a signal would mean all saved games were redundant. Now, in XML format, it will at least try to salvage an older save. The current Loader (V4) will load older timetables without complaint, subject to the simulation data not changing dramatically. Even then, it may just involve the user tweaking a few trains to add or remove a location, change a platform number, or whatever. However, V4 saves the timetable data into the XML format mentioned previously so old exe simulations will not load timetables saved from the Loader. The WTR file is no longer necessary: this data (rules) is now saved in the WTT file. By default the installers will install into [CommonDocs]\SimSig. The actual location of CommonDocs depends on the version of Windows you are using. It's also referred to as Public Documents. Under this structure are two folders, Simulations and Timetables. The simulation data files (*.sim) are installed into Simulations. Under each of Simulations and Timetables is a folder for each simulation - for example, Simulations\KingsX and Timetables\KingsX. Under the simulation's own folder goes the saved simulations (*.ssg), the splash files (splash*.jpg), and any other simulation data. Under the simulation's timetable folder go the timetables (*.wtt - not *.wtr as explained above). Finally, the payware element. I know many of you are disappointed that some simulations are payware. We do spend a lot of time producing the core code and the simulations - several days a week, 8-6, when I have quieter moments in my "other" working life. One cannot be expected to work for free and that is why we now charge for some simulations. But I will repeat: it's some simulations only. Today I re-released King's Cross which I spent about a week updating with the new track layouts and over 50 issues reported with the previous version. But it's still free (or donationware). Some I will charge for; some I will not. Other developers have stated similar plans. Hopefully you'll understand. So I think that's it. I'll be happy to explain anything else. SimSig Boss Log in to reply The following users said thank you: RL1180, rodney30, indian_railways_fan, andyb0607, fsxfaulder, Hawk777, CTCThiago, Hooverman, John, postal, GoochyB, wain77, Javelin395, TimTamToe, peterb, Finger, Dick, Josie, birchy74, jeffh16, sandy221, sorabain, geswedey, UKTrainMan, Gwasanaethau, Wells, NCC1701, welwyn, Roadrunner, Jezalenko, Didcot, memorialstingray, Steamer, Prof Jolly |
| Changes to loader format 06/11/2013 at 12:15 #50771 | |
Javelin395
272 posts |
Would it be worthwhile adding this to the FAQ on the wiki ? I can see it becoming 'lost' as newer topics appear on the forum. And thanks for this Geoff - I know I moaned in the past when you first switched to the loader but this explanation was relatively easy to follow and not too technical. Certainly saves having to wade through several posts to find the relevant info. Log in to reply |
| Changes to loader format 06/11/2013 at 13:34 #50777 | |
|
AndyG
1842 posts |
See also Loader page on the WIKI.
I can only help one person a day. Today's not your day. Tomorrow doesn't look too good either. Log in to reply The following user said thank you: Javelin395 |
| Changes to loader format - feature suggestion 06/11/2013 at 16:27 #50812 | |
|
Entimix
16 posts |
A feature suggestion for the new Loader - I wonder if it might be possible to have SimSig "cache" the compiled sims so it doesn't have to recompile them every time, only when a sim is first installed or updated. Just a thought... 
Log in to reply The following users said thank you: Tempest Malice, postal, Steamer |
| Changes to loader format - feature suggestion 06/11/2013 at 17:23 #50817 | |
|
Danny252
1461 posts |
" said:A feature suggestion for the new Loader - I wonder if it might be possible to have SimSig "cache" the compiled sims so it doesn't have to recompile them every time, only when a sim is first installed or updated. Just a thought... :)I was going to post the same thing - though how easy it is and how much of a saving it would make may depend on how it's handled. Log in to reply |
| Changes to loader format - feature suggestion 06/11/2013 at 21:14 #50845 | |
|
GeoffM
6377 posts
|
It builds the era data at the same time so I'm not sure how feasible that would be. Nice idea; just not sure how it would be implemented - or whether it would save that much time.
SimSig Boss Log in to reply |
| Changes to loader format 06/11/2013 at 23:52 #50876 | |
|
Roadrunner
111 posts |
Thanks to Geoff for explaining the new loader as requested.
Log in to reply |
| Changes to loader format 07/11/2013 at 02:36 #50889 | |
|
GeoffM
6377 posts
|
Also to note that in general the loader sims won't chain to non-loader sims - some might, but there may be issues. Hopefully that'll be an interim issue until the adjacent sims get updated.
SimSig Boss Log in to reply |
| Changes to loader format - feature suggestion 07/11/2013 at 08:26 #50897 | |
|
clive
2789 posts |
" said:A feature suggestion for the new Loader - I wonder if it might be possible to have SimSig "cache" the compiled sims so it doesn't have to recompile them every time, only when a sim is first installed or updated. Just a thought... :)"Compile" is a bit of a misrepresentation. The data has already been compiled - the time taken is to read the XML and create the relevant objects (plus, as Geoff said, apply era choices). I have my suspicions as to why it takes so long but haven't so far had an opportunity to experiment to see if I'm right. If I am, it will be possible to speed it up significantly. Log in to reply The following users said thank you: Peter Bennet, BarryM, Steamer |
| Changes to loader format - feature suggestion 07/11/2013 at 10:01 #50904 | |
postal
5265 posts |
" said:I have my suspicions as to why it takes so long but haven't so far had an opportunity to experiment to see if I'm right. If I am, it will be possible to speed it up significantly.A patient XP user says thank you (currently over 4 minutes to load Marylebone!). “In life, there is always someone out there, who won’t like you, for whatever reason, don’t let the insecurities in their lives affect yours.” – Rashida Rowe Log in to reply |
| Changes to loader format - feature suggestion 07/11/2013 at 14:08 #50927 | |
|
UKTrainMan
1803 posts |
" said:" said:That's still probably less than the time I've either seen or known some staff have to wait just to be able to load up something like P2 (TMIS) on their computers at work; if it even loads at all, and without locking up the computer completely.....! :blink:I have my suspicions as to why it takes so long but haven't so far had an opportunity to experiment to see if I'm right. If I am, it will be possible to speed it up significantly.A patient XP user says thank you (currently over 4 minutes to load Marylebone!). Any views and / or opinions expressed by myself are from me personally and do not represent those of any company I either work for or am a consultant for. Log in to reply |
| Changes to loader format - feature suggestion 07/11/2013 at 14:53 #50937 | |
|
AndyG
1842 posts |
Another wish-list suggestion that might be easier to implement and possibly more useful useful would be to re-connect as a client without having to restart the sim from scratch. With Marylebone, if the host settings aren't correct and you get the disconnect message when trying to connect you then have to restart the sim which can take up to a minute in W7. I can only help one person a day. Today's not your day. Tomorrow doesn't look too good either. Log in to reply |
| Changes to loader format - feature suggestion 07/11/2013 at 15:49 #50945 | |
|
slatteryc
254 posts |
something in the XML parse code perhaps, ran into several issues with DTD parsing in my own line of work.
Log in to reply |
| Changes to loader format - feature suggestion 07/11/2013 at 15:59 #50947 | |
|
slatteryc
254 posts |
May be of interest Win 7 64-bit. Marylebone saved game load Preview 7 seconds Full Game ( from Pressing Start on Preview to when control on full screen is given to user ) - 48 seconds. 8GB RAM, Core i7 laptop quad-core. Log in to reply |
| Changes to loader format - feature suggestion 07/11/2013 at 16:49 #50954 | |
|
Danny252
1461 posts |
" said:Another wish-list suggestion that might be easier to implement and possibly more useful useful would be to re-connect as a client without having to restart the sim from scratch.In a similar vein, the ability to load a save/change TT (or even just start a new game with the same settings) without going through the whole lot - trying to test the problem I'm having with seeded trains is a right pain as I have to keep clicking through exactly the same options and waiting for the sim to load. I imagine bughunting/testing of any sort would be similarly slowed. Log in to reply The following user said thank you: postal |
| Changes to loader format - feature suggestion 08/11/2013 at 09:22 #51003 | |
|
postal
5265 posts |
Following on from Danny252's post, I've been chuntering away to myself overnight. Perhaps it might be possible to nuance things rather than going through the nuclear option of a full re-load each time you want to start something off. Maybe the options in the File Menu could be expanded. Instead of just "New Simulation" (or as a sub-menu when "New SImulation" is called) would it be possible to have options of "Load Saved Game" and "Start New Game"? Then for each of those options a choice to use the existing sim and scenario, same sim but different scenario and new sim. Or perhaps the three sim/scenario options followed by the saved game/new game option. I don't know enough about the mechanics of the loading process to know whether that would be possible and there would need to be safeguards so you can't load a game saved on one scenario to a sim already in memory on a different scenario but if it were all possible it would really help when you are testing sim or TT as well as potentially making things a bit quicker if you want to change your game in normal use. “In life, there is always someone out there, who won’t like you, for whatever reason, don’t let the insecurities in their lives affect yours.” – Rashida Rowe Last edited: 08/11/2013 at 09:23 by postal Log in to reply |
| Changes to loader format - feature suggestion 17/11/2013 at 03:53 #51636 | |
|
GeoffM
6377 posts
|
" said:" said:No, compile is exactly what I mean. You're probably applying it in a strict "turn X into Y" which is not the usage here, nor the only definition of compile.A feature suggestion for the new Loader - I wonder if it might be possible to have SimSig "cache" the compiled sims so it doesn't have to recompile them every time, only when a sim is first installed or updated. Just a thought... :)"Compile" is a bit of a misrepresentation. The data has already been compiled - the time taken is to read the XML and create the relevant objects (plus, as Geoff said, apply era choices). Now, most data is compiled as in turned from one form into another, meshed, cogutated whatever into the .sim file. However, final assembly - as in sticking bits together - and linkage still has to take place while loading. I need to run some sort of analyser to work out where the slowdowns are taking place. I'd hazard a guess that it's the string searching - fundamentally it's a text file underneath and that needs turning into nice Delphi strings, integers, booleans, etc. Add to that if object Alpha references object Beta then the code has to go and look through the list to find the object with that identity. I know the Delphi IndexOf routines are linear and unindexed. Originally we were using MSXML but that was very slow. That has been eliminated but it's not sped things up as much as I would have liked. The load and parse part of the XML is only a couple of seconds on my machine for a biggish sim; the bulk of it does not seem related to the XML. There are some issues we've yet to resolve with restarting without reloading. The safest option currently is to exit and start again. However, initiating a host when not previously selected, or re-connecting as a client should be a lot more do-able. SimSig Boss Last edited: 17/11/2013 at 03:59 by GeoffM Log in to reply |
| Changes to loader format - feature suggestion 17/11/2013 at 06:30 #51637 | |
BarryM
2158 posts |
Could a program be created similar to Windows Hibernate? Just a thought! Barry Barry, Sydney, New South Wales, Australia Log in to reply |
| Changes to loader format - feature suggestion 17/11/2013 at 09:24 #51639 | |
|
Forest Pines
525 posts |
Has anyone done any testing to find out how the code performance scales? That sort of data can sometimes give you an insight into where there might be problems. I mean, compare how fast the loader takes with sims A, B and C; compare the times taken with the number of XML elements in each dataset, and see what sort of performance curve you're getting. I've never played with Delphi, but linear searches on strings can be a pain in any language. When you say the XML load and parse section is fast, I assume you mean that turning the raw text into an XML in-memory data tree is fast and then processing that into the final data structure is the slow part? I do think you might get a quick win by using some sort of quickly-searchable lookup table to resolve references, probably one for each object type in case of clashing IDs. Log in to reply |
| Changes to loader format - feature suggestion 17/11/2013 at 12:28 #51645 | |
|
Jan
906 posts |
I think I've already mentioned it somewhere, but I don't know whether it got picked up: Would it be possible to add some command line options to the loader? Something in the style of SimSigloader -sim <PathToSim>\KingsX.simwould open the loader directly at the scenario selection stage, while SimSigloader -ssg <PathToSim>\KingsX\KingsX-2009.ssgcould be used to open a saved simulation. I'm not quite sure though what the preferred course of action in that case would be: Directly loading the simulation, with the option of running as a server being left to the GUI, or just opening the "Load a saved simulation - Choose file" screen with the filename (and simulation) already filled in. Personally, I'd tend towards the former, but opinions may vary. In conjunction with the appropriate registry entries, this would allow for launching simulations directly from Windows Explorer, or creating direct desktop/start menu shortcuts to your favourite simulations. I've attached a sample registry file below, though you should note that: - It assumes the default installation directory on 64-bit Windows, so depending on your installation directory you might have to change the paths. Maybe in the future, the system files installer could create these entries automatically with the corrects paths. - As the loader currently doesn't support command line options, you'll only get nicer descriptions and the Simsig logo for .ssg, .sim, .wtt and .wtr files. Double-clicking .ssg and .sim files works, but at the moment it only launches the loader without any further actions. [attachment=2016]Simsigfileassociations.zip[/attachment] Post has attachments. Log in to view them. Two million people attempt to use Birmingham's magnificent rail network every year, with just over a million of them managing to get further than Smethwick. Log in to reply |
| Changes to loader format - feature suggestion 17/11/2013 at 15:54 #51657 | |
|
GeoffM
6377 posts
|
Command line options are on the to-do list. Overnight I realised there are very simple ways of searching faster. I'll see if that makes much difference. I also have an old XP netbook which can be slow for the bigger sims. SimSig Boss Log in to reply |
| Changes to loader format - feature suggestion 18/11/2013 at 06:06 #51680 | |
|
GeoffM
6377 posts
|
I've just tried a quick fix to a number of object types. This has reduced loading for one particular sim on my PC from 28 seconds down to 17 seconds. That's nearly a 40% saving! I can't guarantee that the results would be the same for everybody but it's certainly encouraging. :cheer:
SimSig Boss Log in to reply The following users said thank you: postal, Forest Pines, GoochyB |
| Changes to loader format - feature suggestion 18/11/2013 at 13:28 #51691 | |
outofsection
149 posts |
It's funny how some of the really good ideas seem to pop into one's brain either last thing before going to sleep or in the middle of the night! When I used to dabble in Intel 8085 assembly language programming (that dates me!) I lost count of the number of times I had to get up in the middle of the night and write down ideas & possible code changes which had come to me whilst lying awake at night! Hope the new idea works out Geoff: as I'm still on XP, I'll be very happy if the new idea speeds things up considerably! :woohoo: Log in to reply |