Anger's Alternative

myself wrote:will be made sharable when there's something worth sharing.

https://gitlab.com/TheAnger/volt-oven

i've removed the binary / exe since it changing every test build was eating up more room than expected, will add it back in when the engine core is stable enough that it doesn't need to change on a daily basis. instead i've added an output.txt file to show the program trace / debug messages being shown right now (which is about all it will do for the time being, until gfx is redone) - doesn't say much except show that Lua is indeed running in multiple threads.

otherwise... i guess feel free to browse the source code if you're curious how any of it works / will work, i've nothing to hide there (yet). if you have a gitlab account (or make one), feel free to add suggestions / issues etc concerning the engine.

quick update.

Lua inter-thread comms is about half-way done, it involves 3 methods -

1. global lock - global critical section, basically. implemented already, didn't take long.

2. thread message queues

basic idea is that each thread has a queue of values it can read in, which other threads can add to from the other end. adding and reading operations require locks on the queue, so in essence a thread can continue uninterrupted while other threads are sending it values, and vice versa when it needs to read any queued values, other threads don't have to wait until the read happens to continue with their tasks.

Lua threads can already export Lua values to other threads' message queues.
threads can't read their own queues yet, and intermediate value memory management needs a solid once-over to make sure it isn't leaky, though that shouldn't take more than a few days.

3. superglobals (Lua global = thread global; superglobal = global to all threads)

essentially mimicking an Lua table, the fields of which are 'superglobal' / ENV variables.
implementation depends on the same mechanism that allows message queues to hold Lua values.


resource API and superglobals will be merged in some form - they are more or less the same thing, and it's up to the Lua code how to use it / expose it to a game.

that much i should get done by May - the remaining modifications will take a few more days, and a week possibly to design the Lua bindings to create / start / stop threads, send them code / messages and interact with superglobals etc.

all goes well June will be spent on rendering and input rewrites - most of it is going to be cut-and-paste with refactoring but i'm still giving it a month to squeeze in some improvements / additions.

July - will have to see how things look. probably be taking a short break. otherwise will be rewriting the Lua-side interface (formerly GEO) to be as lightweight as possible and store all app-global objects as resources / superglobals.

August - this is looking like the month where the core of the game itself will start to manifest. by this point there should be full model rendering support (sans shaders and multitexturing unless time permits it) as well as easy-to-use game logic constructs for Lua.

September and onwards - see how things go, around then i expect to start adding features to the engine on demand / as needed. also around then i'll be looking at formally kicking off the game project separately from the engine.

very quick update.

this was anticipated, but im still annoyed of a week-long delay due to a conceptual problem with iterating over Lua tables stored in C++.

details here if anyone cares for it. https://gitlab.com/TheAnger/volt-oven/issues/3

estimating by next week i'll have multithreaded Lua at a prototype stage. from there it's a matter of deriving a superglobal class based on LuaTable, writing an Lua interface for all these things (not hard) and documenting it all - still on track for end of April.

short update - gotten about a month / 5 weeks behind due to life things going on.

i've been slowly refactoring the Lua<->C++ read/write functions, finishing that off tonight/tomorrow night. the efficiency of these functions isn't great (nor was it going to be), but compared to IO operations on regular tables, its roughly 2-3x slower; not bad as long as each message contains at least 4 values, where it starts to become worthwhile. BUT - these are not going to be messages passed at random, they're inter-process signaling tools that are not supposed to handle large data traffic, which is what superglobals are for, a sort of "enumerable host memory space" available to all Lua processes created.

conceptually i've got superglobals down in theory, requires a reference counter to hold the data and a special userdata type to hold references to it in Lua and in the message queues.

superglobals can be passed by reference in effect between Lua stacks, so sending references to superglobals is about as efficient as sending individual values.

all goes well and time permitting, this weekend or next i should have caught up with the single-threaded incarnation and can make some proper progress again.

well, the c<->lua data transfer ops are working somewhat. need to clean up the code and revise a few things here and there. few hours-worth of effort.

bit of a change in priorities; moving into getting superglobals working next, because with that done the graphics can be re-written to work with data buffers only. the single-threaded incarnation allowed raw geometry commands, which while adequate do not offer that great performance aside display lists (which i hope to avoid using without clear bounds).

then, game objects will become proper superglobal-type objects from the beginning, which should massively simplify the API from both C and Lua ends.

OK!!!!

been a little while, bad habit of mine to be a hermit. anyway.

https://gitlab.com/TheAnger/zero-volt

I've started over, the original code was too.... intertwined to put it one way. things were being put together faster than they had working code for, so to stop that bullcrap before it got out of hand i've archived the old and started fresh with abstract objects first and better namespacing. but you can see that for yourself.

the new architecture

ive decided it is no longer a priority or need for Lua to be aware it's in a multithreaded environment (as long as the external data interface it uses is aware, which it will be).

instead there will be some number of worker threads running in Lua, processing (effectively but not actual) co-routines for every game object. roughly: every object has one or more logical 'states' it can be in, like operational modes, that determine how it behaves at a high level. this behavior determination is done once per game cycle, per object.

in practice these objects will sit in a pool of processing tasks, that the Lua worker threads pick up and process independently. for this to work of course, the game object data cannot reside in Lua, and will sit behind a thread-aware reference-counting interface.

outside of this going on, the graphics, sound and any other subsystems (eg physics / network) will run parallel. there will be more fine-tuned restrictions on what can run when, but i'm not decided on that yet and tbh it's way too early to be sure yet.

progress so far

picking up from where i left off... the non-local type is now the de-facto standard for Lua / C++ data harmony. internally to the engine, the reference type that Lua and C++ will deal with is ZV::Variant. this has to work before anything else is worth being built, excepting multithreading constructs and their primitives (ZV::Sync, ZV::Primitive).

if i can keep up the pace, and this time around it's not stalling due to technical issues being found, it's looking like sep/oct there will be a working prototype of the engine with graphics and sound.

alternate control methods for the game - TBD. anything Lua can do already for one, i'm not in any hurry to implement network code or file pipe or gamepad support etc... when it's worthwhile, and no earlier than a working engine anyway, then i'll get around to this.

network play - per above, not necessary until much later.

ETA for "much later" - sketchy. at least, november, perhaps by march next year i'd want to be at a point where even if i dont feel like it, it wouldn't burden the engine layout anymore for sidetracking additions.

been a few months since last update. some gloriously productive ones, for once.

above recent link still valid. theres a test window you can look at, and press Esc to test the input (yes it works, was hard ). you get (most of) the source too, if you can improve on it i'll gladly take on the help. odds are ill keep the engine open source but if history is any lesson i'm not ruling out otherwise just yet; too early to tell but even so, there's other ways, like protecting the game content which - unlike the engine - is most certainly proprietary.

progress?

much.

the reason there's even a test window at all is thanks to SFML - it's tricky to get working because i hate microsoft compilers and insist on being different. results? worth it. highly recommend for pet projects involving multimedia programming from basics through to, well, this at least lol.

it set me back a few months deciding on this and implementing it, along with set backs in overthinking the parallelism. breaks were had as this was a frustrating august - october.

since then i've let the ideas settle and picked up where i left off, throwing out a lot of what i've done for now and starting with SFML as a base. over the last couple of weeks i've started adding back in the useful chunks of code tried and tested before.

at this stage it's very close to being a proper game engine. im estimating primitive capacity for a game around next weekend, and around 3-6 weeks from now there's a high chance i'll have started on a game.

presently im finalizing the game object code, and have already concocted a thread / process model for the engine. the game api has been prototyped in a javascript experiment (pm me for link if you really want to see, i dont think its anything impressive, just a proof of concept in its unfinished state) so im very confident it can be built up on once it's finished.

i couldn't get to this part without finishing the lua <-> c interface and a variant type. both have more or less been finished (though not fully tested) to a working point. any gaps can be quickly filled.

very roughly speaking, the engine has 3 phases that it go through, with operations in each phase run in parallel:

1. physics & animation / core object updates

2. graphics, sound, scripting (read-only, writes-queued)

3. object property updates (queued writes), engine IO (keyboard, network, gamepad, etc)

scripting is message-based, but can't say more than that yet. within each phase there may be serialization yet, or more phases, but atm that's what it looks like.

there's definitely no more than 2 weeks of effort (at the pace im going at lately) before at the lowest level it can be used to create games...

in fact it's so close im announcing an invitation for testers. feel free to test what you can see there, but as i say, for another month probably it will be nothing of interest to 99% of people imo.

my ability to stick to a schedule is a legendary failure so feel free to check on it any time. even by that standard, i can taste how close it is now...

hopefully more good news soon...

- anger

and good news of a different sort.

whether by laziness, good fortune the engine design is compatible, or sheer chance i happened to remember about bullet, that i'm now using that physics library for the engine.

there are a lot of things happening with the engine design lately i didn't foresee; i did not expect old lua-based code to be of use for this iteration of the engine, i certainly did not expect i would need almost zero effort to integrate something as impressive as bullet into the engine.... but there you go lol...

going to do that next, because im coming up to finalizing the engine pipeline - ignoring physics would not be wise, best deal with that now.

in saying that, game object class just isn't complete without knowing the physics implementation to be used later, and the game api shouldn't pre-empt what the engine can do (unlike 2 rewrites ago).

after bullet is done, shouldn't take long for a rudimentary implementation, i'll be starting on test demos making use of physics, graphics, sound and input. in the process i'll decide on how the Lua-side of the API will work, and in turn finalize the game engine structure & pipeline.

i really wish i could finish the engine without doing it in this roundabout way, but from experience i work best upwards and towards something, not from a high level, ordered plan but in order of pragmatic need and definiteness; it has approached that point where i need to start using the engine to determine how it should work, hence the call for testers earlier though i didnt think id need to this early...

edit - just to point out how much of a game changer bullet is, it basically adds stability to the engine design with all the unknowns that a specific implementation brings. as a result, im happy to say, the hardest part left of the engine to design, is the network code. everything else is not really critical and can be implemented per-game as needed, and if something happens to be utilitarian or computationally demanding, then it gets upgraded to engine module status. it's just that at the moment, there are no engine modules! i'm writing them right now, well, the core that will allow modules to work at all to be precise. and with that, it means prototyping the API, and I wont be able to do that until i try using it from the ideal conception point of what i want a game script to look like...

alright... so far so good. bullet builds on my machine, and the engine compiles with it being used. things have accelerated (pun).

in doing so ive archived the code i had and am changing the build order for what i hope is the last time. atm the engine has been reset to a clean slate, and i'll build the engine components as-needed from the code i've collected and written so far. this should take a few days to get done.

more updates soon...

alright, final major rewrite going off to a good start.

bullet is working beautifully, with the crappy test demo it has (sphere on a static flat plane, numerical sim only). but... key point is that it built, compiles, links and now confirmed that it functions as it should too (was a bit worried as it defaults to float, and Lua not having an integer format relies on the larger-than-int precision of a double for such things, so i built it with double btScalar to avoid unnecessary conversions internally where possible)

i've revised and slimmed down variants. rather than them having a subclass to store refcounted data, it will now do the refcounting itself for known variant types that would probably be better off being copied explicitly - specifically str, queue and map subtypes.

i've abandoned trying to give Lua any pointer data to game objects. it can only be abused and an id (typedef size_t id) is plenty to refer to game engine objects by context (eg, in a texture bind call, its a texture id, in an object search call, its an object id, etc). I'm still leaving the option for pointers (userdata will be converted to light pointers, memory ownership will not cross Lua/c++ at this time as it makes a bottleneck dependency) it to be transferred in an out of Lua stacks.

in terms of code produced, ive made a quick version of shared_ptr (because gcc and c++11 on windows is head banging grade stubbornness to get working, not even boost likes my system / setup) - zptr. using this as a base im writing a "selective" shared_ptr variant which does the role of both a variable scalar container and a pointer refcounter container. you should get a good idea of how this is achieved from what ive finished just last night - files /src/var/base.* .

when zv::var is finished i'll thoroughly test it including subtypes. if all goes well (i'll know tonight probably, day job and all) it's going to be a head-first dive into creating a Lua helper class. i started on this before this rewrite, but only barely. there was a lot of potential to create a complete Lua interface (that is, turn the c-style calls and a stack instance into a lua stack class with inlined class methods). that is clearly overkill.

however there was an ambiguity that this rewrite resolves - does zv::var or zv::lua classes own the code to read and write to/from eachother? or should it be split? i informally went with "lua depends on var" and in this rewrite it's going to be an explicit dependency.


so.... finishing off variants, testing them. bringing back the lua class i had after that, and building the var <-> lua functions into the lua wrapper class itself. thanks to var's copy, move and swap semantics being an assurance now, it will keep the code almost entirely related to lua itself against the lua wrapper class, otherwise lua or var had to partially pre-empt one another and that meant global operators (ew) with code in yet a third place. no thanks. started doing that, not a good idea, doing it right this time.


i assume the lua import export code ive made 'mostly works' - ive rewritten the damn thing 4 times in 2 weeks, only to stick with a minor variation on what i had before (some loops collapsed, rewrite to reference passing vs ptr, etc). if you've seen the code for it before, you'd understand why im so nervous using it - the import / export i made sure would work with lua even with limited stack size. eg, the export needs 5-7 stack slots at its peak and this includes walking through nested tables (no recursion checks though, im hoping i or other game writers using the engine would be more careful than that).

in any case i'll be testing this tonight / tomorrow, so at some point saturday, the main engine loop and game objects will be started on (assuming few delays / issues up to that point). i have a good idea of the engine pipeline now that all 7 critical engine subsystems are (more or less) able to be built now to well defined specs - scripting, networking, graphics, input, physics, sound, resourcing / object management.

i'll probably do graphics first, so objects can be at least seen with a mesh-compatible structure (texturing too; i have the code to load wrl and tga files done for Lua already so this is not a problem to prioritize). next, they need to be scripted so the graphics can be defined via script, which will invoke building the main engine cycle / loop to handle objects. in turn scripting would be started on, enough to get basics going, followed by linking game objects to bullet objects, exposing some properties (position, orientation) and using them to transform the object for rendering.

it's possible to finish this to a very basic degree - room full of spheres - this weekend or early next week.

from that point extras - like sounds - will be mostly re-using how the above code will work.

networking requires a special touch and will be all but last. it is a topic in and of itself one could spend years on.


so. rambling aside that's the short term roadmap for the engine at this time. and you can see, at this pace it's likely a game could be seriously done using the engine in about 3 weeks time, give or take.

SIGH...... it's been a long weekend.

issues were had alright. never trust large chunks of code written over many sittings, from different assumptions about the host environment. things go to shite and back and waste a lot of time. anyway.

variants tested and working.

lua <-> variants tested and working.

i gave in to my better judgement and made plain-old recursion to get it done. good ideas aren't so good if they don't work, compared with an approach one knows will...

still have a few hours of the weekend, am finally making a start on game objects and the engine pipeline. latest code & test exe updated (nothing fancy. just shows var <-> lua IO in action).

good news is with variants and lua now being quite stable, this shouldn't take too long, as game objects will be built up from this foundation with minor tweaks / additions as needed.

the engine pipeline, because of the central role game objects will play, will take shape in the form of how objects are processed each cycle.

still on track for ~3 weeks, want to say sooner but not to get ahead of myself again....

Wow! after a couple of years of not visiting this site or playing (due to the fact that no one ever does any more), it's great to see this almost done!

A couple of years later for me as well. I absolutely love this game and I wish you all well. Ill stay posted and help support where I can.

ok guys.... the time is approaching.

0V is edging closer to a working state (again). links are dated, i havent been uploading anything in 6 months as i sort this thing out.

TL;DR i've been having a good long think on how to scale the engine, given i now know what does and doesn't work with the architecture choices made so far.

where things are up to:

engine has been split into a core-and-modules design.

core = internal stuff, data storage, threading / sync code, state code, etc.
modules = everything else. graphics, physics, input, audio, networking, other.

the core consists of a several phases:

1. messaging - object message in/out boxes are swapped & inboxes are processed.
2. events - object state machine events are fired (onspawn, onenterstate, ontick and so forth).
3. objects - old objects are removed, new objects are inserted with respect to the object processing master list. relevant data is cleared out / initialized.
4. modules - engine modules have their turn with handling game objects, and in the process the 'game' is materialized through multimedia, input and networking tasks taking place.
5. updates - changes to game objects throughout the engine cycle are now committed.

presently i'm rewriting the core - this is 2 rewrites on top of where i was up to mid-dec. however this is a structural rewrite, i'm not making huge changes but mostly refactoring things around and making it read better. i should be finished with this tonight / tomorrow night.

next, a cursory Lua API for dealing with game objects - creating, destroying, querying etc.

from there, basic input and graphics modules - just bare enough to work. again, wont take long since the fundamental code for these things already exists.

ETA - 1 to 2 weeks before an alpha engine build will be viable. 3-4 weeks before a game is properly started.

from there, it's downhill -

anything the game needs of the engine, it can be built into the engine in a scalable way. this includes new modules or additions to existing modules (this will be 95% of non-game-related coding - catering for the game via engine features).

now... to have such a timeline, and being one person, naturally i'm taking some shortcuts. but as i say i dont mind an incremental game engine as long as it's designed to be incremental (which it is). physics, audio and networking in particular will take a back seat to the game itself, unless it's at a stage where those things should be worked on further (wont be for a while).

lastly i'm looking around IRL for 3d modelers - most urgently i will need model & texture assets to work with, realistically in 6-8 weeks from now (sooner is good too, gives me something to work towards).

so yeah. that's where this is up to.

~2 weeks from engine alpha.

~4-8 weeks from game alpha (earliest playable, far from resembling the final game though).

i'll either revive the dropbox or set up a box.com account to share the engine runtime and game (inv only, not ready yet will advise when).

updates -

engine is now available for download from a consistent location (check my sig).

engine is not yet ready for use. really this is not even alpha build.

source code has been updated as well, i've removed the dll's and lua scripts from the source repo as these will be tracked separately to the engine. you can get the engine binaries in the source repo, but i've moved things around so what the engine exe needs isn't next to it in the repo and "some assembly required" put simply.

finally, with the pace things are moving at there will be a test program / test game coming soon. dont think 'game' so much as 'engine test', like a jet engine being tested while bolted to the floor behind blast shields.

any engine tests will be in the engine public release folder - these aren't games after all...

TL;DR refer to box.com for the engine (available now though not really "working" yet) and games (soon (tm)) unless you're interested in engine source code.

update -

reviewing the engine pipeline, and considering a lot of it has been faithfully reproduced over 3 versions now without asking too many questions, i've found some aspects of it problematic - specifically, object data storage in relation to creating and destroying objects asynchronously from the engine pipeline.

for example, if there are 10 game objects in play, and as part of processing object #2, a #11 is created, that #11 won't be processed in the same game cycle. however, object #2 created it and knows about it, presumably it will also assign some values / properties to it. and this gets messier if you attempt to remove the object before it's instantiated properly into the pipeline (so it can be processed per cycle).

TL;DR - many inter-thread sync operations happen in order to allow maximum freedom with when and how objects can be created and destroyed (and implicitly, how theyre accessed). this is likely to bottleneck performance (in the OS kernel no less) in scenarios where this is to happen frequently - eg, missile spam that goes on for an extended period of time.

i was concerned this might happen but didn't think it through far enough - mainly because i wasnt at the point where i could progress further. two steps forward, one back. now i have a very good picture in mind of how and when objects are interacted with... it's a non-choice to re-do this growing in my mind by the minute as i think about it.

so.

im re-arranging things so the scripting / game object environment is less a 'state' type object and a proper object 'manager' - this means game objects will actually now store their own data. i have a few tricks up my sleeves to make that approach work more efficiently than the ideal possible with the current approach.

in the 'oven' build it nearly went this way, but splitting out the object data storage was something i wanted to see through and i had a sort of hybrid in the version following it. ive hit a wall with how far the performance can go that way, so taking a step back now and re-writing it for the better.

i have a very good idea of what needs to happen, but regardless this does set me back a few days. good news is there aren't too many changes that need to happen - the pipeline in and of itself is fine and dandy for the most part, it's the object data that's now causing grief, but the rewrite will require a rewrite of the pipeline in full etc.

i don't know yet if this will throw off the other ETA's by too much, certainly the short term stuff will be pushed out a week (realistically). but then, i more or less expected something like this to happen this week anyway...

more updates soon.

ok.... it's been a long weekend. much revising, much reading on the subject matter.

wont go into too much detail - i identified the right area for improvement, but using assumptions and thought-trains from a while ago, what i suggested there perhaps is not the best solution.

TL;DR partial states.

the engine already uses the concept of object data copy and object data delta. it sort of does this with the message queue (inbox and outbox respectively). clearly there are two elements to any object data, the working copy and the changes.

if one generalizes the concept there is an active state and a partial state in the game engine.

that's basically how i'm restructuring things atm. the engine has a persistent state for objects, and each thread generates a partial state that is merged into the persistent / active state at the appropriate phase.

due to how i'm storing object data these merge operations are highly efficient:

1. i already use master lists of objects, eg objdata[ i ][ "class" ] = "player";

2. each object already can be modified by only one thread per cycle (well, per phase but still).


so thread one might have { obj1 : { playername : "me"} } and another thread { obj2 : ... } (you get the idea). if you know your mergesort, you'd know that it's trivial to combine two such maps of properties when the maps do not share keys (and the keys are already ordered) and can be done in linear time. (should mention that the way i'm coding this will likely be suboptimal at first, until i'm not too lazy to actually transfer pairs as most likely they'll be reallocated, but time and a place ditto a more efficient new / malloc impl)

i already use this merge trick with object property updates, this is scaling up the behavior to the entire game state.

the benefit is that object reads come from the active state, which now has a guarantee about no-writes during specific phases - in short i can throw out almost all locks during the scripting phases and still have scalable code. the brunt of this is moved into the update phase (needs to be reworked now as a result) which can merge the partial states at the end of each cycle into the active state.

the remaining sync points in scripting are still vital to be there, such as creating new objects, but unlike right now the only reason locks are required is to avoid duplicate object id's. ie, the number of locks goes down from 6 ish to 2.

bonus, due to the restructuring i'm doing the code is being set up for engine modules to be created now.

so yeah. things moving along. slowly but surely. unfortunately as expected this does throw things out by a week...

expecting to start on engine modules this coming weekend.


PS.

if you're wondering why i do not intend on sticking with read/write locks, it comes down to their performance even in the unlocked case.

the logic for a readwrite lock (in my implementation) works like this -

you need an atomic (or mutex-locked) counter C, and a binary semaphore S.

to lock for writes, use the semaphore. to unlock for writes, also just the semaphore. easy.
to lock for reads, increment C and check if this is the first 'reader', if it is then do a writelock, otherwise assume another reader has already.
to unlock for reads, similarly decrease C and see if this is the last reader and if so do a writeunlock.
if reading code is easier - https://gitlab.com/TheAnger/zero-volt/blob/master/Source/Core/Sync/readwrite.hpp

some people say you can do this with just two mutexes, but actually you cannot - a mutex MUST be unlocked by the same thread that locks it, so with a queue or RR of readers that doesn't happen and nothing nice follows.

basically to use a readwrite lock in read mode only, with no expected writes (or rarely occurring writes) you have to use two synchronization primitives, each time. this would have happened every single time an object property was read, albeit for the first time per-cycle per-object but that's still a tremendous number of sync operations per-frame. it'd be far better to have that work lock-free, no?

things are moving along - just finished prototyping the partial state merge process. this will be a backbone of several types of operations, such as integrating network traffic into the game state (much later on) and things like that.

code isn't hard to understand if you're curious how it works - https://gitlab.com/TheAnger/zero-volt/blob/master/Source/ZVOS/state.cpp

in hindsight i should eventually switch from queues to lists, not concerned at this stage because of much larger performance fish.

anyway. with the merge step basically done it's onto re-creating the pipeline, which should take 2-3 days from this point. reason being is there have now opened up a lot of possibilities and opportunities for parallelization using this approach. i'll try not to get stuck optimizing it in this round of changes, but just today i realized it's possible to now carve up the multiprocessing any which way, importantly: being able to combine the message and update phases for a single game object.

right now i'm mulling over iteration and object pooling for threads (same reason as above); soon as i have a good answer for that, the pipeline will write itself just about. trivially i could re-use set-iterators again, probably will if i dont have anything better in 2 days time.

answer found, and sure enough the thing wrote itself in the space of a couple of hours.

ignoring engine modules (which i will get to after a round of testing the architecture) there is mainly just the scripting API to re-implement - something i've done successfully twice before and am mostly just copying code across.

the new pipeline is officially a 3-step pipe (as opposed to 6 before) - object processing, partial state merge from processing-type threads, and engine modules' turn with game data (gfx, sound, input, phys etc). there's ample room for further improvements still but there is no need to go further with optimizing the pipeline just yet - it's already worlds ahead of the previous design and i want to be sure it works before i make further assumptions while optimizing.

ETA - module-stage testing this weekend. if that works, it's a green light for implementing engine modules.

incidentally i've re-arranged things in the source a bit, there is now a clear split between game data and the processing thereof - ZVOS (zero volt object system) is the object API, it does nothing on its own just gives the means to, and the Engine source which is the conventional architecture code (the stuff putting it all together from top-down).

on track for something to show in 1-3 weeks time (depending how testing goes).

making a fair bit of headway, small changes at a time.

the bulk of what the engine does every cycle comes down to this one file - still a work in progress - https://gitlab.com/TheAnger/zero-volt/blob/master/Source/Engine/script-thread.cpp

hard as it is to believe, it won't be much larger than 200 lines, if even that much. most of the magic has been abstracted into other areas.

still to do: Lua API (moved out of core components and into engine implementation), then using said API to finish the above file, and.... that's it! next come modules, the abstract template for modules that is, which cements the engine class and in turn the engine cycle, then the modules themselves.

input and graphics will be the first two to a basic degree. depending how much time i find that could take as little as a day to get working. realistically 3-4 from today. at that point game dev can start though i'll be frequently switching back and forth between working on a game and working on the engine to make working on the game easier still (or at all possible at times).

it's finally happening \o/

ok, it's been a challenge but finally i've re-implemented (and improved upon) the lua api framework.

it's 5am now and i've just gone through 11 or more commits, i dont even remember exactly, im going to sleep. BUT.

when i test this tomorrow, if it goes well, it's onto modules. finally. last few steps and then the fun really starts...

still WIP, but some progress.

it's at that critical stage where the whole thing is being put together using very large and complicated jigsaw pieces and they have to fit and look right. right now they look right but don't quite fit.

more technically - i've found that the scripting portion of the engine is in fact now able to run in parallel with the modules in general. This does simplify things but as usual i've overspecialized with keeping scripting its own beast.

ill be set back another few days re-doing it. on the other hand i now have a crystal clear picture of what the module system will look like, and am restructuring things slightly to push ahead in that direction - fully modular engine design.

i expect partial states will also be reworked to cater for more than just the game state - the concept proved more than useful, it's now invaluable to this sort of engine architecture.

ETA with how things look now: 1-2 days for module system / abstraction to be created, 1-2 days for game state separation from new abstractions (specialized iterators, partial states etc), 1-2 days for re-writing scripting as a module. By the time this is done I'll have started on graphics and input modules. all in all that's 3-6 days away, depending on how i do in between for time.

progress -

partial states - partial states have been successfully abstracted out, and scripting / object states have been redefined in terms of the abstract state class.

modules - basic framework underway, the class structure is basically done and now i'm fattening it up with the actual module interface. the class structure for modules has been tricky, since they require an abstract state for data storage. this means all modules (except scripting for obvious reasons) will need to define their own state type if they also have data that needs to undergo the same run/update phases.

scripting-as-a-module - 50% done, the states have been refactored already, leaving the thread portion. this is waiting on module structure since they have to be done side-by-side with this module.

still to do: finish engine class (25% done), finish module framework (~50% done), finish scripting module (~50% done), start other modules. within the next 4 days, still on track to do so. will be busy this weekend so may be off by another day or two.

ETA's - module framework (by end of week), scripting module (by end of week), engine class (by end of week), framework dry test (weekend), graphics & input module start (weekend), commencement of game(s) (next week some time).

friday update on ETA - making some solid progress. falling about a day - 2 days behind but on the other hand other parts will be faster to write after this round of changes.

you can track what's happening with the code here - https://gitlab.com/TheAnger/zero-volt/activity - without waiting for me to say anything. you'll probably see a commit message to the tune of "holy crap it works" soon enough, then you know its ready

still here, haven't run off anywhere... i'm having to step back and really think this through. sidetracked to create some extra parallel processing constructs but in the last couple of days have been refactoring and reducing the new code to bare minimum.

what it boils down to is a need for Future objects or similar. Promise objects as they're known elsewhere. as opposed to full-fledged thread pools and multi-tiered thread structures.

that way the module run phase is only the following steps: do object actions, notify Future that this module is finished with actions, wait on 2nd Future object to proceed with updates, do updates, notify 3rd Future object that module has finished run phase.

the engine will coordinate this back and forth of asynchronous results, and this is basically all the engine needs to do. the rest is up to what the modules can contribute.

my weekend proved busy and this week has been tiring, hopefully ill get more done this weekend, without elections and family and things.