This is the first of my old “Code Samples” that I’m updating into this new more bloggy form. There will probably be less information that other newer entries, but I’ll try and add more than what was originally present in that post (which was basically just the comments in the file) to talk about why things were designed the way they are, and what I would like to add to this in future.

All code content in this file is © 2017 DigiPen Institute of Technology.

To start with, lets take a look at the public interface for the Resource Manager class.

class ResourceManager
{
  public:
    /*
      Creates a resource manager
    */
    ResourceManager(Engine* engine);

    /*
      Clean up the threadpool, after stopping the threads if necessary
    */
    ~ResourceManager();

    /*
      Add a Resource to the manager with a given load priority, then
      returns a pointer to that resource. Be sure to check the resource's
      GetIsLoaded() flag before using as may be loaded on another
      thread
    */
    template <typename T>
    T* AddResource(
      std::string filename,
      unsigned int priority,
      LoadCallback callback = nullptr);

    /*
      Gets a Resource if it has already been loaded. Otherwise, adds a Resource
      to the manager with default maximum load priority, then returns a pointer
      to that resource. Be sure to check the resource's GetIsLoaded() flag
      before using as it may be being loaded on another thread (its not
      guaranteed to be ready at this call)
    */
    template <typename T>
    T* GetResource(std::string filename);

    /*
      Load any resources that were requested that cannot be loaded on
      background threads. This should be called in the engine's main loop.
    */
    void LoadSyncLockedResources();

    /*
      Spin down the thread pool and prepare for shut down
    */
    void Cleanup();

    /*
      Start up the ResourceLoading worker threads
    */
    void StartLoading();

My primary goal with the public facing interface of this class was to minimize the complexity of the interface, whilst maximizing the potential uses of this system. The two methods that users of the class mainly touched were extremely simple to use, simply requesting a resource and getting back a pointer of that type. I decided to use templates here to prevent the user from needing to use messy and potentially error-prone casts on a generic Resource* type.

private:
    /*
      Function object to use with the toLoadQueue

      Handed to std::priority_queue to use as its sorting operator
    */
    struct ResourceCompare
    {
      bool operator() (const Resource* lhs, const Resource* rhs)
      {
        return lhs->GetPriority() > rhs->GetPriority();
      }
    };

    /*
      Implementation of the engine System interface
    */
    void UpdateSpace(Space* space, float DeltaTime);
    void UpdateSystem(float DeltaTime);

    /*
      Queue of resources to load
    */
    std::priority_queue
    <Resource*, std::vector<Resource*>, ResourceCompare>
    toLoadSyncQueue_;

    /*
      Map of all currently resources that have been requested

      NOTE that a resource being in here is NOT a guarantee of
      that resource already being loaded, just that someone has asked for it,
      so be sure to check resources before using for the first time
    */
    std::map<std::string, Resource*> resources_;

    /*
      Since the load queue will potentially  be accessed by multiple
      threads simultaneously, declare a mutex variable that can be
      checked by each job using this list before using it to ensure
      that things don't get fucked up

      I know there is sometimes a red squiggle here. VS doesn't
      understand that this is a macro call. The compiler likes
      it though, so too bad VS.
    */
    DECLARE_MUTEX_VAR(toLoadSyncQueue_);
    DECLARE_MUTEX_VAR(resources_);

    /*
      Pool of worker threads to handle the loading
    */
    ThreadPool::ThreadPool* workers_;

    //Should the system currently be loading resources?
    bool shouldLoad_;

    DECLARE_JOB(ResourceManager, LoadResource, Resource* resource);

    /*
      Declare the FileWatcher job privately
    */
    DECLARE_JOB(ResourceManager, CheckFilesForUpdate, int delay);
  };

I learned a lot about priority queues from implementing this system. In part, what I learned is that the declarations for the standard libraries priority queue get pretty ugly pretty fast, but thats the price we pay in C++. The priority queue for loading resources made it very easy to ensure they were loaded in an order that respected the importance of the request.

I opted to use a map for the resources because fast random access time was my top priority. This may have been a situation where unordered map would have proved faster, and this will definitely be an area for investigation should I revisit this system again for a future project.

This section was actually simplified significantly by my job system. This was an extremely primitive job system, with all jobs sharing a single queue for the sake of simplicity of implementation, but it did enable this system to be implemented on multiple threads relatively quickly. I wrote this basic job system a long time ago when still had a lot of habits from C, hence the heavy usage of macros for declaring jobs. Believe me when I say you don’t want to see the definitions of those. Perhaps at some point in future I’ll attempt a dissection of that particular monstrosity now that I know all the ways in which it was wrong. Essentially, what the DECLARE_JOB macro does is declare a function that is immediately ready to be submitted to a set of worker threads instead of being executed on the main thread. This function will be defined later with a similar macro.

template<typename T>
  inline T* ResourceManager::AddResource
    (std::string filename, unsigned int priority, LoadCallback callback)
  {
    //If the resource is already in the system, return it
    if (resources_.find(filename.c_str()) != resources_.end())
    {
      return dynamic_cast<T*>(resources_[filename.c_str()]);
    }
    //Otherwise, add it to the appropriate load queue
    else
    {
      Resource* toLoad = new T(filename.c_str(), priority, callback);
      toLoad->SetManager(this);

      if (T::ThreadSafeLoad)
      {
        LoadResource(workers_, toLoad);
      }
      else
      {
        //Gain mutex lock on the queue
        GET_LOCK(toLoadSyncQueue_);
        toLoadSyncQueue_.push(toLoad);
        //Release mutex lock on the queue
        RELEASE_LOCK(toLoadSyncQueue_);
      }

      //Gain mutex lock on the resource library
      GET_LOCK(resources_);
      resources_[filename.c_str()] = toLoad;
      //Release mutex lock on the resource library
      RELEASE_LOCK(resources_);

      return dynamic_cast<T*>(toLoad);
    }
  }

  template<typename T>
  inline T* ResourceManager::GetResource(std::string filename)
  {
    return AddResource<T>(filename, 0);
  }

These two functions are the bulk of the interesting behavior of the system. As you can see, the primary purpose of the templates was to conceal a cast. I opted to use a dynamic_cast so that the system would smoothly return nullptr instead of either failing or returning something with the incorrect pointer type. This way if a user requested “image.png” as a lua script type object they would simply get nullptr back, which they could handle gracefully. I knew that dynamic_cast would be safe to use here, since all Resource types extended Resource, and had to override some pure virtual functions there and so I knew that the virtual function table must be present.

/*
    Constructor, implements Engine System Constructor
*/
  ResourceManager::ResourceManager(Engine* engine) :
  System(engine, "ResourceManager"), shouldLoad_(true)
  {
    workers_ =
      new ThreadPool::DynamicJobQueue(SystemUtilReport::GetNumCores() - 1);
    toLoadSyncQueue_ =
      std::priority_queue<Resource*, std::vector<Resource*>, ResourceCompare>();
    resources_ = std::map<std::string, Resource*>();
  }

  /*
    Destructor, Cleans up threads and removes worker pool
  */
  ResourceManager::~ResourceManager()
  {
    Cleanup();
    delete workers_;
  }

  /*
    Spin down all worker threads and delete resources
  */
  void ResourceManager::Cleanup()
  {
    workers_->ForceStop();
    workers_->WaitForComplete();

    for (auto& resource : resources_)
    {
      delete resource.second;
      resource.second = nullptr;
    }
  }

  /*
    Load the resources that must be loaded on the main thread
  */
  void ResourceManager::LoadSyncLockedResources()
  {
    while (!toLoadSyncQueue_.empty())
    {
      Resource* load = toLoadSyncQueue_.top();
      toLoadSyncQueue_.pop();

      load->Load();
    }
  }

  /*
    Start the loader threads spinning
  */
  void ResourceManager::StartLoading()
  {
    CheckFilesForUpdate(workers_, UPDATE_THREAD_DELAY);

    workers_->Start();
  }

The first half of the cpp file for this class doesn’t really do anything unexpected. The constructor initializes everything, the destructor ensures everything is smoothly shut down and cleaned up, which is only slightly more interesting in a multi-threaded context. Some resource loading is handled here, but its only the things that cannot be handled on another thread. This was used in our game for compiling lua scripts and other things that were unsafe to handle on other threads due to the modification of global state loading them caused.

/*
    Job that runs on another thread that checks files for update
    and then adds itself to the back of the job queue
  */
  DEFINE_JOB(ResourceManager, CheckFilesForUpdate, int delay)
  {
    for (auto& resource : resources_)
    {
      try
      {
        //If a resource is loaded but has changed since it was loaded
        if (resource.second->GetIsLoaded()
          && resource.second->NeedsUpdate())
        {
          //If it is safe to do so, add it to the load queue
          if (resource.second->IsLoadThreadSafe())
          {
            LoadResource(workers_, resource.second);
          }
          //Otherwise, add it to the synchronus load queue
          else
          {
            GET_LOCK(toLoadSyncQueue_);
            toLoadSyncQueue_.push(resource.second);
            RELEASE_LOCK(toLoadSyncQueue_);
          }
        }
      }
      /*
        In order to prevent this process from being stopped, catch any possible
        error and report it to the user.
      */
      catch (...)
      {
        engine_->PushWarning(resource.first + " failed to check for update");
      }

      /*
        Sleep to prevent using all the CPU while idling
      */
      std::this_thread::sleep_for(std::chrono::duration<int, std::milli>(delay));
    }


    //Add this job back into the job queue
    CheckFilesForUpdate(workers_, UPDATE_THREAD_DELAY);
  }

  /*
    Job to load a resource
  */
  DEFINE_JOB(ResourceManager, LoadResource, Resource* resource)
  {
    resource->Load();
  }

Here are those macro instances to match the DECLARE_JOBs from earlier.

CheckFilesForUpdate is designed to check to see if every resource it has already loaded has been modified since it was loaded. If the resource had been modified, it was added back into the queue to be updated. This effectively meant that we could modify the assets of the game while it was running, and see our changes immediately. Designers and artists took advantage of this feature, since it meant they didn’t even have to restart the game to see any changes they had made in engine. This was achieved by having the job, after checking all of the resources and adding them to the appropriate queues, add itself back into the job queue.

The LoadResource job is much simpler, and takes advantage of the polymorphic design of the Resource class to tell whatever type of resource has been requested to load itself correctly. Since this is done on another thread, care must be taken in the implementation of that load function to be thread-safe, but none of that effort is reflected here.


Overall, I’m quite happy with this system. It served its purpose well for the game it was built for. If I were to implement a similar system now, much of it would probably be pretty similar. My main issues are with the implementation of the job system, since I have spent a lot of time attempting to write better such systems since. There are also a few missing features that would present a significant problem in a larger game. The lack of ability for the system to unload resource from memory that are no longer required being the most important.