Jump to content

DVR or data copy? How to deal with large data in AOP projects


Recommended Posts

I'm working on a large project and have run into a quandary about how to best deal with a large shared data set.

I have a set of classes that define 3 data structures.  One is for a script read from disk that the application executes.  Another is the output data from the program, generated as the script is executed.  The last is a summary of the current state of execution of the script.  The first two can be significant in size and are unbounded.

In my application, I have one actor (with sub actors) responsible for reading the script from disk, executing it and collecting the data.  It also updates the summary status data.  Let's call this the control actor.  I have another actor that takes the data and displays it to the user, allowing them to navigate through it while the script is being executed.  Lets call this the UI actor.  The last actor is responsible for communicating the summary to another application over the network.  We will call this the comm actor.

So, the control actor is generating the data and the other two are consumers of the data.  I had originally thought to have all the data stored as state data in the control actor and then as it is updated/created just message it to the other actors.  But then they would essentially have to maintain a copy of the data to do their job.  This seems inefficient.  Then I thought I could wrap the data classes in DVRs and send the DVRs to the other classes.  That way they could share the same data.  The problem with that is they would not control when their data gets updated.  And I am violating the philosophy of actors by creating what is essentially a back channel for data to be accessed outside of messages.  Also, I could block the 'read' actors when I am writing to the DVR wrapped class.  I would have to be careful when updating subsections to lock the DVR in an in-place structure to do the modify and write.  Then comes the question of how to best alert the readers to changes made to the data by the writer.  Simpy send them a message with the same DVRs in it?  Or send a data-less message and have them look at the updated values and take appropriate action?

 

So, any best practices or thoughts on how to resolve this issue?  I appreciate any feedback..

 

-John

Link to comment

I have an application where there is a lot of data moving around and I use DVRs for the data transport and user events to signal updates to any interested processes. Some of the processes are started with the application, while others are spawned and destroyed as users select various things to view in the UI portion (separate application). I haven't really looked at the actor framework to know if this is a violation of the intent, though.

-Chris

Link to comment
  • 1 month later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.