According to my tests on AF this behavior is normal. AF is just not the fastest messaging framework that can be implemented in LabVIEW. It has two components that make is relatively slow:
Uses priority queues. It makes sending atomic messages(messages with little or no data content) about 2 times slower. More on this: https://decibel.ni.com/content/docs/DOC-24589
Uses dynamic dispatch. Once per message a dynamic dispatch VI is called. It has an overhead and it makes atomic messaging 3 - 8 times slower. Read more on this: https://decibel.ni.com/content/docs/DOC-13709#comment-31559
Please don't take my numbers seriously as accurate measurements, but approximately these are what I measured at least on my PC.
If you read all the links you will understand the reason for AF being slow. I think you can get some speed by leaving out priority queues and use normal queues, but I see no immediate solution for the dynamic dispatch problem.
I have a similar system in work, and what I do is to use as much data in a packet as my GUI's refresh rate requires (so for a 1kHz signal to achieve 25 fps you can send arrays of 40 data points). I see that you use this trick for the Controller, but use it for the Hardware actor also.This way the user still experiences "real-time" behavior, but you spared with messages.
Hope it helps a bit.
[this reply was first posted on AF forums, as I didn't notice the discussion already started here]