Joe,
Regarding the question of "What's the next step after state machines?", I believe that it is component-based architectures. These use the OOP principles of encapsulation and modularity to aid in application design and maintenance. Although components may utilize highly complex messaging, internally, this complexity is not exposed to the users of the components.
If you are interested in looking into using OOP techniques in LabVIEW you might find the links, below, useful.
Graphical Object Oriented Programming (GOOP)
OpenGOOP - OpenG's GOOP framework
I would also take a look at the examples of each of the tools on the Advanced >> Synchronization palette. These are the tools that enable messaging and event driven programming. Messaging and Events enable functionality that is not possible with dataflow, alone. It allows you to define timing and data-flow relationships that are not transmitted through LabVIEW wires.
Once you have a grasp of the basic tools for messaging and events, you should invest some time learning the frameworks, architectures, and patterns, which utilize them.
Good luck,
-Jim