Poll on Architecture and Frameworks

[Daklu]

On 10/3/2019 at 3:08 AM, drjdpowell said:

A big part of the payoff of a good framework is making large complex applications simpler. But it's hard to see this when you're on the wrong side of the learning curve.

Hmm... I'd slightly change that to say the payoff of a good architecture is in managing an application's inherent complexity.

IMO, the whole point of a framework is to increase productivity, so a "good" framework is one that does that.  Naturally, the frameworks we're talking about here are neither wholly good or bad, just good or less-good for a specific set of project requirements. 

I disagree with the claim that a good framework makes complex applications simpler.  An application with complicated requirements is going to have some minimum amount of inherent complexity.  The best a framework can do is help eliminate accidental complexity.  However, I believe minimizing accidental complexity is more a product of good architecture than a good framework.

One of the reasons I don't use frameworks is because they force developers to fit the application into the framework and tend to impose architectural decisions that are better left to the system designer.  The best frameworks (again, IMO) do not impose undesirable architectural decisions on the system designer.

 

[Idle thought:  I suppose it might be useful to differentiate between "application frameworks" and "component frameworks."  My comments above refer to application frameworks, which I'm not convinced exist yet in Labview.]

[ShaunR]

28 minutes ago, Daklu said:

Funny you mention this.  Here's a slide from the most recent version of my Fundamentals of AOP presentation.

No link? For shame! 

[Michael Aivaliotis]

1 hour ago, Aristos Queue said:

I've got an 80-slide PPTX on actors as first-class citizens: no queue management, no classes for messages but retaining type safety, no weird error codes, easy handling of parallel loops without custom stop signals, direct execution testing, debug monitoring... but I just don't see it happening in LabVIEW.

Does this mean the Actor Framework functionality will now be represented graphically in NXG, as apposed to a list of hundreds of class VIs in a tree in the project?

[drjdpowell]

2 hours ago, Daklu said:

  The best a framework can do is help eliminate accidental complexity. 

That "accidental complexity" is a killer.  It's a big part of that exponential increase in complexity you showed on your graph a couple of posts ago.  Helping to get that out of the way so you can address the true complexity of the actual problem is no small thing.   

[drjdpowell]

2 hours ago, Daklu said:

One of the reasons I don't use frameworks is because they force developers to fit the application into the framework and tend to impose architectural decisions that are better left to the system designer.  The best frameworks (again, IMO) do not impose undesirable architectural decisions on the system designer.

Yes, I try to do that with "Messenger Library".  Unfortunately, I find that if I stress that it is a library of messaging (with optional "actor" templates) that it gets dismissed as not to be considered when comparing "frameworks", but if I present the whole thing as a "framework", then people assume it is very restrictive.

[ShaunR]

6 hours ago, Daklu said:

However, I believe minimizing accidental complexity is more a product of good architecture than a good framework.

 

3 hours ago, drjdpowell said:

That "accidental complexity" is a killer.

What is "accidental complexity"? This sounds like an excuse given to management.

[drjdpowell]

I think Daklu meant when your program is more complex than the problem it is solving.  My problem with calling this "accidental" is that it sounds easy to avoid if you're careful, when really it is very hard to avoid lots of extra complexity.  

An example that comes to mind is shutdown/cleanup code.  A good framework will handle cleanup simply and easily (this is an area I think Messenger Library is strong).

[Aristos Queue]

21 hours ago, Daklu said:

Is this restricted to AF actors specifically, or would our own actors also be first-class citizens?

I'm not sure how to answer this question. To me, it's equivalent of asking whether LabVIEW will let you add your own definition of VIs. NXG has the ability for you to create your own models of computation. To make your question more confusing, these actors aren't AF actors... the AF is a library where actors are constructed out of bits and pieces of language that we have available. This would be its own thing.

Regardless, it is years away, and I don't want to hijack Powell's discussion on things that exist today.

[Aristos Queue]

14 hours ago, ShaunR said:

What is "accidental complexity"? This sounds like an excuse given to management.

Allow me to introduce you to implied spaces.

When I build a two-story house, I consciously add a staircase between the zeroth and first floors. I add handrails for safety, optimize the height of the steps for average human legs, etc. I spend a lot of time designing the staircase.

What I don't spend a lot of time designing is the room under the stairs. I put a door on it and turn it into a closet, a storage place for the people who live in the house.

Now, the people who live in the house start using that storage space -- exactly as intended. But after a while, they are complaining that frequently, they need something at the back of the storage space, so they have to take everything out to get it and then put everything back in.

You ask me, "Didn't you put other closets in the house?! Why aren't they storing more things in the other closets?" I did add other closets: I wasn't that short-sighted. But it turns out that this staircase closet is taller than any of the others, so it holds things nothing else holds... wasn't intended, just happens to work because it is under a two-story staircase. Also, this is central in the house, so it is closer than the other closets, so the users think that the time needed to pull everything out to get to something at the back isn't *so* bad.

The users of the space made it work, but there is accidental complexity in how they have evolved to use it. I didn't do anything wrong in the design, they didn't do anything wrong in giving me their requirements. It just happened with no one at fault.

With this new understanding of my users, I refactor the house and add a second door on the short end of the stairs so people can pull from either end. Suddenly the under-the-stairs closet is not an implied space but an intended space.

It doesn't matter how much you refine a design, there are always places that are implied within the design that are not spec'd out. It's a macroscopic aspect of Godel's Incompleteness Theorem. Some things aren't designed; they just work the way they work because they're near the things that are designed. And when users start relying upon that implied functionality, that is accidental complexity.

Inspiration for this post from the novel Implied Spaces by Walter Jon Williams and Whit by Ian Banks, two science fiction novels that happened to give me good advice on software design. Accidentally... I think.

Edited October 16, 2019 by Aristos Queue

[ShaunR]

54 minutes ago, Aristos Queue said:

The users of the space made it work, but there is accidental complexity in how they have evolved to use it.

Nah. Don't buy it. This is a change in requirements and there is no added complexity of the space itself.

54 minutes ago, Aristos Queue said:

Some things aren't designed; they just work the way they work because they're near the things that are designed. And when users start relying upon that functionality, that is accidental complexity.

This is still a change in requirements and this is definitely an excuse for claiming value-added when no intention to add exits! Just because a user infers a feature that was never offered; it doesn't mean that the code is more complex. It just means the User (or Product Owner) has identified a new requirement. We were talking about code complexity growth and the term "accidental complexity" implies some sort of hidden cost - unkown or impossible to know, at design time (from what I can tell).

This is why I asked for clarification. I've never heard of it and it just sounds like an excuse.

7 hours ago, drjdpowell said:

I think Daklu meant when your program is more complex than the problem it is solving. 

By that definition, wouldn't the framework itself be an "accidental complexity" rather than the "considered and acceptable" complexity of a tried and tested template for design?

Maybe I'm just getting too hung up on "accidental" and what it implies.

Edited October 16, 2019 by ShaunR

[Daklu]

44 minutes ago, ShaunR said:

Maybe I'm just getting too hung up on "accidental" and what it implies.

The things I've read always refer to it as "accidental" complexity, but you can think of it as "unnecessary", "incidental", or simply "non-required" complexity.  It's just complexity in a system that isn't a necessary to meet the requirement (both functional and non-functional) of the system.  System complexity is a subjective evaluation, so these are more abstract ideas than concrete rules to follow.

[drjdpowell]

54 minutes ago, ShaunR said:

By that definition, wouldn't the framework itself be an "accidental complexity" rather than the "considered and acceptable" complexity of a tried and tested template for design?

What is a "framework" but a tried and tested template for design combined with a support library and hopefully some productivity tools and documentation?  

[Daklu]

1. What application architecture do you use for higher-level organisation?

I'm going to be pedantic and change my answer, since the options listed are frameworks, not architectures.  😋

The architecture I most frequently use is the "hierarchical actor architecture."  If one were to ask for a little more detail, I'd say "hierarchical actors with asynchronous name-data messages."

[ShaunR]

52 minutes ago, drjdpowell said:

What is a "framework" but a tried and tested template for design combined with a support library and hopefully some productivity tools and documentation?  

Exactly, but according to your definition it would be "accidental complexity".

50 minutes ago, Daklu said:

1. What application architecture do you use for higher-level organisation?

I'm going to be pedantic and change my answer, since the options listed are frameworks, not architectures.  😋

The architecture I most frequently use is the "hierarchical actor architecture."  If one were to ask for a little more detail, I'd say "hierarchical actors with asynchronous name-data messages."

This is why I said in an earlier post that people confuse architecture and frameworks. I personally use a SOA architecture but within each service I may use QMH or Actors or whatever framework best achieves the goal. Many people choose one framework and fit everything inside it, making it their architecture. And lets be frank (or brian). Most LV frameworks are written and intended to be just that.

59 minutes ago, Daklu said:

The things I've read always refer to it as "accidental" complexity, but you can think of it as "unnecessary", "incidental", or simply "non-required" complexity.  It's just complexity in a system that isn't a necessary to meet the requirement (both functional and non-functional) of the system.  System complexity is a subjective evaluation, so these are more abstract ideas than concrete rules to follow.

So LVOOP is "accidental complexity"?  (Just teasing)

I don't really think it is a thing by these definitions when talking about complexity. Rube goldberg code exists but it isn't really "accidental". It is the product of brute forcing a linear thought process rather than iteratively designing one. Neither case is "accidental". Bolt-on bug fixes to cure the symptom rather than the cause might be argued as "accidental complexity" but that is just bad practice (of which I'm sure we are all guilty at some point).

From the feeback it seems more of a weazel phrase for inelegant/inefficient code (except AQs take on it) in order to not admit it as such. I suspect this phrase is only used when things go wrong on a project and probably has an unquantifiable quality about it..

Edited October 16, 2019 by ShaunR

[drjdpowell]

46 minutes ago, ShaunR said:

1 hour ago, drjdpowell said:

What is a "framework" but a tried and tested template for design combined with a support library and hopefully some productivity tools and documentation?  

Exactly, but according to your definition it would be "accidental complexity".

No, my original claim (that Daklu reacted against) was that a good framework reduces complexity.  Working without a tried and tested...something (could just be a standard way of working that you are very expert in, but a "Framework" adds a library that supports that standard) leads to code that has extra complexity in it.  Often that extra complexity is not obvious, but that's the worst kind of complexity.  A particular bugbear of mine is NI templates like "QMH" and  "Continuous Measurement and Logging" template/example that come with LabVIEW, which you might think are simple.   I consider them very over-complicated.  I've given more than one talk pointing out weaknesses in CM&L.  

[Aristos Queue]

On 10/15/2019 at 1:42 PM, Michael Aivaliotis said:

Does this mean the Actor Framework functionality will now be represented graphically in NXG, as apposed to a list of hundreds of class VIs in a tree in the project?

No. The AF remains as it is in NXG. This would be something new. 

[ShaunR]

2 hours ago, drjdpowell said:

No, my original claim (that Daklu reacted against) was that a good framework reduces complexity.  Working without a tried and tested...something (could just be a standard way of working that you are very expert in, but a "Framework" adds a library that supports that standard) leads to code that has extra complexity in it.  Often that extra complexity is not obvious, but that's the worst kind of complexity.  A particular bugbear of mine is NI templates like "QMH" and  "Continuous Measurement and Logging" template/example that come with LabVIEW, which you might think are simple.   I consider them very over-complicated.  I've given more than one talk pointing out weaknesses in CM&L.  

OK. But good or bad wasn't the question. I was after the definition of "Accidental Complexity" and what you've just said brings me back to what I said originally

5 hours ago, ShaunR said:

wouldn't the framework itself be an "accidental complexity" rather than the "considered and acceptable" complexity of a tried and tested template for design?

Here I am saying that the underlying complexity of the framework is a necessary evil that has been "accepted and considered" rather than "accidental". What you seem to be confirming from my interpretation of your suggestion is that any hidden complexity is "accidental" in the context of the meaning and therfore a Framework is accidental complexity. 

Anyway. I've pretty much come to the conclusion that it's just more of a woolly buzz phrase like "Synergy" and "The Cloud". It obviously means different things to different people and I've a sneaking suspicion that it's meaning depends on where blame will be apportioned 

[drjdpowell]

Thought I'd show an example of "complexity" of a framework, according to my way of thinking, by comparing the priority messages of the NI Actor Framework and the DQMH:

Looks the same from an API level (they even use similar icons).  Let's look inside; here is the relevant code section for priority messages for the AF:

Yikes!  And here is the same for DQMH:

Ah, much simpler.  Now we can see which framework involves more complexity: the DQMH.

Wait, what, you say?  Isn't the obvious complexity of the AF implementation mean the AF involves more complexity?  Well, no, because I, as User of a framework, care nothing about the implementation, I care about application I am building with these APIs.  So let's consider the task of sending three high-priority messages, A then B then C.   In what order will the three messages be received and acted on?

With the Actor Framework the order will be A, then B, then C, ABC, always.

With the DQMH we have:

1) if the receiver can execute them faster than they are sent, the order will be ABC

2) if the receiver is handling another message the order will be CBA (as we place on the front of the queue)

3) if the receiver is idle, but executing A takes time (allowing C to get before B), the order will be ACB

4) if busy but finishes after B is sent but before C is sent, the order is BCA 

5) as (4) but B is finished executing before C sent, ther order is BAC 

Thus with the DQMH there are 5 possible orderings of execution, with the probability of the various orderings highly dependant on timing of not-directly-related bits of code (other messages being sent).  At best, this is counter-intuitive and potentially confusing during debugging.  At worst, one combination is a rare race condition that doesn't show up during testing and causes near-impossible-to-debug errors in deployed code.

So that is an example of complexity, and it is certainly accidental, as the DQMH designers did not intend unpredictability of message-handling order when they used enque-in-front as message "priority".   

 

Edited October 17, 2019 by drjdpowell

[Daklu]

20 hours ago, ShaunR said:

This is why I said in an earlier post that people confuse architecture and frameworks...  Many people choose one framework and fit everything inside it, making it their architecture.

Yeah, I agree with you for the most part.  Maybe that's because the applications LV developers are typically asked to write are a small fraction of the kinds of software that can be written.  My projects almost always boil down to some sort of machine control or acquire/analyze/present.  (I've not heard of anyone building a load-balancing web server, a cool 3D game, or a spreadsheet using LV.)

 

20 hours ago, ShaunR said:

And lets be frank (or brian). Most LV frameworks are written and intended to be just that.

Eh... that may be true for some of them.  There's an inherent impedance in connecting non-AF code to AF code that tends to encourage people to use AF for the entire app.  (At least there was last time I looked into it 6-7 years ago.  It might be better now.)  Most of frameworks listed I haven't used enough to evaluate how easy it is to connect to them with non-framework code.  (DQMH, Aloha, DCAF, etc.)  I agree that it would be nice if it were easier to connect components that use different frameworks.

 

20 hours ago, ShaunR said:

So LVOOP is "accidental complexity"?  (Just teasing)

It certainly can be, but it's not necessarily the case.  Remember that that app has to satisfy all requirements, not just the functional requirements.  I believe any technical problem that can be solved by LVOOP can also be solved without it.  But my projects also have requirements like the number of hours I can spend developing it and it must be maintainable by John Doe.  So, I choose to use LVOOP because I don't have to build the functionality I'm looking for and John Doe doesn't have to learn my custom LVOOP-ish code.

 

20 hours ago, ShaunR said:

I don't really think it is a thing by these definitions when talking about complexity. Rube goldberg code exists but it isn't really "accidental". It is the product of brute forcing a linear thought process rather than iteratively designing one. Neither case is "accidental".

I agree the terminology is bad, but you're getting too hung up on the common meaning of "accidental."  In this context it is used simply as an antonym of "inherent complexity."  (I've also seen "inherent complexity" referred to as "essential complexity," if that helps conceptualize it.)

[Daklu]

1 hour ago, drjdpowell said:

Thought I'd show an example of "complexity" of a framework...

That's a great example, and I'll add that I think your conclusion holds even if you didn't look at the implementation.

On the surface the conceptual model presented by the AF seems more complex.  It has multiple priority levels available whereas the DQMH has only a single priority level.  But the AF implementation behaves predictably--messages sent in order (with the same priority level) execute in order, while the DQMH implementation may not.

On the other hand, I think Fab strongly recommends only sending Exit/Abort messages as priority, and as long as one follows that rule you could argue DQMH is simpler.

------

Thinking about it a bit more, in my mind there are at least three different kinds of complexity:

1. Code complexity:  How hard is it to understand a block diagram.
2. Behavioral complexity:  How hard is it to understand how a single component will behave under various conditions.  (Or how complicated is the conceptual model of the component.)
3. System complexity:  How hard is it to understand how changes in one part of a system affect other parts of the system.

Seems to me the DQMH priority message is simpler than the AF priority message in 1 and (arguably) 2, but takes a huge hit to 3 when multiple priority messages are sent.

Edited October 17, 2019 by Daklu

[ShaunR]

2 hours ago, drjdpowell said:

Thought I'd show an example of "complexity" of a framework, according to my way of thinking, by comparing the priority messages of the NI Actor Framework and the DQMH:

As far as I'm aware. There is no guarantee (or expectation) that priority queues enforce ordering, only that higher priority messages will be executed before lower priority messages.

I'm not familiar with the internal workings of the AF but if what you say is true (that order, at the same level, is guaranteed) then more of what you term "complexity" happens when that isn't required. An emergency stop springs to mind where you may not want the previous buffered messages to be executed, just the E-Stop. With the AF (based on your description) the user has to categorise different messages to different levels and I would suspect you woud also argue that is a "complexity". I wouldn't, however. Neither would I for the DQMH.

I take your point about debugging and difficult to diagnose for the DQMH under certain conditions but it is a limitation of the design, and probably adequate for most scenarios if you don't make the guaranteed order assumption. The AF code also means practical debugging complexity due to the code paths which doesn't exist in the DQMH. So it's all 6 of one and half-a-dozen of the other to me. I expect the reverse order is more surprising to most people but it probably compiles and executes significantly faster than the AF one (just a hunch) so the limitations may have been a compromise to that. If you need a priority queue that guarantees order then that feature in the DQMH is not for you but I go back to my original statement that "here is no guarantee (or expectation) that priority queues enforce ordering, only that higher priority messages will be executed before lower priority messages." (this is a discussion that crops up in task schedulers too, by the way).

Edited October 17, 2019 by ShaunR

[Daklu]

6 minutes ago, ShaunR said:

As far as I'm aware. There is no guarantee (or expectation) that priority queues enforce ordering, only that higher priority messages will be executed before lower priority messages.

The AF priority queue does enforce ordering.  The DQMH priority queue does not.

(I'm not advocating for one or the other... personally I think priority queues are a bad idea.)

Edited October 17, 2019 by Daklu

[ShaunR]

6 minutes ago, Daklu said:

The AF priority queue does enforce ordering.  The DQMH priority queue does not.

(I'm not advocating for one or the other... personally I think priority queues are a bad idea.)

Yes. I'm saying the "concept" of priority queues doesn't.

[ShaunR]

1 hour ago, Daklu said:

Thinking about it a bit more, in my mind there are at least three different kinds of complexity:

1. Code complexity:  How hard is it to understand a block diagram.
2. Behavioral complexity:  How hard is it to understand how a single component will behave under various conditions.  (Or how complicated is the conceptual model of the component.)
3. System complexity:  How hard is it to understand how changes in one part of a system affect other parts of the system.

Seems to me the DQMH priority message is simpler than the AF priority message in 1 and (arguably) 2, but takes a huge hit to 3 when multiple priority messages are sent.

This I can get behind!

2&3 also map onto Whitebox and Blackbox testing. There is a test for 1 that I have seen (can't remember off hand what it was called) but it was mainly for c/c++ and counted things like the number of if/else or entries in Case statements to arrive at a figure for "complexity"

Edited October 17, 2019 by ShaunR

[drjdpowell]

I'd expect there to be a very strong expectation that anything called a queue enforces ordering, as that is what the word queue means.