What do YOU want GCentral to be?

[joerghampel]

After discussing with some of the GCentral team and other community members at NIDays Europe in Munich yesterday, I would like to give feedback here so you guys can make of it what you will.

Frankly, I didn't really understand that the "User Personas and User Stories" thread is meant for that - I wouldn't have looked into it if Fab had not pointed me towards it. I would like to suggest to rephrase the title of the feedback thread to something that is more easily understood by the majority of people, not only by the top notch of LabVIEW software developers.  

Moreover, I don't feel comfortable giving user stories for my feedback. That feels to me like phrasing requirements, and at this stage,  I only have vague ideas. If those undeveloped musings are not of interest to you, then just ignore this 🙂

[joerghampel]

The first discussion with Fab, Dani and Jerzy Kocerka was about where to keep the code. We quickly came to the conclusion that it would be great if GCentral did not host their own repositories on their own servers, but rather was able to tap into existing services like Github, Gitlab, Bitbucket and so on.

That might also help with acceptance. Personally, I would like to keep our code in our own repos at gitlab.com. We have our readme's, our documentation platform, etc etc. But if there was an easy way to plug into the GCentral website of available code, I'd love to register our offerings (whatever that might be worth!) and see it featured there.

And also the other way around: I'd like it if I could find not only properly packaged code from the three main repositories (VIPM, NIPM, GPM) on GCentral, but also other offerings in other formats. We like to keep as many dependencies as possible inside our project directories, so we work a lot with packages that are not installed via a package manager but either extracted/copied into the project directory or maybe sometimes linked as git submodules. 

[joerghampel]

The second discussion started - again, with Fab - after James' and my presentation on Open Source and Inner Source with LabVIEW. With Github, for example, it seems rather easy to a) publish an open source project and b) contribute to it, once you've learned the very few steps needed to fork a repo and create a pull request.

I have no idea how that aligns with what you guys have in mind, but it would be great if there was a way to combine the ease of use of existing platforms with a central repository of LabVIEW code and tools greatness.

[Michael Aivaliotis]

GPM extracts code into your project, BTW.

VIPM, NIPM and GPM provide built products of reusable code, among other things. I believe what you are saying is that you consider existing, cloud-based source code control tools such as GIT as a viable option for reusable code distribution. This shouldn't be limited to packaged stuff.

[joerghampel]

10 hours ago, Michael Aivaliotis said:

GPM extracts code into your project, BTW.

Yes, I was quite excited about that! But last time I checked, there was no way for private or local repositories. Is that still the case?

 

10 hours ago, Michael Aivaliotis said:

VIPM, NIPM and GPM provide built products of reusable code, among other things. I believe what you are saying is that you consider existing, cloud-based source code control tools such as GIT as a viable option for reusable code distribution. This shouldn't be limited to packaged stuff.

Thanks for rephrasing. Yes, I think this is what I'm saying. 

[JamesMc86]

I have a few things that might fit a user story - but also some more freeform feedback too -

I'm excited to see some effort and thought going into this area - although early discussions sounds like a lot of technology work where I think there is a solid base already.

• To backup Joergs point as well - I want to use github/gitlab etc. for my open source projects - leveraging what is already there makes it easier to find resources, help and allows me to translate my experience between languages. There should really only be LabVIEW specific elements where that is absolutely necessary IMHO.
• Discoverability is kind of interesting - but when all the other package managers already have sites for this, it doesn't feel like very low hanging fruit.
• Pushing collaboration feels like a great approach. Getting more people owning code and publishing it in a way that people can collaborate feels like something that is lacking. There are so many projects that appear as forum posts or packages on the tools network that lack a public issue tracker, code repo and other things that can impact collaboration. Better still this is mostly an education play requiring less investment and can leverage loads of great existing resources like opensource.guide.

Having a non-gated repo would be of interest though - even if you can just link to a package to simplify the infrastructure that would be a great start.

 

[John Medland]

I am very excited about the potential for a platform that encourages opensource collaboration on LabVIEW code.

My main experience of non LabVIEW package managers is with NPM for Node.js. NPM is an organisation which provides two things - a tool which is the mechanism for managing what packages are used in a project and a registry that allows for anyone to publish their packages to.

I believe that NPM supports private packages for enterprise customers but open source packages are generally hosted on github and when a package is uploaded to the NPM registry it simply pulls in the README to provide the package documentation. The github link is also provided on the NPM page so that users can easily see where the library comes from and if they want to open issues or submit fixes then they do that on github.

I have not had much of a chance to look at it but it appears like GPM would/does follow similar mechanics  to NPM and compared to VIPM and NIPM I am certainly most excited about the GPM model.

I see GCentral as a organisation that could provide the registry for packages and ideally be the one place for opensource LabVIEW code (including NI-community page hosted code) with clear signposts as to where to find the source for issue raising and forking.

One issue that many text based languages don't have is that users with older versions of labVIEW cannot even open code made with newer versions of LabVIEW, let alone run it - so maybe GCentral could provide some computing power (and licences) to automatically convert VIs to older versions -  even if they didn't run, at least a user could open them.

Edited November 27, 2019 by John Medland

[Michael Aivaliotis]

I feel this is somehow related. GitHub supports packages as described here:

https://help.github.com/en/github/managing-packages-with-github-packages

Do you think this is something that we can utilize for LabVIEW package distribution?

 

[JamesMc86]

Yeah that would be cool but it is a front for different package technologies i.e. npm, nuget

It would involve persuading GitHub to run a LabVIEW package server - maybe not impossible - but probably not high up there list.

Many package managers can just read GitHub repos as a package though - so rather than just entering a name you enter a path to GitHub - that seems more achievable if GPM takes off

 

[joerghampel]

+1 for allowing GitHub as a GPM repo (and GitLab, too).

I watched Derek's CLA Summit presentation on What's new in GPM and was intrigued by the local/private repositories! Don't get me wrong, we already share our own libs and might also do so via GPM, but this makes it probably viable to use it for our customer projects. I will definitely have a play with it in the holidays (and am looking forward to it 🙂 )

[ShaunR]

 

5 hours ago, Michael Aivaliotis said:

I feel this is somehow related. GitHub supports packages as described here:

https://help.github.com/en/github/managing-packages-with-github-packages

Do you think this is something that we can utilize for LabVIEW package distribution?

Aren't all the addon installers (VIPM, GM and the one I was playing with) about to be made reduntant with the NI Installer now it supports addons?

[JamesMc86]

As yet it doesn't really support code add-ons is my understanding (at least not well). For example I think it still lacks any support for palette editing.

[The Q]

@joerghampel,

GPM does allow local repos and GPM is open source. 
 

Link to MGI’s GPM Wiki:

https://gitlab.com/mgi/gpm/gpm/wikis/How-Tos/Filesystem-Based-Registries

Link to the GPM project:

https://gitlab.com/mgi/gpm

Even I have contributed to GPM along with some of the Composed Systems guys.

@JamesMc86

I’m working on a palette editor now which can be used stand-alone but I’ll work with MGI to add it to GPM. 

Edited December 3, 2019 by The Q

[The Q]

On the topic, I want to be able to publish my code, have others contribute to my code, and contribute to others code as easily as possible. This is regardless of packaging technology. 
 

Also, I want to learn what is out there already, quickly and easily. So I don’t reinvent the wheel in ignorance. 

[Michael Aivaliotis]

On 12/2/2019 at 11:22 PM, JamesMc86 said:

Yeah that would be cool but it is a front for different package technologies i.e. npm, nuget

Well, it seems to be more of a back, than a front. GitHub is becoming the package repo. GitHub is not making a front end manager.

On 12/2/2019 at 11:22 PM, JamesMc86 said:

It would involve persuading GitHub to run a LabVIEW package server - maybe not impossible - but probably not high up there list.

Agree this would be hard or impossible to achieve.

On 12/2/2019 at 11:22 PM, JamesMc86 said:

Many package managers can just read GitHub repos as a package though - so rather than just entering a name you enter a path to GitHub - that seems more achievable if GPM takes off

Ok, ya this is a more attainable goal and a possible path forward. But GPM is not widely adopted and has limitations.

On 12/3/2019 at 1:59 AM, ShaunR said:

Aren't all the addon installers (VIPM, GM and the one I was playing with) about to be made reduntant with the NI Installer now it supports addons?

NIPM will win by default, because it is made and fully supported by NI. However, it has a long way to go to support all the features we (as developers) need. It's a dumb installer and has no smarts related to the LabVIEW environment. For example, you cannot target LabVIEW by version and cannot allow up compiling of code. NI is investing resources to make it better over time and we need to keep pushing them to add features we need. However, NI moves like a big company does, very slow. There are currently 3 package formats: VIPM, NIPM and GPM. It's a mess, and by reading this thread, it seems people are open to ditching packages all together.

[ShaunR]

55 minutes ago, Michael Aivaliotis said:

 VIPM, NIPM and GPM. It's a mess, and by reading this thread, it seems people are open to ditching packages all together.

NIPM is the odd one out. The rest are all zip files (including Github) but NIPM is multi-wrapped GZip and the zlib wwrapper we all use doesn't support it out of the box. I can already install all them (VIPM, GPM, raw zipped archives, Github, Sourceforge etc) except the NI one. But whilst looking at that I investigated SystemLink. That is the way forward, IMO - being able to manage addons, programs and installs from there and push them out if necessary (either from the cloud server or your local server). I'm very impressed with SystemLink and think it's an industry game-changer.

1 hour ago, Michael Aivaliotis said:

NIPM will win by default, because it is made and fully supported by NI.

About time too  I never understood why they went for the JKI one in the first place. I moaned like hell when they first said that the JKI one was to be NIs preferred solution. So I guess they are only about 7 years late.

[Chris Cilino]

There has been a lot of discussion, which is great, but I feel the need to reiterate GCentral's vision and mission.

GCentral envisions a LabVIEW community making the best version of itself by improving its capability through collaboration.

GCentral is a non profit organization:

1. for programmers who need to find, share or collaborate on G reusable code or software engineering tools.
2. that provides a platform for G code packages and collaboration resources.
3.  that is independent and driven by community experts.

GCentral's Mission

Enable LabVIEW programmers to collaborate by

1. removing barriers to finding / using code designed for reuse (packaged code)
2. removing barriers to contributing code designed for reuse (packaged code)
3. removing barriers to co-developing code
4. using code with confidence

GCentral is package technology agnostic / SCC agnostic 

1. GCentral does not endorse or encourage the use of one package manager over the other, nor will we. Each community member can package their code according to their preference.
2. GCentral does not endorse or encourage the use of one Source Code Control Provider (local or cloud based) over the other nor will we. Each community member can use the SCC they prefer.

GCentral will ease the pain we all feel when attempting to find and use packages by

1. index the currently available public repositories (Tools Network, GPM, JKI Tools, NI Packages) 
2. by indexing an new, un-gated, cloud based storage location that can house any package type. 
3. by displaying the index results in a web page / APIs, etc (see https://www.gcentral.org/ for the prototpye)

GCentral will ease the pain we all feel when attempting to contribute packages by

1. creating new, un-gated, cloud based storage location that can house any package type (not source). 
2. MAYBE creating software to transport built packages from build machine to the new cloud storage location

GCentral will ease pain we feel when attempting to co-develop code by

1. Creating template projects for each of the major online SCC. (GitHub, etc).
2. Coming pre-configured to build the package type of your choice and upload to the GCentral package server. 

GCentral will inspire confidence by

1. Making any submitted package always available. Once submitted, a package cannot be deleted apart from a GCentral administrator. As a result, you can depend on a package without fear of it ever missing.
2. Product pages per package designed to educate on the package and author.

The above is a summary of the CLA summit presentation I gave (https://sites.google.com/gcentral.org/website/about-gcentral)

 

The advent of the GitHub Package Registry is very interesting. I've reached out to GitHub to provide clarity on how extensible their framework is. At time 29:44 in the presentation Michael linked above the presenter says "We have a great extension framework for adding support for new registries, which will be opening up in the future". That MAY mean we can provide plugins for their registry to recognize NIPKGs, VIPs, GPKGs. And that may completely solve the "find/use" pain point i mention above... so long as the community is ok putting their packages in GitHub AND sacrificing confidence that the package will always be available to use or link against. 

In conclusion, GCentral's aim is to impose the least amount of infrastructure on a community member while enabling us to find/use, contribute, co-develop packages designed for reuse. GCentral will use already existing technologies to accomplish its goal and create new technologies where needed.

[The Q]

This discussion on package manager capabilities definitely needs to happen but just to summarize Chris' post:

Quote

GCentral's aim is to impose the least amount of infrastructure on a community member while enabling us to find/use, contribute, co-develop packages designed for reuse. GCentral will use already existing technologies to accomplish its goal and create new technologies where needed.

Therefore, will use what we have and improve as we go.

Edited December 4, 2019 by The Q
Added bottom line

[Michael Aivaliotis]

On 12/4/2019 at 4:21 AM, ShaunR said:

About time too  I never understood why they went for the JKI one in the first place. I moaned like hell when they first said that the JKI one was to be NIs preferred solution. So I guess they are only about 7 years late.

NI caused this problem themselves. NIPM should have provided all the features of VIPM from the start and then added GPM features. Lack of investment. Now they're playing catchup, but technology is moving on.

[Rolf Kalbermatter]

On 12/5/2019 at 11:21 PM, Michael Aivaliotis said:

NI caused this problem themselves. NIPM should have provided all the features of VIPM from the start and then added GPM features. Lack of investment. Now they're playing catchup, but technology is moving on.

I think it is a little far fetched to request full VIPM feature completeness from any new package management system. But NIPM was more an NI attempt to do something along the lines of MSI than VIPM despite their naming choice and marketing hype. As such it hasn't and still doesn't really support most things that VIPM (and OGPM before) where specifically built for. It was just NI's way to ditch the MSI installer technology which got very unwieldy and can't really handle NI software package installs well (I find it intolerable that a software installation for the LabVIEW suite with moderate driver support takes anything from 4 to 6 hours on a fairly modern computer. That's some 6GB of software to install and shouldn't take that long ever!) Also MSI can't ever be used for support realtime installations as it is totally built on COM/ActiveX technology which is a total no go for anything but Windows platforms. 

Unfortunately NIPM had and still has many problems even in the area where it was actually built for. It doesn't often work well when you try to install a new LabVIEW version alongside a previous one. That was something that worked at least with the old MSI installers pretty well. And yes despite its name it hasn't really any VIPM or GPM capabilities specifically targetted at LabVIEW library distributions and the configuration part to build such packages is very lacking.in the 

I suppose they were eventually planning on making NIPM the underlaying part of installing software through System Link and not have it any user facing interface at all, but had to do something in the short term. As it is it doesn't look to have enough features for anything but for NI to distributing their software installers.

Edited December 8, 2019 by Rolf Kalbermatter

[Michael Aivaliotis]

7 hours ago, Rolf Kalbermatter said:

I think it is a little far fetched to request full VIPM feature completeness from any new package management system.

It was not developed in a bubble. There was already a close relationship between the VIPM team and NI. So they knew the requirements and the need. It's been over 5 years since the release of NIPM, so not much movement however...

7 hours ago, Rolf Kalbermatter said:

But NIPM was more an NI attempt to do something along the lines of MSI than VIPM despite their naming choice and marketing hype.

That might have been the first step, but hardly the long-term plan. In reality, I think the main reason for lack of development on NIPM is that people got shuffled around and new management came in and the original development plans for NIPM got put on a shelf. I can tell you that NI had the goal of full VIPM replacement.

There is much churn internally at NI on where to take NIPM next. They are back to wanting to add features to facilitate better reuse library installation for current LabVIEW (how to achieve this is not clear). For sure however, this is the clear case with NXG.

I suggest you watch this video: 

 

[ShaunR]

13 hours ago, Michael Aivaliotis said:

I suggest you watch this video:

Looks like users are expecting it to solve their version control (or lack thereof)  problems. I don't think NI should be entertaining that for a package builder.