Jump to content
Michael Aivaliotis

OpenG Library Exported to Github

Recommended Posts

Cool!

What tool have you used to import the whole SVN repo with all the revision into the Git repo?

Is it a feature in Github?

Edited by Antoine Chalons

Share this post


Link to post
Share on other sites

I followed these instructions:

I did that because my goal initially was to export to bitbucket, which I did. Then I changed my mind and decided that Github would be better for a community project like this. So then, since it was already in a Git format it was simple from within the Github website to select "import" and just point to the Bitbucket URL.

Note: Those instructions work best if executed from a linux machine. I quickly spun-up an Ubuntu VM to do this.

  • Like 2

Share this post


Link to post
Share on other sites

Well, I was hoping someone would continue the discussion, so great! We can redo the conversion. But is it really that critical to migrate the history intact? I question the need for that. If not we can start fresh.

  • Authors - The conversion I went through had the ability to add email addresses to the author names. I just don't know what the email addresses are for the authors in Sourceforge. I've attached the author list if anyone wants to help flesh out the email addresses.Then we can rerun the conversion.
  • Branches and tags. - I looked at the SVN repo and there's only one branch, SVN does not due branches well, so I'm not surprised nobody used that feature. There are a few tags and those are very old, circa 2007. Not sure if anybody cares about those. It seems the tagging procedure (if any) was dropped long time ago. You should be tagging with every release but that does not appear to have happened.
  • One repo - The original SVN was a single repo, this is why i kept it the same. The conversion is a lot simpler. Breaking up a single SVN repo into multiple GIT repos and keeping the history intact seems complicated if you have commits that include files that cross library virtual boundaries. If you can think of a way to do this, that would help.
  • Commit messages. - The commit messages are all there. It's the URLs inside the messages that are not pointing correctly, but I'm not sure what they should be point too and how to fix that during the conversion. Also, what if SourceForge changes the URL structure later? Again, is this important?

Well, this is a good starting point. The alternative is to start fresh and create multiple GitHub repos, with the latest revision of the source. Then the SVN repo can be an archive if anyone wants to get at it.

I welcome your help if you can create scripts to solve some of the above problems.

 

authors.txt

Share this post


Link to post
Share on other sites
59 minutes ago, Michael Aivaliotis said:

Authors - The conversion I went through had the ability to add email addresses to the author names. I just don't know what the email addresses are for the authors in Sourceforge. I've attached the author list if anyone wants to help flesh out the email addresses.Then we can rerun the conversion.

The simplest way is to add "@users.noreply.sourceforge.net" to make them valid addresses that don't collide with any existing GitHub user (GitHub and SF both support these). It should be sufficient for this task. Here is my version: https://gist.github.com/LogMANOriginal/c4109873a5d524387d3fb46f5b83aa0a

59 minutes ago, Michael Aivaliotis said:

Branches and tags. - I looked at the SVN repo and there's only one branch, SVN does not due branches well, so I'm not surprised nobody used that feature. There are a few tags and those are very old, circa 2007. Not sure if anybody cares about those. It seems the tagging procedure (if any) was dropped long time ago. You should be tagging with every release but that does not appear to have happened.

I agree. From what I can tell the reason it didn't work correctly is because each project has its own branch/tag subfolder. None of the standard systems support these (including git-svn). Find below a solution that can handle them.

59 minutes ago, Michael Aivaliotis said:

One repo - The original SVN was a single repo, this is why i kept it the same. The conversion is a lot simpler. Breaking up a single SVN repo into multiple GIT repos and keeping the history intact seems complicated if you have commits that include files that cross library virtual boundaries. If you can think of a way to do this, that would help.

Done, see below.

59 minutes ago, Michael Aivaliotis said:

Commit messages. - The commit messages are all there. It's the URLs inside the messages that are not pointing correctly, but I'm not sure what they should be point too and how to fix that during the conversion. Also, what if SourceForge changes the URL structure later? Again, is this important?

Sorry, my explanation wasn't very clear. My point is, that the original commit message actually doesn't have these URLs inside (see SF vs. GitHub). It was probably added during the Atlassian conversion process.

59 minutes ago, Michael Aivaliotis said:

The alternative is to start fresh and create multiple GitHub repos, with the latest revision of the source. Then the SVN repo can be an archive if anyone wants to get at it.

I welcome your help if you can create scripts to solve some of the above problems.

Here you go: https://gist.github.com/LogMANOriginal/fa1e59703c41e27758bcb935f15bea21

The script must be placed in an empty folder next to 'authors-transform.txt'. It must be marked as executable (runs in bash, not sh). svn and git must be installed of course. Then start the script via './openg_import.sh' and it should work (unless it breaks of course).

It does a few things:

  • Lookup all projects, branches and tags
  • Import all projects into dedicated Git repositories
  • Cleanup the Git repository (i.e. tags are imported as branches, so they need to be fixed)

The output are 39 bare git repositories, one for each project with full history, branches, tags and commits. Enjoy 😉

Edited by LogMAN
  • Like 1
  • Thanks 1

Share this post


Link to post
Share on other sites

Great news @Michael Aivaliotis! @LogMAN thanks for supporting the process and publishing transformation code. 

Should we then wait with potential forking an merge requests for all the git-housekeeping of the repo?

Share this post


Link to post
Share on other sites
13 hours ago, Michael Aivaliotis said:

Thanks @LogMAN. After I get paying work done, I will try this out. For anyone else following along. If you are currently an OpenG developer ( @Rolf Kalbermatter?) and want to be added to the OpenG team on GitHub, send me a PM.

Well I happen to have committed code in the last weeks to the lvZIP library including yesterday evening. Do I have to redo that to get it in the new GitHub stuff?

And I'm still busy trying to add support for symlinks and utf8 filenames to the lvZIP library. The basic work to support this is mostly done. Just need to properly integrate it in the lvZIP library and then do some corner case testing on the non-mainstream platforms like Mac OSX, Pharlap and VxWorks. MacOSX because it is different to Unix in some ways and Pharlap and VxWorks since they do not support symlinks and multibyte encoding at all.

My SF account name is labviewer.

Edited by Rolf Kalbermatter

Share this post


Link to post
Share on other sites
2 hours ago, Rolf Kalbermatter said:

Do I have to redo that to get it in the new GitHub stuff?

I suggest informing all active contributors, that way everyone involved has a chance to push their changes before import.

2 hours ago, Rolf Kalbermatter said:

And I'm still busy trying to add support for symlinks and utf8 filenames to the lvZIP library.

The import script supports branches. If you upload your current status as a branch (inside the /branches/lvzip/ folder), it will be detected and imported. Once the repository is uploaded to GitHub you can simply continue working on the branch using Git.

2 hours ago, Rolf Kalbermatter said:

My SF account name is labviewer.

You have a chance to match it with your GitHub account (see 'authors-transform.txt'). This is only possible during transformation, however.

Share this post


Link to post
Share on other sites

@Rolf Kalbermatter, I will be redoing the export today based on the new scripts, if they work. So if you committed code recently, it will get included. I think you are the only one working on any code at the moment. I will add you as a collaborator on the repo so you don't need to fork. Once the conversion is complete:

  • Make a branch on the git repo.
  • Checkout the branch
  • Do a file diff between your local SVN and the Git branch.
  • Copy over the changed code from SVN to Git.
  • Commit the branch.

@LogMAN, unfortunately GitHub does not have a good way to group related repositories, so they visually seem to belong together. The only distinction is organizations. There's the Project feature, which I think just helps with development workflows. Having numerous OpenG repositories will make this a bit messy. But I guess there's no way around that. It seems GitLab and Bitbucket have a slightly better approach with this.

Share this post


Link to post
Share on other sites
3 hours ago, Michael Aivaliotis said:

GitHub does not have a good way to group related repositories, so they visually seem to belong together... Having numerous OpenG repositories will make this a bit messy.

A poor man's way of doing that is to prefix all of the repository names with "OpenG". Users can then use the "search" feature within the Organization to find all repos related to OpenG. For example, https://github.com/ni?q=niveristand vs. https://github.com/ni

There's also GitHub Topics, although it crosses organization boundaries: https://help.github.com/en/articles/classifying-your-repository-with-topics

Share this post


Link to post
Share on other sites

@Michael Aivaliotis You are right, organizations are the only way to group projects. Just as @JKSH said, a prefix is very helpful in finding things.

On 10/5/2019 at 9:00 PM, Michael Aivaliotis said:

@LogMAN, Is it possible to remap a source forge username to a GitHub username?

On 10/5/2019 at 9:43 PM, Michael Aivaliotis said:

I've started adding repos:

https://github.com/Open-G

Looks like you figured it out :)

Share this post


Link to post
Share on other sites
3 minutes ago, LogMAN said:

Looks like you figured it out :)

Sort of. I think Github uses the email address, not the username. I've asked @Rolf Kalbermatter for his GitHub email, because it didn't work with his username.

I decided to leave the repos under GCentral and use the OpenG prefix as suggested.

Share this post


Link to post
Share on other sites

I noticed all of the repos have their own ToDo.txt. Many are empty but for those that are not, you might want to create an issue for each item.

I don't know if you can create a wiki for an organization but I would also want to be able to find information on how to contribute and some specifics like what LabVIEW version is being used for development, expectations on testing, and build/release process.

Share this post


Link to post
Share on other sites
9 minutes ago, jacobson said:

I noticed all of the repos have their own ToDo.txt. Many are empty but for those that are not, you might want to create an issue for each item.

I don't know if you can create a wiki for an organization but I would also want to be able to find information on how to contribute and some specifics like what LabVIEW version is being used for development, expectations on testing, and build/release process.

Good points. You can't create a wiki for an organization. I've setup an OpenG team, but even there, no wiki support. Limitations... However, each repo has its own wiki. Which is good for content related to that specific repo. It's possible now with separate repos, to have each one have it's own LV version and build/release process if needed. However, if not, then each wiki can simply have a link to some global page that covers the overarching policies. This could be on LAVA, but I think the LabVIEW Wiki could provide support here. There could be a single page there covering everything, which is also community editable.

Creating issues can be done by anyone. So if you want to do that, go for it. If they are invalid, they can be closed later. We should also search here son LAVA and create issues for stuff reported by people in the past.

Share this post


Link to post
Share on other sites
34 minutes ago, Michael Aivaliotis said:

You can't create a wiki for an organization.

You can create a GitHub-hosted website for an organization: https://pages.github.com/

Share this post


Link to post
Share on other sites

Here are some common files that could be helpful:

All of these files, if they exist, add to the GitHub experience. For example: https://github.com/microsoft/vscode

  • Like 1

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.