ROSE Compiler Framework/Source Code Repository

Motivation

Why are we considering this?

A single monolithic git repository for ROSE becomes too large
- not productive to clone big repo if people only care about a single portion of it
We have many collaborative projects using ROSE. Many project contributors are forced to live with the same high-standard as the core ROSE contributors for commits
- Same rigid regression tests by Jenkins for different platforms, different versions of GCC, boost, etc. This is not always necessary for some projects.
We have some LaTeX documentation and HTML web pages within ROSE's git repository. They essentially can be standalone. But even minor fixing typo for them will require lengthy Jenkins tests (~10 hours or more) to be made effective. This really turns down our appetite to add/update our documents.

Have lean, clean, and well organized source code repositories for the ROSE project

lean (<=100 MB): only essential things should be stored in the central core repository
clean: only code-reviewed and tested things are allowed be added into central repositories
well organized: intuitive, standardized directory layout

Facilitate collaborations

allowing incremental/alternative testing requirements for collaborative, experimental projects

Other requirements: mostly already taken care of by using git

Find out how other big projects manage multiple repositories ??
remove unnecessary big files: remove from the repository and from the history
split the single repository into smaller ones.
- Allow users to download/clone only the things they care.
- make sure they all work together and tested as a whole when needed
prepare sample repositories to demonstrate how different types of repositories should be used

The biggest concern is that by having multiple repositories,

it is hard to make sure non-core repositories always work with the core ROSE repository
Or changes to the core repository may break other repositories without being noticed
make sure the accurate linking between a project and the version of librose.so

How to prevent these problem?

Commits to ROSE core repo will trigger tests of all official projects. Only commits passing all project tests will be accepted into the core repo.
Similarly, changes to any official project will trigger tests against latest ROSE. Only commits passing all ROSE tests will be accepted into the individual project
Better versioning of librose.so: non-official (external) projects can easily detect if a version of librose.so is compatible.

The absolutely necessary things for 80% ROSE users should be put into the core repository of ROSE.

It should contain only

Submodule:

Questions

Why do we want to separate some documentations out of the ROSE repository?

Right now, any changes (even fixing typos) to documentations within ROSE require the same rigid regression tests by Jenkins. This is not necessary and wasteful.

We may have two choices:

Having a single monolithic repository for all projects.
Allowing individual repository for each project. So people can just clone what they want. Each project can have its own lifecycle (abandoned, merged into rose, separated for ever, etc.)

I think in a long run, most projects should be treated as

They are important and useful, but they somehow are not merged into the core of ROSE due to various reasons.

The current idea:

Individual projects should be separate repositories, configurable "--with-rose=/path/to/rose/installation" to make the ROSE-to-project connection.
Collections of all official projects should be contained in a monolithic repository (e.g. "rose-projects"), where each project is simply a submodule

To be discussed

We may allow users to contribute projects which only use simple Makefile. Staff members or paid interns can be in charge of converting the build system to use autotools.

Common test input files, benchmarks we use to test robustness of the core of ROSE.

Projects can also leverage the same set of tests.