Hosting Git and Mercurial Hg with Heptapod07.11.24

Heptapod installation guide

# Heptapod installation guide

## Important message about migrations

Do **not** migrate directly from Heptapod 0.8 to 0.12.
See the dedicated section at the end of this document

## Installing from source

Heptapod installation is similar to GitLab's except that

- some components have to be taken from our Mercurial repositories
   and built specifically.
- we have additional components for Mercurial.
- some specific configuration has to be set in place.
- we have more constraints, many of which will be hopefully lifted in
  forthcoming versions

### Limitations

the Rails application, GitLab Shell, Mercurial and the repositories must
be in a single system.

### Fetching the standard GitLab components

Please use the mirror URL for initial and repetitive clonings.

- Rails application (replacement for `gitlab`): https://mirror.octobus.net/heptapod/heptapod
- Heptapod Shell (replacement for GitLab Shell): https://mirror.octobus.net/heptapod/heptapod-shell
- As of Heptapod 0.20.0rc1, Heptapod Workhorse sources are bundled with the
  Rails application and don't have a separate repository any more. See the
  dedicated section below for how to build it.

All these have version numbers that are independent (but sometimes coincide) with
their upstream counterparts:

- `VERSION` is the upstream version
- `HEPTAPOD_VERSION` is the Heptapod specific version
- The tag for Heptapod version x.y.z is `heptapod-x.y.z`.
- All tags taking the form `vx.y.z` are upstream tags and must be ignored.

See heptapod#352 for explanation and rationale of the version policy.

As is the case with upstream GitLab, the repository for the Rails application
plays the role of the main repository. It contains additional version files to
specify the versions of Heptapod forked components:

Example:

- checkout the Rails application at the wished Heptapod version

```
     $ hg clone https://mirror.octobus.net/heptapod/heptapod
     $ cd heptapod
     $ hg update heptapod-13.7.0

- check version files:

```
     $ ls HEPTAPOD_*_VERSION
     HEPTAPOD_SHELL_VERSION
     $ cat HEPTAPOD_SHELL_VERSION
     13.7.0
  ```

- checkout a forked component for the wished Heptapod version

```
     $ hg clone https://mirror.octobus.net/heptapod/heptapod-shell
     $ cd heptapod-shell
     $ hg up heptapod-13.7.0
  ```

All other standard GitLab components (Gitaly, etc) have to be taken
from the canonical upstream locations, at the version specified at the root
of the Rails application sources (`GITALY_SERVER_VERSION` etc).

### Building Heptapod Workhorse

Warning: this is new in Heptapod 0.20 series, representing a major change from
previous instructions. Don't hesitate to refer to the version of this file
shipping with the exact version you are installing.

Heptapod Workhorse is the Heptapod fork of GitLab Workhorse, a specialized and
mandatory reverse proxy sitting between the ingress and the Rails application,
taking care of diverting HTTP requests to static files or repositories after
authorization by the Rails application. The sources are provided [with the
Rails application](workhorse/).

1. Choose a directory path to host the built binaries, for example
   `/path/to/workhorse/bin`, but make sure this directory does not exist yet.

Note: the rake task will actually initialize the target directory
   with a `git clone` command, just to bring in a README file.
   If the directory already exists and is
   not a Git clone already, this will result in an error.

2. Launch the rake task from the Rails application checkout:

```
   bundle exec rake 'gitlab:workhorse:install[/path/to/workhorse/bin]`
   ```

(some shells don't need those quotes. `zsh` does.)

3. Check that this is really Heptapod Workhorse:

```
   $ /path/to/workhorse/bin/gitlab-workhorse -version
   gitlab-workhorse heptapod-0.20.0rc1-20210221.191739
   ```

### Additional required external executables

HGitaly, the internal service exposing Mercurial content to other components,
relies on several executables to be available. The paths can be adjusted via
configuration.

A full list is available in [HGitaly README](https://foss.heptapod.net/heptapod/hgitaly#external-executables). You might want to check the same file in the
tag corresponding to the exact HGitaly version.

### Installing Mercurial components

We need a specific Mercurial version, a WSGI server (usually Gunicorn), and
some additional Python packages from source.

You can install them system-wide if that suits you (Mercurial has very few
dependencies, it's very unlikely it'll clash with system libraries) or with
`pip install --user` or in a virtualenv. What matters for proper operation is
that they are all importable from the final Mercurial executable.

In the sequel `$PIP` is `pip --user` or `some/virtualenv/bin/pip`, or just
`pip`, according to your choice.

Mandatory prerequisites:

- `pip<23` (version 23 breaks support for the needs of Mercurial with Rust)
- Rust toolchain. The minimum version is 1.65 as of this writing.

#### Python versions

As of Heptapod 0.36, Heptapod supports Python 3.9 and 3.10 only.
Python 3.11 is not supported yet.

1. Install Mercurial enhanced with Rust:

```
$PIP install --no-use-pep517 -r python/hg-rust.req
```
2. Install other Heptapod Python components

```
$PIP install -r python/common.req
```
(notice how this doesn't pass the `--no-use-pep517` option)

3. check: `/path/to/just/installed/hg debuginstall`. Here's an excerpt of
  the expected output:

```
  checking Python security support (sni,tls1.0,tls1.1,tls1.2)
  checking Rust extensions (installed)
  checking Mercurial version (6.4.5)
  checking Mercurial custom build ()
  checking module policy (rust+c)
  ```

"module policy" is the configuration for the runtime usage of native Python
  modules. In the context of Heptapod 0.36, it has to be `rust+c`.

4. take note of the exact HGitaly version

```
  $ $PIP freeze | grep hgitaly
  hgitaly==0.33.2
  ```

(HGitaly is the main internal service exposing Mercurial content to other
  Heptapod components, quite similarly to what Gitaly does for plain GitLab).

### Build RHGitaly

RHGitaly is a partial reimplementation of HGitaly in the very efficient and
safe Rust language. It is used by default from Heptapod 0.38 onwards, but can
be disabled by the `rhgitaly`
[feature flag](https://docs.gitlab.com/ce/operations/feature_flags) if needed.

0. Read the wished RHGitaly server version

```
   RHGITALY_VERSION=`cat RHGITALY_SERVER_VERSION`
   ```
1. Download the sources tarball, for the exact same version than HGitaly (see
   above)

```
   wget https://download.heptapod.net/rhgitaly/rhgitaly-${RHGITALY_VERSION}.tgz
   wget https://download.heptapod.net/rhgitaly/rhgitaly-${RHGITALY_VERSION}.tgz.asc
   ```

Note: RHGitaly is not released yet to crates.io, hence cannot be downloaded,
   built and installed from scratch with a simple `cargo install` command.

2. Check signature

This assumes you already got Heptapod [package signing keys](https://heptapod.net/pages/keys.html):

```
   gpg --verify rhgitaly-${RHGITALY_VERSION}.tgz.asc
   ```

3. Unpack and build

```
   tar xzf rhgitaly-${RHGITALY_VERSION}.tgz
   cd rhgitaly-${RHGITALY_VERSION}
   cargo build --path rust/rhgitaly --locked --release
   ```

4. Install to the wished target directory.

(check you have enough rights on the target directory)

For example:

```
   mv rust/target/release/rhgitaly /usr/local/bin
   ```

Note: you may also use `cargo install` with the `--release`, `--path` and
`--root`  for steps 3 and 4 above, but it would perform the whole with
the privileges needed to write to the install directory.

### Configuration

You need to

1. define Mercurial settings, including some necessary paths
2. create Mercurial internal services
3. configure GitLab components to use the proper Mercurial with those settings
4. take care of few special cases, depending on the Heptapod version

#### Defining Mercurial settings

The `heptapod` Python project, listed in `requirements.txt`,
ships with a configuration file, `required.hgrc`, that performs most
of the required and default Mercurial settings for Heptapod
(using the `heptapod` extension, permission policy, etc.)

You'll find it where your `$PIP` has installed `heptapod`, which you can
find out like this:

```
export PY_HEPTAPOD=`$PYTHON -c "import heptapod, os; print(os.path.dirname(heptapod.__file__))"`
```

where `$PYTHON` is the python interpreter that's consistent with `$PIP`. So,
for instance if you're using a virtualenv, and `$PIP` is
`/path/to/virtualenv/bin/pip`, then `$PYTHON` is
`/path/to/virtualenv/bin/python`. If you're using
the global pip, then `$PYTHON` is most likely just `python` or `python3`.

You'll need to use `$PY_HEPTAPOD/required.hgrc` and add your local
installation details.

A simple way is to create a `heptapod.hgrc` file using the `%include` directive.
See the example in `$PY_HEPTAPOD/heptapod.example.hgrc`.

##### Configuring the repositories root
Heptapod's Mercurial services have to know where the repositories are.
This is done in the hgrc file, like so:

```
[heptapod]
repositories-root=/path/to/repositories
```

This is very similar to configuring the repositories root in plain upstream
GitLab (Gitaly).

##### Configuring Mercurial for interaction with the Rails application

Various Mercurial processes need to ask the Rails application for permission
on behalf of the end user, and notify changes in the repositories.

For that to work, you need tell it where to find the `.gitlab_shell_secret`
file and configure the internal API address. This is done in the `[heptapod]`
section of your `heptapod.hgrc`:

```
[heptapod]
gitlab-internal-api-secret-file=/path/to/.gitlab_shell_secret
gitlab-internal-api-url=http+unix://%2Fpath%2Fto%2Fgitlab.socket
```

Notes:

* the default internal API URL is `http://localhost:8080`.
* the older `heptapod.gitlab-shell` setting is still supported to find
  the `.gitlab_shell_secret` file

If you're upgrading from Heptapod 0.12 *and* the Rails application is binding
to `localhost:8080` (the default with unicorn) then the configuration should
be readily compatible.

### Internal Mercurial HTTP service

HTTP access to Mercurial repository content is provided by an internal HTTP
service that sits behind the Heptapod Workhorse reverse proxy.

Warning: do NOT have it listen to any other interface than the loopback:
its security model is to trust Heptapod Workhorse blindly.

It also needs to load your hgrc file(s). Here's an example, assuming `gunicorn`
is on the path.

```
  HEPTAPOD_HGRC=/path/to/your/heptapod.hgrc gunicorn -w 5 \
      -b 127.0.0.1:8000 \
      --error-logfile /tmp/gunicorn.log \
      --capture-output \
      heptapod.wsgi:hgserve
```

Again, this `gunicorn` must be able to load the Heptapod Python packages, so
you should use the one that's been installed with `requirements.txt`.

Notes:

- Heptapod is not able yet to use Unix domain sockets in this service
- If you want to use another loopback URL, you must also pass it to Heptapod
  Workhorse with the `-hgBackend` option (whose default is
  ``http://localhost:8000``)

### HGitaly service

Mercurial repository content is exposed to various GitLab components by the
HGitaly service. Eventually, it will become the only way GitLab can access it,
except when proxying HTTP user requests.

Again, it needs to know about your HGRC. Here's an example, assuming the `hg`
executable to be on `$PATH`:

```
  HGRCPATH=/path/to/your/heptapod.hgrc hg \
      --config extensions.hgitaly= \
      hgitaly-serve \
      --listen unix:///path/to/hgitaly.socket
```

Startup and shutdown messages will go to stdout and stderr, as well as
potentially rare cases of uncatched exception (a bug in itself).

Actual logs are emitted as configured in the HGRC file.

Here is a sample systemd service unit file, just fix the paths and/or the
system user and it should do the job.

```
[Unit]
Description=HGitaly, internal Heptapod service for Mercurial handling
After=network.target

[Install]
WantedBy=multi-user.target

[Service]
User=heptapod
Group=heptapod
Environment=HGRCPATH=/path/to/your/heptapod.hgrc
ExecStart=/path/to/virtualenv/bin/hg --config extensions.hgitaly= hgitaly-serve --listen unix:///path/to/hgitaly.socket
Restart=on-failure
```

Note: you may also configure HGitaly to bind on a TCP port
(e.g., `--listen tcp://127.0.0.1:9237`), and in that case, you'll get better
scalability because it will launch several worker processes.

### RHGitaly service

RHGitaly only implements a few requests of the Gitaly protocol, but these
are the most common ones, and it is 50 to 100 times faster than HGitaly,
non-blocking, and has a very small memory footprint.

As of this writing, RHGitaly configuration is entirely done with environment
variables and consists only of the path to
the repositories base directory and the address to bind to.

Logs are emitted to `stdout` and/or `stderr`.

Here is a minimal systemd unit file:

```
[Unit]
Description=Heptapod RHGitaly Server
After=network.target

[Service]
User=heptapod
Group=heptapod
Environment=HGRCPATH=/path/to/your/heptapod.hgrc
Environment=RHGITALY_LISTEN_URL=unix:/path/to/rhgitaly.socket
Environment=RHGITALY_HG_EXECUTABLE_PATH=/path/to/hg
# If HGitaly binds to tcp://host:port, then replace the sidecar
# address below by http://host:port
Environment=RHGITALY_SIDECAR_ADDRESS=unix:/path/to/hgitaly.socket
Environment=RHGITALY_REPOSITORIES_ROOT=/home/hg/repositories
ExecStartPre=/usr/bin/rm -f /path/to/rhgitaly.socket
ExecStart=/path/to/bin/rhgitaly
Restart=on-failure
```

(notice how we need to clean any lingering socket. Another option would be
to configure with a TCP URL, such as `tcp://localhost:9238`, but that doesn't
provide any scalability benefits, contrary to the HGitaly case).

#### Using Mercurial in GitLab

Edit the configuration files for the Rails application
(see `config/gitlab.yml.example`) and GitLab Shell (see `config.yml.example`),
to provide

- the full path to your `heptapod.hgrc` file and to the `hg` executable in
  the `mercurial` section.
- the path to the HGitaly and RHGitaly sockets in the `repositories` section

Notes:

- in those YaML files, `hgrc` must always be a list. This is for separation
   in several files with different persistence properties.

### Validation with the functional tests

- start from an empty set of data (do not even set the instance root password)
- install tox (Python3)
- grab the functional tests:
  `hg clone https://dev.heptapod.net/heptapod/heptapod-tests`
- run the tests as the same user as the GitLab components, with something
  like:

```
  tox -- --heptapod-ssh-port 2222 \
           --heptapod-url http://localhost:3000 \
           --heptapod-source-install \
           --heptapod-repositories-root ~/heptapod/gdk/gl10-5/repositories
  ```

(for a quicker first test, you can add `-k push_basic`)

## Migration notes

With GitLab migrations in general, it's not possible to jump between too much
different versions.

See the detailed migrations page [on Heptapod web site](https://heptapod.net/pages/upgrade.html)

https://foss.heptapod.net/heptapod/heptapod/blob/branch/heptapod/INSTALL_HEPTAPOD.md

Anonymous

Hosting Git and Mercurial Hg with Heptapod07.11.24

Information Epoch 1772890306